User experience (UX) survey best practices

What is a user survey?

A user survey is a questionnaire designed specifically for use in user testing and UX research. While surveys have been a core component in applications like market research and academic studies, they’re less central to the field of user experience.

Your user survey questionnaire will be circulated to user testing subjects, who might be existing customers, members of a research panel or early adopters you’ve recruited at an initial stage of product development. It will capture their thoughts and feelings about the thing you’re testing.

User testing surveys can be completed independently in the participant’s own time, or you can use them as a framework for moderated lab-based or live online research, with the researcher using the questionnaire as a basis for their interview.

Get started with our free survey maker tool

Why are surveys so valuable in user research?

Surveys are used time and again as a research tool because they offer a combination of rich insights and structured, consistent data. Using a survey, you can get a detailed description of how someone feels about an experience, then take their data and combine it with the responses of many other participants. Since every questionnaire contains the data in the same format, it’s easy to analyze the combined data and discover trends and patterns within it.

A well-constructed questionnaire can be reused again and again, so you can make like-for-like comparisons between the responses to one iteration of your product and the next, or between two variants of a creative design you’re comparing head to head.

Surveys provide self-report data, which means they don’t give you the same kind of knowledge as testing that involves observing a user’s interactions. They aren’t a replacement for other forms of user research and user testing, and should be considered part of your best practice tool kit alongside moderated and unmoderated testing, interviews, observation, diary studies, card sorts and other techniques.

Best practices for user surveys

1. Use plain, neutral language

Your wording needs to avoid skewing the tone of the question towards a positive or negative response. Descriptive words, and even the order of the words in a sentence, can introduce bias. Then there’s the risk of acquiescence bias – the tendency to agree. For example, if you phrase your question positively (‘I like this website’) you may get more positive responses than if you frame it negatively (‘I dislike this website’).

2. Mix up your question types

Not all questions are created equal. Some question types, such as open field responses, are high-effort for your respondents to answer, whereas others, such as multiple-choice, are less taxing. Make sure you use a range of different question types in your survey to help participants get through to the finish line. Different question types also yield different kinds of data, so your results will benefit too.

3. Add open-field responses

They may be high-effort to answer, but open field responses also play an important role in picking up context and specific detail from your respondents. They are valuable additions to a multiple choice question, as they give respondents the chance to explain a ‘none of the above’ or ‘other’ response.

4. Ask one thing at a time

It sounds obvious, but make sure each question addresses just one topic. It can be tempting to combine multiple aspects of the subject matter in a single question, especially when you’re trying to keep things short and sweet. But the result is a confused user and data that isn’t useful for your research. For example, if you ask “how do you feel about the taste and texture of this ice-cream?” you may get confused responses from people who loved the taste but hated the texture.

5. Keep your questionnaire short and focused

The longer your questionnaire, the more likely it is that people will drop out before they get to the end. Make your questionnaire only as long as it has to be to get the information you need. It’s also best if the survey as a whole is coherent and consistent, addressing a specific theme rather than adding in a range of different subjects.

6. Be specific

Your question wording should steer your respondent towards providing the type of information you need. Add specifics to help avoid ambiguity. For example, rather than asking ‘what’s your favorite restaurant?’, which might prompt them to tell you about a great place in another state that closed in 1992, try asking ‘thinking about restaurants you’ve visited in Arkansas in the last year, which is your favorite?’

7. Keep an eye on question order

Question order matters because questions early in the survey can influence how the respondent answers later ones. To help offset question order bias, you can try randomizing questions, or if appropriate, blocks of related questions, so that not everybody gets them in the same order.

8. Let people skip questions

Making the wrong questions mandatory can cause participants to drop out because they don’t want to answer. Always give them the opportunity to a skip a question unless it’s something essential, such as the consent question at the start of your survey.

Questions to ask in a user survey

For many researchers, the first port of call when designing a survey for user testing is the System Usabilty Scale (SUS), a standardized 10-item questionnaire that’s well-known in the UX industry.

The SUS is specific to usability, and returns a number score based on the results from the 10 questions. These are presented in a Likert scale format (so participants can agree strongly / agree / neither agree nor disagree / disagree / strongly disagree). Notice that the way the questions are phrased alternates between positive and negative statements.

1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.

Of course, the SUS isn’t the only questionnaire template out there, and you don’t have to stick to it if it doesn’t suit your purposes. If you choose to write your own survey questions, think about the kind of data you want to get to and work backwards from there.

For example, if you want to ask about the navigation of an app or website, you could ask things like:

• It’s easy to find what I am looking for in the menu (agreement scale answer)
• The menu items make sense (agreement scale answer)
• How easy is it to navigate this website? (very easy – not at all easy scale)

Or for a more general impression of your app’s design:

• I understand what the app does (agreement scale)
• I know how to use this app (agreement scale)

You may be interested in the emotional reactions to your product or service, and ask questions like:

• This website is frustrating (agreement scale)
• This website makes me feel: (multiple choice + open field)
• I like the way this website looks (agreement scale)

Whichever approach you take to writing survey questions, remember to check and re-check your questionnaire before introducing it to users, as this will save you time and effort in the long run. We recommend that you always pretest your survey before sending it.

Choose a powerful, easy-to-use platform for your user testing survey program

You can design and field your user testing survey with a free Qualtrics account, which includes everything you need to get started.