How to Use Surveys for Your Business and Why They are Important?
When is it good to use surveys for user research?
Surveys are a powerful tool for gathering initial quantitative data about people's problems. You can use that data to point toward problems to investigate more deeply, validating your ideas using methods like user interviews.
Surveys are also handy when you don’t have the time or budget to go deep into user research – you can still gather some actionable insights, and it’s better than nothing.
What are the pros/cons?
Cons
No qualitative data – we know where the problems are but not necessarily why
Can lead to wasted time and budget if questions aren’t carefully formulated
Some users might not not take your survey seriously because they just want the reward
Can’t ask users clarifying questions or dig deeper into their responses
Pros
Cheaper than other methods
Easier to create and deploy
Lots of results in a short period of time
How to plan a survey?
Start with some initial research - or existing info about the product if it’s already been shipped - to help devise your questions.
Decide on what is your purpose – and build your questions based on that goal.
Aim to keep your survey focused around one topic unless you’re offering a large incentive, nobody is going to be focused enough to fill out a survey with 100 questions.
Users are likely to give more meaningful responses to short surveys, stick to 12-15 questions at most.
How to get good quality survey answers?
Pre-screening
Helps to qualify users by sharing a small survey before the main survey and inviting people whose responses showed they are a suitable audience.
Open-ended questions
Out of my typical 15 questions maximum, I tend to include two to three open-ended questions.
Open-ended questions in your survey provide useful qualitative data. This will help researchers filter responses and eliminate people who just filled the space to receive their rewards.
How to write good survey questions?
Survey good practices
1.Limit your survey to 12-15 questions
Asking too many questions can tire users and reduce the quality of responses.
Don't make users focus on too much at the same time – figure out the 12-15 most important things you need to know and ask those.
2. Simple questions
Make your questions as simple as possible to receive concrete answers.
3. No ambiguity
Questions should be interpreted in one way only. Show your survey to 2-3 colleagues or friends and assess if they understood your question. If not, rephrase your question.
4. No double negatives
5. Avoid subjectivity
Asking about how happy a user with something can be very subjective. For one person, “maximum happy” might mean one thing while for another that could be something very different. It would work better if you have scale or choose an objective variant of answers.
6. Phrase questions about difficulty carefully
Users often don’t like to admit that something is difficult. To receive that kind of answer, it’s often better to ask something like “Which task was most difficult?” Even better would be the user testing, where you give users a task and watch how they perform. When you’re looking over their shoulder, users can’t lie – if something was difficult, you can easily see that.
7. Every question should have a goal
It’s common for surveys to ask questions just for the sake of asking them. Is it really important for you to know your user's date of birth, favorite color, and dog’s name? Your questions should be aligned with your research goal. As you’re formulating them, think about what you want to learn from each one. If a question doesn’t align with your goals, consider removing it.
8. Group questions into meaningful sections
Group your questions in 2 - 3 sections for a smooth flow of topic. Group them appropriately, so the person doesn't have to jump from question to question.
How many people to recruit?
Much like user interviews, the answer here varies. Sometimes you might get meaningful data from a smaller number of people. This is case to case basis.
Typically, I start with 60-100 users, I look for meaningful patterns and repeat with more users as necessary. Avoid changing questions so that you can combine your data.
What will be the outcome? How can we use the results?
Most of my surveys produce two main outputs: some quantitative data and the responses to my two to three open-ended questions. Those questions can produce some varied responses that you can group together and even extract some numbers about the frequency of particular responses.
How can we use survey results?
If the insights are clear or you don’t have time to do more research, you can use survey responses to inform your optimization efforts. If you formulate your questions well and 90% of the respondents’ answers validate your hypothesis, you’re probably safe to make decisions based on that.
Survey results are also great for further research. Perhaps something is unclear after the survey, or perhaps it uncovered a problem you’d missed until now. From there, you can use it to build your research plan, go deeper into qualitative research, and write effective interview scripts to uncover more details.
How to analyse the results?
I usually start by analyzing each question and making a simple pie chart of the answers so I can easily scan it.
Then I write my conclusions - what I observed from each question’s responses – and I figure out what insights we can take from that and our next steps.
It’s best to avoid mentioning the percentage only! Always include the actual number of users alongside the percentage because we include the fraction of all the potential users base in the research.
What tools to use for creating surveys?
The most popular ones are Typeform and Google Forms. They're straightforward and simple to use for beginners, so I’d recommend those.
Where to share them?
Just like other research methods, the best thing is to recruit users wherever your target audience can be found. If you’re an eCommerce company selling Anime collectibles, then maybe a car forum isn’t the right place to find users.
This will vary from business to business – you might consider a small pop up on your website, send an offer out to your email list, or employ specialized panels like Respondent.io or userinterviews.com
Can we rely on survey results only?
Ideally not! In general we try to triangulate issues by confirming them using multiple research methods.
But if you don’t have time or budget for more research, you can sometimes get away with it IF:
You are sure you recruited the right people
You are certain you have the correct survey questions that can only be interpreted one way
AND it is clear that the survey results confirm your hypothesis
If options are available, it’s better to check your conclusions with other methods.
What to do if we are not sure how to interpret some answers?
When in doubt, it’s always best to check. If there’s ambiguity about some answers, it’s best not to include them in your results or check your results using other methods.
Tools like screen recordings and heat maps can help clear things up, make sense of your responses, and save time and budget vs. expensive user interviews.
Long form vs pop up survey
If I can, I usually choose longer surveys because they provide more information. Short form surveys could be very useful if you’re trying to get specific insights about a limited topic.
Short surveys have some advantages – they can be good for asking one or two questions and it’s probably easier to recruit people - you can get more responses faster. The downside is that with just one or two questions, you’re very limited.
Short surveys can also come in handy where the target audience isn’t very collaborative and isn’t likely to fill out a long survey.
It’s important to carefully consider the placement and timing of popup surveys - think twice before presenting a full-screen popup as your customer is checking out.
Sometimes clients are reluctant to use surveys at all. On a few projects, we’ve used these short surveys to break the ice and show stakeholders that asking a couple of simple questions doesn’t upset users and produces useful insights. Then we leverage that to secure buy-in to engage with users more.
Often we’ll do that just after a purchase so it doesn’t interfere with the flow – the transaction’s done but the user’s memories and experiences are still very fresh.
A popular approach is to ask users something like “What is that one thing that stopped you from completing a purchase today?” on exit intent. If ten users respond with something like “my preferred payment method isn’t covered” then you might have an issue.
This is probably most useful for repeat visitors – if you ask users on a jewelry or luxury goods site their very first visit “Why didn't you purchase?”, the most likely answer would be “Come on! I'm just browsing, give me a chance to think!”
Of course, popups are annoying and plenty of users will just click out. You’re likely to have more success if you offer an incentive, like a 5% discount or free shipping on their next order.
This type of question can be very useful, but you should remember there are NO silver bullets. Asking “What is the single issue which almost prevented you from making a purchase?” probably isn’t going to double your conversion rate. If CRO was that easy, we’d be out of job.
That's why going and speaking to users is the best because you can ask several things in different ways and receive unexpected outcome – and that’s often where my best insights come from.
