What is UX Research and Why is it Important for CRO?
UX (user experience) research is a collection of techniques used by designers and researchers to understand how users interact with products, websites, or anything used by people.
In short, UX research is all about improving the experience of people who use your product or website.
In Conversion Rate Optimization, UX research is all about improving user experience by identifying sources of friction in our website design. Unnecessary friction in your design and conversion funnel gets in the way of customers taking the action we want – that is, to buy your product or perform micro conversion actions that lead to the main goal.
Why is UX research important for CRO?
UX research is a vital weapon in the CRO arsenal with several massive benefits:
Helps the team and stakeholders to understand users
Helps CRO practitioners make decisions on the real evidence from the users
Helps save time and money on development
UX researchers can use wireframes to identify problems before baking them into an expensive full build. After that, rapid iteration can be done – saving weeks (or months) of development time.
UX research downsides/limitations
UX research doesn’t always produce the results you need
UX research generates friction in decision makers
It’s common for decision makers to question the value of user research or push your team to compromise on methodology or precision to obtain quick results.
Dealing with other people can be unexpected, results aren’t always guaranteed
While performing research respondents, no-shows and recruitment problems should be taken into account because they might happen.
Researchers need to analyze results properly, understand which feedback will be taken into account and which will be irrelevant. There is a need to control the researcher biases as well.
UX research methods for CRO
There are numerous methods to choose from, but the most common tool in your toolbox as a UX researcher include:
Surveys – great for providing quick, quantitative insights
User interviews – useful to dig deeper into user behavior and unearth qualitative insights
User sessions recordings and heatmaps analysis
Usability testing (moderated and unmoderated) – when there is a live product or prototype ready
It’s typical to combine two or three research methods and build a clearer picture.
Choosing the right method for UX research isn’t simple. You have to do it on a case-by-case basis and the methods you choose might change as the project evolves. You have to understand the research questions, your client, your company, and the system you’re researching.
A vital part of that process is creating your research plan.
How to do UX research for CRO?
WHEN to do UX research for CRO
If you’re looking for the best results in design, user satisfaction, retention and the all-important conversions, the short answer is that user research should never stop.
Baking user research into your product development and iterations from day one will produce the best results by:
Discovering the real needs/pain points to help people with
Saving development time
Gaining early traction
Revealing design problems at an early stage
But if you didn’t do it before, it’s better late than never. UX research can provide powerful insights to remove friction and drive conversions at any stage of your business’ development.
UX Research for CRO step-by-step process
Step 1: Understand the objectives. Understand the client and his/her request. What are the goals to be accomplished?
Step 2: UX audit and heuristic analysis
Step 3: Fix the low-hanging fruit you identified in your audit
Step 4: Further research planning: determine what methods you’ll need to get the data you need, document the research goals, think on the timeline
Step 5: Recruit respondents
Step 6: Execute your plan
Step 7: Document, organise, analyse and present your findings
Research plan
It is a document that contains flow of ideas, hypotheses, goals, and the methods you're going to use.
It should contain all the possible information:
A list of hypotheses you want to check
Your goals and objectives of what you want to find out in general
Any materials you’ll need
Research material
Recruitment criteria (how many users to recruit, where we will recruit them, budget for incentives etc)
Deliverables that will be generated as a result of the research
It’s also helpful to have interview scripts for user interviews and testing scripts for user testing. This doesn’t mean you’ll always stick to the script, but having a framework is critical to show you the direction.
How NOT to do UX research
No research plan
Casting around in every direction with no clear plan is just a waste of effort. You have to strike a balance between sticking to the plan and being ready to revise your hypothesis.
No prescreening
Failing to prescreen users can also lead to bad results when incentives are involved. When people are just doing it for the money and especially when you give them too much info beforehand, they’re prone to telling you what you want to hear or filling out your survey responses at random. Ask different questions during the pre-screening (including the ones not related to your topic) so that the person answering couldn’t guess what you are looking for.
Allowing your emotions and biases influence the results
As a researcher, you have to control your biases that can influence your results. Everybody wants research to prove the initial hypothesis but it doesn’t always happen. There should be a courage to admit that instead of manipulating the raw data to get desired results that didn’t happen in reality.
Helping people and answering their questions during the research process
If you are using a moderated research method, there is a risk of giving hints to the respondents or feeling inconvenient to leave their questions unanswered. It is necessary to control yourself to allow maximum natural user experience, like the respondent is using your product at home without any guidance available.
Trying to fix every little problem in the product
There can be several findings that don’t hurt the experience much. From all the changes you recommend after the research phase there should be a clear list of ‘High-impact’ ones and others. It’s nice to make things look better or work a bit better but we want to receive the significant results at the end.
UX Research tools for CRO
You can easily get started with UX research with a stack as simple as Zoom, screen sharing, a screen recorder, and Google Docs.
But for teams who want to look deeper or need a specific functionality, there is a huge variety of tools which can make user research faster or provide additional insights:
What questions should we ask?
Avoid leading questions
The easiest route to bad data in UX research is asking leading questions. That is, questions designed to elicit responses which fit your hypothesis and expectations.
Imagine you’re designing an app to help people who are scared to fly and you ask them “It’s scary to fly, isn’t it?” Most people will probably respond “yes, of course!”. But is that what they were really thinking?
To get an effective answer, you need to be a lot more objective, paying attention to your own biases to make sure you’re not leading the user to your conclusion.
If you see that a user is lying or being too polite, you might need to rephrase the question from a different angle to reach their true feelings.
Open-ended vs closed questions
If you want to know what users are thinking, you’ll do better with open-ended questions. The idea here is to get users to open up and share their story. You’ll be surprised what you learn when you encourage users to tell you what’s on their mind without offering them your pre-made variants.
Open-ended questions will often pull the best insights out of users, but we can use close-ended questions to clarify or introduce a topic.
Good questions:
Open-ended questions designed to get users tell you their story
Asking about past experience, not future plans or hypothetical situations (Tell me about the last time you did x)
Asking why person is doing something (can be asked multiple times to follow the 5-whys method)
Bad questions:
Suggestive or leading questions
Hypothetical questions – “What would you do if…?”
Ask respondent to provide solutions to the problem (it’s designer’s job to think of solutions)
Is it easy to use? (Respondent will answer yes/no and you won’t get a lot of details. Better to ask What was the most difficult/the easiest?)
Asking 2-3 questions at once. Respondents will usually pick one to answer and forget about the rest.
Recruiting users for UX research
Compensation
The days of recruiting users with little more than the promise of some free refreshments are long gone. Most users aren’t going to give you half an hour of their time for nothing – they’d rather spend time with their family, read a book, or do nearly anything else.
It might cost $250-500 to get 10 users in for testing, but it’s important to remember that this information can be invaluable to your research, often producing ROIs vastly exceeding your expenditure.
Contributing to the cause
“I recently researched sexual roleplay. So we were developing a system for sexual roleplay to help couples rejuvenate their sex life through sexual roleplay.
So in order to design the game that I was working on, I had to go to the users. So I contacted users, and I said, “Hey, guys, I know you guys don't want to talk about it. I know it's an intimate thing. But we want to contribute, like we want to help people get into what you already love. And for that I'm going to need your input.”
And not a lot of like, I had three or four respondents without even offering financial compensation. I had three or four people contacting me. Yeah, man, no worries, I'll talk to you about it openly. Like no problem. It doesn't have to be anonymous, I don't care.
So just just saying, “Hey, I want you to contribute, makes a lot of people sign up. “
- Filip, UX Research Consultant
Where to recruit
Where is your audience at? That’s where you should recruit them. Options include:
Asking the product owner to send invitations to their email list
Hotjar form attached to the site “Can we ask you questions for 30 minutes in return for this incentive? If you’re interested, please leave your email”
Social networks e.g. Reddit, LinkedIn
Social networks like Facebook and Reddit are great channels where you can sometimes recruit highly engaged users for next to nothing - but this may cause an issue. A lot of communities have a standing ban for surveyors and researchers. So, you have to be crafty – consider reaching out to the moderators: “Hi, I want to contribute to roleplay - that's the goal of my research. Can you please make an exception?” Usually, they're fine with that, and the specific answers you get from their super-engaged communities are UX research gold.
User recruitment panels
There are paid user recruitment panels out there like usertesting.com or even Amazon Mturk which will help you recruit a large number of users quickly.
You’ve got to be careful if you're doing quantitative research with services like Mturk. People are there to earn, and they will often just fill everything out as fast as they can. So, you have to mitigate that by e.g. asking open questions which require users to write something.
Info to cover when inviting users
We don’t want users to skip reading your email because they see a wall of text that nobody wants to read. So, the idea is to give very general information which includes only the exact details the user needs to know.
How long will this take?
What will this be? (user interview, usability testing, etc.)
What is the incentive?
Why are we doing this? (without going into too many details so we don’t influence the user – aim for a maximum of 1-2 sentences e.g. “we want to hear feedback on your shopping experience”)
A link to book a session in the calendar
Urgency – i.e. this is a time-limited offer, once we book the users we need you’ll lose out on your chance for the incentive
No-shows
When recruiting users it’s important to account for no-shows. This problem will vary from project to project – one audience will be super organized and committed, the next one will be 80% no-show.
So you should always have some “bonus” users to cover those who won’t show up. Worst case scenario, even this won’t be enough and it may break the timeline of your research project. You can’t always predict it and sometimes you just have to roll with the punches.
How many users do we need for UX research?
The wider the niche your product works in, the more people you will need.
Methods you’re using are also relevant – e.g. surveys will require many more users vs. usability testing because it’s a qualitative method of research relying on hard numbers.
So while you might be able to get away with 10 users for usability testing, you would need anything from 60 to 100 users to get meaningful survey results.
Qualitative testing should be researched to the point of saturation. Saturation means that point when, after we can group our results and see certain repeating patterns, we notice that no new patterns are emerging that can be grouped together. As long as we are seeing new behaviors that can be grouped, we keep getting more people to interview.
Consider how much variation there’ll be in the users of a search engine or a job site compared to an eCommerce store selling specialty art supplies.
Three or five users might be enough – but most often not. Typically, you're looking at anywhere between 7 to 15 people for qualitative research of any sort, while edge cases might demand interviewing 100 people.
Can we save time by interviewing users in a group?
This is a contentious question and you’ll find heated debates about this in marketing and UX communities online. The truth is that very often it’s not a good idea to interview users in a group because group dynamics can influence users. There are always people who are more outspoken who might influence other participants. But there ARE exceptions.
Let’s say you’re doing research into how a team uses Slack. It might actually make sense to interview the whole team, because it’s a lot closer to how they actually use the app. A team is a system like any other, so seeing how the dynamics of that system work in practice can be very useful for UX researchers.
UX research resources
Stick with the basic: Research Design by John Criswell and David Cresswell
Designing for the Digital Age: How to Create Human-Centered Products and Services by Kim Goodwin and Alan Cooper
The Mom Test : How to talk to customers & learn if your business is a good idea when everyone is lying to you by Rob Fitzpatrick
