Planning an effective UX research | A complete step by step guide

Reading Time: 13 minutes

What is UX research?

UX research is the first step in the user experience design process. UX research enables the designers and product owners to take a more strategic view and focus on the core aspect: user experience.

UX researchers identify their users’ goals, their users’ environments and the business context of the product – indeed, anything that affects a user’s experience with a product or service.

Two questions that we must answer with UX research

Fundamentally, all UX research answers these two questions:
(a) Who are our users and what are they trying to do?
(b) Can people use the thing we’ve designed to solve their problems?

The first question: Who are our users and what are they trying to do?

If you don’t know who your users are, then you cannot start the UX design process. It is like shooting an arrow in the dark. We can identify our target users and know their behaviour from field research.

Many people mistake field research with carrying out surveys/interviews in a controlled setting like a coffee shop or a mall. In fact, it enables you to observe the users in their real environment.

It lets you understand what they actually do, rather than just listening to them say what they do. To draw an analogy, if you want to understand animal behaviour, an interview is like a visit to the zoo whereas field research is like going on a safari.

The second question: Can people use the thing we have designed to solve their problems?

This question can be answered by carrying out a usability test. It focuses on how people do specific tasks and the problems they experience when using a particular system. Usually, it is conducted in a lab, but it can be practically conducted anywhere (including the field). Typical research locations include:

• A participant’s home or workplace
• Public spaces, like coffee shops and libraries (so-called “pop-up research”)
• Research studios or labs
• Meeting rooms
• Your desk (using a laptop or phone for remote research)

Usability testing is fundamentally inward-looking: you give your users a prototype and a set of tasks and you see if they can complete those tasks. The key difference between a usability test and a field visit is that with a usability test you’re evaluating a specific design idea with your users.

A field visit is like turning on the lights and a usability test is like observing under a microscope. Field visits give you the big picture whereas a usability test lets you evaluate a specific solution.
With that being said, let’s look into the steps you need to follow to plan a UX research project.

Steps in planning a UX research project

1. Defining your UX research problem
2. Approaching desk research
3. Conducting effective stakeholder interview
4. Identifying user groups
5. Screening the participants
6. Starting with the first research activity

1. Defining your UX research problem

Without a clear understanding of a research problem, one cannot expect UX research to deliver useful findings.

Many designers fall into the trap of assuming that they already know what they are trying to solve. But in most cases, this assumption is either incomplete or totally flawed.

And that’s a rather worrying thought because companies spend a lot of money on customer and UX research and it would be prudent to define the research problem as clearly as possible.

We shall now look at characteristics that make a research plan flawed. A poorly designed research plan is one that is:

• Merely gathers data and regurgitates it.
• Deals in generalities and superficial surveys, avoiding depth and analysis.
• Asks no analytical questions.
• Does not advance knowledge, but is happy to summarize what’s already known.
• Is boring.

So before you jump right into starting with preparing the questionnaire or interviewing candidates, you need to think first about the problem at hand first. So how will you go about defining the problem?

Here are four techniques you can use to help you better understand a research problem, determine clear objectives, and sharpen the research question:

Find out what other stakeholders need to know:

Even though the client provides a brief for the research, it isn’t just enough. You need to keep in mind the client’s project manager or project owner is representing a larger development team, and the team has a lot of knowledge and experience. 

This is an invaluable opportunity and you must include them also while defining the research problem. You need to find out what they want to know and how they think about the research problem.

Deconstruct the construct:

Another way of defining a research problem is to deconstruct the phenomenon that is being investigated. Most of the phenomena that UX researchers are likely to study are constructs. That is to say, they do not exist in any physical sense and cannot be directly observed. 

Usability is an example of a construct. You can’t weigh it or put it in a box like you can with, say, pencils or jars of marmalade. Quality is also a construct. So are emotions, desires, intelligence, attitudes, preferences and the propensity to buy. 

This doesn’t mean we can’t research them or measure them, but in order to do so, we have to deconstruct them to reveal their constituent elements and then find ways to operationalize those elements. not only is this an essential step in designing research, but it’s also really the essence of what’s meant by “drilling down” into a problem.

Measure something:

The objective of UX research is to measure some aspects of human behaviour. Understanding a problem and what can be measured are interconnected with each other. Focusing on the measurements is a way of clarifying the nature of the problem.
So to define the parameters, you may ask questions like:

• What specifically do we need to measure?
• What kinds of metrics will differentiate specific concepts or different levels of a variable?
• What will my dependent variable be and what will I need to manipulate in order to detect differences?
• What variables will I need to control?
• Will these kinds of data convince the development team?
• Can I just use subjective rating scales or are there some objective behavioural measures I can use?
• How will I analyze the data?
• How can I connect my metrics back to the business?

You need to analyze the data properly. There might be hidden gems. You have to interrogate the data and make it work for you.

Shake out the issues:

UX research can require a sizable investment in time and costs. The outcome will dictate the direction of a project and influence its success. There is too much at stake to risk mishaps or misunderstandings happening during the research.

Although it seems increasingly common in the corporate world to skip this step, you should always conduct a pilot study prior to commencing the full research project.

The term “pilot” derives from the Greek word for ‘rudder’. It refers to steering and adjusting the course of something. TV shows are always piloted to get an early audience reaction; engineers test jet engines on the ground before they use them to fly aircraft.

Doing research is no different. Typically, a research pilot test is conducted quite late in the preparation stage and resembles a rehearsal that actors would perform before the actual show. It is typically used to check aspects such as:

test design will return valid data,
give the test administrators and data loggers an opportunity to practice,
making sure the timing and logistics are in order, and
check for any potential glitches in testing or recording equipment.

2. Approaching desk research

There are mainly two types of research activities: primary research and secondary research.

Desk research is another name for secondary research. Desk research is not about collecting data. Instead, it is about reviewing previous research findings to gain a broad understanding of the research question.

So how should approach desk research?

Organizations are generally poor at creating a shared repository of knowledge. They rarely teach staff how to use the knowledge base or where past reports might be located.
So within your organization, you should:

• Talk to your stakeholders. Get to know the product owner and understand their goals, vision and concerns.
• Examine web analytics (if it is set up currently).
• Talk to the front line, customer-facing people who currently interact with users.

The next step is to look for more generic information about your users, the environment in which they’ll use the system, and the kinds of goals your system will support.

• What research has been done with your users, even if it’s not directly relevant to their goals when using your system?
• What research has been done on the kind of goals your system will support, even if the research has been done with a different user group?
• What research exists on the kinds of environment where you expect your system to be used (environment means hardware, software and the physical and social environments in which your system will be used).

3. Conducting an effective stakeholder interview

Most of the time, clients just provide a brief about what they want to achieve through the research. But honestly, you need to dive deep into exploring more about the research, and this can be achieved by carrying out stakeholder interviews.

We too often think we have understood a stakeholder’s needs properly— especially if we work for the same company—when, in fact, we haven’t quite grasped something. We often make assumptions without even realizing that they are just assumptions. So it’s better to keep the guessing game at the bay and know the real deal.

The problem is most stakeholders are looking for a quick fix, or a magic bullet because they have already self-diagnosed their needs. At times, the client hesitates to carry out a face to face discussion. This is because the client has already decided what needs to be done, has written down the solution, and now just wants someone to deliver.

But the real difficulty lies in the fact that they may not know how your design or user experience process works, so they don’t know what you need to know before you can start thinking about designing a solution.
So how can you get to know your stakeholders’ expectations better?
First, let them think about the solutions to the problem and bring it to the table.

At this stage, you can ask them questions like:

• “It’s likely that we can design a new X, but what issues are you hoping a new X will address?”
• “By not having an X, what kind of problems are your users (or your company) experiencing?”
• “If we designed the best X imaginable, what problems will it solve for you?”
• “Let’s say we designed the world’s best X. What results would you achieve that you can’t get today?”

The stakeholder may respond by giving you a list of problems or may go off track. But you need to bring them back to your question. You can do it by asking:

“Thanks for describing the general situation. Let’s get a bit more specific. What particular problem have you identified that you think a new X will solve for your users?”

The next step is to have them generate a list of issues. The items you write on the whiteboard should describe a specific problem or a desired result or outcome. If you are working with a group of stakeholders you can ask individuals to write down the main issues on sticky notes, then have them place them on a whiteboard. You can keep asking questions till the list gets completed.

When the stakeholders have exhausted their input, you, from your expertise, can add by questioning: “I notice no one mentioned anything about (for example, improving ease of learning). Is that an issue for you?”

Next, you need to know which issues matter the most. Sometimes the stakeholders will easily identify the main issues. Other times they may say, “All the issues are important.” Respond to this by acknowledging that you want to understand all of the issues, and then ask: “Which one would you like to talk about first?”

Finally, you need to double-check all the items on the list so that all the important ones are covered. You can ask:
“I think we have covered all the issues here and have nothing else to add. So shall we now proceed to discuss what you think the solution could be?”
If the answer is no, there’s still something you’re not being told. Find out what it is and add it to the list.

Due to the COVID-19 pandemic, it is not possible to meet every participant or stakeholder personally. So in our article on remote UX research methods, we have explained how you can conduct UX research remotely with help of the latest tools in the market.

4. Identifying user groups

The success of any UX research depends on identifying the user groups correctly. Many products fail, just because they are aimed at everyone. But that is not the case.

Even though it seems that everyone will be using your product, there is still an area of target users that needs to be defined correctly.

So where do you start?

You can conduct this activity with your team. You can ask each person to write down at least five different user groups for your product. For example, a photography app might have groups like day-trippers, food enthusiasts and Instagrammers.

Using the grid

Once the groups have been generated, remove any obvious duplicates and then organize the remaining in the following grid:

You’ll start with a group of users from the top right of this grid. Think of these as the “gold” user groups since these are users who will teach you a lot about their needs from your product and who are also easy to set up.

You should be able to arrange sessions with these groups of users in the next few days. Once you start talking to these users you’ll discover some assumptions you made that are wrong. You’ll also start to learn new stuff that’s relevant to the use of your product. You’ll begin to generate new assumptions. This is validated learning in action.

But let’s not forget the other important quadrant: users who you expect to learn a lot from but who are hard to get to. Think of these as your “silver” user groups. You should add these groups to your UX research backlog. While your existing research is in progress, take time to plan visits to this group of users. This will give you the chance to test more of your assumptions.

We could characterize the user groups in the lower right quadrant as our “bronze” user groups. Consider the “bronze” user groups only once you’ve exhausted your “gold” and “silver” user groups.

And what about that final quadrant: users who are hard to get to and who probably won’t teach you much? Given that there’s never enough time or budget to do research with everyone, don’t feel bad about skipping this group of users entirely.

5. Screening the participants

“Participant screening” is where you sift through all the possible candidates to identify people who are truly suitable for the participants.

You can follow these guidelines to screen for the participants effectively:

a. Screen for behaviours, not demographics:

When faced with a new product or website, a user’s past behaviour will predict their performance much more accurately than their demographic profile.

Demographic factors like gender may be important segmentation variables for marketers but they have very little impact on the way someone actually uses a product.

But which behavioural variables should you use? On web projects, two behavioural variables used by every UX researcher are:
(i) digital skills (the user’s experience of web idioms like navigation, form filling and search) and
(ii) task knowledge (the user’s knowledge of the domain, for example, photography, share dealing or genealogy).

Design your screener so you can classify each candidate’s ability as “high” or “low” on both these factors, and recruit people accordingly.

b. Ask precise questions:

If you want to distinguish “high” and “low” digital skills, don’t just ask people how long they spend online. One person’s “frequently” is another person’s “sometimes.”

Instead, ask what they do online and whether they do it on their own or with assistance. People with high digital skills probably buy things online using a credit card, download and install software, belong to social networking sites, manage their photos online, actively manage their privacy, and comment on blogs. People with low digital skills will do fewer of these activities or seek help from a friend or family member.

c. Identify unsuitable candidates early:

Broadly speaking, screeners contain two kinds of exclusion questions:
i. Yes / No Type Question: The answer will exclude the candidate, such as answering “Yes” to the question, “Do you work for a competitor company?”
ii. Rating scale type question: Where you want to get an equal number of people in different categories, for example, a balance of “high” and “low” digital skills.

You need to ask these exclusion questions at the early stage to filter out unsuitable candidates as quickly as possible.

Sometimes candidates can pass all your screening questions but still not be what you want. For example, one of our colleagues was testing a website that sold apparel. Everything went fine during the usability test until it got to the product selection part of the test. It turned out that, in real life, the participant said he took his best friend along to the store to choose his clothes.

So if you’re testing a B2C website, make sure that your participants make the purchase decisions too (or alternatively, make sure they bring their influential partner along to the test).

d. Manage each participant’s expectations:

Before starting the actual research, you need to ensure that your participants realize that answering screening questions is a prerequisite for taking part in the research, not the research itself. You need to clarify that the screener isn’t the actual deal.

Once you have recruited participants, manage their expectations for the session. Most people’s preconception of consumer/user research is the focus group, so if you’re running a typical usability session make the participants clear that it will be an individual interview.
At this time, you can inform participants that the session will be recorded on video, and have to sign a consent form and a non-disclosure agreement. If any of these are deal-breakers, now would be a good time to find out.

At the same time, don’t reveal too much in case of participants decide to do their own research prior to the test. For example, if you tell participants that you’ll be asking them to evaluate the company’s website, this gives them the opportunity to go to the site and work on it in advance of the test. It is just like “practising” before the test. So, provide enough information to reassure the participant of the nature of the study, but not the specifics.

e. Pilot test the screener:

After having filtered out the participants you don’t want to include in your research, you need to make sure the remaining ones fall into the right category. You also need to ensure that the internal stakeholders agree and sign on the screener list. This helps you to prove the validity of the study if someone challenges you that you didn’t get the right participants.

6. Starting with the research

Once you have all these steps correctly, now you can proceed with conducting the actual research. Given that “user experience research” implies we need to involve users in our research, the most (scientifically) logical UX research activity is a field visit, especially in the early stages of design. This gives you the opportunity to test out your hypotheses around user needs.

Conclusion

UX research is the first and the most important step in the UX design process. Hence, you need to plan it meticulously to ensure that UX design meets its core purpose. Without planning, UX research is nothing but assumptions and you cannot expect a product to be successful just based on the assumptions. So what are you waiting for? Follow the above-mentioned steps and nail your next UX research project.