Whether you like it or not, there is no way around the topic of UX testing when creating an IT product. Any specialist who does not give a damn about his work at least wants the results of the man-hours spent to be appreciated by the end user.
Some of the methods of UX research are so obvious and straightforward that some of the IT specialists, when they first get acquainted with the classification of tests, will have a logical question "Why, someone doesn't do this?" followed by "Have they been fired yet?"
Let's face it, we all test UX, sometimes without even explicitly setting ourselves such a task. Someone calls it the pretentious word "research", while others do not call it in any way, but it still tests on the sly.
This article has collected 21 methods of UX research. Some of them are simple and commonplace, some are more sophisticated. Choose the one that suits your taste. But first, let's see what characteristics they differ in.
Method differences
Quantitative qualitative. Part of the research gives a measurable result, for example, collected by the analytics system or calculated based on the results of a survey. We call such research quantitative. In other cases, only a qualitative assessment is possible, i.e. consisting of the judgments of the researcher or participant (convenient - inconvenient, simple - confusing, etc.).
This is logically related to the number of subjects. The more of them (quantitative method), the more difficult it is to analyze each case separately and the easier it is to operate with averaged figures. The fewer the number of test subjects (qualitative method), the less sense it makes to rely on numbers, since with small samples they can vary greatly. Then you have to resort to a qualitative assessment and find out not “how many” people did or said something, but “why” they did it or said.
Behavioral / Relational. Behavioral methods answer the question "What are users doing?", Relational methods answer the question "What do users think?" The answers do not always coincide, but depending on the purpose of the study, both are equally important.
Product involvement. According to the degree of participation of the product in the study, methods are distinguished:
- with natural product use
- using the product according to a pre-created scenario
- without product participation
- mixed, consisting of combinations of the above methods
Stage of creation. Let's highlight three stages of creating a product: planning, development and debriefing. Research will differ depending on these stages.
We'll start with the methods used early on and finish with the methods that are required at the end of the product cycle.
UX research methods
- Proof of concept
- Sorting cards
- Focus group
- Ethnographic research
- Involvement in design
- Tree testing
- Assessment of preferences
- Laboratory research
- Eye tracking
- Research "diaries"
- Remote research
- 5 second test
- Unmoderated panel research
- A / B testing
- Usability Benchmarking
- Expert review
- Clickstream
- Collecting feedback
- Email survey
- Exploring True Intent
- Interview
1. Concept Testing
Qualitative / Quantitative: Both
Are Possible Behavioral / Relational: Relational
Product Participation: Mixed
Stage: Planning
Overview
The method focuses on highlighting the key qualities of the product to determine if they meet the needs of the target audience. Anyone familiar with the concept of MVP (minimum viable product) probably understands what this is about. It can be performed one-on-one or with a larger audience. The main goal is to understand if there is at least some need for such a product.
When to use
To prove the viability of a concept before developing it into a full-fledged product.
2. Card Sorting
Qualitative / Quantitative: Both
Are Possible Behavioral / Relational: Relational
Product Involvement: Natural
Stage: Planning
Overview
We can say that this is involving the user in the design of your product at minimum. The product information architecture is sketched. Users receive sets of cards indicating the data found on the site or in the application.
Guided by their own logic, they divide them into semantic groups, which in the future can become interface screens. Users also come up with group names.
When to use
In the early stages of product design, especially one that has no direct analogues on the market. Also to detect errors in an already formed information structure.
3. Focus Groups
Qualitative / quantitative: qualitative
Behavioral / relational: relational
Product participation: no participation
Stage: planning, development
Overview
A group of 3-10 participants, led by a moderator, discusses their views on the future product. The role of the moderator is to maintain the flow of opinion rather than direct it. A focus group can answer a few basic questions, but should not turn into an interview (see method # 21).
When to use
At the very beginning of a project, to understand what users expect from the end result. Useful for making important strategic decisions and forming a general concept.
4. Ethnographic Field Studies
Qualitative / Quantitative: Both Possible
Behavioral / Relational: Both Possible
Product Involvement: Natural
Stage: Planning, Developing
Overview
Ethnographic research is conducted using an existing product on the market that is similar to what you want to do yourself.
Your task is to catch the user in their natural environment. This includes, for example, monitoring employees of an organization while working with a CRM system.
It assumes the physical presence of the researcher and studies not so much the interaction of people with the product as the interaction of people with each other. Requires more preparation time and may include passive observation, participatory observation, and contextual interviews (questions during work).
When to use
To observe similar products on the market in the early stages of your own development. Provides critical insights into user needs, allowing you to refine your overall product concept.
5. Involvement in design (Participatory Design)
Qualitative / quantitative: qualitative
Behavioral / relational: relational
Product participation: natural or scripted
Stage: planning, development
Overview
Users are encouraged to structure their application on paper or on a magnetic board. Knowing the purpose of the product, they try to present it in the most convenient form for themselves.
Cards with functional elements, decorative materials and markers are used as materials. Both the target audience and the client's employees, managers, shareholders, etc. can participate.
When to use
As a product design brainstorming session. Allows you to find unexpected useful ideas that can help improve the UX solution.
6. Tree Testing
Qualitative / Quantitative: Quantitative
Behavioral / Relational: Behavioral
Product Participation: No Participation
Stage: Planning, Development
Overview
Participants work with a text version of a site or application, the entire structure of which is presented in the form of a tree. Top-level categories expand into nested categories, etc.
The task is to find this or that menu item, guided by this scheme. The result is the distribution of clicks by category. Shows how misleading product structure can be.
When to use
To find possible navigation difficulties and better understand user logic.
7. Assessment of preferences (Desirability Studies)
Qualitative / Quantitative: Both
Are Possible Behavioral / Relational: Relational
Product Participation: Natural, Scenario
Stage: Planning, Developing
Overview
Participants are shown several design options. They must evaluate each of them with a set of quantitative and qualitative characteristics. For example: speed, simplicity, usefulness, predictability, freshness, authority, etc. Specifications are provided in advance, often on cards for convenience.
When to use
When you need to understand how users feel about a design. It is advisable to carry out in the early stages of product development.
8. Laboratory research (Usability-Lab Studies)
Qualitative / Quantitative: Qualitative
Behavioral / Relational: Behavioral
Product Participation: Scenario
Phase: Development
Overview
Participants perform proposed tasks in the product under the supervision of a researcher. To do this, they are provided with a script of the necessary actions with an explanation of their goals. Important details of the operations (sequence, time, etc.) are recorded.
You can read more about how such a study is carried out in the article "UX-research of RBS: our experience, mistakes and discoveries".
When to use
During the development of a UX solution, to validate and refute assumptions that arise. Can be done already at the prototyping stage. Suitable for proprietary applications that are not available to most users.
9. Eyetracking
Qualitative / quantitative: both are possible
Behavioral / relational: behavioral
Product participation: natural or scripted
Stage: development
Overview
A special device - eye-tracker - marks the user's gaze fixation points when working with a website or application, as well as transitions between them. Often used are infrared-illuminated monitor eye-trackers that track reflections from the eyes with a camera.
Research reveals interface elements that receive more attention. It also allows users to find areas that are confusing for users (for example, confusing menus where the eye does not immediately find the desired item). May include prepared assignments or be conducted without instructions.
When to use
While working on a product, with a valid version. To optimize the structure of the interface and navigation around it.
10. Research "diaries" (Diary / Camera Studies)
Qualitative / Quantitative: Both Possible
Behavioral / Relational: Relational
Product Involvement: Natural
Stage: Development
Overview
Study participants regularly record information about the times in their lives when they think about your product. Together with this, they write down the actions, events, the situation that led to such thoughts.
This can be accomplished in a variety of ways: using a paper journal, an application for notes, or audio and video messages. Users are instructed on how to take notes and, if possible, set up daily reminders.
When to use
When you need to better understand the context in which the user can encounter our product. As well as the motivation of users and their daily habits.
11. Remote research (Remote Usability Studies)
Qualitative / quantitative: rather qualitative
Behavioral / relational: behavioral
Product participation: by scenario
Stage: development, debriefing
Overview
Analogue of laboratory research, carried out remotely, for example, using software for remote access. Sometimes moderated, when participants receive comments and questions from a specialist, and unmoderated, when tasks are performed independently.
When to Use
To save resources on laboratory research. Allows you to invite more participants and get more realistic results by keeping users physically in a comfortable environment.
12.5 second test
Qualitative / quantitative: qualitative
Behavioral / relational: relational
Product participation: no participation
Stage: development, debriefing
Overview A
blitz test on the first impression of the design. Participants in the study see a fragment of it for exactly 5 seconds, after which they answer a series of questions. For example, about what the main elements were remembered, what idea of the brand developed, what is the purpose of the page, who is the target audience, etc.
When to use
When to make sure the vision of the designers aligns with the vision of the users. If the discrepancy is noticeable, it is better to have time to correct the concept.
13. Unmoderated Remote Panel Studies
Qualitative / quantitative: both are possible
Behavioral / relational: both are possible
Product participation: natural or scripted
Stage: development, debriefing
Overview
The panel is a specially trained group of users who independently work with the product and “think out loud”. To do this, they use video recording applications. Consider this a simplistic alternative to ethnographic research.
The advantages of this approach are time savings, the ability to test more actions, a fresh perspective. Cons - difficulties in analyzing the results.
When to use
Suitable for researching products with a long cycle of use (fitness apps, health monitoring, etc.). Also useful for analyzing pending events (such as notifications) and when scheduling updates.
14. A / B testing (A / B Testing)
Qualitative / Quantitative: Quantitative
Behavioral / Relational : Behavioral
Product Involvement: Natural
Stage: Design, Debriefing
Overview
A popular method of comparing two versions of a site or application that differ in one or more elements. The audience is randomly split into two segments, each of which sees only one version.
After reaching statistical significance, it is concluded which option won for the selected KPI (for example, in-app purchases). Held in dedicated services such as Google Optimize for Websites and Optimizely for Apps.
When to use
To optimize the working version of the product, that is, either at the last stages of development, or after the release. Helps with “fine-tuning” critical interface elements such as CTA buttons or navigation elements.
15. Usability Benchmarking
Qualitative / Quantitative: Quantitative
Behavioral / Relational: Behavioral
Product Involvement: Scenario
Phase: Development, Debriefing
Overview
Laboratory research that is conducted regularly to track changes in UX. The methodology is similar to the previous point, the main difference is in comparing the results. You can compare the performance with previous versions of the product or with its competitors on the market.
When to use
When to track or demonstrate progress in product optimization. To assess the effectiveness of the solution itself, especially when moving between development cycles.
16. Expert Review
Qualitative / quantitative: qualitative
Behavioral / relational: relational
Product participation: natural or scripted
Stage: development, debriefing
Overview
The product is reviewed by an experienced UX specialist, filing their observations into a detailed report with illustrations. The document describes the strengths and weaknesses of the solution, the identified problems and their causes, recommendations for elimination and best practices.
When to Use
For a fresh look from the outside - it helps to notice the little things that the "blurry" eye was not paying attention to. Also suitable when the UX design team lacks their own expertise.
17. Clickstream Analysis
Qualitative / Quantitative: Quantitative
Behavioral / Relational: Behavioral
Product Involvement: Natural
Stage: Design, Debriefing
Overview
Analyze data on which pages and in what order the user visited. Easy to conduct using the analytics system Google Analytics, Yandex.Metrica, Firebase, Mixpanel. Allows you to identify problems related to the navigation of the site or application. Doesn't help with finding their causes. To find out the reasons, it is worth using usability research (methods # 8, # 11).
When to use
When to check if the product is being used as intended. The research can be carried out in the final or intermediate version.
18. Collecting feedback (Customer Feedback)
Qualitative / Quantitative: Both
Are Possible Behavioral / Relational: Relational
Product Involvement: Natural
Step: Debriefing
Overview A
random sample of users are sent out a questionnaire about their experience with the product. There are many channels for this, from web forms and pop-ups to polling services and email. You can combine closed and open questions, the main thing is not to overdo it.
When to use
After a meaningful action on a site or app, such as placing an order or requesting support. Launching a questionnaire earlier can complicate the path to conversion and alienate the visitor.
19. Email Survey
Qualitative / Quantitative: Both
Are Possible Behavioral / Relational: Relational
Product Participation: No Participation
Stage: Summing Up
Overview Survey of
users by email, as opposed to collecting feedback, is general and not triggered. Refers to previous interactions with the product, so it only evaluates its perception by the audience. Easier to conduct than interviews (see method # 21), but more likely to fail. The number of questions usually does not exceed 10, so as not to scare away the respondents.
When to use
To analyze the effectiveness of an existing product and compare with competitors.
20. Research of true intention (True-Intent Studies)
Qualitative / Quantitative: Quantitative
Behavioral / Relational: Both Possible
Product Participation: Natural
Stage: Summing Up
Overview
Survey of random users of the site / application before and during work with it. At the entrance, they receive a question about the purpose of the visit, and after a while - about whether they managed to achieve it.
Sometimes the survey also asks for demographic or other relevant information. It is important not to overload the questionnaire, otherwise visitors will not have a desire to fill it out.
When to use
After product launch, when sufficient traffic volume is reached. To find out if the development meets the needs of users and if it fulfills its intended functions.
21. Interviews
Qualitative / Quantitative: Qualitative
Behavioral / Relational: Relational
Product Involvement: No Participation
Stage: Debriefing
Overview
Personal interview with people with experience with the product. The task is to understand what impression the site or application leaves on users. Since we are talking about past actions, it will not be possible to obtain accurate numerical data.
When to use
When planning a redesign or upgrade of a working solution. Allows you to assess the perception of the product and brand, as well as draw attention to the main technical weaknesses.
Outcome
As you can see, there is a whole arsenal of methods for analyzing and identifying the reasons for user behavior and product perception. Each approach can play a critical role in creating a UX solution if you know when to apply it. Or maybe not. Hope this guide was helpful.
This article was written for you by: Denis Elianovsky , Stanislav Lushin.
Thanks to Elena Efimova for the infographics, Tatiana Kitaeva for editing.
Read in English
List of Sources:
When to Use Which User-Experience Research Methods
7 Great, Tried and Tested UX Research Techniques
Five second tests
Tree Testing: Fast, Iterative Evaluation of Menu Labels and Categories