Usability Testing
Last updated
Was this helpful?
Last updated
Was this helpful?
Usability refers to the ease with which users can effectively and efficiently interact with a product, system, or service to achieve their goals. It is a critical aspect of user experience (UX) design, focusing on making products intuitive, accessible, and satisfying for users.
Usability models are conceptual views that lay down the focus areas to demonstrate the usability of the existing software.
The shackle model in 1991 gave the four important characteristics of usability namely:
Nielsen, in 1993, studied and recognized usability as an important attribute that influences a product's acceptance. He divided acceptability into practical and social acceptance and gave four important characteristics of usability: effectiveness, learnability, flexibility, and attitude.
The ISO standard, which, across the years updated itself, has instead the following usability models:
Usability Methods are a set of research methods whose purpose is to test these models to verify them to improve the usability of the product. There are various research methods used to test the usability, such as:
User interviews
Direct conversations with users to gather in-depth insights about their experiences, needs, and pain points.
Provides detailed and qualitative insights.
Allows exploration of users' thoughts and feelings
A/B testing
Comparing two versions of a product to determine which one performs better in terms of usability and user preference.
Provides direct comparison between different design choices.
Quantifies the impact of design changes
Eye tracking
Using eye-tracking technology to measure where users look and how they navigate a user interface.
Provides objective data on user attention and navigation.
Helps identify areas of the interface that attract or distract users.
Clickstream analysis
Analyzing the sequence of clicks users make while navigating a website or application.
Provides insights into user navigation behavior.
Helps identify problematic areas in the user flow.
Heuristic evaluation
Experts evaluate the product against established usability principles (heuristics) to identify issues.
Provides a quick and cost-effective way to identify usability issues.
Leverages expert knowledge.
Surveys and questionnaires
Collecting quantitative and qualitative data from users through structured forms.
Can reach a large number of users.
Provides both quantitative and qualitative data.
Card sorting
Participants organize topics into categories that make sense to them
Improve information architecture.
Reveals user language and terminology
Focus groups
Group discussions with users to explore their opinions, attitudes, and experiences with the product. Benefits:
Generates a variety of perspectives.
Encourages interaction and discussion among participants.
Cognitive walkthrough
A usability evaluation method where experts walk through the user interface and identify usability issues by focusing on user tasks.
Helps identify usability issues early in the design process.
Focuses on the user's problem-solving process.
Contextual inquiry
Observing and interviewing users in their natural environment to understand how they interact with the product.
Provides insights into real-world usage.
Helps identify contextual factors that influence usability.
Another usability method is called “Task-related measurements” which measure several aspects that affect the usability, among which:
Time spent to complete a task
Number of errors
Time spent recovering from errors
Partly for:
Historical reasons - Internet boom and its rapid diffusion
Cultural reasons - Such as the large influx of web designers, graphic designers, and lesser presence of UX Architects within the world of internet products
Educational reasons - Lack of universities and professional schools who teach the profession
Today, when we talk about Usability, we hear the echo of the words: Learnability, Efficiency, Memorability, Errors and Satisfaction but testing the first three it’s often forgotten. So we do not hear “Usability Methods” and then pick an attribute to test, rather we hear "Usability Testing" to refer to the "Task-related measurement" aimed at identifying the number of errors a user makes during a given task assigned to them and so consequently understanding why is happening and then refining the design.
Usability testing is a popular UX research method. It is a technique used to evaluate a product or service by testing it with real users.
Identifying usability issues in the design of the product or service
Uncovering opportunities to improve by gathering qualitative and quantitative data
Learning about the target user’s behavior, preferences and satisfaction
The ultimate goal of usability testing is to create something that is useful. So we conduct a “Usability Test” to:
Identify usability issues - Usability tests help uncover usability issues, such as confusing navigation, unclear instructions, or functionality errors, that may negatively impact the user experience.
Gather user feedback - Usability tests provide valuable feedback from real users, allowing designers to understand user needs, preferences, and pain points and incorporate this feedback into the design process.
Improve user experience - By identifying and addressing usability issues early in the design process, usability tests help create products and interfaces that are more intuitive, efficient, and user-friendly.
Evaluate design decisions - Usability tests validate design decisions by testing assumptions and hypotheses with real users. This ensures that design choices are grounded in user feedback and align with user expectations.
Even the best UX designers can't create a perfect user experience without iterative design based on real user interactions. Designing a modern user interface involves many variables, and human interactions add even more complexity. The only way to ensure effective UX design is through testing:
Early in the design process to validate initial concepts and ensure the design direction aligns with user needs.
During the mid-design phase to refine and improve design based on user feedback before development begins.
Pre-Launch (before release) to identify and fix usability issues in the final design before it goes live.
Post-Launch (after release) to continuously improve the product based on real user interactions.
When introducing major changes to ensure that significant updates or redesigns enhance the user experience rather than detract from it.
Periodically (regular intervals) to ensure ongoing usability and keep the product aligned with evolving user needs.
Facilitator: Guides the test, gives tasks, observes behavior, and asks follow-up questions.
Tasks: Realistic activities for participants, delivered verbally or in writing.
Participant: A realistic user, providing behavioral and verbal feedback.
Environment: the setting in which the usability test is conducted.
Data collection: methods used to capture user interactions, behavior, feedback.
Tasks for usability testing are crafted through usability test scripts. These tasks are designed to assess a participant’s capability to accomplish specific tasks. There are two prevalent task formats. Depending on the test plan and the researcher’s objectives, these formats can be used individually or in combination.
Begin with direct tasks - Direct tasks are more technical and clearly outline the primary goals and actions the platform should facilitate.
Incorporate context - Direct tasks lay the functional groundwork for your scenario tasks. If the context in which participants use your product or service is significant for your project, consider adding context to your direct tasks to create scenario tasks.
Avoid spoilers - When crafting tasks, avoid using overly descriptive or technical language that reveals the solution.
Anticipate queries - Encourage participants to verbalize their thoughts (Think a loud) and ask questions. If the question is pertinent, keep probing with “why” until you understand the root cause of their behaviour or attitude towards the topic. If they deviate from the topic, gently steer the participant back to the task at hand.
Maintain authenticity - Aim to recruit participants from your target audience. The direct tasks and scenario tasks should be realistic and relevant to both the project and the participants.
Before initiating any test, it’s important to formulate a usability test plan that delineates the scope, objectives, and logistical specifics of your session beforehand.
Test plans can range from highly detailed to succinct, depending on the research objectives and the project’s stage. At the onset of a project, it’s advisable to keep things concise. This allows for better time management and the ability to conduct numerous quick tests while iterating prototype enhancements.
Scope - What is the focus of your testing? Registration, Selecting a specific Meditation etc.
Schedule - When and where will you meet with test participants? Reach out to them beforehand to arrange a suitable place and time.
Sessions - Session durations can vary based on the project’s complexity, usability test, and tasks. For your project, aim for ten-minute sessions with 5 participants from your target demographic.
Metrics - For this project we will use the metric proposed by Nielsen’s error severity rating scale, which we discussed earlier.
Direct tasks - Direct tasks are an excellent way to articulate the most critical user goals in a clear, technical, and actionable manner.
A well-structured guide is invaluable when introducing yourself and the test objectives to your participants. Assure them that the focus of the test is the prototype, not their performance, and that there’s no such thing as a mistake in this context.
If you plan to record the session, either through audio or video, inform them in advance and obtain their consent via a consent form You can find a template for a Usability Test Script here: https://sensible.com/download-files/
It’s a common practice for a pair consisting of an interviewer and a note-taker to conduct usability tests. The interviewer engages directly with the participant, while the note-taker records observations about the participant’s demeanor and actions.
In your project, you might need to take both roles. However, if you plan to record the video call and you obtain the consent signed from the user, the number of notes you will need to take will probably decrease.
Real-time insights - Taking notes allows you to capture insights and observations in real-time. It can help you identify patterns and issues as they emerge during the test.
Focus on key points - While a video recording captures everything, notes allow you to focus on the key points and significant moments of the test. This can make the analysis process more efficient.
Backup - Notes serve as a backup in case there are issues with the video recording. Technical glitches can occur, and having notes ensures you don’t lose all your data.
Ease of reference - Notes can be easier to refer back to than videos, especially when you need to quickly look up specific points or share findings with your team.
Pay close attention to any frustrations or obstacles participants face when attempting to complete your tasks. After the test, take some time to rephrase and restructure your notes in a way that makes sense.
One of the most challenging aspects of conducting a successful usability test is identifying and prioritizing issues to facilitate actionable improvements to the prototype.
Use your test notes to pinpoint and rank key issues using Jakob Nielsen’s error severity rating:
After ranking the severity of the issues, propose solutions to address them. This is your first chance to offer your insights, as all the notes you took during the test should have been purely observational.
These proposed solutions are essentially hypotheses for enhancing your prototype. In a real-world project, these hypotheses would be validated in the next round of usability testing, forming a continuous loop of creation and validation through direct engagement with your target audience.
Introduction: Welcome the participant. Explain that we are seeking their feedback to improve a certain product or service.
Ask questions: Ask the participants about their daily routine and what technologies they use. Ask the first questions about the topic.
Introduce prototypes: Show the prototype.
Explain the task: The participant performs tasks and thinks aloud.
Closing/Debrief: Ask the participants how their experience on the test went and thank them for their time.
You can leave comments here.
Understandability
Learnability
Operability
Attractiveness
Usability compliance
Direct tasks
It provides explicit instructions to accomplish a desired action without any contextual information
E.g. Register a new account.
Select a meditation to improve sleep
Scenario tasks
They incorporate real-world context into task instructions. They are more frequently used as they add a touch of realism to usability testing.
E.g. You keep forgetting to start your daily meditation and you want to set up a reminder. Locate the area of the app where you can set up a reminder for your daily meditation.
