Usability testing – this weird, wonderful technique/skill/process/method that can reveal insights to how your users really use your website, prototype, product and so on.
But what is usability testing?
Teaching you everything about what usabiltiy testing is, to go from newbie to competent in one article, is next to impossible.
But the next best thing is a guide that provides you actionable information and resources to achieve usability testing expertise on your own.
This is that guide.
Before we can guide you, we must define what it is…or are we already guiding you by defining it? 👀
Usability Testing Definiton
The end goal is to discover usability problems, collect qualitative data and measure the participants ease of use, learnability and satisfaction with your website, prototype product etc.
Layman terms: you find out how people use your website, product, prototype and so on.
There’s this misconpcetion of when conducting usability testing, if a participant can’t complete a task, use your product or navigate your site, it’s the particpants fault.
So, why does usability testing matter?
Who wouldn’t want those kind of returns?
Now, usability testing isn’t just important so you can increase financials – that’s a bi-product from the insights you derive from testing.
However, usability testing is important for the following:
- Check if your product is meeting user’s expectations
- Identifying and removing bugs from your website or product
- Revealing user frustrations in completing their tasks with your product
- Matching your business decisions with real-world product use.
Ultimately, it matters to your business and your users.
START TESTING YOUR WEBISTE/PRODUCT
Do you feel usability testing is just to important, you need to start testing now?
If you do, go here.
Enter the URL you’d like to test > Select your testers > Create an account > You’re good to go!
What is the purpose of usability testing?
The purpose of usability testing is to test the ease of use of your product or website – to reveal the learnability behind your product and the satisfaction a user (person) has when using your product or website.
A successful round of usability testing should explain the why behind your analytics data.
As an example: you run an ecommerce site and have a 60% drop off from add to bag to checkout.
A usability test should reveal the why behind that drop-off.
There are a couple of usability testing methods that can reveal the why behind your what.
Usability Testing Methods
UX researchers and devoted practicioners in the UX industry, have developed many techniques and skills in order to validate or in-validate what they thought they knew about their user’s behaviour.
Some of the most popular techniques are usability testing, card sorting & A/B testing:
1. Usability Testing
Often referred to as “user testing”, usability testing provides a way to understand how real users experience your product or website.
It is one of the few methods that provides flexibility to what information you can gather, due to it’s multiple use cases e.g.
- Test how users would interact with your prototype
- Reveal how customers find information about delivery
- Compare competitors to see who has better navigation or search
There is often a debate between whether you should use moderated usability testing or unmoderated remote usability testing – the decision should be made on per situation a basis, as each method has advantages and disadvantages.
Moderated Usability Testing
Mostly used during the design phase of a website or product, moderated usability testing allows a moderator to sit live with a participant – faciliating tasks, asking questions and see live reactions/actions as a user users a website or prototype.
Tip: It’s recommended to have two moderators when testing – one to ask questions, the other to transcribe insights.
- Control: you’re able to control and guide the participant during a test. As an example, the participant may say or do something you find interesting – you’re able to probe deeper by asking further questions around that behaviour.
- Time with participant: as you’re with the participant, you’re able to set the expectation that the test will run for 45mins to an hour. Likewise, incentivisation in MUT is generally higher and therefore, participants are comfortable with providing an hours worth of feedback.
- Participant recruitment: with MUT, you usually recruit your own participoants or use a recruitment research firm. This allows you to specify exact criteria that represents your real users e.g. recently bought X product, shops on our site 3 times a week, AOV > £100 etc.
- Expense: generally speaking, MUT is expensive compared to it’s counterpart, URUT. When you factor in the time to setup, travel expenses, testing labs, incentives and time to analyse the feedback, your testing session can be on the other side of £1,000+.
- Time: compared to URUT, there is a lot of time involved to gain user insights e.g. time to plan session/s, schedule participants, run session and then analyse and report on findings.
- Bias: we’re human and all have natural biases. Unconscious body language, miscues and non-vebal cues can lead to false-positives in participant results.
- Location: it’s quite common for MUT to take part in some form of “testing laboratory”, unless you have on-site facilities. These “testing laboratories” are, unfortunately, unnatural and are unlike the participants real environment e.g. their home, in a shop etc. The unnatural environment can hinder test results and feedback – causing errors and non-realistic feedback.
When to use moderated usability testing?
“Moderated usability testing is best to be used during alpha, beta and live phases, to test prototypes or the service you’ve built. You can also use it in the discover phase to learn about problems with an existing service.” – Gov.uk.
Where to learn?
Learning how to do moderated usability testing best comes from practice. The more tests you run, the better you’ll get.
However, I won’t leave you out on a limb…
Here are some resources on where to learn moderated usability testing:
- Laura Klein Webinar Series: Moderated Testing Best Practices
- 20 tips for your next moderated usability test
- Gov.uk: moderated usability testing service manual
- Remote moderated usability tests: how and why to do them
- Calendly or Doodle – organise timings
- OBS, iOS Screen Recorder and DU recorder – device screen recorder
- Descript – audio transcription service
Unmoderated Remote Usability Testing
As the name implies, unmoderated remote usability testing, is usability testing that is completed with the participant in their location of choosing, without a moderator.
Due to having no moderator and participants being able to complete tests remotely, unmoderated remote usability testing offers, fast, accurate and inexpensive insights.
- Cost: tools such as UserTest.io, allow you run tests with 5 testers for as little as £90 (excluding VAT). Due to the low cost, this allows you to afford more tests and in turn, conduct more.
- Insights: gaining more insights is a bi-product of the cost of URUT – lower cost, afford more tests, run more tests, have greater insights.
- Time: once you’ve created your test and put your test live, participants will begin their test in a matter of minutes. Reducing your time to receive insights from a day, to minutes/hours.
- Location: participants peform remote testing in their natural environment e.g. home, office, supermarket, resulting in relaxed participants that are representative of your real users. This also alleviates pressure from the tester and results in more accurate test results.
- Test control: with unmoderated remote usability testing, you have to predict where the participant will be on your site/prototype, at a given task. This can lead to mis-match of tasks and questions, confusing the participant. Likewise, you don’t have the back-end-forth conversation you can have in MUT.
- Time on test: as tasks are completed when the participant decides, tasks may end prematurely, resulting in inaccurate time-on-task data.
- Recruitment: while you can recruit more participants from larger demographic panels, you are sometimes limited to just demographic data. Demographic data isn’t a full representation of your actual users.
RECRUIT YOUR REAL USERS BY SCREENING FOR BEHAVIOURS
With UserTest.io recruitment you can recruit your participants by demographics and offline/online behaviours.
- Shops online 5 times a week
- Recently got a mortgage
- Is going on holiday in the next 3 months
When to use unmoderated remote usability testing?
Having said that, URUT is best used when:
- You need answers to short, focused research questions
- You have budget contraints – you can derive the same level of insights from URUT as you would in MUT
- Easily examine competitors and benchmark the competition
- Time is of the essence and you need quick feedback
- Need large sample sizes
- Have a global user base and need to test many partcipants at once
Where to learn?
- UIE: Remote Usability Testing
- How to conduct remote usbility testing
- Nielsen Norman Group: Usability Testing Course
Tools to use
2. Card Sorting
There are three main types of card sorting, which are:
1. Open Card Sort: participants sort cards into categories that make sense to them, and label each catory themselves.
When to use?
Use open card sorting when designing the information architecture of your website, or aiming to improve it. This method is more generative than evaluatuve; providing you with ideas on how to structure and label your categories.
2. Closed Card Sort: participants sort cards into categories you label.
When to use?
3. Hybrid Card Sort: participants sort cards into categories you label, but are also given the option to create and label there own categories.
When to use?
Where to learn?
3. A/B Testing
When to use?
A/B testing is best used when you want to test and change the user behaviou. As an example, you see from your data, those who use search, convert at twice that of those who only use your navigation. Now, you can do a couple of things and change the user behaviour:
- Test a variation of search e.g. make it larger, more bold – get more people using search
- Test a variation of your navigation – make it easier for people to find what they’re looking for, restructure your categories and products
Where to learn?
- Research-Drive Conversion Optimisation by CXL
- Conversion Optimisation Minidegree by CXL
- A/B Testing by Google
- Conversioner Blog
Usability Testing Process
Whenever I’m asked, where do I begin when conducting a usability test?, my answer is always the same.
Start with a plan.
This part of the post won’t go into much detail on conducting a usability test from start to finish (that is for another time).
We will cover in detail, how to write a testing plan.
Starting with a test plan has a couple major benefits over preparing your designs, finding your participants etc, including:
- Test clarity: Writing a test plan, that covers your testing objectives, the type of participant you’re after and the testing script you’ll use, allow for clarity. Anyone in your team can pick up the test plan and know exactly what’s going on.
- Test revision: Planning your test before you begin recruiting or executing the test, will allow you and stakeholders to make any necessary changes, without wasting time and money. Planning ahead of time, will give you greater flexibility later on.
Script writing: Defining the objectives for your test, allows for easy script writing later on. By defining your objectives, you know what questions must be answered in order to achieve the objectives, and in turn, you’ll know what tasks to provide in your test script.
As an example, an objective for a usability test on Amazon may be: Identify usability issues when finding a book.
A question that needs to be answered is: What functionality of the site did the user’s use to find a suitable book?
So, your test script may include a task, such as: Imagine you’re searching for a book about Google. Using this site, find a book about Google. Feel free to use the navigation of search to find your book.
- Participant recruitment: Within your test plan, you’ll define the type of participant that is suitable for your test, e.g. shops online 5 times a week, buys technology items from Amazon. Defining your ideal participant early, allows changes to be made and recruitment services enough time to find the participant for your test.
USE USERTEST.IO QUALIFIED TESTERS FOR SPECIFIC TESTER RECRUITMENT
Finding the right type of tester for your test can be tricky busy! You need more than just demographic data to match the type of person who uses your site, to the tester who will take your test.
We offer a solution that allows you to find your exact users – Qualified Website Testers.
Filter the testers by demographics and online/offline behaviours e.g. shops on Amazon, recently went on holiday, plays golf once a month.
How to write a usability testing test plan?
One of the first steps when conducting a round of MUT or URUT, is to construct a test plan.
The test plan should document:
- What are you going do?
- How are you going to run the test?
- What is your objective for this test?
- What and how many participants are you going to test with?; &
- What testing script will the participants use?
- Specify what you are going to do: what are you testing? when are you testing it? what device/s are you testing on, how many participants do I need and what type of participants?
- Define your objective for the test and what questions must be answered, in order to achieve the objective/s.
For example, our objective may be: Identify usability issues when finding a book on WHSmith.
The question you would need answering is: What problems did the users encounter when finding a book?
- Write your testing script and keep in mind your objective and the questions that need answering to achieve your objective. This will allow you to keep your test central to what you’re trying to achieve/wanting to know.
- Usability testing is not about testing the user
- Invest in usability testing and you’ll see massive ROI
- Successful usability testing should explain the why behind your quant data
- You have several testing methods to choose from – choose one that suits your time, budget and what data you’re after
- Start with a plan before conductng any usability testing – save yourself from wasted time and resource