How does user testing work?

• You ask a number of people to perform a number of typical tasks on your website or intranet.
  Or on a mock-up if you’re in the process of building a new one.
• The test is performed on an individual basis.
  So it’s not like a focus group where there’s a bunch of people giving you feedback all at once.
  Please, don’t ever call a focus group a user test.
• The test participant should belong to your target audience. Recruit for engagement, not demographic representation.
  Ideally, the test participants are existing or potential clients.
• Everything the test participant does and says is recorded.
  That way you and your team can review and analyze the data afterwards.
• Encourage the test participant to act just like they would if they’re at home or in the office.
  Simply let them do or say whatever they would do anyway. If people don’t talk much, don’t push it. Observing participants’ behaviour is more important than urging them to talk aloud the whole time.

Why is user testing more valuable than an expert review?

Always remember: you are not your user.

4 reasons why a good usability expert will often recommend to do a user test.

1. Experts don’t know everything

No matter how much expert knowledge and experience you have, you can’t know everything.

You can’t always judge situations perfectly. That’s a lesson I’ve learned in more than 17 years as a usability expert. You can read more on that topic in our article ‘Experts don’t know everything, not even usability experts‘.

2. Facts trump opinions

Clients can easily dismiss expert advice with the argument “Yeah sure, but that’s just your personal opinion, right?”.

User tests reveal facts.

And facts trump opinions any day of the week.

3. Ignoring your own clients is difficult

Or at least, ignoring your own clients is more difficult to do than ignoring expert advice.

If you have seen with your own eyes that 5 out of 6 people have trouble using the filter navigation in your webshop, there’s no way you can’t realize this is a problem you have to fix.

User testing creates empathy with your users. If they’re struggling on your site, I can tell you that’s very painful to watch. You get to share in that struggle.

4. Convincing and durable proof

A video that shows your customers and potential customers having trouble with your website, cursing your interface and company? That has immense power. That kind of video will shut up even the most die-hard, ego-tripping know-it-all experts and directors during board meetings.

Also, you can re-use the recordings at any stage of the project. Whenever that same old discussion rears its ugly head again, you just whip out that recording: “Remember, this is what we saw during user testing…”.

Keep hammering on that nail.

Most common types of user testing

1. In-person moderated user testing

Launching a website without any user testing is like playing Russian roulette.

The classic user test set-up is in-person moderated user testing. The moderator sits next to the test participant and asks questions.

This old-school method still has a lot of benefits. Observing the user in person and being able to ask them additional questions based on what they’re doing or saying at a certain point, can give you very deep insights into what’s really wrong with a website.

To me, that is priceless information. And it can’t be replicated by any other method. There are some things you simply can’t get out of click and scroll maps or user session recordings.

Having said this, good moderating skills are obviously key in moderated user testing. The way the moderator asks the questions and even just the way she sits or looks at the user can heavily influence your test results.

2. Remote moderated user testing

The biggest disadvantage of the ‘classic’ in-person user test is that your test participants either have to come to you or you have to go to them. And time is simply money.

Remote moderated user testing is an excellent alternative for in-person user testing.

With remote user tests, the moderator and test participant can be wherever they want to be, as look as they’re looking at the same website on some screen or other. With screen sharing software like Zoom or Whereby that’s a piece of cake nowadays.

If your user has a webcam, you can also see their face. Which makes it a good alternative for in-person moderated user testing.

We often do remote moderated testing if we need test participants from different countries. It’s also a great solution if you only want to test something small: one particular page or a step in a process.

Or, you know, when there’s a global pandemic going on.

3. Remote unmoderated user testing

As you probably already guessed, there is no moderator in this type of testing.

There are 2 main variations of remote unmoderated user testing where you have footage of the test participant’s actions on your website as well as feedback:

• Recorded image and sound
  Users are asked to think out loud. To be fair, I’m not a big fan of this method. It relates to classic user testing like sardines to caviar.
  Many test participants you get with these tools sound as if they’re semi-professional test users.
  Thank heavens you can also test with your own user panel. Which is definitely an improvement on using the tool’s testers.
• Recorded mouse movements and time
  In this method, you don’t see the test participant and you don’t hear them. They see the task on the screen, perform the task and fill out a questionnaire to give additional feedback after each task.

My biggest issue with this type of testing is that it leaves out any real communication or interaction with the user. If they make a comment you don’t really understand, you can’t ask them that all-important question: why? If a test participant says something during a moderated user test you want to get to the bottom of, you should keep asking why until there are no why’s left. And you can’t do that in unmodertaed testing. That’s why I’m not a big fan of unmoderated testing for usability issues.

If you’re after feedback on content and messaging, unmoderated testing can be very useful. Because you get to hear in your customer’s own words what resonates with them and what doesn’t. So if you’re a conversion copywriter, unmoderated user testing can be very valuable. Credit where credit’s due: I learned this from Joanna Wiebe, the original conversion copywriter.

And what about eye tracking?

We’ll discuss eye tracking in a separate article. Eye-tracking can be very interesting, but it’s all too often misused and abused.

Other user research tools

There are of course lots of other user research tools out there.

I really like HotJar‘s user session recordings if you want to get a quick look of how people use your pages.
Sometimes a problem is so obvious all you need to see is those erratic mouse movements and rage clicks.

For our information architecture projects, where we often have mock-ups we like to test on, I’m a big fan of Optimal Workshop’s first click testing tool Chalkmark.