The idea of ‘group’ user testing goes against everything that user testing is designed to do, but research techniques should be adapted to the needs of the project
I like experimenting with new techniques. Adapting research methods to specific projects is exciting and can uncover new insights for clients. One method that seems to have stuck is group user testing – user testing in a focus group.
It’s a technique I use in two different scenarios:
When products are used in groups
User testing is carried out with one person because it is designed to observe individual behaviour in a functional way. We never usually use digital products or services in a group, so logic suggests it makes no sense to conduct user testing in a group.
But some of the most popular digital products are designed to be used in groups; for example, messaging apps, online games, console games or workflow collaboration tools.
The experience one has when using these products is often closely connected to the interplay of someone else. It makes sense to bring users together and observe how they interact using them.
To evaluate emotional UX
Group user testing also works well when you want to evaluate the emotional impact of using products. Where standard user testing evaluates the ability to complete tasks in a functional way, group user testing is more effective to determine if the experience is enjoyable, standard or disengaging.
In other words, your digital product may have a perfect functional experience, but it bores users. They use it with very little emotional engagement and feel little connection with your brand.
Issues to consider when using it
I have run group user testing for many years but there has been a learning curve. Most recently I have used it for a gaming client who wanted to observe how customers used their redesigned mobile app.
On this occasion the customers really liked the new design, the client was happy and the research went well, but I felt it didn’t deliver on the user testing elements.
Like any technique, group user testing has strengths and weaknesses, so here are some tips if you want to use it:
- Make sure it fits the brief. It’s primarily designed to uncover emotional usability issues – the elements that delight and disappoint users, rather than the way that they perform tasks
- The technical setup is important. Consider that the client team wants to see each of the screens being used by users. Ideally, you need one very large screen in the viewing room, tiled to show each user’s screen.
- Too many screens and it becomes too difficult to monitor. The very maximum number of participants should be 6.
- Design each session carefully, clearly separating the discussions and tasks so that you can move participants through the session professionally.
- Make sure you have enough time for the tasks. Ideally, over the course of 90 minutes, 4 medium-complex tasks will be about right. If you include too many, there will be a distinct lack of insight.
- For each task, write clear instructions for participants and ask users to start specific tasks at the same time. Viewers can then compare how well each user completes tasks at the same time.
- Use metrics like expectation pointing and SEQ, so that you can easily analyse each user’s experience. You will have 24 users in a 4-group study, so this data will add structure to analysis.
- Try to stop participants looking at their neighbours’ screens – if they do, it’s a sure sign they are in trouble!
- The real value in running group user testing is the discussion before and after tasks, so ensure you build in enough time to cover everyone’s POV.
Summary
For the right product and research brief, group user testing can provide you with the behavioural data of user testing supplemented with the emotional data of a group discussion. It’s not for the faint-hearted researcher but with careful preparation, this technique will deliver masses of insight.
Of course, you could just run user testing AND groups separately to cover those different objectives, but you may be missing out on some valuable emotional insight.