What is Agile User Research?

User research in a tight timeline and budget is not impossible. In fact, it is already happening now. All you require are the quality voices of a handful of customers to test and validate your work using an agile user research method.

So what is the core difference between agile and a full user research method? Fewer number of participants are being tested in agile as compared to the full method. But does that mean lesser quality data? No.

One of the early usability gurus, Jakob Nielsen’s research suggested that with only 5 users, 85% of usability problems can be found. For a full user research method, 12 users can find almost 99% of the usability problems. For those who think that user research is too costly and elaborate, a small and agile user research method with frequent testing suits better (as many as the budget allows).

The other difference between agile and full user research is that there will less tasks covered during the testing. To overcome this, test and iterate the product’s features and functions in smaller chunks until it achieves its bigger goal, which is part of the agile manifesto.

Planning and communication are the keys to conducting a great agile user research. Early strategizing occurring at the previous development cycle helps. All of these information and ideas in the early planning phase should be communicated frequently to the user research team so that any issues can be ironed out quickly and for resource management to occur efficiently.

Here in Objective Experience, the entire testing to reporting phase takes only 2 days.  The planning beforehand from the kick-off workshop takes 2 days. Ideally, everything happens within 4 days as illustrated below.

Agile user testing in Singapore

For agile user research, there is no need for testing a large number of users as then it defeats the purpose of the word ‘agile’, which means quick. Testing 5 users who are selected carefully and thoroughly screened to ensure the best participant quality of the targeted user segment is good. Each testing session covers around 2-3 main tasks or user flows within 45 minutes.

It is compulsory for the product design and development team members to sit in and observe the testing sessions as it goes on. Why? To immediately get a sense of what users actually need and iterate on the spot or the next day.

In our agile user research sessions, we also use eye tracking as a way to gain direct insight into how the product is used and what users struggle with. Eye tracking allows observers of the testing to see users’ unconscious behavior in real time, and enables stakeholders to make instant decisions about solutions to interface problems.

At Objective Experience, we have the facilities for team members and other stakeholders to observe the live sessions in person at our viewing room or remotely via a web link. The remote viewing link is great if you have overseas members interested in observing what goes on during the user research. We’ve got a really comfortable space complete with refreshments too!


Take a peek at our viewing room!

After all the testing sessions are done, a brief workshop with the research moderator is held with the observing team members to discuss the key findings from the users, brainstorm some solutions together and actualize the results for the next development. The next day, a report cementing the top 10 most impactful findings with the actionable design recommendations will be produced.

oesg_agilereportsampleSLIDELet us help you make incremental improvements to the user experience of your products, thus driving business growth. Drop us a line at infosg@objectiveexperience.com or call +65 67374511 to discuss your needs now.

Save time and money with Rapid Usability Tests

In 2014, We have been having incredible success with Rapid Usability Test and Eye Tracking! In Singapore, companies like Singtel and DBS Bank run these Rapid usability tests weekly. They iterate designs on various projects from mobile app to transactional websites.

“Rapid Usability Testing in UX is a powerful method. It is cheap and can be done quicker than traditional methods. It is easy to incorporate rapid usability testing within tight project schedules.” 

Agile usability testing saves time and money

How do we do Rapid Usability Testing with Eye Tracking?

When we are working in Agile development projects for a client who wants to iteratively validate the concept for, say a mobile app. The objective is to get quick feedback, from real customers, on the concept.

We run the projects like a formal usability test, which includes:

  • Formal usability testing scripts that are signed off by the client
  • Use of highly accurate Tobii X2-30 Eye trackers
  • Formal lab setting with viewing from our special Brainstorming Room or live online with Adobe Connect.

We have to do this quickly and we mandate that all relevant project team members come to the session and actively participate in solutioning during and after each test. When they leave for the day most UI improvement decisions have been made and our summary report becomes checklist of things that were agreed.

Here’s the steps to run an Rapid Usability Test

Step 1: Decide on what you want to usability test

The best part about Rapid Usability testing is that you can get started with just about any working (or non-working) products! For the test that we conducted, the concept was rather high level, with no working prototype to speak of. So we did the next best thing, mobile paper prototypes!

Mobile app paper prototypes that were used to Rapid Usability test the concept

Mobile app paper prototypes that were used to Rapid Usability test concepts

Since the product is in its earliest stages, it can literally be shaped by user expectations. You must tailor the test based on the insights you want to gain.

For instance, our client wanted to gauge the audience’s response to an unorthodox method of online shopping. We asked our client the following:

  • What user journey are you expecting to deliver positive business outcomes?
  • Which pages/features are of most interest?
  • Are there specific copy, navigation or UI elements we want feedback on?

Tasks were therefore created around navigation and and customers filtering their search down to a specific product. This helped establish if the user journey was as smooth as intended.

Step 2: Pick your usability test participants quickly

As with any usability test you should still have a specific demographic, you won’t have time to be super specific.  We use our SG Research Network platform to recruit people really really quickly using social media. We’re talking about recruiting people in one-day, the day before the test! If you like, we offer recruitment for your projects too.

Step 3: TEST!

The Rapid Usability Test for this mobile product was done with 5 or 6 people, individually in a lab or it could have been done in a spare room at our client’s office.

As with formal usability testing, prepare a script that details each task. Keep tasks short and focused and test for about 45min each.  You can fit about 3 tasks in one Rapid Testing session.

This video shows how eye tracking can be used on a mobile app.

We do find that eye tracking facilitates a better Rapid Usability Test, as it allows the observers to see exactly how people do things in real time. They can interpret very easily why participants are having trouble and begin making their design decisions immediately.

Here’s 5 Rapid Usability Testing tips for the session

1. Introduce yourself and the purpose of the test: Say hello! Explain what is about to happen. It is important to stress to the participant, that the product is being tested, not them. Tell them they are being video recorded.

2. Keep it light: It is important to put the participant at ease in order to get honest feedback.  Crack a joke, be genuinely interested in the participant’s interests and have a genial approach.

3. Get them on your side: Allow participants to contribute to the product. Ask them what their solution would be to a certain issue they were having.

4. Try not to lead participants: It’s easy to get subjective about a product, especially if you are the creator!

Some leading and closed questions are:

    • “Does the slowness of the application frustrate you?”
    • “Does the colour theme of the UI frustrate you?”
    • “Are you unhappy because you are unable to figure out the navigation?”

Here are some open-ended questions that fit into almost every user interview:

    • What do you think?
    • How do you feel about using it this way?
    • What would you do next?
    • How would you expect it to work?

Users open up with interesting points of view when questioned in this manner. It is also important to use silence effectively during user interviews. Silence is a powerful tool that naturally prompts the interviewee to fill the void.

For Rapid Usability Testing you MUST mandate stakeholders watch and participate in vigorous discussion: It always helps when people who are involved with the product development, are invited to watch tests. The result is almost always a more understanding development team and managers who will take key decisions soon after the testing. It also means you don’t need to write a long report. They already know what needs fixing!

Here’s some tips for effective Rapid Usability testing observation:

  • Ensure observers can see the participants eye gaze live on a large screen
  • Check in with clients to see that they are happy with the facilitation process and modify it if necessary
  • Use lot of Postit notes to jot down findings
  • Use a white board to sketch solutions to UI issues
  • Print our key screens of interest and draw on them
  • Encourage discussion
  • After each session run a brief discussion of key findings
  • At the end of the day summarise the findings and, more importantly, they key updates that will be made.

You must use an observer (2nd consultant) for the testing, and they need to be able to facilitate client discussion around the usability observations and potential fixes.

User Experience Consultants and clients brainstorming in our usability lab during a usability testing session

User Experience Consultants and clients brainstorming during an eye tracking study in Singapore

All said and done, lean testing is very flexible. There is no perfect way of doing it.  The heart of the matter is to get quick insights from real people.

For more information about Rapid Usability Testing, please complete the form below.