Conducting User Testing Remotely

Ideally, user research and testing is conducted in person at a location where a person performs an action or task. There are multiple factors that affect user behavior. As a user researcher, you need to read and understand your user’s body language, and know where it is appropriate to probe and when. How is this research impacted when it’s not possible to be physically located with your test audience? With the rapid pace of innovation and increasingly global focus of organizations, how can we adapt our approach?

In some cases (and I’m seeing this increasingly more), in-person user testing is not always possible. Personally, I believe that remote research is better than not having a user testing sprint at all. Testing products where users are geographically dispersed, scheduling a series of online remote user testing sessions can be preferable -- and far less costly -- than traveling each time an iteration needs testing. A remote testing approach allows your team to move faster and iterate quickly -- allowing you to concentrate on building the product your customer really needs.

Participants can be from any geographic area rather than concentrated in one location, which makes recruiting faster and easier. Remote user testing allows the users to use their own computers for the study, letting you see how they set up their desktop, navigate between programs, and use tabs. This gives insight into how people work with their machines and also what interaction they have with other programs while they use your product. Here are a few things to consider before getting into the tools for remote user testing:

• Create scenarios — Instead of giving the users tasks to perform, it would be better to give them scenarios. Design the scenarios in a way we can achieve multiple interactions with the product. This would ensure we are testing with humans, not with a robot
• Persona match — While designing the scenarios, make sure that the scenario is for the user persona you are testing with. Giving wrong scenarios for a wrong persona would mean wrong results - plus frustration for the user
• Mock test — Before conducting a fully fledged user test with the end users, it is better to have a round of mock user testing sessions with team members or product owners who may not familiar with the system you are developing
• Fix a time and send a calendar invite — Better to have the time fixed with the user and send a calendar invite as reminder to avoid no shows
• Check audio and user access — Before getting into the user testing session, check the audio systems so that you don't miss those vital things the users say. Also check if you have solid network connectivity. Most importantly check if the user has access to the system before hand
• Get users’ consent — Always let the users know what the user testing is done for and what you would be using the recordings for. Send in a consent form for the user to read and agree to it
• Incentivize the user— Incentivizing the user to take the test with things like amazon coupons or beta licenses for the product. You are after all taking time from them. You may add it to the screener for recruiting the users
• Moderate it — Many people suggest unmoderated user testing for remote user testing (a selection of the method of user research also depends on what you intend to get out of it). In my experience I have seen moderated user testing fetch more understandable results than unmoderated ones. It gives flexibility for the user researcher to probe when a new user behavior is spotted right then and there. But caution is to be taken not to ask leading questions to the user
• Win user confidence — A key to getting desired results from the user testing is to win their confidence. Users often open up and let you know answers for questions which you haven’t asked
• Think aloud protocol — While the user is performing the actions, ask them to think aloud and talk about what they are doing and why. This will help you understand their actions at a deeper level
• Follow up — If the user has asked some questions which you do not have answers, make sure you do follow up and answer them
• Feedback and Retrospect — Get feedback from the users about the session and retrospect so as to make necessary changes

While there are many online tools for unmoderated remote user testing, for moderated user testing a simple online meeting tool would work. Any VOIP tool that allows real time video recording like GotoMeeting, Webex, Fuze etc. can be used.

Once you have performed the user testing, now what? You need to analyze the results, watch the user testing video over and over again, understand the users’ actions and pain points. See if there is a pattern of user behavior appearing across different users whom you have tested. It is said that you just have to user test the product with only 5 users to get a pattern of behavior.

Having heard to answers for the why’s and how’s of remote user testing. Now the question comes up when to conduct the user testing sessions? It is better to have the user testing as part of a research sprint before the actual sprint starts. This should answer open questions for the upcoming sprint and verify if the previous sprint produced a usable product. The necessary changes for the product that came from user testing can be accommodated in the upcoming sprint as well. However having guerilla user testing sessions conducted during the iterative development of the product is also good, just we need to make sure that the necessary changes for defects or needed enhancements that came out of user testing results are tracked and implemented.

These are just some methods that have had successful outcomes for me, what’s worked for you?