As Adapt grows, our projects become bigger and more complex along the way. This year has seen the Conversion Optimisation team undertake a variety of overseas user research projects across Europe, Canada, Australia, Mexico, and Japan.
To help us, our remote user testing tool of choice has been Userlytics. However, there are plenty of other tools on the market that do a very similar job, which we have covered here...
Below is what we’ve learned through our international user testing so far, including our 5 top tips for getting the most out of your own testing.
International User Testing: Our Observations
During our research, we’ve noticed that user behaviours from the UK, Australia, and Canada were more comparable. This is likely due to cultural similarities.
Whereas, users from Western Europe, Mexico and Japan responded well to their native language, but expressed frustration if they identified any inconsistent use of English.
Users from western countries prefer interacting with websites that focus on visuals to deliver the essence of a company, and they like reading clear and concise copy.
On the other hand, Japanese users come from a high-context culture, meaning they prefer text-heavy websites with long product descriptions and an accessible company history.
International User Testing: 5 Top Tips
When you need to do user testing for multiple countries, it’s crucial to have a testing plan, especially when the project has more than one moderator/analyst. If you’d like to start by learning a bit more about remote user testing generally, just head here...
Your testing plan serves as a blueprint of how to plan and run the moderated or unmoderated user testing sessions. Below are our top 5 tips for getting your testing plan right the first time.
1. Overcome the Language Gap
This is an important one. Whether you decide to conduct international usability tests in person or remotely, you should consider what language the participants are going to be most comfortable using.
Recruit Bilingual Users
This is not the perfect solution, but in practice is often the easiest to implement.
Make sure that you don’t get unrepresentative users who are, for example, expats and have the possible linguistic and cultural traits your keen to avoid.
To solve this, remote user testing tools such as Userlytics or Usertesting give you the option to return user testing sessions that have not met your targeting requirements and request a replacement.
Recruit Native Speakers
This would be effective if you speak the language reasonably well, or if your company has translation capabilities. Otherwise, it can get very expensive.
Adapt has translation capabilities, which we have used to run moderated user testing in Japan. This was because the user testing pool for bilingual users isn’t as big as in western countries.
Employ a local usability consultant
The final option is to either have a local usability consultant or an employee from your branch office conduct the tests. The could, for obvious reasons, be quite convenient.
2. Write User Testing Scripts for an International Audience
Writing a user testing script comes with challenges when it’s aimed at international users whose mother tongue isn’t English.
First of all, you need to bear in mind that most of these users will be most familiar with American English as it’s more accessible in mainstream media.
Therefore, avoid using British jargon, slang or idioms while writing the user script, and especially when conducting the usability testing sessions, to avoid inviting misinterpretation.
Additionally, provide clear, detailed written instructions on how to complete complex tasks. Remember, they cannot ask questions during an unmoderated user test, so each question must be clear and concise to avoid confusion.
Finally, we’ve noticed that international user testers tend to forget they must speak English at all times, especially when they have to review a website in their native language. Make sure to emphasize in the introduction that they should provide feedback in English throughout.
3. Set Up Strong Screener Questions
We can’t talk about user testing scripts without discussing best practice for setting up screening questions. To ensure a careful screening, we recommend using open-ended questions.
Using open-ended questions will remove any bias. Closed questions could result in a high volume of exaggerated answers from those who want to successfully reach the goal incentive – usually a financial reward.
On top of that, you can add distractors within the response options to get a more accurate target audience.
Luckily, if you’re doing remote user testing, the tools we recommend offer detailed user screening capabilities. For more information on how to write successful screening questions check this NN Group article.
4. Be Patient When Recruiting International Users
It tends to be that non-English countries are the most difficult to recruit, due to additional language requirements.
If you’re used to running remote user testing in the UK, you’ll know that slots tend to get filled within a day. So, you’ll need to arm yourself with a great deal of patience when recruiting international user testers – it can take two to three days.
Some markets have a much smaller pool of online testers. For instance, with Japanese sites, we tend to wait longer than a week, but it’s worth it.
Also consider differing time zones. Ideally, you conduct research during your own working hours, which unmoderated testing can facilitate.
However, if you’re looking to speak to users directly, then it could be challenging to run research projects in countries that have more than one time zone, like America or Australia.
Be flexible with your agenda, and use a tool like World Time Buddy to ensure your time zone matches with that of your testers.
Interestingly, when we ran moderated user testing in Western Europe (Italy, Spain, France, The Netherlands) sessions during lunchtime would be the most popular.
5. Be Aware of Cultural Differences
Finally, it’s important to know your audience when running both moderated and unmoderated user testing internationally. Even though most Western users would be similar to UK testers, the users from the Asia-Pacific region have different expectations.
Based on our experience of running research in Japan, testers would rather participate in usability tests if they don’t require a camera, as they have different privacy expectations.
They also favour written responses over spoken responses, as they tend to be very straight to the point while providing spoken feedback. We had to send many tests back to our user testing provider because we’d get sessions with very minimal feedback.
On the other hand, Japanese testers would provide more feedback when they had multiple-choice questions as they felt the need to explain their choice.
Final Thoughts
International remote usability testing can provide you with valuable insights and cutting-edge recommendations. For one, it helps companies with multiple offices across the world to set up a seamless website.
It also allows other companies to test their products in other markets as well as understand users’ needs and priorities. If you’re looking to conduct international user testing but don’t have UX resources available, come and have a chat with us.
