What if your target users are all over the world and your UX group is here in the U.S.? TecEd’s multinational clients need to conduct user research with more than American audiences, but what’s the best way to do this? Although our goal—rigorous high-quality research that yields rich insights—applies to most UX research, many of the methods, techniques, and tips for achieving it on a global scale are unique to multinational research.
This post describes how to manage a successful UX research project in multiple countries and languages, some of the challenges and pitfalls you may encounter, and how to address them.
Local Languages
The greatest challenges—and the greatest rewards—occur when multinational UX research is conducted in the local languages, especially when the product or service is (or will be) localized. Using local languages, we can emulate more accurately the actual context of use. But most American UX researchers are not bilingual, and probably none of us speaks the half-dozen languages of our primary target users—so TecEd works with partner researchers in different countries, and effective coordination with these partners is the cornerstone of multinational research.
Research Design
Like all research design, a multinational research design describes the goals of the research, the tasks or activities we’ll observe target users perform, and the criteria for recruiting appropriate participants. In multinational research, we also use the research design as the primary screening tool for our research partners: we send them a version of the research design as an RFP, asking for their feedback on methodology, participants, and schedule, as well as a budget. For each new country where we conduct research, we identify several bilingual partner candidates and select a partner based largely on their understanding of the methodology in the research design.
Then we develop a “translation-ready” participant recruiting survey and research protocol (moderator’s script) for the project. Translation-ready documents use consistent and unambiguous language and sentence structure, both to facilitate the translation process and to maintain consistency of content as much as possible between languages.
In TecEd’s experience, the most successful multinational research projects are designed with a U.S. component first. That approach enables us to pilot-test the translation-ready protocol and revise it (and if necessary, to adjust the research design and methodology) based on actual users, even if they are not yet users from the other target countries.
Although we are always prepared to provide research protocols in local languages, most of our bilingual research partners prefer to translate the moderator’s script themselves. The translation activity enables them to gain more familiarity with the tasks, and often results in useful questions about the protocol.
Participant Recruiting
TecEd considers participant recruiting a critical element of high-quality UX research. If research participants don’t accurately reflect the characteristics of the target audience, then we cannot be confident that our results are valid.
A complication in multinational research is that participants’ roles have different names in different countries—and often the backgrounds of people with the same role vary in different countries. For example, on a project for a medical equipment manufacturer, we learned that most job roles performed by radiation therapists in the U.S. are performed by physicians in China and Japan.
As a result, it’s often not possible to have a single translation-ready participant screening survey. We may need one screener per country, and stakeholders from the research sponsor will need to participate in helping to achieve appropriate consistency in participant selection.
Conducting Research Sessions
A key question in conducting multinational UX research is whether the stakeholders want to observe sessions in real time. If so, there are two challenges, time zones and language. Many research sessions we conduct in other countries take place late at night for U.S. stakeholders. Will your stakeholders really want to observe then? If so, the language challenges become greater.
If the sessions are observed in real-time, either the bilingual researcher pauses every few minutes to interpret in English to the TecEd and stakeholder observers, or we arrange for simultaneous translation. There are trade-offs for each approach: we have concerns about the impact on the participant of a stop-and-start method, and simultaneous translation has logistic challenges.
If stakeholders do not need to observe in real time, they can read transcriptions a few days after the sessions, a much easier and more economical approach. TecEd typically uses “manually interpreted transcription,” where a bilingual transcriber listens to the local language recordings and transcribes them directly into English. The results are not exact enough for a legal record, but are usually accurate in reflecting the content.
Finally, schedule considerations vary among countries. It’s necessary to be aware of holidays and vacation times in different countries. For example, several of TecEd’s clients requested usability testing to be conducted in China during the Spring Festival, when most people—including our research participants—plan their holidays to include a week before and after the Spring Festival itself.
Data Analysis and Reporting
TecEd often provides partner researchers with a data logging template, to be filled out in English, either during or after the sessions. Regardless of whether a data logging template is appropriate, we prepare “Instructions to Researchers” documents for each country which not only describe the session logistics, but also give detailed instructions about the data analysis and reporting the partner researchers should provide. This effort ensures consistency of deliverables.
Get in Touch
TecEd has designed, conducted, and managed multinational UX research projects for more than 30 years, with participants in the UK, France, Germany, Holland, South America, Singapore, India, Japan, and China. Recent projects include both remotely moderated and in-person sessions in France, Chile, China, and Japan. Contact Us to learn how we might support your multinational research initiatives!
