Successful Mobile App Localization Requires User Research

An International Localization Case Study in Four Parts; Part Two

What are the requirements for a mobile app to be successful in multiple languages? This is the second post in which we address that question as we describe UX research on localization  that TecEd and Cisco worked on together.

Cisco Systems developed a mobile application, SalesConnect, for its internal sales force of tens of thousands, as well as partners across the globe. Before investing in costly localization, the team wanted to learn:

• What is most important to sales users for a localized app?
• When and why do these users want materials in English versus in their local language?
• When do they search using English versus their local language?

This case history, originally presented at the 2016 ACM SIGCHI conference, describes how Jennifer Lee Carlson and a multidisciplinary team at Cisco worked with TecEd on this research project to learn what constituted a good localized application for sales users’ content needs. It describes the methods we used, the challenges we overcame, and what we learned.

Part 1:  Introduction

Part 2:  Project Work Process, What We Did Steps 1-4

Part 3:  Project Work Process, What We Did Steps 5-9

Part 4:  Lessons Learned and Key Takeaways

Part 2: Project Work Process: What We Did Steps 1-4

The demands and constraints of a multinational project had a major impact on our work process. Some activities required additional steps, and almost all were more complex than we initially anticipated. To summarize, we:

1. Designed the research
2. Engaged bilingual partner researchers in different countries
3. Coordinated with Cisco cross-functional teams
4. Created a “translation-ready” research protocol
5. Designed data-logging templates for consistent reporting
6. Ran pilot-test sessions in each language
7. Conducted 22 participant sessions
8. Translated the session recordings
9. Analyzed and reported the research results

Each of these activities had its own objectives and challenges, and this installment describes Steps 1-4 and the next installment, Part Three, will cover Steps 5-9.

1.    Designing the research

This localization research for requirements definition was primarily concerned with what makes the app useful and valuable to the target users. We wanted to learn the priorities and considerations for the localized application to fit these users’ needs and workflows, more than how easily people could navigate the user interface.

The localization research project began with working sessions to make decisions about the scope and focus of the research. Cisco and TecEd reviewed together the initial design concepts and discussed in detail what requirements definition and concept validation information Cisco wanted to learn from the localization research sessions.

The key decisions in the study design were which languages to include in this localization research, as well as how many participant sessions we wanted to conduct in each language. The sample size was chosen based on the job roles of the target users; we had learned in previous studies that Cisco account managers and systems engineers have different content needs, so both were represented in this research.

2.         Engaging bilingual partner researchers in France, Chile, and China

Cisco judged several geographic areas particularly important to them for sales. We initially considered conducting the research in French, German, Japanese, Mandarin Chinese, Latin American Spanish, and Brazilian Portuguese; and TecEd identified and interviewed bilingual research partners in all these geographic areas.

To manage the project scope, Cisco then selected three high-priority segments that represented major market segments of Cisco’s customers: China (specifically, speakers of Mandarin Chinese), France and French- speaking Canada, and Spanish-speaking South America. Since Cisco is headquartered in California, our project spanned 15 time zones! Coordinating across these time zones was a major challenge throughout the project.

To define the engagement with our partner researchers in China, France, and Chile, we created a detailed project plan of exactly what we wanted them to do on what schedule. The project definition had designated a specific week for sessions—which became our first multicultural problem. The scheduled session week was immediately after Chinese New Year, when virtually all businesses in China close for 10 days. It also spanned a holiday in France. Dozens of emails flew between TecEd and our Chinese partners, while we negotiated how quickly their team could return to work after the holiday.

3.         Coordinating with Cisco cross-functional teams

In conducting UX research, sales participants are as hard to recruit as physicians (every hour a salesperson on commission spends with a research team can reduce his or her time with the customer). High-tech salespeople are a valuable resource that must be treated with care.

Cisco was especially careful to respect their sales users’ time; a senior Cisco seller engagement manager was the participant recruiter. The recruiter’s first priority was keeping the salespeople happy, while coaxing them to participate. During the project, we also coordinated with the Cisco UX designer, product owner, project manager, IT specialist, content specialist, localization specialist, sales experience architect, and director of Sales Research and Design.

4.         Creating translation-ready research protocol

The final research protocol included 40 mostly open- ended questions about salespeople’s practices, preferences, and reactions to the design concepts. TecEd’s roots as technical communicators enabled us to write clear, unambiguous text intended for translation, to help ensure that the translated protocol was both accurate and consistent among the languages. Authors without training in professional communication often do not even realize they are using slang.

We also made some changes in the work process as we negotiated with our research partners. We had originally planned to provide the facilitator’s script in Chinese, French, and Spanish to our partners. In fact, the bilingual research partners preferred to make their own translations from our English translation-ready protocol. They judged it would help them become more familiar with the protocol before the sessions, since they were not themselves designing the research.

Also, the partner researchers could ask us questions in advance about any content that needed clarification.

Because this research was to be moderated by partner researchers, rather than directly by TecEd or Cisco, the research protocol included not only the facilitator’s script and questions, but also instructions for conducting the session. For example, we said, “Please keep in mind that only the audio portion of the session recording will be forwarded to a transcription/translation service, so if the participant is indicating something on the screen, please say aloud what they are pointing to or referring to.“

In addition to the research protocol, the design concepts were also translated into all the target languages. In addition to the research protocol, the design concepts were also translated into all the target languages.

Next Installment  Part Three: Continuation of our Project Work Process: What We Did Steps 5-9

Permissions

(Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

CHI 2016, May 7 – 12, 2016, San Jose, CA, USA. Copyright is held by the owner/author(s). Publication rights licensed to ACM.)