Most UX teams would agree that in user experience research, it’s not optimal to run a single study with multiple facilitators.
Yet oftentimes, situations dictate that this is in fact the case.
Timing is usually a major factor. As an example, a large-scale study might require dozens of sessions, with stakeholders who need the results urgently. For practical reasons, there might be insufficient time for both planning and execution of the study. One option is to shorten the schedule for conducting sessions by using more facilitators.
Another example is a study that takes place in multiple locations. Oftentimes, research staff are based in those locations. Budget considerations might dictate that the team member in each location is the most cost-effective choice to serve as a facilitator.
Given these scenarios, is it possible to reduce the risks that jeopardize the data in multiple-facilitator studies?
Seeking Alignment
In our experience providing usability testing and evaluation, there are specific techniques to bring multiple facilitators into close alignment on how they facilitate a session.
The key is having a detailed script, then using practice, script refinement, and trial sessions to bring consistency to what facilitators say, how they say it, and other behavioral factors that otherwise introduce variation.
It’s also a good approach to seek out researchers who are the best fit for a multiple-facilitator study. They must be both rigorous researchers and team players. The researcher should also be experienced in adhering to a script, rather than the “winging it” style of facilitation.
Thoroughness, Consistency
In terms of data collection during a user testing project, the facilitators must be willing to record details of what happened in each session (participant clicked this button, entered that keyword, made a face, uttered this comment, etc.), not just a higher-level interpretation of the observations (typical path, no real problems).
Just as important, the facilitators must offer a ruthless devotion to consistency. Team members must be willing to discard session data (for example, after an unsuitable participant or technical difficulties) or to admit a script deviation and discuss solutions with team members.
In all, the keys to establishing consistency in multiple-facilitator studies are teamwork, practice, and structured data collection.
Learn More
Download a comprehensive TecEd white paper for more information.
The white paper “Multiple Facilitators in One Study: How to Establish Consistency” shares insights into how to control the disadvantages of multiple facilitators while achieving the advantage of meeting the deadline with high-quality results.
About the Author
