Infotainment Usability Testing

Automotive User Experience Testing Considerations

User experience research and testing with automotive systems often involves special considerations and challenges that don’t apply with in-lab testing.

On your journey to learn from users what they view as an excellent customer experience here are some things you may need to consider:

Off-road or on?

Some tasks need to be performed while the participant is actually driving, but many can be performed in a parking lot. If you include on-road testing, you should make sure you have the appropriate insurance coverage. You’ll also want to make sure the participant has a valid driver’s license. Even for tasks done in a parking lot, you should mention that you should never do this while driving!

Subscription services

If you’ll be conducting tests with the entertainment or navigation system, you may need subscriptions if you plan to test those components. For example, you may want a SiriusXM or Pandora Plus account for the vehicle or for any smartphone used during testing.

New technologies

Be prepared to define technologies the participant may not be familiar with, if you plan to test them. Some participants may not yet be familiar with technologies such as Apple CarPlay and Android Auto, or HD radio stations that have subchannels.

Blind study?

If you plan to mask the identity of the auto manufacturer, you’ll need to tape over any identifying logos or insignia both outside and inside the car. Also, in a blind study, the moderator may need to start the vehicle instead of the participant, so the participant doesn’t see identifying logos on the infotainment screen or elsewhere on the dashboard when the vehicle starts up. In addition, make sure your moderator’s script doesn’t include the manufacturer’s name in the main text or in the header or footer. You may also want to consider a special tent to mask the entire vehicle itself to the outside world. If you do this, don’t forget to vent the exhaust to the outside! You can use a special exhaust pipe attachment for this purpose.

Video recording?

Setting up video cameras to record a series of usability sessions can be a challenge. Consider what you need to record. If your focus is on the console between the front seats, you’ll need professional windshield suction mounts for your recording devices. If your focus is on the dashboard, such as on an infotainment screen, you’ll need a video recorder mounted securely in the backseat or on a headrest. If your testing includes on-road tasks, you’ll especially need to ensure that the cameras and mounts won’t vibrate excessively. Point out to the participant where the cameras are, so they don’t inadvertently obscure the recording.

Terminology

Nowadays, vehicles of many brands share common international symbols and words in vehicle user interfaces. As with any usability testing, avoid framing questions or task instructions using words that appear in the interface (for example, instead of using “Repeat,” say something like, “Let’s say you just love this song, and you’d like to hear it over and over and over again. Show me what you’d do to make that happen.”) In some cases, you can just point to something and call it “this,” rather than name it.

Screenshots

You can’t use screenshot or recording software to capture information on a vehicle’s screens. You’ll need to rely on video recording and perhaps also a separate digital camera, the latter especially for photographing unusual locations, such as where a participant says the voice-activation microphone might be (because the fixed video recorders won’t be pointed there).

Background questions

It’s a good idea to get information from the participant on their own vehicle—make, model, year, and mileage, for example—as well as how they currently use features such as Bluetooth, steering wheel controls, and preset radio stations.

Participant comfort

To accommodate all body types, make sure before each session that the driver’s seat is positioned as far back as it will go, and let the participant adjust the seat to their liking. If it’s raining, it’s a nice touch to offer an umbrella when walking out to the vehicle.

Winter testing

In winter in northern climates, you may be doing in-vehicle testing in less than ideal conditions. Obviously on-road testing will be affected, but even with stationary testing, you’ll probably want the heater on, and the heater fan can make speech and video recordings harder to hear.

Ambient noise

In-vehicle testing, whether off- or on-road, involves an inevitable amount of ambient noise—the sound of the engine, the sound of the heater fan, outside traffic noise, sirens, and—if you’re doing infotainment system testing—the audio system. Make sure you test your recording parameters and levels under these same conditions.