Interception! Best Practices for a Field Study
Usability testing is not limited to labs with one-way glass windows and video cameras; every now and then a product needs to be tested that either is too unwieldy for a lab or needs to be used within a specific environment or context. Enter the field study, a methodology borrowed from social science ethnographic studies: observing people in their “natural habitat,” rather than in the contrived environment of a lab. The expectation is that more natural behavior will occur when an item is used in a typical environment—such as a workplace office or a home—under ordinary conditions. Recently I (along with a colleague) conducted our first field study; as I was unable to find any “how-to” material that addressed our specific needs, I decided to summarize some of the things I learned on this project in this blog post.
For this project, the client wanted customers at its retail stores to try out a prototype of a phone application. This method, which is often used for marketing research, is sometimes called a “mall intercept,” since the participants are intercepted by the researchers at a designated location. Compared to working in a lab, this field study involved considerably more preparation. However, some things were determined on the fly: it wasn’t always possible to accurately predict ahead of time how things would transpire, so flexibility was a must for this type of testing. We noticed, for example, that many customers were in a hurry and tended to ignore us.
By experimenting with different approaches, we found we could catch their attention if we held out the phone—with the app displayed on the screen—toward the customer. This was also a way to forestall objections from customers who said they did not have time to install the app on their phones. We set up a schedule and the locations early in the preparation process, since the client wanted to test in 4 different stores over 4 different days. To lessen any bias on the results due to the day of the week, time of day or store location, we devised a schedule that alternated these factors: for example, we went to Store A in the morning and Store B in the afternoon of Day 1; then on Day 3, we visited Store B in the morning and Store A in the afternoon.
We also scheduled stores that were located close to each other on the same day to minimize travel time. Although the managers for these retail stores had been informed of the testing, I contacted each one directly to introduce myself and inform them of the scheduled dates and times we would be at their stores. It took me several days to get hold of 4 managers, so it’s important to allow plenty of time for this step. Even though not all of the managers were on the premises during testing, contacting them ensured that other employees, such as assistant managers, were aware of our study.
We therefore had a frictionless reception at each location, meaning we spent less time explaining to store personnel what we were doing. In contrast to lab tests, we had no control over the ebb and flow of customer traffic in the stores. One store emptied out after 6 p.m. on a Sunday, just an hour after the check-out lines were spilling over into the product aisles. Fortunately we reached our daily quota for participants just before this lull. From this inaugural field study, I gained an appreciation for the controlled aspects of lab tests, even as I learned to survive and thrive under fluid and challenging conditions. I’m already looking forward to discovering more insights on the next project “in the field.”