Lean Ethnography: Is That Even a Thing?
How does a heavyweight research technique like ethnography fit into a lean user research process?
Hint: It doesn’t. We cheated. #worthit #sorrynotsorry
Tasked with the broad initiative of reimaging the monograph, we at JSTOR Labs knew we needed a lot of rich, detailed information about how scholars currently use monographs. After all, you can’t solve complex problems for someone until you have a deep understanding of those problems.
Normally, to prepare for a project, JSTOR Labs does a handful of hour long video chats. That’s leaner and quicker than ethnographic research, but still provides some of the contextual information about our users that helps us understand the users and their problems. For this project, though, the scale of the question we were asking was a great deal larger than we’ve typically faced. “How are books used in research” is considerably broader than something like “How do undergrads find articles and books about Shakespeare”? We didn’t think our usual approach would be rich enough.
Luckily for us, the Labs team works within the larger JSTOR and ITHAKA organizations. We were able to partner with the wonderful Christina Spencer, JSTOR’s User Research Manager, to craft and conduct the user research the project demanded.
A full-fledged ethnographic study may take years, perhaps with dozens of participants, and days, weeks, or months with each participant. This allows for time to build trust and observe the full spectrum of activities and interactions that those participants have. That methodology is extraordinarily valuable, but it surely isn’t lean. With Christina’s guidance, we found a happy medium. Our approach was lean enough to let us move forward quickly but in-depth enough to collect nuances from scholars’ environments and workflows that remain hidden in a Skype interview. Over the course of a couple weeks, Christina spent a day each with six history scholars, identified for us by AHA, who use monographs in their research. She documented everything they did in their regular work -- reading, taking notes, searching stacks at the library, writing, flipping through books, photocopying pages -- even taking breaks to play PokemonGo and watch TV. The goal of this type of observation is to learn how this work really happens and why. People are often unaware of the details of their processes, particularly workarounds they have developed over time; you have to watch them, and not just ask, to get that information.
We wanted to share what she learned both at a workshop we had planned and within the white paper we planned to draft. The trouble was, there was so much information! We spent days reading and re-reading notes to see where similarities, differences, themes, and surprises appeared. After significant reflection and discussion, the findings for the study were consolidated into profile infographics for dissemination. The profiles present a portrait of each scholar [names and identifying details changed for privacy], with particular emphasis on how books fit into their daily work. To see them in more detail, check out the appendix of Reimagining the Monograph (.pdf).
I won’t go through every piece of information in the profiles here, but I will use one as an example. It was a case of a “simple” question having a complex answer that was a challenge to represent visually. Before Christina observed scholars, we were curious for which tasks scholars preferred print vs. digital books. What she observed was that there was a spectrum of preference for each task performed with a book. We created a graphic with accompanying text to represent where each scholar fell on the spectrum for each book task and why.
This scholar, for example, preferred digital books for most tasks, but she found close reading easier with a print book. It’s more reliable to infer the type of book someone is likely to use for citation mining after a couple of hours of watching them mine citations and confirm with them what you saw, than to explicitly ask them in 20 seconds if they use digital or print books to mine citations.
We won’t conduct ethnographic studies for all, or even most, of JSTOR Labs’ new projects going forward. It takes a lot of time and effort, and is often overkill for the types of questions we’re attempting to answer. But, I’m glad to know it’s in our toolkit for the next time we come across a vast, murky problem we need to make sense of. Heavens knows there are a lot of these kinds of problems out there that need solving. #ethnographyftw #evenifitsnotlean