Embracing Agility in User Research

Plans have a purpose, but I’m always afraid of being too constrained by them in research, strategy and design work.

In academic research, plans help to ensure scientific rigor and secure funding, etc. In UX research, plans help manage expectations, coordinate teams, assure the bosses that we’re making good use of time and money, and generally maintain a degree of professional diligence.

But just as ‘no battle plan survives contact with the enemy,’ research plans have a way of changing once participants get involved. It’s not at all uncommon for interviews to go in an unexpected direction in the first few minutes, even when participants have been carefully screened. The uncertainty is both a bit scary and exhilarating — especially when it plays out in front of clients.

“Everybody has a plan until they get punched in the mouth.”

– Mike Tyson

During one session with a couple of continuing education students, it became clear that the participants were much more dependent on the print catalogue than we expected. Most of their comments revolved around making a 200-page PDF easier to find — despite also expressing frustration about it. I thought we’d be getting feedback about the website but they hardly used the site at all. Most of the prepared questions didn’t make any sense in that situation.

The participants’ fixation on finding the PDF and leaving the site as fast as possible felt like a classic case of “customers don’t know what they want.” The planned discussion was mostly a dead end, but we’d been given an unexpected gift: now we had an opportunity to observe people using the site somewhat naturally but in ways we hadn’t seen, based on real needs, to get deeper insight into the experience. So I turned my laptop around for the participants to use in some unplanned usability scenarios. The unexpected change gave us a better opportunity to see what people really saw and thought while on the site.

Uncertainty and change don’t just affect individual sessions; the whole process often needs to be open and adaptable to unexpected discoveries.

If you’re too rigid you often end up missing the best opportunities to learn. Over-planning reinforces biases and assumptions in a closed feedback loop that makes transformational insights harder to get. Rigid plans are often fine for evaluative research, like when you want to know how satisfied people are or whether they know where to click on an interface, but rigidity constricts the exploration and generation of new ideas. Transformative discovery is the art of serendipity — putting yourself in a position to be surprised by new questions.

“Chance favours the prepared mind.”

– Louis Pasteur

Planning helps anticipate some of the more predictable surprises. Planning forces you to visualize how an interview or group session or set of survey results might go, then tailor your questions, methods and materials accordingly.

A while back I started drafting a short questionnaire to recruit and screen participants for a personal project on how people listen to music. In the process of drafting the questionnaire I thought, “wait, what if someone says they ‘listen in the car’ but mainly to news and talk shows?” Podcasts and talk radio often do the same job that music does. “Would the question make sense? Would the answers be credible? And why should this study be limited to music?” It became clear that I needed to do some more open-ended exploration — talking to people — to better understand the scope of the challenge I wanted to address.

“Plans are worthless, but planning is everything.”

– Dwight D. Eisenhower

The plan itself is no guarantee that every contingency is covered, but the planning process helps anticipate problems and opportunities to learn. Planning makes us think carefully about the purpose and goals of our research. What decisions will we inform? What are the underlying questions we need to answer? Those answers then become the pole star we can use to navigate through uncharted terrain.

The most effective UX research plans, I think, adopt Agile and Lean principles.

Agile principles dictate that work should be iterative, collaborative, and open to change — with value delivered early and often. You might have a research roadmap (or loose plan) looking months ahead, but the plan is subject to change based on early findings and decisions. Instead of waiting to package everything in one big, hairy deliverable, insights are shared and applied as work progresses.

Lean principles go a step further and make the process more continuous. In a lean system, generally, work is pulled in small batches in response to demand. Lean research is a response to decisions that need to be made, questions that need to be answered, and assumptions that need to be tested — all of which continue to emerge during and after implementation.

The most adaptable research is a process of continuous discovery — not as an excuse for bad planning but as a natural response to evolving questions and needs.

Original post: https://res.im/blog/embracing-agility-in-user-research