Creating user conscious questions

If you decide to use methods that involve asking questions that users respond to (asking people, survey, interview, etc) the framing/design of those questions should focus on the sorts of things that participants would be able to comment on from their own experience. While you may be trying to get information that allows you to answer your research questions (and ultimately meet your research objective), the research question itself may not be something that the user would be able to answer if posed directly.

• Good user focused questions frame things in a way that makes it easy for individuals to respond by framing the concept in the way that makes the most sense for them and/or by providing a ready onramp for their comments.
• Poor user focused questions are ones which individuals have difficulty answering because they do not have a ready thought/experience/opinion at hand.
• The best user questions are based on users telling you about something that they have themselves previously experienced (done or felt). While it may be tempting to put weight in user comments about a proposed idea, people often struggle to report accurately on what they might do in a speculative future scenario (they might think they know but the ultimate result might actually be quite different).
• Open ended questions enable discussion and give participants the opportunity to explain things in their own words (questions starting with how, what, who where, when). Sometimes it might be possible to follow up a short participant answer with a ‘why?’ or ‘how’ to prompt them to further expand on the information provided.
• Questions should allow for a single answer. Try not to create double barrelled questions, as these ultimately cause confusion in relation to where.

In the process of question design, try to keep in mind that participants have their own agendas and biases. What this means is that, whether aware of it or not, participants will tell you:

• What they think you want to hear.
• What makes them look good (or feel good).
• What doesn’t incriminate them/put them in a bad light.

This is particularly important to remember if you are in a position of influence – students are not going to report accurately about their poor study habits to their teacher because they feel that it will reflect badly on them and have an impact on their grades. This means that sometimes it is necessary to conceal the nature of your true question so that you can get the information that you are after without being impacted by peoples’ natural biases. While noting that filtering bias is also part of the interpretation stage (discussed below), it is best to avoid as much as you can in the question stage to make sure that you’ve got the most useful data to go into the next stages with.

Learning how to write user focused questions is a skill that takes time to develop and relies on seeing for yourself which questions work and don’t work in practice. Take a shot at it and refine your practice as you go. Iterate and fail fast!