Interpreting information and data

Interpreting the collected information/data is a critical stage in the process, whether that information is analytic data or direct text comments from students (i.e., quantitative or qualitative). Acting on comments without taking time to interpret them is a common mistake that can lead to actions that don’t solve the issue.

While students (and people in general) are pretty good at knowing how they feel about an experience, they are typically less adept at articulating why they feel that way and what would be the most effective way to resolve a problematic one. Despite this, feedback is often offered by users as a specific suggestion about what should be done to change a situation. Some survey questions also actively invite suggestions; the Student Feedback Survey (SFS), for example, asks students to ‘suggest any improvements that could be made to this subject‘. It’s not uncommon to respond to such feedback by implementing these student suggestions directly. It seems like the logical step – the students appear to be telling you what they want. Consequently, a student feedback response such as ‘I think the lectures should be shorter‘ might be met with a reduction in lecture time from an hour to 30 minutes, for example. Without understanding the exact problem with the lecture experience, this change could trigger comments after the next semester like ‘The lectures are too short – I don’t feel like I’m learning much from them’. So what do you do to unpack the suggestions provided by students?

By taking a moment to interpret what students might mean through their feedback suggestions you may start to see that there is a different core problem at hand, and due to this there may be a better way to respond. If ‘I think the lectures should be shorter’ actually means ‘I’m getting bored‘ or ‘I can’t focus on this sort of content in this way for an hour’, the learning design solution may look very different from a simple reduction in lecture time. Alternatives might be to provide the same hour of learning, but include some interactive elements and opportunities for students to discuss in small groups, for example. You may ultimately decide that your students’ survey suggestions are relevant and appropriate, but in that instance you are doing it because it is your choice not theirs. The more you know about your users needs, goals and experience in general the better you will be able to figure out the real why’s.

So when looking at user data and comments ask yourself:

• Why do they want that or why have they responded in that way?
• How does it line up with the other goals that users have (that you know about)?

Affinity diagraming is a HCD method that is useful for drawing core issues from a body of (typically qualitative) data. It is based around writing findings down on individual notes and then identifying and summarising themes to gather them into a few key points of need.

The process goes like this.

1. Using a set of coloured notes (say yellow) write down each individual qualitative datapoint from your research (a common example would be a user quote from an interview) and lay them out.
2. Spend time clustering the notes into different theme areas based on the sort of needs they are indicating. Each area should not have more than ~6 notes – if there are more there is probably additional definition that could occur.
3. Use another coloured note (say pink) to summarise a particular cluster of notes and do so by writing a comment in the voice of the user – so starting with ‘I need…’, ‘I like/don’t like’ etc. These become you key user needs.
4. If you wish you can then add another layer to the hierarchy (blue notes) to collect together different sets of pink note ideas.

Ultimately this method allows user needs to bubble up from the data rather than being driven by questioning and thinking that comes from the top down. This helps to draw out ideas that appear ‘between the lines’ of what people are saying, but which become more apparent when looked at collectively.