Bridging the gap between user research and behaviour
I’m a software designer that helps improve the experience of using digital products. I help companies understand their customers and create solutions that fit their needs.
Understanding customers is critical, because there is a surprisingly significant difference in the way each of us interprets and responds to the world around them. Consequently, the look and feel of an application can make or break it; if a user doesn’t interpret and respond to an interface in the way that is expected, it won’t work as well. Without understanding, I have to make assumptions based on my own life experience, which exposes the risk of error of judgement, known as cognitive bias.
Every human is susceptible to cognitive bias. We naturally fill gaps in our knowledge with assumptions if we don’t have the right information. For example; we might assume that the people we’re designing for understand the technical jargon of our own industry.
To fill the gaps in my customer knowledge, I take time to observe how they use technology, consider the environment they use it in, note their challenges and listen to feedback on design ideas. Interestingly, the insights I gather from observing and speaking to customers don’t always turn out to be accurate. There is a dissonance between what people say they do and what they actually do. This article explores why and how this happens and suggests techniques to mitigate the issue.
How & Why People Don’t Speak Accurately
Subconscious Actions
There is a major gap between what people say and what they do. In research interviews, customers tend to filter their thoughts and rationalise their actions to create seemingly logical explanations. However, their actual behaviour is heavily influenced by subconscious thought processes that favour safety, clarity and habit. People are not aware of their subconscious thought processes, so they apply a legitimate-sounding narrative to their past actions to help make sense of them.
When interacting with an app, most behaviour is driven by subconscious muscle memory, micro-interactions and triggers (like a notification). When asked later, "How did you complete this task?", people struggle to recount the hesitations, disorientation or confusion they experienced; they simply summarise the final outcome.
Nielsen Norman Group proved the role of the subconscious in user behaviour with a study known as The Background Colour Experiment. They interviewed people about a design asking what background colour they preferred. People confidently stated an opinion. However, when their actual behaviour was monitored via usage analytics and eye-tracking, their performance, reading speed and navigation choices often completely contradicted what they claimed.
Bias Blindness
We have a skewed perception of our own behavior; typically, we recall our own actions in a more positive light than reality. If asked to recall personal life events, people tend to remember more positive memories than negative ones, or rate positive personal memories as stronger. Unpleasant memories also tend to fade faster over time than positive ones to help maintain our own psychological well-being.
Culture, Communication & Language
Our cultural background influences the way we communicate with each other. In some cultures, such as British society, people tend to soften criticism and accentuate positive feedback so as not to appear ill-mannered. Speaking with people from different cultural backgrounds can also lead to misunderstanding, as the difference in communication and behavioural style can trigger subconscious emotional responses that inaccurately interpret a message.
Even shared culture and language is prone to misunderstanding. As powerful as language may seem as a communicative tool, interpreting the meaning of a phrase is still dependent on the mood of the conversing parties. Sarcasm, for example, can be missed in transcription, or misinterpreted by somebody in a different mental and emotional state. Education levels also play a part in comprehension; without full knowledge of the meaning of every word in a dictionary, misinterpretation is a risk.
Can We Trust What Users Tell Us?
Yes, but we should be aware that the answers people provide represent their personal perception - not reality. People are good at explaining what they believe, but don’t explain their unconscious decisions. This distinction is important. It’s the reason research needs to include more than just interviews to truly understand customer behaviour.
Interview Effectively
Open vs. Closed Questions
The wording of a question can influence the answer. “Do you like this feature?” encourages positive feedback, which we don’t neccesarily need. “What happened the last time you used this feature?” asks for an experience. The latter is an ‘open question’.
Open questions are usually more useful as a a starter because they encourage stories rather than a one word answer. However, they can sometimes be too wide; ask about specific events rather than general behaviour - they help to focus the answer on the relevant topic.
After gathering some answers from open questions, narrow down the details with closed questions, such as “When did you last do this?”, “What were you trying to achieve?” or “What did you expect to happen?”. These questions encourage more specific, near-binary responses. These questions move the conversation from opinion towards evidence.
Observe Behaviour
Test Your Product
To reduce the gap between what people say and do, observe them using your product. Instead of describing a task, ask them to complete one. The way they interact may reveal friction that language can conceal.
Try to simulate real life as accurately as possible. Give people space so they don’t feel pressured and avoid explaining or helping too quickly. Mistakes tend to offer more valuable insights. A participant who cannot find something demonstrates a problem.
Contextual Research
Context changes behaviour. Our surroundings influence our capacity for attention, motivation and patience. For example, using an app at home is different to on a train; factors such as network connection, privacy and distractions differ between these environments and have significant influence over how an app is used.
Observing people in their natural environment can reveal problems that a research room cannot. A feature may work perfectly in controlled scenarios, but fail in the real world. For example, it may render incorrectly on different devices, crash on slower machines or be impossible to use for users with unique accessibility needs.
Verify Theories
Research becomes stronger when different studies tell the same story. It’s harder to dismiss theories when backed up with evidence from multiple sources.
Interviews can reveal potential problems, but they need to be verified by other means. Distill interview findings into hypotheses such as “Our registration form is confusing people and causing abandonment. Improving message clarity will increase form completion rates”. These can be verified by measuring quantitative data that proves/disproves the theory.
Measure Analytics
A theory can be backed up with analytical data, but it’s important to think carefully about which metrics to monitor. For example, a high number of registrations doesn’t necessarily mean your app is a success - you should measure the amount of traffic starting and the amount completing the process to get an understanding of efficiency.
Consider A/B testing your ideas by releasing a new design to half your users and comparing performance against the original.
Review Customer Support Trends
Recurring problems can often be spotted in customer support records. Frequently asked questions can be a great source of direction when weighing up what to prioritise as a designer.
Design for What People Do
The ultimate purpose of research is not to understand people perfectly - that’s impossible. The goal is to make better decisions with better evidence. Users will not always behave as they expect. They will not always explain their behaviour accurately. They may not even understand why they made a decision. That is normal.
Good UX does not require perfect users. It requires designers who understand the difference between intention and behaviour. Listen to what people say. Watch what they do. Design for the gap between the two.
A.I. and User Insights
The Summariser
When using AI to analyse user interviews, be aware that while it is an incredibly powerful efficiency tool, it’s a summariser, not an interpreter. Relying on it blindly risks producing inaccurate or ungrounded insights. On top of this, relying on AI to read and process documents removes you from the thinking process, which is where the best ideas come from.
Garbage In, Garbage Out
It’s essential to distill accurate insights from poor quality information. Flawed or poor-quality data will always produce a faulty or useless output. If you put bad information into a system, you’ll get bad results no matter how smart the system is.
The phrase “Garbage In, Garbage Out” became popular in the early 1960s, though the idea can be traced back to computing pioneer Charles Babbage in the 1800s. It applies to everyday life, business decisions, data science, and artificial intelligence, where bad inputs lead to bad choices.
Hallucinations & Blindspots
Although we can be sure that AI systems are not under the influence of psychoactive chemicals, we can’t be sure that their forefathers were and that they have immunity to distortions of reality. AI systems are known for subtle fact fabrication. Hallucinations in data analysis are often quiet and hard to spot. An LLM might smoothly merge two different users' thoughts into one cohesive quote or subtly tweak a participant's sentiment to make it sound more polished.
Sycophancy Bias
AI models are prone to giving you exactly what you want. If you prompt an AI with "Find examples where users struggled with our checkout page" it may over-index on minor complaints or take neutral comments out of context just to satisfy your prompt.
Non-textual Context
Transcripts only capture spoken words miss behavioral data, such as a user sighing in frustration, hesitating for five seconds before clicking a button, or their facial expressions during a video call.