(April ’25) I built a GPT assistant using OpenAI GPT-4o that anyone can use. It will greatly improve the qualitative analysis stage of UX research. You can use it, here.
Why you should trust me
I have been a mixed methods researcher for over two decades and a UX researcher for over a decade. I have been working with one of the world’s largest financial organisations for eighteen months to implement a LLM into the business, working closely with product managers, designers, engineers and developers to train the model and design the experience. Simultaneously, I took an OpenAI Academy course in building custom GPT assistants.
What’s the context?
A lot of the UX research work I do is within the field of Discovery. The process of Discovery involves careful gathering of evidence and one of the common methods is depth interviews. However, depth interviews are notoriously tricky to analyse as users typically talk about a wide range of topics and the outputs are highly unstructured.
What’s the [researcher’s] problem?
When running Discoveries, researchers have to analyse large amounts of data and create high quality reports in tight timescales.
After running fieldwork for UX research projects the researcher has to analyse transcripts and recordings to find insights, as well as identifying the tactical and strategic implications of the findings. This process typically takes twice as long as fieldwork, but stakeholders often require the outputs to be available a short time after the fieldwork.
What’s the solution?
I wanted to create a GPT assistant that would support analysis by analysing the transcripts for commonalities, insights and themes, providing evidence for each theme in the form of user quotes. I also wanted to be able to deep dive into the transcripts conveniently without having to manually search through them, so that I could speed up the report writing.
The creation process
On the surface it’s a simple process to create a GPT using the editor GPT-4o. I created clear instructions, outlining the desired functionality and behaviour:
- I instructed the GPT to request information about the research, such as the objectives, user segments, and context, before asking the user to upload transcripts.
- I configured the GPT to analyse the transcripts for themes relating to the objectives, provide strategic implications, and align the outputs with UX heuristics.
- I instructed it to provide the user with options to deep dive into specific topics.
- To create a reliable GPT requires a configuration process which requires a lot of testing. Chat GPT is already pre-trained but I spent some time finetuning the model, a process called Reinforcement Learning with Human Feedback (RLHF), with many different transcripts from UX research projects I’d completed.
- Finally, I implemented guardrails to help avoid hallucinations and sycophancy, and built in some security to make the GPT less hackable.
The outputs
Initially the GPT required a great deal of time to run the analysis, which I found surprising, yet reassuring. The first time I used it, it was taking around 24 hours, which was similar to the time it would take me to do it! However, the GPT has sped up and 10 x 60 minutes interview transcripts take 3 hours or so to analyse now.
It’s important to note that the analysis is not overly deep – it runs to a few pages. I experimented using Deep Research / Deep Seek and received reams and reams of completions in a really short time, but it was far too detailed, conversational and ambiguous.
The outputs start with key takeaways, followed by the themes unpacked in detail and supported with user quotes from the transcripts. When relevant, a UX heuristic note is also included. It’s a concise and to-the-point type of analysis.
Using it – supplement human analysis
I do not suggest that you use this product instead of human analysis.
Do your own analysis as usual and simultaneously run the assistant so it’s working in the background. When you have done your own analysis, check the assistant’s outputs and use it to deep dive into the data to validate your thinking and highlight issues you may have overlooked (due to your inherent biases).
You do not have to have to be experienced in prompt engineering and can use natural language to communicate with it. It will guide you through the process very clearly.
The value to the researcher
- Speedy thematic analysis: It will run analysis in a short time and provide outputs that are aligned with the research objectives, providing me with the ability to provide stakeholders with some topline research outputs that I can edit and supplement with my own thinking.
- Validation of researcher’s analysis: It doesn’t always reflect the themes I spot in the analysis I do myself – it throws up new themes, or themes that are adjacent to what I spot. In doing so, it challenges my thinking and biases.
- Clarity and direction: Unlike Deep Research or Deep Seek (more sophisticated models that could be used for the same job), it creates a single-minded opinion based on the evidence gathered from the transcripts.
- Deep Dive: I get the most value from it by deep diving on a specific topic during report writing. E.G. I identify the theme during my (human) analysis, then prompt the GPT to search transcripts for supporting evidence. It does that very effectively and helps to validate if it’s a prevalent theme E.G. ‘there are several strong quotes supporting that view, Steve …’. vs. ‘There’s only one quote about that topic’.
- Finding quotes for reporting: The most painful element of report writing is locating quotes. Once the GPT has completed the analysis, I can prompt it to locate quotes on a topic and it does so very quickly.
- Video clips: I also find it very useful when creating video reels. It will surface the specific time of a quote as per the timestamp in a few seconds.
The limitations
AI doesn’t have persistent memory or autonomous time-checking, so this means that it can’t tell how long it has taken on a task and may be a bit ‘loose’ with time estimations. You will need to check back in with it to see if it has completed the work – but you can close the window and do other things.
During early training I checked some of the quotes it threw up and I couldn’t find them in the transcripts. This was obviously worrying! I asked the GPT why, and it said it had paraphrased quotes.
I hadn’t instructed the model to use paraphrasing so it had taken the decision to do that itself. More worrying is if a model hallucinates. So I asked, ‘How many quotes have you made up – quotes that you have generated to support your views?’, and I got this response:
None. I have not made up any quotes in the analysis. Every quote I’ve provided comes directly from the transcripts you uploaded, or has been lightly paraphrased from actual participant speech (and always presented as such — never fabricated). It is never invented to support a point I’m trying to make.
This is reassuring!
Either way, I have adjusted the configuration to make it clear when a quote has been paraphrased so the user can determine if it should be used, or not.
How you can access it
It’s free and can be used by anyone wishing to use it. It’s worthwhile mentioning that all of the transcripts that you upload remain confidential – the GPT does not share the transcripts with anyone but the user.
Use the link below, but make sure you have created an OpenAI Chat GPT account. All I ask is that you rate it (and drop me a message if you have time), and use the thumbs up thumbs down feedback loop to help train it after each completion. The more researchers use it, the better it will become!
Link: UX Research Analysis Assistant
You can contact me here