I used AI to analyse how my job has changed over the past 10 years. It reveals greater levels of sophistication and the impact of adopting research automation.
(January 2026) At some point many years ago I started recording the basic details of each research project I did in an Excel file, and I recently had the bright idea of using AI to reflect on the patterns, progress, and lessons learned.
Taken together, the data tells a story of sustained, hands-on engagement with users across many contexts, markets, and product stages:
- 85 total projects – just over 8 per year
- Over 3,500 users users engaged across AMERS, EMEA, and APAC
- Work spanning B2B and B2C
By stage of the product lifecycle
- 32 x Discoveries
- 26 x Alpha
- 11 x Beta
- 14 x Live
- 2 x Training courses
By type of research
- 23 x Exploration & Concept Development (including quantitative)
- 15 x Prototype Testing
- 14 x Design-related research
- 10 x Usability studies
- 9 x Concept Development projects
- 9 x Quantitative / Metric projects
- 5 x Persona projects
What’s changed?
I’ve worked for 12 different organisations over this period, either on a freelance or contractual basis. Each one had a culture shaped by leadership that distilled throughout the organisation into the UX teams that I worked with; Innovative, trend-chasing, disorganised, traditional, laser-focussed.
Most often I’ve worked as a researcher in multi-disciplinary product design teams, but also as a freelance gun for hire, in future focussed 10X labs, within a GDS design team, and in the UK’s largest client side research team (39 researchers, in 2019, before the pandemic).
So I have a lot of context and variable experiences to draw on. I used Chat GPT to analyse my spreadsheet and provide evidence of how things have progressed.
Remote research as the norm
One of the most obvious adjustments has been the lack of face-to-face research. I’ve only run a handful of in-person research studies since the pandemic. Before that, it was common to do research interviews in a lab with participants using products whilst they were filmed on cameras pointing at their face / hands / screen as they performed tasks. Insight walls and wash ups followed. But remote research has become the norm now, unless you absolutely have to be in the space with the user.
Stakeholder maturity
Stakeholder skepticism used to be part of the job. UX research maturity was poor and the challenge wasn’t gathering insights, but demonstrating why those insights mattered. These days those relationships have become less of a battleground and the friction tends to be about getting time and access to stakeholders rather than whether research is needed at all. That’s a meaningful evolution. The debate has moved from legitimacy to logistics, more collaborative and less critical.
Product team involvement
In terms of stakeholders I work with, there’s been a shift that has resulted in more work with product owners. I team up with product people a lot more than ever and in some companies they are driving the research requirements as much, if not more, than the designers. They are seeking to determine user needs or pain points for their products, in order to prioritise specific improvements.
I think this shift may have been driven because designers, without the deep understanding of product and engineering, proposed solutions that were not supported by incumbent technology; Creative solutions that were not able to be built. I get the feeling that, these days, the product team is driving the direction of travel more than design.
Complexity and measurement
Something else that’s become more evident is working with complexity, particularly around data and measurement. Earlier work rarely flagged complexity as a challenge but recent projects do because evidence of product effectiveness has become more important. The most obvious driver of this is OKRs (after the book ‘Measure What Matters’ was published in 2018).
I’ve noticed a shift in research at the different stages of the product lifecycle and there’s more focus on product performance than in the past. This means the work I do has become more sophisticated and I’m doing much more with analytics, surveys, metrics and mixed evidence.
Research automation
This research sophistication has been supported by increasing automation. Ten years ago, a lot of my time went into mechanics: transcribing interviews, manually clustering themes, analysing quantitative data, and generating themes into something intelligible. Those tasks were necessary, but they were also slow, repetitive, and mentally draining. Tools that support automated transcription and quantitative theme generation have made it faster to get from raw data to structured insight. Because the grunt work is lighter, I can hold more complexity at once, providing faster and richer insights.
Automation has infiltrated most areas of research. In the last 12 months, AI-moderated interview tools and synthetic respondents have launched. It feels like startups launch every week, promising to revolutionise UX research with ‘all-in-one, AI-driven solutions’.
AI Automation pain
I’m still learning how to make best use of AI automation for analysis. I regularly use Dovetail, Figma, Chat GPT and Co-Pilot to generate themes from raw data and it’s very tempting to just let it do its thing. But I find they can wrap you up in knots.
AI tools regularly over-assume e.g. refers to ‘users’ plural, confuse the context e.g. confuses real life products with prototypes, use hyperbolic tonality e.g. ‘significant desire’ or ‘critical issues’, and create unnecessary noise e.g. over analysis, circular-referencing.
I find the time I was saving doing the grunt work is now being eaten up by perfecting the prompts, critiquing the completions and mending / stitching the AI outputs into a format which I feel confident in before sharing with stakeholders. That said, the overall time saved is astonishing.
Recruitment pain
The final observation is that recruitment is still as difficult as it was a decade ago. B2B user recruitment, particularly in finance, increasingly means navigating tighter schedules, more complex compliance concerns, and gatekeepers who don’t understand customer research.
Consumer recruitment was never easy and it became a lot more complex after GDPR was introduced in 2018. On the one hand there’s been a huge rise in online panels of users with lower costs and automated screening, but most researchers still complain that the research industries dirty secret, the ‘professional respondent’, is alive and kicking. No matter how well we design screening surveys, those dedicated enough know how to game the system.
Summary
When you look back at a decade of research, it’s tempting to say, “I’ve got this now.” The reality is more nuanced. What used to be painful has been replaced by other challenges.
I no longer have to cram users into a lab for a day / week of interviewing back to back. I no longer spend a week analysing dozens of user testing recordings and another week reporting, whilst being chased for the report. And I don’t have to analyse hundreds of open ended survey comments for themes, or create PowerPoint charts of findings.
Those former sources of pain have been replaced with greater complexity and amount of work. The number of projects I do has increased significantly: In the last year, since I’ve been using automation and AI to support analysis, I’ve run 15 projects at various stages of the product lifecycle. That’s double the number I was doing a decade ago.
The work involves an increasing amount of evidence that needs to be analysed and I have had to completely adjust the way in which I conduct analysis and create reports. Most recently I have struggled with the way AI presents its analysis and spend more time adjusting the outputs than I would like.
But the overall direction of the work I do is positive. The outputs feel strong and the work is received well. I’ve stopped having difficult conversations about how important research is to the product design process and instead I get a strong sense that the research is valued and implemented into roadmaps more readily than ever before.
Contact me here to learn more about the work I do.