Using AI Responsibly for User Knowledge Mobilization

I’ve spent a lot of time lately thinking about knowledge, or more specifically, what happens to it after a research study wraps up.

I’m not talking about the debrief. I’m not talking about the deck that lives in a Confluence page no one looks at. I mean the actual findings ( the nuanced, contextual, sometimes-contradictory things users told us) and whether they ever actually inform decisions at scale.

Spoiler: often, they don’t. Not because the research wasn’t good. But because knowledge mobilization is hard, most orgs haven’t cracked it.

So when I started thinking seriously about building a “User Need” bot (an AI-assisted tool that lets product teams query research findings the way you’d ask a knowledgeable colleague) I got excited. And then I got nervous. And then I realized the nervousness was actually the important part.

The Promise is Real and so is the Risk

Here’s the appeal: a PM is prepping for a roadmap session at 10pm. They need to know what users have said about a feature. Instead of pinging a researcher or digging through three years of UXR notes, they ask a bot and get a synthesized, sourced answer in seconds.

That’s useful. That’s what knowledge mobilization could look like.

But research repositories (if they even exist!) are not neutral archives. They reflect who we recruited, what we asked, when we ran the study, and what the researcher chose to highlight. If you feed that uncritically into an LLM, you don’t get insights, you get confident-sounding misrepresentations.

“Users find our UI confusing.” Sure. But which users? In 2021, on a flow we deprecated? With a sample of 6 people recruited from a certain persona that skews heavily towards that demographic?

Stripped of that context, that finding is not just unhelpful… it’s risky.

Garbage In, Garbage Out And Worse

Before research teams even think about training a bot, we need to sit with an uncomfortable truth: a lot of what lives in our knowledge base isn’t ready to be democratized.

Not because we did bad work but because insights were never designed to travel without their author. They were built to be presented and contextualized in real time by the person who ran the study.

So step one is a critical audit. Not just of the data, but of the methods used to gather it. Were the right techniques used for the questions being asked? Is the insight well-supported across multiple studies, or is it a signal from one session that somehow became a “finding”? Is it still relevant, or has the product or the user base changed significantly since?

This is unglamorous work. It doesn’t make for a flashy internal pitch. But it’s the foundation everything else rests on.

Training the Bot to Think Like a Researcher (Not Just Retrieve Like a Search Engine)

Once you’ve got clean, critically evaluated data, the next challenge is making sure the bot handles it the way a careful researcher would.

That means building in humility. Tagging findings with confidence levels, including whether the finding is established across multiple studies or emerging from a single one. Flagging methodology. Applying appropriate hedging when confidence levels are low. Surfacing scope conditions. Preventing inappropriate generalizations. Making the bot say “this finding comes from a study of 8 enterprise users in 2022 and may not generalize” rather than just “users prefer X.”

Basically, the bot should answer the way your best researcher would answer a stakeholder question. Curious, precise, and completely comfortable saying “it depends.”

This isn’t just a prompt engineering challenge; it’s a design philosophy. The goal isn’t a confident assistant. It’s an appropriately humble one.

The Researcher Still Needs to Be in the Room

I want to be direct about something: building this bot is not a path to removing researchers from the process. If anything, it’s the opposite.

Someone has to architect the knowledge. Someone has to make the call on what goes in, how it’s tagged, and what limitations get surfaced. Someone has to be accountable when a high-stakes decision gets made based on a finding the bot retrieved.

That someone is a skillful user research leader.

What changes is where the research team’s judgment gets applied. We may spend less time answering “what did we learn about X” at the stakeholder readout, and more time doing the foundational work that makes the bot trustworthy in the first place + staying in the loop for the decisions that matter most.

I actually think this has the potential to be an even more interesting version of the job. But it requires us to take ownership of it.

When the Loop Works

Here’s what we could build toward: a state where a PM, designer, or engineer can query research findings and actually trust what they get back. Not because the bot is perfect, but because a researcher put careful work into making sure what went in was worth using.

When that loop is closed with a quality audit, critical annotation, nuanced model behaviour, and clear escalation paths for complex queries, then you have something worth sharing broadly. Non-research folks can explore findings with real confidence and effectively use insights to inform decisions.

This Has Always Been Our Responsibility

I note one last thing that feels obvious but is easy to lose track of in the excitement of new AI opportunities...

Preventing costly decisions made on misunderstood evidence has always been core to what researchers do. That’s not new. A “User Need” bot or AI integration tools don’t change that responsibility… it just makes the stakes of getting it wrong a lot higher, and the consequences a lot faster.

If done right, this is one of the most impactful things a research team can provide. If done without much thought, it may be a quick way to scale mistakes.

Start with the research quality loop. AI comes second.