Using AI research tools without fooling yourself

Most of the big AI assistants now have a research mode. ChatGPT, Gemini and Claude can each search the web, read a pile of pages and come back with a written summary and a list of sources. For a small team without a research budget, that’s genuinely useful.

It’s also easy to be fooled by. A report that looks thorough isn’t the same as one that’s right. This note is how we use these tools in our own discovery work, and the habits that keep us honest.

Where AI research tools help

Desk research is the reading you do before you talk to anyone: what’s already been written, what similar organisations do, what the rules are. AI tools are good at the first pass of that work.

• Getting the lie of the land on an unfamiliar topic or sector.
• Finding what comparable organisations offer and how they describe it.
• Summarising long public documents, such as reports, guidelines and policies.
• Suggesting questions you hadn’t thought to ask.
• Turning your own messy notes into a structure you can work from.

Used this way, they save time and widen your view. The trouble starts when the summary becomes the finding.

Check every source that matters

AI research tools can cite a page that doesn’t support the claim, misread a statistic, mix up two organisations with similar names, or rely on an article that’s years out of date. The writing stays just as confident either way.

Our habit is simple: if a fact will appear in anything we publish or use to make a decision, someone opens the original source and reads the relevant part. Not the summary of it, the source itself.

• Open the link and find the sentence that supports the claim.
• Check the date. Rules, prices and products change.
• Prefer the primary source, such as the standard, the law or the organisation’s own page, over someone writing about it.
• If you can’t find a source for a claim, leave the claim out.

It’s slower than copying the report, but much faster than correcting a mistake after it’s published.

Watch for bias and blind spots

These tools reflect what’s easy to find online. That tends to favour large organisations, English-language sources, overseas examples and whoever has the best search ranking. Small Australian community groups, First Nations perspectives, and people who don’t write about their experience online are often missing or misrepresented.

The way you ask matters too. A leading question tends to get an agreeable answer. Try asking for the strongest case against your idea, or for evidence that would prove you wrong, and see what changes.

Mind what you paste in

Anything you put into an AI tool is sent to its provider. Before pasting in interview notes, survey responses, client documents or internal plans, check what the tool’s terms say about storing your data and using it for training, and whether your plan lets you switch that off.

As a rule, we don’t paste in anything that identifies a person, and we strip names and details from notes first. If your organisation works in health, education or community services, check your own policies before you start. The Office of the Australian Information Commissioner has guidance on privacy and AI products that’s a sensible starting point.

Where you must talk to people

No AI tool can tell you how your own visitors, members or clients actually think, because it has never met them. It can tell you what people in general tend to say online, which is a different and much weaker thing.

The most useful research we do is still a handful of conversations with real people: watching someone try to find something on your site, or asking a volunteer what slows them down. Our note on what five conversations can teach you covers how to do that on a small budget.

AI can help around those conversations. It can help draft an interview guide, or sort anonymised notes into themes, as long as a person checks the themes against what was actually said. It can’t stand in for them.

Our rule of thumb

• Use AI for the first pass, not the final word.
• Open and read every source behind a claim you’ll rely on.
• Ask for the counter-argument, not just confirmation.
• Never paste in anything that identifies a person.
• Talk to real people before you decide anything that affects them.

Where to start

Next time you research something, try running the question through an AI research tool, then check three of its sources by hand. You’ll quickly get a feel for how far to trust it.

If you’re planning a redesign or a new service and want research you can rely on, have a look at our research and discovery service, or tell us what you’re working on.