Thinking Critically with AI in Research
Thinking Critically with AI in Research
A practical guide to using AI responsibly in academic research, with principles for maintaining intellectual rigor, verifying evidence, and protecting research integrity.

Dr. Benita Olivier
Professor of Rehabilitation Oxford Brookes University

AI is now part of the research landscape. The challenge is not whether to use it, but how to use it well. Used thoughtfully, AI can support learning and sharpen thinking. Used uncritically, it can undermine the very skills researchers are trying to develop.
Most researchers reach a point where AI produces something genuinely impressive — a summary, a structure, a first draft — in the time it takes to make a cup of tea. The reaction is often mixed: relief, and then a flicker of unease. If it can do that, what is my role here?
This guide does not sidestep that question. It is built around it. Because how you use AI — and what you allow it to do on your behalf — will shape the kind of researcher you become.
Disclaimer
Read this guide in the context of your own institution, discipline, and programme.
There is a distinction worth noting as you read this guide: the difference between doing research and directing it.
Doing is the execution layer — the searching, drafting, summarising, and formatting. AI is already competent here, and increasingly so. Directing is something else: deciding which question deserves your attention, recognising which direction within a project is most promising, knowing when a result is genuinely good rather than merely plausible. Directing requires a deep familiarity with your field — a feel for the terrain that tells you where to go next.
This is important for both ethical and practical reasons. AI is most powerful in the hands of researchers who can direct it well — who know what to ask for, how to evaluate what comes back, and what it means for the work they are actually trying to do. That directing capacity does not come from having access to the tools. It comes from having developed the judgement to use them. And judgement develops through engagement with the substance of research — exactly the engagement that AI makes it easiest to skip.
Pause and reflect: what are you developing?
Think about the research tasks you have used AI for recently — or are tempted to. For each one, ask: is this a task where the doing itself builds something in me? Or is it genuinely routine?
The goal is not to avoid AI. It is to be honest about when the engagement matters — and when it does not. Some tasks are worth protecting. Others not.
Evidence-based practice depends on a hierarchy of evidence — systematic reviews and meta-analyses at the top, expert opinion and anecdote at the bottom. AI outputs occupy no recognised place in this hierarchy. They are not a source of evidence. They are a tool for processing, organising, and communicating evidence that must originate elsewhere.
In practice, this means:
Any claim that originates from or has been shaped by AI must be traced back to a verified primary source before it can be treated as a research finding.
AI-generated summaries of the literature reflect patterns in training data, not a systematic or methodologically sound synthesis. They cannot substitute for a proper literature review.
The distinction often comes down to whether you remain cognitively active — interpreting, questioning, and deciding. This applies across all phases of research, not just the writing-up stage: data collection, instrument design, qualitative analysis, and knowledge translation into practice are equally governed by these principles.
Appropriate use
Not appropriate
Reading a paper yourself, then using AI to challenge or refine your understanding
Asking AI to summarise a paper and relying on the summary without reading it
Using AI to create summaries during initial scoping — before deciding whether a paper is relevant
Citing a study based on the findings in the summary without reading the paper
Using AI to suggest structure when the argument is already yours
Asking AI to write or construct the argument on your behalf
Asking AI to improve the clarity of a sentence you have written
Accepting AI-generated text containing reasoning or claims you did not generate
Using AI to help draft interview questions, then reviewing them against your methodology
Using AI to generate themes from qualitative data without engaging directly with participants' words
Asking AI to explain a difficult concept, then testing your own understanding
Asking AI to appraise a paper's methodology on your behalf and accepting that appraisal unchecked
Using Consensus to discover relevant papers, then reading and critically appraising them
Treating Consensus results as a synthesis of evidence without reading the primary sources
A useful test: your name is on the work. The expectation is that the ideas, reasoning, and interpretations are your intellectual contribution. If AI is constructing the argument or speaking in your place, it is no longer supporting your work — it is replacing it.
Pause and reflect: are you still in the driver's seat?
After your next AI-assisted session, try this test: set the output aside and write — without referring to it — one paragraph explaining the argument, finding, or idea in your own words.
If you can do it fluently, you were cognitively present. If you struggle, that is useful information: not a reason for guilt, but a signal that the engagement needs to go deeper before you move on.
Systematic reviews are the gold standard of evidence synthesis in health research. They operate under strict methodological protocols — including PRISMA reporting standards, PROSPERO registration, structured database searching, and formal risk of bias assessment. AI is increasingly being used at various stages of this process, and the risks are specific and significant.
What AI cannot do in a systematic review
Replace structured database searching. Clinical databases — MEDLINE, CINAHL, Cochrane, EMBASE — remain the methodological standard for evidence retrieval. AI search tools do not currently meet the requirements for reproducibility, transparency, and comprehensiveness in systematic review methodology.
Screen abstracts or extract data without validation. If AI is used to support these tasks, the process must be validated against human screening, and the method must be explicitly reported.
Conduct risk of bias assessment. This requires methodological expertise, contextual judgement, and accountability that AI cannot provide.
Declaration in methods
Any AI involvement in a systematic review — including abstract screening, data extraction, or synthesis support — must be declared explicitly in the methods section, not simply noted in a general disclosure statement. Reviewers and readers need to be able to evaluate the methodological implications.
The methodological landscape is evolving, and a small but growing number of published reviews have incorporated AI with explicit validation and reporting. If you are considering AI involvement, seek current field-specific guidance — but the principle of transparency is non-negotiable.
The interpretive, reflexive engagement with participants' words and experiences is not a preliminary step in qualitative research — it is the research. Using AI to generate themes from interview data, summarise field notes, or suggest codes risks introducing researcher distance from the data and undermining the methodological rigour that gives qualitative work its validity and trustworthiness.
This does not mean AI has no role in qualitative research. It may legitimately support literature searching, help structure a methodology section you have already drafted, or improve the clarity of your writing. But the analytic process itself — the sustained, reflexive engagement with data — must remain yours.
Data protection in qualitative work
Uploading participant interview transcripts or field notes to a public AI system may breach your ethical approval, your institution's data protection policy, and GDPR — even when participants are not named. Contextual details in qualitative data can be identifying. When in doubt, do not upload.
1
Stay cognitively active
Question AI outputs, compare them to your own understanding, and refine them. Sit with uncertainty rather than using AI to generate premature answers — tolerance of ambiguity is a genuine research skill, and one that AI can quietly erode.
2
Verify everything — and know where to look
AI can fabricate citations, misattribute ideas, and inaccurately summarise research — generating authoritative-sounding text with no basis in reality.
What a hallucinated citation looks like
AI might generate: 'Smith, J. (2019). Methodological pluralism in qualitative health research. Journal of Nursing Studies, 45(3), 112–128.' — complete with volume and page numbers. That paper may not exist. Always locate sources before citing them.
3
Use AI after thinking, not before
Form your own view and attempt the task before involving AI. This preserves the learning value and ensures that what AI supports is genuinely yours.
4
Protect your voice
If a revised passage no longer sounds like you — or contains reasoning you did not generate — revise it until it does.
5
Know your tools
There is an important difference between retrieval-augmented tools and generative AI. Consensus searches indexed academic databases and returns real, citable papers — making it significantly more reliable for source discovery than a general-purpose language model. But retrieving real papers is not the same as synthesising evidence. Consensus can show you what exists; it cannot tell you whether those studies are methodologically sound, whether their findings are applicable to your context, or what they collectively mean. That judgement is yours.
General-purpose large language models (such as ChatGPT or Claude) generate text based on their training data and can fabricate sources with confidence. They are useful for drafting, explanation, and exploration — not for evidence retrieval. Using them to identify or summarise research without verification is methodologically indefensible.
Use the right tool for the right purpose. And with every tool, read the original.
6
Keep a reflective log
Answer these questions in your reflective log: what did I use AI for, what did I accept or modify, what was my reasoning?. This is good reflective practice, and it provides a defensible record should your methods ever be questioned.
7
Check guidance on grant writing and ethics applications
AI use in grant applications and research ethics submissions is an area where institutional policies are often unclear or still developing. Check with your institution before using AI to draft these documents, as some funders and ethics committees have specific requirements or restrictions.
Do not input unpublished data, participant information, interview transcripts, or any confidential material into public AI systems. Doing so may breach your ethical approval, your institution's data protection policy, and GDPR.
Disclose AI use as required — but note that disclosure does not make a use acceptable. Policies vary across institutions, programmes, and journals, and are still evolving. Disciplinary norms also vary: what is standard in computational research may be inappropriate in qualitative health research. Seek field-specific guidance alongside this framework.
Journal and publisher policies on AI-assisted writing differ, including on which tools must be declared. Check these before submission.
Before inputting anything that touches participant data into any AI tool, ask three questions:
1
Does my institution have an approved agreement with this provider?
2
Does my ethics approval permit this use?
3
Could any of this data identify a participant, even indirectly?
When using AI, or including AI-supported work in your research:
#
When using AI — ask yourself
Response
1
Am I using AI to support my thinking, or to replace it?
Yes / No / Unsure
2
Do I understand the output — could I explain it confidently in my own words?
Yes / No / Unsure
3
Have I verified all claims and references against a reliable, verified source?
Yes / No / Unsure
4
Does this reflect my own voice and reasoning?
Yes / No / Unsure
5
Is this aligned with my institution's, programme's, or journal's current guidance?
Yes / No / Unsure
6
Have I checked this against my ethical approval where data is involved?
Yes / No / Unsure
7
If this involves evidence synthesis, have I used appropriate clinical databases?
Yes / No / Unsure
8
Have I declared AI use in my methods where required — not just in a general statement?
Yes / No / Unsure
9
Could I defend this work as genuinely my own intellectual and professional contribution?
Yes / No / Unsure
If any answer is 'no' or uncertain, pause and re-engage with the task.
Pause and reflect: the researcher you are becoming
Looking at your research over the past few months — not the outputs, but the process — where do you feel your judgement and expertise are genuinely developing? Where do you feel it is being quietly outsourced?
Researchers who stay intellectually in charge of AI are not those who use it least — they are those who remain curious about what they are becoming in the process.
The researchers who benefit most from AI are not those who use it most — but those who use it most wisely, keeping their own intellectual development and professional accountability at the centre.
You are the researcher. AI is the tool. Keep it that way.

Become a Consensus MCP expert.
For courses and more information how to use the MCP, check out our guide below.

