Thinking Critically with AI in Research - Consensus

Thinking Critically with AI in Research

Thinking Critically with AI in Research

A practical guide to using AI responsibly in academic research, with principles for maintaining intellectual rigor, verifying evidence, and protecting research integrity.

Dr. Benita Olivier

Professor of Rehabilitation Oxford Brookes University 

Thinking Critically with AI in Research

01

Why this matters

01

Why this matters

AI is now part of the research landscape. The challenge is not whether to use it, but how to use it well. Used thoughtfully, AI can support learning and sharpen thinking. Used uncritically, it can undermine the very skills researchers are trying to develop.

Most researchers reach a point where AI produces something genuinely impressive — a summary, a structure, a first draft — in the time it takes to make a cup of tea. The reaction is often mixed: relief, and then a flicker of unease. If it can do that, what is my role here?

This guide does not sidestep that question. It is built around it. Because how you use AI — and what you allow it to do on your behalf — will shape the kind of researcher you become.


Disclaimer

Read this guide in the context of your own institution, discipline, and programme.



02

The core principle: augmentation, not substitution

02

The core principle: augmentation, not substitution

There is a distinction worth noting as you read this guide: the difference between doing research and directing it.

Doing is the execution layer — the searching, drafting, summarising, and formatting. AI is already competent here, and increasingly so. Directing is something else: deciding which question deserves your attention, recognising which direction within a project is most promising, knowing when a result is genuinely good rather than merely plausible. Directing requires a deep familiarity with your field — a feel for the terrain that tells you where to go next.

This is important for both ethical and practical reasons. AI is most powerful in the hands of researchers who can direct it well — who know what to ask for, how to evaluate what comes back, and what it means for the work they are actually trying to do. That directing capacity does not come from having access to the tools. It comes from having developed the judgement to use them. And judgement develops through engagement with the substance of research — exactly the engagement that AI makes it easiest to skip.


Pause and reflect: what are you developing?

Think about the research tasks you have used AI for recently — or are tempted to. For each one, ask: is this a task where the doing itself builds something in me? Or is it genuinely routine?

The goal is not to avoid AI. It is to be honest about when the engagement matters — and when it does not. Some tasks are worth protecting. Others not.


AI should support your thinking, not replace it.

AI should support your thinking, not replace it.



03

AI outputs are not evidence

03

AI outputs are not evidence

Evidence-based practice depends on a hierarchy of evidence — systematic reviews and meta-analyses at the top, expert opinion and anecdote at the bottom. AI outputs occupy no recognised place in this hierarchy. They are not a source of evidence. They are a tool for processing, organising, and communicating evidence that must originate elsewhere.

In practice, this means:

Any claim that originates from or has been shaped by AI must be traced back to a verified primary source before it can be treated as a research finding.

AI-generated summaries of the literature reflect patterns in training data, not a systematic or methodologically sound synthesis. They cannot substitute for a proper literature review.



04

Appropriate and inappropriate use

04

Appropriate and inappropriate use

The distinction often comes down to whether you remain cognitively active — interpreting, questioning, and deciding. This applies across all phases of research, not just the writing-up stage: data collection, instrument design, qualitative analysis, and knowledge translation into practice are equally governed by these principles.


Appropriate use

Not appropriate

Reading a paper yourself, then using AI to challenge or refine your understanding

Asking AI to summarise a paper and relying on the summary without reading it

Using AI to create summaries during initial scoping — before deciding whether a paper is relevant

Citing a study based on the findings in the summary without reading the paper

Using AI to suggest structure when the argument is already yours

Asking AI to write or construct the argument on your behalf

Asking AI to improve the clarity of a sentence you have written

Accepting AI-generated text containing reasoning or claims you did not generate

Using AI to help draft interview questions, then reviewing them against your methodology

Using AI to generate themes from qualitative data without engaging directly with participants' words

Asking AI to explain a difficult concept, then testing your own understanding

Asking AI to appraise a paper's methodology on your behalf and accepting that appraisal unchecked

Using Consensus to discover relevant papers, then reading and critically appraising them

Treating Consensus results as a synthesis of evidence without reading the primary sources

A useful test: your name is on the work. The expectation is that the ideas, reasoning, and interpretations are your intellectual contribution. If AI is constructing the argument or speaking in your place, it is no longer supporting your work — it is replacing it.


Pause and reflect: are you still in the driver's seat?

After your next AI-assisted session, try this test: set the output aside and write — without referring to it — one paragraph explaining the argument, finding, or idea in your own words.

If you can do it fluently, you were cognitively present. If you struggle, that is useful information: not a reason for guilt, but a signal that the engagement needs to go deeper before you move on.



05

Systematic reviews and evidence synthesis

05

Systematic reviews and evidence synthesis

Systematic reviews are the gold standard of evidence synthesis in health research. They operate under strict methodological protocols — including PRISMA reporting standards, PROSPERO registration, structured database searching, and formal risk of bias assessment. AI is increasingly being used at various stages of this process, and the risks are specific and significant.

What AI cannot do in a systematic review

Replace structured database searching. Clinical databases — MEDLINE, CINAHL, Cochrane, EMBASE — remain the methodological standard for evidence retrieval. AI search tools do not currently meet the requirements for reproducibility, transparency, and comprehensiveness in systematic review methodology.

Screen abstracts or extract data without validation. If AI is used to support these tasks, the process must be validated against human screening, and the method must be explicitly reported.

Conduct risk of bias assessment. This requires methodological expertise, contextual judgement, and accountability that AI cannot provide.


Declaration in methods

Any AI involvement in a systematic review — including abstract screening, data extraction, or synthesis support — must be declared explicitly in the methods section, not simply noted in a general disclosure statement. Reviewers and readers need to be able to evaluate the methodological implications.

The methodological landscape is evolving, and a small but growing number of published reviews have incorporated AI with explicit validation and reporting. If you are considering AI involvement, seek current field-specific guidance — but the principle of transparency is non-negotiable.



06

Qualitative research: specific considerations

06

Qualitative research: specific considerations

The interpretive, reflexive engagement with participants' words and experiences is not a preliminary step in qualitative research — it is the research. Using AI to generate themes from interview data, summarise field notes, or suggest codes risks introducing researcher distance from the data and undermining the methodological rigour that gives qualitative work its validity and trustworthiness.

This does not mean AI has no role in qualitative research. It may legitimately support literature searching, help structure a methodology section you have already drafted, or improve the clarity of your writing. But the analytic process itself — the sustained, reflexive engagement with data — must remain yours.


Data protection in qualitative work

Uploading participant interview transcripts or field notes to a public AI system may breach your ethical approval, your institution's data protection policy, and GDPR — even when participants are not named. Contextual details in qualitative data can be identifying. When in doubt, do not upload.



07

Practical rules for responsible use

07

Practical rules for responsible use

1

Stay cognitively active

Question AI outputs, compare them to your own understanding, and refine them. Sit with uncertainty rather than using AI to generate premature answers — tolerance of ambiguity is a genuine research skill, and one that AI can quietly erode.

2

Verify everything — and know where to look

AI can fabricate citations, misattribute ideas, and inaccurately summarise research — generating authoritative-sounding text with no basis in reality.


What a hallucinated citation looks like

AI might generate: 'Smith, J. (2019). Methodological pluralism in qualitative health research. Journal of Nursing Studies, 45(3), 112–128.' — complete with volume and page numbers. That paper may not exist. Always locate sources before citing them. 


3

Use AI after thinking, not before

Form your own view and attempt the task before involving AI. This preserves the learning value and ensures that what AI supports is genuinely yours.

4

Protect your voice

If a revised passage no longer sounds like you — or contains reasoning you did not generate — revise it until it does.

5

Know your tools

There is an important difference between retrieval-augmented tools and generative AI. Consensus searches indexed academic databases and returns real, citable papers — making it significantly more reliable for source discovery than a general-purpose language model. But retrieving real papers is not the same as synthesising evidence. Consensus can show you what exists; it cannot tell you whether those studies are methodologically sound, whether their findings are applicable to your context, or what they collectively mean. That judgement is yours.

General-purpose large language models (such as ChatGPT or Claude) generate text based on their training data and can fabricate sources with confidence. They are useful for drafting, explanation, and exploration — not for evidence retrieval. Using them to identify or summarise research without verification is methodologically indefensible.

Use the right tool for the right purpose. And with every tool, read the original.

6

Keep a reflective log

Answer these questions in your reflective log: what did I use AI for, what did I accept or modify, what was my reasoning?. This is good reflective practice, and it provides a defensible record should your methods ever be questioned. 

7

Check guidance on grant writing and ethics applications

AI use in grant applications and research ethics submissions is an area where institutional policies are often unclear or still developing. Check with your institution before using AI to draft these documents, as some funders and ethics committees have specific requirements or restrictions.



08

Data, transparency, and institutional alignment

08

Data, transparency, and institutional alignment

Do not input unpublished data, participant information, interview transcripts, or any confidential material into public AI systems. Doing so may breach your ethical approval, your institution's data protection policy, and GDPR.

Disclose AI use as required — but note that disclosure does not make a use acceptable. Policies vary across institutions, programmes, and journals, and are still evolving. Disciplinary norms also vary: what is standard in computational research may be inappropriate in qualitative health research. Seek field-specific guidance alongside this framework.

Journal and publisher policies on AI-assisted writing differ, including on which tools must be declared. Check these before submission.

Before inputting anything that touches participant data into any AI tool, ask three questions:

1

Does my institution have an approved agreement with this provider?

2

Does my ethics approval permit this use?

3

Could any of this data identify a participant, even indirectly?



09

Decision-making checklist

09

Decision-making checklist

When using AI, or including AI-supported work in your research:


#

When using AI — ask yourself

Response

1

Am I using AI to support my thinking, or to replace it?

Yes / No / Unsure

2

Do I understand the output — could I explain it confidently in my own words?

Yes / No / Unsure

3

Have I verified all claims and references against a reliable, verified source?

Yes / No / Unsure

4

Does this reflect my own voice and reasoning?

Yes / No / Unsure

5

Is this aligned with my institution's, programme's, or journal's current guidance?

Yes / No / Unsure

6

Have I checked this against my ethical approval where data is involved?

Yes / No / Unsure

7

If this involves evidence synthesis, have I used appropriate clinical databases?

Yes / No / Unsure

8

Have I declared AI use in my methods where required — not just in a general statement?

Yes / No / Unsure

9

Could I defend this work as genuinely my own intellectual and professional contribution?

Yes / No / Unsure

If any answer is 'no' or uncertain, pause and re-engage with the task.


Pause and reflect: the researcher you are becoming

Looking at your research over the past few months — not the outputs, but the process — where do you feel your judgement and expertise are genuinely developing? Where do you feel it is being quietly outsourced?

Researchers who stay intellectually in charge of AI are not those who use it least — they are those who remain curious about what they are becoming in the process.



A final thought

A final thought

The researchers who benefit most from AI are not those who use it most — but those who use it most wisely, keeping their own intellectual development and professional accountability at the centre.

You are the researcher. AI is the tool. Keep it that way.


Become a Consensus MCP expert.

For courses and more information how to use the MCP, check out our guide below.

Example: