A PhD student finishes a manuscript after four months with a week to go before a special-issue deadline. Her supervisor is travelling. She asks a generative AI assistant to “review my manuscript” and gets a long paragraph of praise with a few vague style suggestions. She tries again: “Act as a sceptical reviewer for a methods journal. Give the five strongest reasons to reject this paper. No praise.” This time point two lands exactly where she was most worried: her introduction asks about “effectiveness,” but her design has no control group.
An AI assistant is not an expert reviewer: it does not reliably know what is new in your field, and it can be wrong about statistics. But it can be a demanding reader available at midnight, as long as you ask the right way and filter what comes back.
What AI test-reading does well, and badly
| Does fairly well | Does poorly or unreliably |
|---|---|
| Spotting mismatches between research question, methods and conclusions | Judging novelty against recent literature in your field |
| Comparing abstract with body: do numbers and conclusions match? | Verifying complex statistical analyses |
| Flagging inconsistent terminology | Knowing the norms of a particular journal or small community |
| Finding missing limitations or unexplained method choices | Judging whether the contribution suits a given journal tier |
| Asking the questions an outsider would ask | Suggesting references (they may be invented) |
So use AI to check internal consistency and clarity, and leave scientific quality and novelty to people in your field.
Before you paste: confidentiality and co-authors
- If you have co-authors, ask them first; the manuscript is not yours alone.
- Prefer institution-licensed tools or those that commit not to train on your data.
- Strip out raw data containing personal information; text and summary tables are enough.
- Never do this with a manuscript someone sent you to review.
Do not ask it to “give feedback”
AI assistants tend to please: open questions return compliments and minor wording tweaks. Assign a role and a specific task, forbid praise, and cap the number of points so it has to choose what matters most.
Five reviewer roles and prompts
1. The methods reviewer
“Act as a reviewer specialising in research design. Read the research question, methods and conclusions. List any conclusions this design does not support. Maximum five points, ranked by severity.”
2. The editor with three minutes
“You are an editor reading the title, abstract and last paragraph of the introduction to decide whether to send this out for review. Give any reasons you would desk-reject it, and identify which abstract sentence leaves the contribution unclear.”
3. The outsider
“You are a researcher from a neighbouring field. Mark every term, abbreviation or assumption in the introduction you do not understand.”
4. The consistency checker
“Compare numbers, sample sizes and variable names across the abstract, results, tables and discussion. List every mismatch with its location.”
5. The toughest reviewer
“Give the three strongest objections to the main claim. For each, say whether the manuscript already answers it and where.”
A 45-minute review session
- Minutes 0–10: role 2 on the title, abstract and final paragraph of the introduction. If the “editor” cannot restate your contribution in one sentence, fix the abstract before anything else.
- Minutes 10–25: roles 1 and 5 on the whole paper, each in a separate session so the feedback does not blur together.
- Minutes 25–35: role 4, then open your tables and check every reported mismatch yourself.
- Minutes 35–45: copy all comments into one table, sort them into the four groups below and note what to do.
Triage the feedback before you edit
Not every comment deserves action. Put each point into one of four groups:
- Right and important: fix it now (the abstract says n = 312, Table 1 says n = 298).
- Partly right: the comment points at a genuine ambiguity even if its reasoning is off. Clarify the wording.
- Wrong for lack of field knowledge: for example, demanding a large sample for a qualitative study. Ignore it, but ask whether a human reviewer might misread it the same way.
- Unverifiable: for example, “this method is outdated.” Ask someone in your field or check the literature; do not edit on that basis alone.
Do not let the “reviewer” rewrite your paper
Once the AI identifies a problem, it is tempting to ask it to “just fix it.” For numerical inconsistencies, fixing them yourself is quickest anyway. For arguments, rewrite in your own words: machine-written sentences tend to be smooth but blur exactly the nuance a real reviewer will probe. And having AI rewrite substantive passages may require disclosure under the journal’s policy.
You still need people
An AI read-through should be the first pass, not the last. An efficient order: reread your draft after a day’s break, run the five roles above, revise, then send it to a colleague or supervisor. They receive a version already cleaned of consistency errors and can spend their time on what only an expert can say: is this paper worth reading?
Next step: run role 4, the consistency checker, on the manuscript you are about to submit. It is the lowest-risk check and often catches exactly the errors that cost an editor’s goodwill on page one.
Câu hỏi thường gặp
Should I ask AI to review my manuscript before submitting?
It can be a useful first pass for consistency and clarity, provided co-authors agree and the tool protects confidentiality. Scientific quality and novelty still need expert human judgement.
Why does AI mostly praise my manuscript?
AI assistants tend to please when asked open questions. Give them a specific reviewer role, forbid praise and cap the number of points to force prioritisation.
Is AI feedback on a paper reliable?
Partly. It is good at catching numerical mismatches, inconsistent terms and conclusions that outrun the design, but it can be wrong about novelty, field norms and complex statistics. Triage each comment before acting.
Can I use AI to help review a manuscript I received as a peer reviewer?
No. A manuscript sent to you for review is the authors’ confidential work, unless the journal explicitly permits AI use.
Do I need to disclose it if AI rewrites parts of my paper?
It depends on the journal, but many require disclosure when AI writes or rewrites content. Rewriting arguments in your own words is safer and keeps the nuance.
