Reading and critiquing a research paper
The capstones are where you stop reading and start doing. The first skill is the one every researcher uses daily: reading a paper critically. Not hunting for flaws, but separating what was claimed from what was shown, checking that the controls support the conclusion, and noticing where the abstract or press release reaches further than the data. This chapter gives you a referee’s rubric and walks it through two real papers from this book.
On the plate: two papers annotated with the rubric.
The rubric
For any paper, write down: (1) the claim in one sentence, in the authors’ strongest form; (2) the evidence offered for each part; (3) the controls: what alternative explanations were excluded, and how; (4) the statistics: sample sizes, error bars, whether the analysis was fixed in advance; (5) the baseline: what the result is compared against, and whether it is the strongest available; (6) the overreach: where the title, abstract or discussion claims more than the results support; and (7) the next experiment that would settle what remains open. A referee report is these seven items, written politely and specifically. Established as method.
Two worked examples
Choose a paper: , then step through the rubric items with 1. The plate shows this book’s annotation for each item. Both are strong papers; both are careful about their own limits; and in both, popular coverage claimed more than the papers did. The annotations reflect Chapters 169 and 204. Debated for the judgements, which are this book’s.
Your turn
Pick a paper from this book’s sources that you have not seen annotated, for example the 2007 FMO coherence paper (Chapter 167), the 2018 xenon isotope study (Chapter 175) or the 2025 KRAS paper (Chapter 200). Write a one-page referee report following the rubric, with at least one specific, constructive request. Then find the published follow-ups and check how your report compares with what the field concluded. That comparison is the most useful calibration you can get. Established as method.
What you now have
A seven-item referee rubric, worked annotations of two real papers, and an exercise that calibrates your judgement against how the field actually responded.
· · ·
Three questions to carry with you.
And one to work out. A paper compares two groups of 40 with standard deviation 0.25 each. What is the standard error of the difference in means, ?
Sources: J. Xu et al., Nature 594, 535 (2021). J. Robledo-Moreno et al., Sci. Adv. 11, eadu9991 (2025). Annotations are this book’s judgement.