Writing for the Peer Reviewer: Anticipating and Addressing Methodological Criticisms

The peer reviewer is, in most cases, a senior scholar in the same field who has agreed to evaluate the manuscript without compensation, under a deadline, alongside her own research and teaching commitments. She is also a particular kind of reader. By the nature of her training and her role, she is looking specifically for the places where the argument is vulnerable: where the methodology is insufficient for the claims being made, where the literature engagement is selective in ways that distort the picture, and where the findings are interpreted beyond what the evidence supports.

That calls for preparation. An author who submits a manuscript to a peer-reviewed journal without having first subjected it to the kind of critical reading the reviewer will apply is submitting before the work is done. The revision process that follows a first-round review, often months after submission and often requiring substantial changes, is a less efficient path to publication than internal revision that addresses the most predictable criticisms before the manuscript leaves the author's hands.

The methodological criticism checklist. The criticisms that appear most frequently in peer review are predictable enough to constitute a checklist. Sample size adequacy: whether the sample is large enough, or sufficiently diverse, to support the claims being made. Instrumentation validity, meaning whether the measures used actually measure the constructs the study claims to be studying. Analytical rigor. Are the statistical or interpretive methods applied appropriate to the data and the research questions? Alternative explanations — whether the author has considered and addressed the most plausible alternative interpretations of the findings. And generalizability, which asks whether the claims extend beyond the specific sample, setting, and time frame of the study in ways the design actually supports.

Addressing criticisms in the manuscript. The most effective response to a predictable criticism is to address it in the manuscript before it is raised, through honest, specific acknowledgment of the study's limitations in the appropriate section, with a clear account of what those limitations imply for the interpretation of the findings. A reviewer who raises a limitation the author has already addressed, and addressed honestly, has no basis for a rejection on those grounds. Silence cuts the other way. A limitation the author has ignored or minimized gives the reviewer a strong basis for requesting major revisions.

The literature gap. Peer reviewers in specialized fields are aware of the major contributions to the literature, and they notice when a manuscript ignores work that is directly relevant to its argument. The most common literature-related criticism is not that the author has cited too few sources but that the author has not engaged with a specific body of work: a competing theoretical framework, a methodologically similar study with different findings, a recent contribution that changes the conversation. The manuscript that survives peer review most cleanly belongs to the author who has read widely and engaged honestly with the literature that complicates the argument as well as the literature that supports it.

The resubmission as argument. When peer review does require revision and resubmission, the response letter is itself a scholarly document. It should address every reviewer comment specifically and directly. It should identify every change made in response and where in the manuscript the change appears, and it should make a clear argument for any instance where the author has chosen not to make a requested change, and why.

Vague, defensive, or incomplete response letters waste the opportunity. They also signal to the editor that the revision may not have been taken seriously.