Insights into the AJNR Review Process

Manuscript peer review is a critical process to ensure that published manuscripts are scientifically and methodologically sound. While the general process is straightforward, the specifics can be complicated and, from the outside, very opaque. The type of peer review is not standardized across journals, and the number of reviewers and the impact of reviewers' recommendations on an editor's decision vary across the industry. This variability can be frustrating to both authors and reviewers, particularly when reviewers disagree on recommendations. At times it may leave authors wondering how a decision is rendered and how influential reviewers' opinions are on the final decision to accept or reject the manuscript.

Given this variability, we sought to assess manuscript reviews and outcomes for the AJNR. Manuscript data were acquired for both the past year and past 5 years. We tallied the total number of manuscripts submitted, the initial decision, and the number of revisions as well as outcomes of those revisions. The number of reviews and impact on accept/reject rates were assessed. Reviewer recommendation concordance and discordance were assessed relative to manuscript decision outcome.

During 5 years, 7328 manuscripts were submitted with initial manuscript decisions as follows: reject (74.8%), major revision (14.5%), minor revision (7%), and accept (3.6%), with similar rates during the past year. In addition to the rejection rate of 74.8% during the past 5 years, we found a 73.1% rejection rate during the past year, with nearly all (99%) manuscripts receiving initial decisions of minor or major revisions eventually being accepted. Previously, Rosenkrantz and Harisinghani¹ retrospectively reviewed 696 Original Research manuscripts submitted to the American Journal of Roentgenology in 2012 with the goal of providing authors with useful metrics from first submission to eventual publication. They found that >90% of articles needing minor revision and about 50% of those needing major revision were eventually accepted. The latter clearly differs from our findings here, where 99% of manuscripts needing major revision were accepted. It is important to note that the “Major Revision” decision is entitled “Provisional Acceptance Pending Major Revision” for AJNR, suggesting that reviewer recommendations and Senior Editor decisions with this designation imply acceptance at the outset, in contrast to other journals. Rosenkrantz and Harisinghani also found that the more revisions, the more likely that acceptance would be obtained. Nearly 100% of revisions were performed by authors when requested. The overall AJR acceptance rate was 25.3%, and the authors proposed reducing this rate to cut down the length of time from acceptance to publication. This acceptance rate is nearly identical to that found in our study for AJNR (ie, 25.2% for the past 5 years, 26.9% for the past year).

We devoted a large part of our analysis to the reviews and reviewers. First, we found that most manuscripts were reviewed either by 2 reviewers or the Editor-in-Chief or Senior Editor alone. This finding was expected as the Senior Editor is tasked with sorting through a portion of all submitted manuscripts and selecting only a subset to be sent to reviewers while rejecting the rest. The Editor-in-Chief will also accept some manuscripts without review, generally limited to Editorials, Brief Reports, or Perspectives rather than Original Research articles. We also found that the more reviews a manuscript underwent, the less likely that it would be accepted. This may be related to the complexity of the topic, a niche topic, or the greater likelihood of reviewer recommendation discordance influencing the Senior Editor. The trend during the past year to fewer 2-reviewer reviews and more Senior Editor-only and 3+-reviewer reviews likely relates to variability in types and subtypes of manuscripts submitted with slightly fewer Original Research articles when comparing the past year with the past 5 years.

We also looked at how reviewer recommendation discordance impacted the editor's decision for the manuscript. Approximately half of all manuscripts with ≥2 reviews demonstrated reviewer discordance and approximately half demonstrated concordance. As anticipated, discordance was higher among 3+ reviewers compared with 2 reviewers. Discordance led to about a 5% higher rate of rejection relative to all manuscripts during the 5-year period for both 2-reviewer and 3+-reviewer cases. For the past year, the rejection rate in 2-reviewer discordant cases has been nearly identical to that of all manuscripts, while it has been about 13% higher for 3+-reviewer discordant cases (ie, a final acceptance rate of only 13.6%). Additionally, when we broke down the data by manuscript type and subtype, Brief/Technical Reports had a much lower likelihood of acceptance for 3+-reviewer discordance (15.4%) versus 2 reviewers (35.7%), with a similar discrepancy seen for Review Articles (26.1% versus 38.8%). The lower acceptance for Brief/Technical Report discordant reviews likely relates to the relatively higher bar for acceptance overall for this the type of submission versus the primary focus of the AJNR on Original Research articles.

Finally, we found that if reviewers agreed on a recommendation, the likelihood of rejection was lower than if they disagreed on a recommendation, suggesting a certain degree of influence on the editor's final decision, especially with regard to rejection. The impact of the reviewer's recommendation on the editor's decision has been previously studied. Vintzileos et al² retrospectively assessed the influence of a reviewer's recommendation on the final editorial decision to accept or reject an obstetric manuscript for publication. Five reviewers' 635 reviews were analyzed, with the highest correlation found for a reject recommendation with a reject final decision (93%). Accept with minor revisions had a 67% acceptance rate, whereas accept with major revisions had a 40% acceptance rate. No variations among the reviewers (including the quality of the reviews) were found to influence the final decision—that is, it did not matter if the reviewer was more senior or had performed more reviews in the past. This is similar to our findings on reviewer experience.

Similar to Vintzileos et al², Kravitz et al³ explored the relationship between reviewers' recommendations and the final manuscript decision for the Journal of General Internal Medicine between 2004 and 2008. Assessing 2264 manuscripts and 5881 associated reviews, the authors found that reviewers' recommendations agreed or disagreed with one another just barely beyond chance. When all reviewers agreed on rejection (6.9% of the time), 88% of those manuscripts received a final decision of rejection. In contrast, when all reviewers argued against rejection (47.7% of the time), only 20% of those manuscripts received a final decision of rejection. The overall rejection rate was 48%, arguing that the reviewers' recommendations significantly influenced the editor's decision to accept or reject the manuscript. When any level of disagreement was present among reviewers' recommendations (45.4% of the time), the rejection rate was 70.6%. This is similar, though slightly less, to our findings on reviewer discordance and rejection rate.

In summary, AJNR acceptance/rejection rates have been stable during the past 5 years. Manuscripts with initial decisions of major revision are almost always eventually accepted. The more reviews, the less likely a manuscript will be accepted. Manuscripts with discordant reviewer recommendations had about a 5% higher rejection rate during the past 5 years and 13% higher during the past year if there were 3+ reviewers. Manuscripts with concordant reviewer recommendations were less likely to be rejected, suggesting that reviewers have an influence on the final decision. We hope that these observations help provide more transparency to prospective authors about the AJNR peer review process and provide valuable insight to the AJNR editors and reviewers.

• © 2021 by American Journal of Neuroradiology