In this article a pharmacist with experience in research and publishing discusses what peer review is, providing reasons to become a peer reviewer and advice for performing peer reviews.
Authored by: Timothy P. Gauthier, Pharm.D., BCPS-AQ ID
[Last updated 24 May 2018]
What is peer review and how does it work?
Peer review refers to evaluation of scholarly work by individuals qualified to verify the validity of the work. In doing so, peer reviewers assist in ensuring the credibility and accuracy of work accepted for publication. Published content that is peer reviewed is generally considered of higher quality, since it is held to a higher standard as compared to non-peer reviewed work. Not all published content in medical journals is peer reviewed. Which sections of a journal’s publications are peer reviewed are typically identified within a journal’s website.
When a manuscript is submitted to a journal it is typically received by one of the editors, who screens it for relevance and quality. Sometimes it is also presented to a committee of editors for consideration. If the submission is positively received, an editor will alert the authors the submission has been accepted for review and send emails to potential peer reviewers requesting for their assistance with the task. The peer reviewers who accept the request to review the manuscript (usually at least 3 are needed) assess the submission and provide their opinion to the journal, including a recommendation on whether to accept or reject it. If the submission requires revisions, the peer reviewers may be asked to perform a second or third review after edits are made. A good visual of a typical peer review process is provided here. The peer review process can take many months and can vary greatly from journal to journal.
Why should I consider becoming a peer reviewer?
There are many rewards that come with being a peer reviewer.
First is the opportunity to serve your profession and the medical community at large. Your peer reviews are needed to allow for advancements in science and it can be personally rewarding to contribute to enhancing the quality of publications.
Second, participating in peer review provides insights on the publication process, which can greatly enhance your ability to compose work acceptable for publication. Especially for aspiring authors, the lessons gained from refereeing journal articles can prove critical. Identifying mistakes made by others is a great way to learn to avoid them yourself. Being a peer reviewer also helps to learn about the review process in general and manuscript formatting requirements.
Third, being a peer reviewer can enhance your critical eye. As a peer reviewer you are tasked with assessing a manuscript from a to z. As you review published literature in general you can utilize the skills obtained from being a peer reviewer to enable a greater understanding of new literature and how it should impact your practice.
Fourth, participating in peer review can be exciting and create amazing opportunities. In some instances you may be offered the chance to peer review an article that represents leading medical research in your area of interest. If the work is of sufficient quality and is acceptable for publication, this is akin to seeing a good movie before it is released in theaters. In addition during the review process you may be invited to write an editorial commentary regarding the article you reviewed. I once reviewed an article about colistin +/- rifampicin for XDR Acinetobacter for my favorite journal (Clinical Infectious Diseases) and subsequently was able to publish an editorial commentary to accompany it. I consider it one of the top professional achievements of my career to date. It never would have happened if I had not signed up to be a peer reviewer.
How can I become a peer reviewer for a medical or pharmacy journal?
In short, becoming a peer reviewer is simple and in many cases all you need to do is go to a journal’s website and sign up. Some journals may have some basic qualifying criteria, but that really depends on the journal. Prior to signing up as a peer reviewer, consider a few things…
First, contemplate your qualifications and experience. Journals will ask you to identify key words for topics you would like to be a reviewer for. This can be a drug, practice setting, or other topic. For example, I usually select the “colistin” and “antimicrobial stewardship”, since those are topics I follow in the literature.
Second, identify which journal(s) you want to be a reviewer for. A good way to find the journals that may be a good fit for you is to put your key words into pubmed and see what journals are publishing articles on the topic(s). Once you have found a few journals of interest, you can differentiate them by reviewing information provided on their websites.
Impact factors are another way to differentiate journals, but this is a flawed method of differentiating journal value and should be used with caution. Essentially the more citations a journal’s publications have, the higher the impact factor the journal has. This is meant to identify journal relevance. Journals with very high impact factors (e.g., New England Journal of Medicine) may be poor starting points for new reviewers.
What are some things to know about being a peer reviewer?
Becoming a good peer review takes a ton of practice. Picking up helpful tidbits on how to be more effective in peer review is something to keep an eye out for. Here are some tidbits I have picked up along the way and have found helpful…
1. Only accept reviews when you can provide an unbiased response
Be aware of your actual or potential conflicts of interest. If you have a close peer who is one of the main authors on a submission you are asked to review, it may be a bad fit. In addition, if the article is investigating a product for which you have a financial relationship, it is probably not appropriate for you to review it. Keep ethics in mind as you participate in peer review.
2. Review the journal’s criteria for reviewers upon accepting the review
Most journals will have their reviewers log into their manuscript management system as a reviewer, at which point the submitted manuscript can be downloaded and the reviewer can submit the results of their review. Many journals have forms that are electronic for the reviewer to fill out. It is helpful to know what questions will need to be answered prior to setting out to complete the review.
Note that it is important to comply with journal confidentiality requirements, but including a student or resident in your review process can be reasonable and a good way to help trainees learn about the strengths / weaknesses of the peer review process.
3. Start by reading the article straight through from start to finish
Try not to do the review the same day of accepting the task. One of my mentors advised to read the complete manuscript once within a day or two of accepting the review. Come back to it a few days later after you have had time to ponder the core of the study question. Having thought about the topic and how you will respond can make it easier for composing commentary to provide back to the editor and authors.
While it is good practice to take the time necessary to evaluate the work, also do your best to respond with your review in a timely manner. Typically within 10 days is a good goal.
4. Prepare your review using a structured process
One way to approach this is to have a brief introduction, followed by a “major comments” section, then a “minor comments” section.
In the introduction, thank the journal/authors for the opportunity to review the work and briefly note the purpose of it. This can be a good place to briefly identify how important you feel the work is and your general opinion of it. It is also appropriate to acknowledge the efforts of the authors, who likely put considerable time and effort towards composing the submission.
In the “major comments” section have bullets where you comment on things that must be addressed for the article to be published. This is a place to identify fatal flaws in the study design or issues you feel need considerable attention. Your decision to offer a recommendation of accept or reject should be primarily based on the content in this section.
In the “minor comments” section have bullets where you comment on things that are notable, but that do not weigh as heavily on your final recommendation for the work. Here you can identify the paper requires copy editing (no need to nit-pick spelling errors), note areas where you feel more or less commentary is needed, and provide other considerations to the authors. It is acceptable to give kudos to the authors within the review, if you feel it is warranted. It is also acceptable to acknowledge your own limitations and if you feel the data requires evaluation by a statistician, say so.
As you complete your review, try to be as concise as possible. This makes it easier for the authors and editors to interpret.
5. Be honest, but do not be a “reviewer #2”
People in academia like to talk about “reviewer #2” as a fictional character that has some wacky, inappropriate, or overly demanding request for the authors. Try not to be that person.
While it is important to be honest in a review, there is no need to be rude or condescending or otherwise obnoxious. If you have considerable concerns you typically have the option to provide some comments directly to the editor and that may be a better way to address anything you find particularly bothersome.
6. Consider how readers will interpret the work
Many times reviewers can point out opportunities to enhance the clarity of how data is presented. Try to identify whether the paper follows a logical pattern and provide guidance that may improve the reader experience. If there is too much detail in one section, then not enough in another, it is appropriate to identify this during peer review. Also, if there are too many figures and tables, perhaps some can be combined or incorporated into the text. Data in figures and tables should not be duplicated in the text.
7. If the conclusions are not supported by the data, you must not accept
At the end of the day the conclusions the authors make need to be justified by sound methodologies that produce reliable data. If this is not the case then the work should not be accepted.
If the case is that the methods and data are sound, but the conclusions are over-stated, then reviewers must identify this and can help the authors and editor to develop a more appropriate conclusion.
8. Be mindful of accuracy with figures and tables
Many manuscript formats demand tables and figures be placed at the very end of the manuscript. By the time the reviewer reaches this section fatigue may have set in and this can lead to things being overlooked. To manage this, try to review tables and figures at the time they are acknowledged in the text. A study by Margalida et al. investigating key errors missed by the peer review process found that errors were frequently associated with figures, so beware!
Closing comments
The peer review process is an important component of what makes advancements in science possible. Hopefully the information provided here has been helpful in identifying this and answering basic questions about being a peer reviewer.
Recommended Readings & Resources
- The BMJ. Reviewer training materials.
- Wiley. Step by step guide to reviewing a manuscript.
- Margalida A, Colomer MA. Improving the peer-review process and editorial quality: key errors escaping the review and editorial process in top scientific journals. PeerJ. 2016; 4: e1670.
- Zellmer WA. What editors expect of reviewers. American Journal of Hospital Pharmacy. 1977; 34: 819.
- DiDomineco RJ, et al. Improving peer review: what reviewers can do. American Journal of Health-System Pharmacy. 2017; 74: e521-5.
- Groves T. Quality and value: how can we get the best out of peer review? Nature. 2006.
- Dowdall M. How to be an effective peer reviewer. The Pharmaceutical Journal. 2015.
I would like to acknowledge infectious disease pharmacist Dr. Khalid Elijaaly (@Khalideljaaly) who tweeted about this topic, which subsequently inspired the development of this article.
RECOMMENDED TO YOU
