It would be difficult for research to advance without reviewers during the peer review process. At Open Research Europe, our peer review process is formal, invited, and open – so reports are published alongside the article, with reviewer names and affiliations. But what makes a good reviewer? In this blog post, we explore the role of a reviewer and what it takes to guide research articles through the peer review process, with first-hand experience from Dr. Johann-Mattis List, an Open Research Europe author and reviewer from the Max Planck Institute for the Science of Human History.
What is peer review?
Peer review is the process where two or more experts in the field (i.e reviewers) assess a researcher’s manuscript and “validate” the findings. Reviewers ensure that the research confirms and/or expands on current knowledge in the field and bases the research conclusions on effective and suitable investigations. This process can involve several phases of feedback between the author(s) and reviewers.
What does a reviewer do?
When a reviewer accepts a peer review assignment, they are not only assessing novel research but also agreeing to provide expert constructive feedback on the manuscript. This includes specific comments on methodology and interpretation of the findings. The review process can also highlight important literature that the authors can incorporate into their findings. Once the reviewers’ assessment is complete, they share their opinions with the authors in an open report.
What is peer review on Open Research Europe?
Peer review on Open Research Europe is fully transparent, meaning that the reviewer reports, their names, and their affiliations are available for all to see. A reviewer’s report will have its own digital object identifier (DOI), making it sharable and citable as soon as it is live. When producing the report, reviewers are required to answer a series of questions about the article, and the questions focus on the subject area it falls within. These questions are specific to the article type submitted by the authors. For example, when reviewing a Research Article, the questions focus on ensuring the work is original and the conclusions are drawn from the results. Meanwhile, a Method Article requires a stronger rationale for developing a new method specifically (device, process, tool, and so on).
Where appropriate, reviewers can co-review an article with a colleague, recognizing their contributions. Co-reviewing is particularly beneficial for introducing peer review to early career researchers and allowing them to gain experience alongside colleagues who are well-accustomed to peer review. It can also help divide the workload of reviewing a manuscript between two reviewers.
What makes a good reviewer?
Authors understand that their manuscript may not be perfect before it goes into the peer review process and often expect feedback from reviewers to improve their findings. Whether there are major or minor concerns, a reviewer’s comments should always be constructive to advance the Research Article. If the reviewer’s comments do not improve the article or the feedback shifts the focus away from the research question, the author(s) can challenge the reviewer’s commentary.
A good reviewer should always aim to be constructive and prompt with their feedback, yet patient when awaiting responses from the author(s), especially if further experimentation is suggested. Furthermore, having a good understanding and keeping up-to-date with current research is imperative to ensuring a good reviewer can be consistent with their involvement in the peer review process.
Peer reviewing: a case study
This peer review report from Dr. Annette Horstmann on the article looking at improving the touchscreen-based food approach-avoidance task by Hannah van Alebeek et al. is an example of good peer review practices as it provides constructive feedback and reasonable improvements. Overall, the reviewer’s comments improve the article, highlighting the main benefit of the peer review process.
Reviewer tips and tricks
We spoke with Dr. Johann-Mattis List from the Max Planck Institute about his experience as an Open Research Europe author and reviewer.
‘Far too many studies are not replicable in my field of research, and it makes me particularly sad if I see that there are still many papers published without the data and code upon which they were based’.
Dr. List confirms that good reviewers should provide authors with an opportunity to improve their work and believes reviewer reports should:
- Enhance research
- Strengthen arguments
- Clarify research perception
As a reviewer, Dr. List aims to:
- Ensure scholars share their data and code
- Confirm the study is replicable
- Check if researchers follow data FAIRness
Open Data and the FAIR data principles
Open Research Europe endorses Open Data as a framework to promote the broadest reuse of research data, following the relevant Horizon Europe policy. More information can be found regarding our data guidelines.
Understanding TACT
At Open Research Europe, we encourage and support our reviewers to follow TACT throughout the peer review process:
. Timely – assess the manuscript and deliver the report in a respectable timeframe.
. Analytical – examine the manuscript in detail, specifically the methods and results.
. Constructive – present feedback as an opportunity to build rather than destroy.
. Thorough – cover all aspects of the manuscript that the authors submitted.
Resources for reviewers
For training on becoming a good reviewer, our partners Eurodoc and the Global Young Academy have led a variety of educational webinars, such as Eurodoc’s webinar on ‘Open Peer Review’ and The Global Young Academy’s workshop on ‘The Future of Peer Review’.
Clarivate cover all aspects of peer review with a focus on specific subject areas as part of their online training – whilst the Open Science Framework details ‘Good Practice in Peer-review‘.
For general information, we suggest reading the Scholarly Kitchen blog on ‘How to be a good peer-reviewer‘.
At Open Research Europe, we will continue to facilitate open and good peer review while crediting reviewers for their work and supporting the development of their networks.
