Peer Review Practice & Technology
When a federal, state, or private research funding organization disseminates scientific information or awards grants, it is critical that the proposed information be scientifically feasible and have verifiable technical merit. ORAU manages scientific peer reviews that provide critical, independent assessment of research proposals and program performance to help sponsors determine the quality of research proposals and accuracy of scientific information.
The peer review process involves selecting experts in the same field as the proposals to evaluate, comment, and score proposals based on their knowledge of the subject and understanding of the impact each proposal would make if funded. Focusing on continuing process improvement, research collaborations exist to evaluate the entire peer review process—from planning and reviewer identification to evaluating proposals and implementing improvements. Gaining a better understanding of what motivates reviewer participation can bring insight and greater credibility to peer reviews and program evaluations.
Research focus areas
- Literature review of the peer review process to assess norms for rates of invites/participation, honoraria, etc., to identify gaps and/or problems in the field.
- Create reviewer typologies of the characteristics and motivations of different types of reviewers in order to aid reviewer identification and recruitment for different types of projects.
- Evaluate the process of conducting a review from the perspective of the reviewer.
- What are the steps?
- How long does each step take?
- What are the barriers to participation?
- What slows things down?
- What does or doesn’t add value to the process?
- Assess participation in ORAU reviews to identify incentives/barriers to participation, characteristics of repeat reviewers, patterns in reviewer characteristics across different review types and domains, the value of participation to the reviewer, etc.
- Research the effects of “repeat reviewers” on scores.
