CPR Briefing Highlights NIH Peer Review Process
The COSSA-led Coalition to Promote Research (CPR) recently organized its second congressional briefing of 2015 (see Update, March 24, 2015) designed to provide an overview of the National Institutes of Health’s (NIH) peer review process for congressional staff.
The briefing, NIH Priority Setting: How Peer Review Assists the NIH in Selecting the Best Science, highlighted the process used by the NIH’s Center for Scientific Review (CSR) with the help of scientific experts from around the country. Briefing speakers included CSR director Richard Nakamura and Danielle Li of Harvard University. Felice Levine, executive director of the American Educational Research Association (AERA), served as the moderator.
Nakamura: The Sputnik Moment for U.S. Biomedical Research and NIH’s Formula for Success
Nakamura plays a key role in leading NIH’s effort to ensure that the billions of dollars in grants awarded to more than 2,500 institutions across the country support research with the most promise for advancing science and health. He explained how experts from academia and industry review grant applications submitted to NIH from scientists.
He posited that the U.S. has reached its “Sputnik moment,” noting that China has doubled its funding for medical research every five years for the last 30 years, putting it on track to surpass U.S. investment in the next five years. He highlighted areas where NIH peer-reviewed research has paid off in economic benefits as well as scientific and health advances, including:
- NIH extramural funding generated $57.8 billion in economic output nationwide in 2012 (the latest available data).
- Research-related gains in life expectancy from 1970 to 2000 have an estimated economic value of $95 trillion, or $3.2 trillion per year.
- 1.35 million deaths are prevented each year due to NIH research advances in treating or presenting cardiovascular disease, stroke, cancer and diabetes.
Nakamura also explained why the NIH peer review process has been successful, notably its transparency, the focus on funding ideas or people as opposed to institutions, ideas generated by independent researchers, the use of scientists external to NIH as the primary judges of scientific merit, and the high value placed on fairness. He added that “NIH peer review might not be perfect, but it remains a proven way to help the U.S. make the public investments it needs to empower researchers across the country to advance U.S. science, health, and economic growth.” To maintain the quality of the peer review process, he explained that CSR uses the tools of science to evaluate its effectiveness and efficiency, modify review procedures to better ensure reproducibility of research results, and enlists the scientific community to help assess review groups and practices. Nakamura’s presentation is available here.
Li: How Effective is Peer Review?
Li, whose work focuses on research and science policy issues, research ethics, data access and sharing, the scientific and academic workforce, and higher education, discussed her recent research on the efficacy of NIH peer review and the subject of a recent paper in the journal Science. Li explained that “good peer review can be defined in many ways.” Her research examined how well review scores predict outcomes and the extent to which peer review tells us something about the quality of an application that could not be determined otherwise.
Li explained that her research, which examined more than 130,000 NIH grants awarded between 1980 and 2008, found proposals that earn good scores lead to research that is cited and published more and more likely to be published in high-impact journals. The findings held even when controlled for confounding factors, such as institutional affiliation, history of funding, experience, degree, age, or gender. Li’s full presentation is available here.
The briefing’s sponsors included:
Ad Hoc Group for Medical Research
American Association for Dental Research
American Educational Research Association
American Psychological Association
American Sociological Association
Association of American Medical Colleges
Association of Public and Land-grant Universities
Coalition for Life Sciences
Coalition for the Advancement of Health through Behavioral and Social Sciences Research
Consortium of Social Science Associations
Federation of Associations in Behavioral & Brain Sciences
HIV Medicine Association
JNCL-NCLIS
National Communication Association
Population Association of America/Association of Population Centers
Society for Research in Child Development