- What Scientists and Engineers are Saying about Criterion 2
- Questions Surrounding Criterion 2
- Advice on Addressing Criterion 2
- Workshops on Criterion 2
In 1997, the National Science Foundation (NSF) updated their merit review criteria, which are used to judge research proposals for the allocation of funds. Since the adoption of the new criteria, although Criterion 1 (“Intellectual Merit”) has enjoyed general understanding, acceptance, and employment by the scientific community, reaction to Criterion 2 (“Broader Impacts”) has been mixed. Initially, many proposers and reviewers simply ignored the second criterion altogether or assigned it little weight in their proposals or reviews: some suggested it was unclear, irrelevant, or even in conflict with Criterion 1.
Reaction from Congress was swift – in 1998 and again in 1999, Congress directed NSF to engage the National Academy of Public Administration (NAPA) to examine and report on the “effectiveness” of the new criteria. Based in part on the findings and recommendations of the NAPA Report*, published in 2001, NSF has consistently moved to increase proposer and reviewer attention to the second criterion.** There remains considerable controversy about the larger meaning of this criterion and how it actually affects funding decisions. Somewhat more certain, however, is that researchers who can give an account of the larger societal benefits of their work stand a better chance of receiving funds than those unable or unwilling to do so. The two criteria with explanations are listed below:
What is the intellectual merit of the proposed activity?
How important is the proposed activity to advancing knowledge and understanding within its own field or across different fields? How well qualified is the proposer (individual or team) to conduct the project? (If appropriate, the reviewer will comment on the quality of prior work.) To what extent does the proposed activity suggest and explore creative and original concepts? How well conceived and organized is the proposed activity? Is there sufficient access to resources?
What are the broader impacts of the proposed activity?
How well does the activity advance discovery and understanding while promoting teaching, training, and learning? How well does the proposed activity broaden the participation of underrepresented groups (e.g., gender, ethnicity, disability, geographic, etc.)? To what extent will it enhance the infrastructure for research and education, such as facilities, instrumentation, networks, and partnerships? Will the results be disseminated broadly to enhance scientific and technological understanding? What may be the benefits of the proposed activity to society?
Click here to view the 2001 NAPA Report (pdf).
Click here to view NSF’s “ Important Notice: Implementation of new Grant Proposal Guide Requirements Related to the Broader Impacts Criterion”
Click here to view NSF’s suggestions for “Representative Activities” for satisfying Criterion 2.
Technology in Society 27 (2005) 437–451
Assessing the science–society relation: The case of the US National Science Foundation’s second merit review criterion
by J. Britt Holbrook
Science, Vol 276, Issue 5309, 26-0 , 4 April 1997
Merit Review: NSF Adopts New Guidelines
by Jeffrey Mervis
Science, Vol 291, Issue 5513, 2533-2535 , 30 March 2001
NSF Scores Low on Using Own Criteria
by Jeffrey Mervis