High cost of bias: Diminishing marginal returns on NIH grant funding to institutions

Scientific output is not a linear function of amounts of federal grant support to individual investigators. As funding per investigator increases beyond a certain point, productivity decreases. This study reports that such diminishing marginal returns also apply for National Institutes of Health (NIH) research project grant funding to institutions. Analyses of data (2006–2015) for a representative cross-section of institutions, whose amounts of funding ranged from $3 million to $440 million per year, revealed robust inverse correlations between funding (per institution, per award, per investigator) and scientific output (publication productivity and citation impact productivity). Interestingly, prestigious institutions had on average 65% higher grant application success rates and 50% larger award sizes, whereas less-prestigious institutions produced 65% more publications and had a 35% higher citation impact per dollar of funding. These findings suggest that implicit biases and social prestige mechanisms (e.g., the Matthew effect) have a powerful impact on where NIH grant dollars go and the net return on taxpayers’ investments. They support evidence-based changes in funding policy geared towards a more equitable, more diverse and more productive distribution of federal support for scientific research. Success rate/productivity metrics developed for this study provide an impartial, empirically based mechanism to do so.


46
There is strength in diversity. Diversity in scientific research includes the perspectives and creative ideas that are harnessed, the model systems and experimental tools employed, the 4 of funding to each institution over the ten years was divided by the number of investigators who 140 received funding in one or more years to yield overall funding per investigator. The overall funding per investigator at each prestigious institution was higher than that per investigator at institution was larger than that for each less-prestigious institution, giving investigators at the 146 prestigious institutions, on average, 1.5-times more dollars per award each year ($466,000 vs 147 $310,000, p < 0.001) ( Figure 2D).

149
In summary, from 2006 to 2015, each of the prestigious institutions outperformed, by every 150 metric, each of the less-prestigious institutions in securing NIH research project grant funding.

152
The placement of institutions into prestigious and less-prestigious groups was part of the 153 experimental plan, which was laid out before any data were acquired, and the assignments 154 were based on published rankings (Bastedo & Bowman, 2010; US News & World Report, 155 2016). Nevertheless, these groupings could be considered arbitrary and might affect the 156 results, so the data (Supplemental Table S1) were also analyzed as continuous variables 5 dollars of funding, p = 0.003). Of course, it is possible that the scientific impact of publications 189 might differ between institutions.

191
To gain insight into this possibility, the relative citation ratio (RCR) (Hutchins et al., 2016) was 192 compiled for each grant-supported research article during the survey period. Citations to value, which is being used by the NIH to assess portfolio performance and to guide funding 195 decisions (e.g., Lauer, 2016aLauer, , 2016bLauer, , 2016dLauer, , 2017, is a time-normalized, field-normalized

206
In summary, from 2006 to 2015, the overall, funding-normalized productivity of the less-207 prestigious institutions was greater than (35% based on citation impact) or substantially greater 208 than (65% based on publication rate) that of the prestigious institutions. I conclude that the 209 scientific output-based of value of these institutions to the national research enterprise does not 210 justify the strong disparities in allocations of funding (significant differences in success rates, 211 funding rates, award sizes, and funding per investigator) between the prestigious and less-212 prestigious institutions.

214
It should be emphasized that the differences in productivity do not necessarily mean that 215 investigators at the less-prestigious institutions are "better scientists" or are "more meritorious" 216 than those at the prestigious institutions. Reasons for this are documented in a subsequent 217 section of the Results and are described in the Discussion.

219
A more comprehensive measure for the magnitude of disparity

221
Previous studies of funding disparities have focused primarily on differences in grant application 222 success rates (e.g., Ginther et al., 2011;Kaatz et al., 2016). However, results of this study and 223 those recently reported elsewhere (Murray et al., 2016;Wahls, 2016) show that there are also 224 disparities in amounts of funding per award. When investigators who are in a group that is 225 disadvantaged by lower success rates do get their applications funded, they often receive 226 substantially less money per award (e.g., Figure 2D). Moreover, there can be substantial 227 differences in productivity between groups (e.g., Figure 2E-2F), which is germane to whether 228 differences in success rates and award sizes are warranted. These various factors can be 229 evaluated simultaneously by using the SR/P value, which is success rate divided by 230 productivity. Differences in SR/P values for investigators grouped in any way that is desired 231 (e.g., by race, gender, age, institution or state) and using any measure of productivity that is normalized, funding amount-normalized, scientific output-normalized magnitude of funding 234 disparities.

236
metric, below) of each prestigious institution exceeded that of each less-prestigious institution ( Figure 2G-2H and Supplementary Table S1). When publications were used as the basis for 7 per investigator ( Figure 2C). Third, the scientific productivity of the disfavored institutions 285 exceeds that of the favored institutions (Figure 2E-2F) and there are robust inverse correlations 286 between funding (total, per award, per investigator) and productivity (Figure 3). These findings 287 provide important new insight into causes and consequences of disparities in federal funding for 288 scientific research, and they support evidence-based changes in funding policy.

292
The extreme disparities in NIH funding to institutions (e.g., 1% of funded organizations get about 293 34% of the dollars), which favor a tiny minority and disfavor the vast majority (Figure 1) (Figure 2). It thus appears that the NIH funding