Racial Disparity in NIH Grants: Solutions from Your Humble Narrator

A comment by Halophile on a prior post asks:

I'm honestly interested in how a hypothetical NIH Director DrugMonkey would handle the situation.

This was in response to my expressed skepticism that putting together an advisory panel was the right solution. Especially when they are tasked with:

Its charge will focus on five key transition points in the pipeline: (i) entry into graduate degree programs; (ii) the transition from graduate degree to post-doctoral fellowship; (iii) the appointment from a post-doctoral position to the first independent scientific position; (iv) the award of the first independent research grant from NIH or equivalent in industry; and (v) award of tenure in an academic position or equivalent in an industrial setting. The Committee will provide concrete recommendations to the NIH Director on ways to improve the retention of underrepresented minorities, persons with disabilities, and persons from disadvantaged backgrounds through these critical periods.

These are age-old concerns of academia and the solutions are not at all simple. And the obvious stuff comes up over and over again. With little success*. I am making the leap here that the primary motivation of this new panel is a response to the recent revelation that African-American Principal Investigators who apply to the NIH for research grant funding are having poorer outcomes than are white PIs. Correspondingly I think they should be focusing on the proximal problem and not letting themselves get distracted into the larger problem. Yes, even if the proximal issue is a death-of-a-thousand-cuts type of dealio. Yes.

My take with respect to the NIH response is as follows.

i) and ii) are things the NIH is supposedly already doing and tracking with their NRSA programs, MARC training grants and assorted other topics. So first, this is being taken care of elsewhere in the institution. Second, it is pretty damn far away from the prize and you would have to start with a hypothesis that somehow the good African-American scientists are being disproportionately shelled out at grad school and postdoctoral transitions. Thus, those that make it through to apply for grants are only the mediocre ones and this relationship differs for white scientists. I think this is a low probability hypothesis. Move on.

iii) is certainly a concern, particularly with the recent focus of the NIH on transition, creation of the K99/R00 mechanism and general worry about "the next generation". At this point I'm doing the facepalm and thinking "yeaaah, that should have been an obvious component of this K99/R00 flurry but damn I bet it wasn't." However. Much like my response to items i and ii, this is still a bit distant from the grant outcome issue that has been revealed.

iv and v are where the money is. This is where the NIH needs to focus. The fate of existing junior and not-so-junior faculty particularly where it comes to winning grant support from the NIH. Stop getting distracted with "tenure" though, except where you can identify that clearly with disparity in NIH grant awards, the need for African-American investigators to submit more times for a given award, etc.

I am dismayed that the charge is not to delve further into the causes of disparate grant review outcome. To go right at the proximal problem, now, with great dispatch and figure out how to fix it. As soon as possible. Why? Because this news has an immediate and discouraging effect on our current trainees! Not to mention the PIs...

Now, maybe further data mining has already been conducted or is in the works but we don't know that. Tabak and Collins referred to some preliminary analyses that may or may not go beyond what Ginther et al reported. Here's what I would commission from the CSR data miners.

-Outcomes for black versus white applicants on a study section by study section basis.

-Participation of African-American and white reviewers on each panel; heck throw in the same racial/ethnic breakdowns used by Ginther while we're at it.

-Outcomes on a IC by IC basis as well.

These are important bits of information. Is there any relationship with particular scientific domains, review panels or review panel makeups? That would recommend one set of fixes. If not, we'd have to move on to other considerations.

Next up.....um, why not actually talk to the applicants? Supplementary Table 2 of Ginther says there were only 2,942 applications with Black PIs from 2000-2006, 285 awards. I think the 16 member panel could make a serious dent in interviewing this population of investigators. A couple of emails and badabing! What do they think has been helpful and detrimental to their successes and failures as applicants? Where do they point the finger? I'm not saying they will have the same viewpoint or even necessarily be correct (with respect to the broader problem) but for goodness sake this is a place to generate some hypotheses, is it not?

Finally, I come to the most immediate response of all. Programmatic alterations in funding priority. This disparity is a bias in the system, like it or not. Not dissimilar to the bias against new investigators that has been a repeated topic of my blogging. Familiar, in fact, to an older (yet sadly ongoing) discussion of bias against female investigators.

The NIH has responded quite directly in the most recent case of new investigators by putting a finger on the scale to rebalance outcomes. The less established investigators are still getting crappier scores from review panels (and maybe even worse), but the NIH is choosing to pick them up out of the review order in recognition that poorer review outcome is a bug in the system.

This is no different. So my clearest answer to the question raised by Halophile is that I'd put out the word, informally if I had to**, that I was expecting the IC heads to fix the problem pronto. To examine their portfolios of scores in each round to make sure they were prioritizing*** black PI apps in their above-payline pickups.
__
*Sorry but I am exhausted by us majority white institution folks fruitlessly pointing the finger at the supply chain below us. It dilutes the blame and gives a handy out when you fail to improve. There are two methods, only, that work. They are related to each other. The pull side, "attractiveness" of a career, a University, a training program, etc, solves itself as a self-fulfilling prophesy. When women, gays, the poor and minorities inhabit specific roles in society, that makes them more imaginable to subsequent generations. Make the job "look" diverse and dismantle any real hurdles that exist and you are most of the way over the hump. How to get there? I am on record as a big fan of creating overt, visible diversity by any means necessary. This brings me to the second method- money. Making your graduate program, postdoctoral environment or faculty ranks more diverse is simple- outbid the competition for the limited resource.

**think, rightwing anti-affirmative action political bigots

***By way of disclaimer, yes, this would theoretically benefit people who's careers are of direct personal and professional interest of mine.

15 responses so far

  • femalephysioprof says:

    Empower study section panel members to be aware of their own unconcious bias with respect to both race and gender by taking a demo test at ProjectImplicit (https://implicit.harvard.edu/implicit/demo/). Try it yourself. It is pretty compelling. (Note also this research appears to have been funded in part by NIH...).

  • drugmonkey says:

    That's something Tabak and Collins alluded to pursuing. I'm really curious as to how this is supposed to help.

  • anon says:

    I also wonder if they can quantify distinctions in the actual reviews. I can't locate where I just read it, but a female faculty blogger was complaining how reviewers regularly suggest she needs more senior collaborators even when there are such names on the grant.

    There are enough lists of stock critiques floating around. It should be possible to collapse each grant review report into counts of each stock critique & the various scores. Is there a bias regarding how often certain critiques appear? Such a finding could point to real reviewer biases or places where African-American PIs might need more help. For example, if they really do have smaller support networks, this could legitimately lead to the "too few senior collaborators" critique I mention above. The response could be to either decrease the importance of collaborators on grants or make sure applicants know they need such collaborators & try to facilitate those connections.

  • drugmonkey says:

    The idea of having a StockCritique quotient for each study section has me salivating.

  • Halophile says:

    Thanks for your well-reasoned response. I do think that there is merit in looking at the "pipeline" effects as maybe, just maybe, this issue is not bias in the reviewers. Therefore, there may be some deficiencies in the training of African-American scientists who go on to become PIs. I guess I have a very difficult time imagining that reviewers would first, recognize that the applicant was Black, and then second, assign the application a lower score.

  • drugmonkey says:

    I do think that there is merit in looking at the "pipeline" effects as maybe, just maybe, this issue is not bias in the reviewers.

    While you are not wrong, I have seen this *repeatedly* over the years as a way for MajorityLilyWhiteInstitutions to avoid actually having to *do* anything. Like, for example, hire some underrepresented faculty even if it costs them a 10% or 20% salary premium to do so. So I score "pipeline" handwringing in the category of looking busy without actually accomplishing anything.

    I guess I have a very difficult time imagining that reviewers would first, recognize that the applicant was Black

    In NIH review, the applicant investigator(s) is(are) a major review criterion. The focus is most assuredly on this person. Oftentimes, the person is known professionally to the reviewers. If not, they can Google that PI's faculty page in a trice. I would argue that because some of the applicants *are* well known to you as a reviewer you are obligated to do a little bit of work to familiarize yourself with the person in question. It may sound crazy but oftentimes you know someone's face from poster, platform or other meeting interactions but don't necessarily connect it with the name later on. Especially us old farts when it comes to the next generation of scientists. So I think you *should* do a Google on the applicant if you don't know who they are from the name. Again, it is only fair given that the more-famous people, or your tight science homies, already enjoy this benefit.

    then second, assign the application a lower score.

    Yeah, all kinds of people are surprised to find that racism still exists....and in the strangest of places. Some of it is not even conscious. That's why we need data such as were provided by the Ginther et al report.

  • becca says:

    So, why isn't it time for a pilot program where the investigator is *not* a major review criteria? Why not try double-blind peer review and see there's still a racial disparity?

    I get why the way we do things this is neigh impossible. But for a small scale effort, with infinite budget, you could do it. Instead of submitting a biosketch, you submit a packet with the top three most relevant recent publications- with all authorship identity removed (if you want to get fun, remove the journal name as well). You put the review panel up in a hotel room for a few extra days, and get them all to read the grants and supplementary info there, without internet access. You ask them if they recognize the publications and therefore have identified the author- and ask them to recuse themselves from that application under that scenario.
    I'm sure there are a host of other reasons this is an impossible experiment... but I wish they could do it.

  • Paul Orwin says:

    I wonder how entangled this problem is with the problem of bias in hiring at universities. We know from many years of study that unconcious bias is pervasive, and perhaps one way it emerges is that faculty of similar apparent quality (ie pub record, grad school reputation, post-doc rep, etc) do not end up at the same level of university at the same frequency - in other words, you have to be even better if you are black/hispanic/female etc to get the R1 faculty job, so the net result is that the underrepresented person ends up at a "lesser" institution. Does the paper under discussion take that sort of thing into account (the reputation/academic level of the schools from which grant applicants are applying)? There are other factors involved here besides bias in decisionmaking. For example, some schools will have higher teaching loads or make it harder to get released from teaching, resulting in longer lag to pub and lower productivity; some schools that are technically at the R1 level may have small/poor Ph.D. programs (in terms of $$) that make it hard to recruit top grad students or post-docs. All of this can lead to disparate outcomes in grantsmanship that have nothing to do with racism/bias at the panel, but are still the result of more diffuse/pervasive/difficult to deal with bias. This isn't a vote against doing something, by the way - if a problem is everywhere, you have to start somewhere!

  • drugmonkey says:

    They tried to account statistically for type of institution. Main effect still held. I see where you are going with increasingly complex interaction terms- yeah could be that. You can always invent new factors which "really" explain the data. *in the meantime* however, until a factor is proven, I think we need to accept bias in review as the null.

  • Paul Orwin says:

    Fair enough, and probably you are right that bias in review needs to be examined and eliminated. My concern is that if the problem is pervasive (and I think it is) then it probably manifests at every level, meaning that dealing with it at this point (panel review) might have a small or even negligible effect, which might lead people to think that the bias must not exist, which is incorrect - a bunch of small things all pushing in the same direction can lead to a big disparity. I don' t think anything "really" explains the data, in the same what that no one enzyme is responsible for oxidative phosphorylation. We can see the obvious truth of that statement in biochemistry, but when it comes to society, we seem unable to grasp it. None of this is to say that the NIH trying to improve its process is bad, just that the problem seems likely to me to be bigger than that. Just my 0.02

  • drugmonkey says:

    Why not try double-blind peer review and see there's still a racial disparity?

    It is far beyond time to attempt double-blind peer review. It will generate the entirely predictable outcome that the reviewers know full well who the applicant is. This will be fantastic so that we can retire this tired nostrum that is raised by people who refuse to think it past the shiny-pretty part.

  • This will be fantastic so that we can retire this tired nostrum that is raised by people who refuse to think it past the shiny-pretty part.

    The only people who ever suggest this idiocy are scientific toddlers who have never actually peer-reviewed anything in their lives.

  • [...] offered solutions before, after expressing skepticism about the Advisory Panel [...]

  • [...] the wake of the Ginther report, this is a very nice step forward. It was not something I had considered before as a response to the Ginther [...]

  • […] the disparity in grant funding that was identified in the Ginther report in 2011. See here, here, here for blog comment on Gither et al. Nakamura is, of course, the current director (pdf) of the Center […]

Leave a Reply