The NIH system stifles creativity. Why not empower the R21?

May 24 2012 Published by under Academics, NIH, NIH Careerism, NIH funding

Dario Maestripieri pointed to an Opinion by Fred Southwick in The Scientist over at The Creativity Post. Southwick's point, in part:

The maladaptive insistence that research scientists obtain their financial support by continually writing federal, state, and private grants often prevents scientists from stepping outside the boundaries established by this grant process. Because the success rate is now down to 10 percent, academic scientists are forced to spend much of their time writing proposals, rather than performing creative research.

The balance of his remarks seem mostly directed at what University changes are needed.

Maestripieri adds a bit more finger-pointing at the NIH (and other major granting agencies):

Obtaining research grants has become an increasingly competitive business. Many more people apply for grants now than in the past while the amount of funds available has remained the same or decreased. Federal funding agencies fund only about 10% of the applications and grant review panels have become highly risk-averse. They tend to fund proposals by well established investigators, which often represent replications or minor extensions of previous work, while creative, original, and riskier proposals by young researchers are penalized. The funding issue has its origin outside of academia and its solution must also come from outside of academia: a political decision to allocate more funding to research.

More money, yes. But let us not take our eye off the ball.

Those of you* who have been with me for all these years should remember from whence this blog voice came.

One of the problems that I have always had in my head, driving my comments about fairness to young investigators, is that of the inherent conservatism of the NIH granting system. This comes from two major sources. First, the peer review by those who are already well-ensconced in the system. That means people who were selected by that system for success. You have to have been awarded an R01 to serve on a review panel, mostly speaking. This means you have to be able to at least fake writing the grants the way the established folks "expect" to see them. If you can't....you are going to fail. It means that you have to propose science that those who are already on the inside "get" in some way. Sorry, but even in the 25 page days, that wasn't necessarily long enough to get three people totally on board with your ideas. It helps a lot if the reviewers are already predisposed to see things your way.

The second major driver is the review culture often expressed in words such as "risky", "feasibility", "fishing expedition", "insufficient supporting preliminary data", "untried investigator"....and "highly productive scientist can solve any problems that may arise"

It can sometimes** be appallingly conservative on panels.

Being something of a student of the past few decades of research supported in my areas of interest, mainly by NIDA, I've been bothered by all of this and the Street Lamp Problem***. The SLP is common in science, far too common. And before you get all huffy, yes we all suffer from this on occasion. It may not be such a bad thing for labs but it is death for a broad-based funding agency such as the individual ICs of the NIH.

To give you a flavor, that has trickled through my posts now and again, the preclinical drug abuse world has two theoretical poles that have driven much of the research modelling. One is the "feel good" or reinforcement side. The notion being that if we understand why drugs make us feel good and why they may stop making us feel quite as good (for a given dose) then we're on to something. The other pole says, nonsense, this is about feeling bad. Everyone feels good on drugs but not all become addicted****. What really matters is that some people start to feel so crappy when they are not intoxicated that it drives them to take more just to feel normal.

For a very long period of time the reinforcement types ruled the day. They had the tools, the simplest models, the biological targets were obvious. Cocaine, meet dopamine transporter. easy peasy.Ditto heroin. Methamphetamine? No problem. And the current midcareer "reinforcement" butt kickers sucking up all the NIDA money are at least three scientific generations removed from the start of this.

umm, "feeling crappy"? where's the target? how do you even define that in a mouse?.

Anything rats won't readily self-administer intravenously (THC, nicotine) or where the targets weren't known (THC, Alcohol)....well, shrug. The light isn't very good over there. So they received short shrift. At NIDA. Luckily in the case of alcohol there was a whole 'nother IC devoted to it. It is no coincidence that one of the the main drivers of the "feel bad" pole is dug in much more deeply at NIAAA than ever at NIDA.

Things eventually opened up. The CB1 receptor was cloned and the streetlight flickered into life above THC research. Some folks finally worked out what was UP with nicotine self-admin and that started rolling. People flogged the hell out of "reinstatement" and "escalation" models to try to get past a couple of problems with the straight-up acute "feel good" models.

Funding-wise, it took some heroic efforts, if you ask me. Remember, all the while the scientists were working through this, the Program staff was getting their best intel from those scientists. People made discoveries and published papers and kept getting high grant scores. So Program thought they must know what they are talking about. The fringe complainers? Bitching about their biased grant reviews? Failing to publish their ideas convincingly (because they can't get the awards, of course)? Chalk them up to lunatic fringe, right? "Come back when you convince us all", they are told.

But we haven't learned our lesson, I have little doubt. In fact, we (as a whole enterprise- scientists and Program staff are doubling down. Kill the R21! Circle the wagons on feasible projects where the "significance" and "likely impact" is obvious to all. Demand more Preliminary Data. Save the Small Town Grocer and the Noonans. Just keep plodding along with your models, the same as half a dozen others, and you deserve "your piece of the pie".

"Innovation" can only come from within a defined space of the "feasible".

I could cry for wholesale changes. Dismantle everything. Break down the large, established groups, radically restructure grant review panels and turnover your entire Program Staff with their "established relationships". But that's not going to happen.

What we can push for is the restoration of the R21. The expansion of it, perhaps. With the very, very firm goal of making sure the ones getting high scores are "Exploratory", "Developmental" or both. It can be done. The simplest way would be to give extra bennies on an R01 application that arose from a prior interval of R21 support. But you could get the right reviewers and instruct them properly as well. Some of us get R21s. Some of us can't help but talk about supporting data and whether it is going to "work". SROs know who is who.

__
*PP and maybe one or two others.

**often. Not always, often.

***Policeman comes across obviously drunk guy searching through gutter at night under a street lamp and inquires as to the problem.
"Lost m' keys" slurs the drunkard.
"And where did you lose them", queries our intrepid flatfoot.
"Up the block on my way home from the bar." is the reply.
"So why on earth are you looking here, my friend?", puzzles the copper.
"The light is better", comes the response.

****Ok, in fairness while that is my take, it is only recently that people are coming around to this extent.

22 responses so far

  • Isis the Scientist says:

    Damnit with the damned footnotes already!!!!!

  • juniorprof says:

    I'm with you till the R21 part... 2 years for a pittance? How is that going to inspire more creativity? Sure the idea is right on the R21 but you can't get anything done in that amount of time or for that amount of money (IMHO).

  • Beaker says:

    The risk aversion is caused by study section group behavior in the context of low funding rates. The program officers I've talked to would love to fund more innovative/risky research, but they can't because those proposals usually get shot down in study section.

    So, if you want to make the R21 mechanism do what it was designed for, you need to create better mechanisms by which programmatic pressure/rules can enforce/adjust the scoring criteria used by the study section. As it stands now "approach" is all that matters; the other criteria are just window dressing--especially innovation and environment.

    And if we go down that route, are we not degrading the strict separation between peer review and programmatic review that is the cornerstone of NIH funding decisions?

  • becca says:

    So stop awarding them in study sections. Get the peer reviewers to triage the worst of the bunch, and then give them out by lottery.
    #ChaoticGoodApproaches.

  • drugmonkey says:

    The risk aversion is caused by study section group behavior in the context of low funding rates.

    Uh-huh. This is what was going on back when success rates were in the mid 20s....in the midst of the doubling.

  • drugmonkey says:

    you need to create better mechanisms by which programmatic pressure/rules can enforce/adjust the scoring criteria used by the study section

    exactly what I've been saying. fix review at the point of review, this post-review band-aiding stuff is inferior

  • They tend to fund proposals by well established investigators, which often represent replications or minor extensions of previous work, while creative, original, and riskier proposals by young researchers are penalized.

    You know, this fucken shitte comes out *everyfuckentime* there is a discussion of the battle between the olds and the youngs for grant moneys. There is absolutely *NO* evidence that I am aware of that the olds are less "creative, original, and risk[y]" than the youngs. The conflation of these things is fucken absurd and a distraction.

    The need to encourage risky research and the need to support young investigators so that they can grow up to become old investigators are completely orthogonal. My personal opinion is that the most "creative, original, and risk[y]" investigators are the mid-career scientists who have renewed one or two R01s and are *both* confident enough in the stability of their careers *and* experienced enough to see where to open areas for major advances lie.

    (Dude, enough with the goddamn motherfucken footnotes! Thatte shitte is unfuckencomperhensible!!! With the fucken scrolling and unscrolling, sheesh, motherfucken!!)

  • Dr Becca says:

    If my R21 gets funded, then there is absolutely nothing wrong with the review process. If not, then I agree with you on all counts, DM.

  • Joe says:

    I'm with juniorprof. Add $50k/yr to the R21.
    Also, couldn't SRAs beat people over the head in training with "Innovation is a super important scorable factor for R21's" and ss chairs force reviewers to talk about innovation?

  • drugmonkey says:

    Do y'all youngsters know the R21 used to be three years, $100K per?

    I liked that version better, actually.

  • drugmonkey says:

    regarding PP's point, it really comes down to $$. Stability of funding permits greater creativity. Greater slush amount promotes creativity. Doesn't matter who has it...

    This is another point about one-project labs....you have to service the project. Less extra hands/time/equipment to just screw around....

  • Pinko Punko says:

    Massively in favor of the 3 year R21. The best way to fund ideas/concepts that could be interesting. It should be considered seed/incubator money.

  • juniorprof says:

    I like that version better too, would be half-way worth it actually. I think the time is more important than the money, that extra year supporting a postdoc and money to do the work could make the difference in transitioning to the R01. Then again, who gets 5 years on their R01s anymore?

  • Joe says:

    If you like the 3yr R21, don't spend all your money right away, and no-cost-extension it into year 3. I've done it, and it works ok.
    What's with the "...who gets 5 years on their R01s anymore?" I thought every experienced PI got 5yrs.

  • drugmonkey says:

    No, Joe. There are apparently ICs that routinely chop a year off. Of everyone. Maybe even experienced PIs preferentially.

  • drugmonkey says:

    Wrt the no-cost extending, did you propose the timeline that way?

    Point being that some projects take time and reviewers know that.

  • Joe says:

    DM, No, I didn't consider asking for 3yrs up front. Do you think that would worK? I just had more money than I was expecting, so it made sense to just extend the R21 for another year.

  • drugmonkey says:

    My point is that that is not a solution to many o the reasons for preferring the original R21.

  • Grumble says:

    Look. The problem with grant review is that reviewers have NO FUCKING CLUE about which grants are likely to yield unexpected results that really move the field forward. That's because those results are UNEXPECTED, even to the PI. Yet they are incredibly important for science. The more experiments you try, the more things you investigate, the more likely that something surprising will pop up. Well funded investigators will often yield a steady stream of provocative results, yet even they often cannot predict whether something really important will arise from a specific line of experimentation.

    So why is grant review always so concerned about the exact experiment and exact interpretations? Well, duh, that's what's proposed: experiments and expected interpretations. As DM has discussed before, it's the approach score that matters most for overall score. Yet, ANY grant review process that is strongly influenced by considering approach will NECESSARILY result in lots of funding of incremental research.

    OK, so let's imagine what happens if NIH were to explicitly remove approach as a review criterion for some portion of grants (say, R21s). Investigator, environment, innovation and significance would be left to judge the grant. Investigator and environment usually get high marks (hey, we're a bunch of PhDs in labs). We can usually judge innovation when we see it, but not all grants need to be innovative to produce excellent results. And I'd argue that we are terrible judges of significance, partly because by definition you can't predict an unexpected result, and partly because "significance" is so subjective it's almost meaningless.

    So, what's the alternative? I've been saying, since I started reading DM's ramblings, that some portion of the grant pot should be awarded based solely on productivity, under the theory that, unlike the stock market, in science previous results are an excellent indication of future results. What if you were given an R21's worth or two of money just for having published reasonably proficiently in the last few years - and I mean gratis, without having to spend days, weeks and months writing and re-writing grants to get it? My guess is that the quality and quantity of the resulting science would be at least as good as awarding that many R21s based on proposed experiments and expected interpretations.

  • pinus says:

    Isn't that called a MERIT award?

  • drugmonkey says:

    I have proposed in the past that if they want to decrease the churning of applications, that expanding MERIT R37 extensions would really help. Naturally only if I were to receive it. Otherwise it would be clear evidence of BIAS and INCOMPETENCE at the program level...

  • [...] drawerspecial sectionJohn Ioannidesspecial topicRegistered Reportstrong feelingamongresearchersthis postBlackawton Beesvon [...]

Leave a Reply