Archive for the 'NIH Budgets and Economics' category

The NIH Grant "Have" States Resist Sharing

Apr 04 2014 Published by under NIH, NIH Budgets and Economics, NIH funding

From the Boston Globe (of course):

Two dozen rural states stretching from Maine to Mississippi and Montana are clamoring to increase their share of federal research dollars now disproportionately awarded to Boston-area institutions and scientists.

Whaddaya mean, "disproportionately"? WE DESERVE IT!!!

“There’s a battle between merit and egalitarianism,” said Dr. David Page, director of the Whitehead Institute, a prestigious research institution in Cambridge affiliated with MIT.

Yeah, pure merit versus affirmative action quotas for lame ass science from Universities we've never heard of maaaang. There couldn't possibly be any bias in grant review and award that puts a finger on the scale could there?

In one of the efforts, Senator Susan Collins, a Maine Republican on the Appropriations Committee, is proposing that funding for the special program to benefit rural states, formally called the NIH’s Institutional Development Award, be raised to $310 million, up from the current $273 million. The current amount equals just 1 percent of the institute’s research grants — a drop in the bucket compared with what Boston researchers win each year.

Last time I checked Massachusetts Congressional District 8 for NIH funding (probably a number of FY ago), Brigham and Women's Hospital was pulling in $253,333,482 in NIH grants. MIT? $172,184,305. Harvard Medical School? $168,648,847. The list goes on in this single Congressional district.

and while the Globe has this scare passage near the top:

The coalition of states that benefits from the NIH special program for rural states doubled the amount of money it spent on lobbying in the last decade, to $590,000 in 2013 from $300,000 in 2003. That number does not include direct lobbying by universities in those states.

this is going to barely manage to tread water against the combined might of the richest of "have" Universities and institutions:

Representative Michael Capuano, whose district encompassing the Boston-area research hospitals wins more NIH money than any other congressional district, said the Massachusetts delegation is playing defense right now.

“The system works reasonably well but it’s under attack in a serious way,” Capuano said.

Massachusetts is mobilizing. Hospital executives, university presidents, and Washington lobbyists make routine trips to the Capitol. Their not-so-subtle message: Boston is on top because its elite institutions offer the best chances of big scientific breakthroughs.

then there is classic misdirection and the usual conceit that the NIH award process is purely about merit, uncontaminated by self-reinforcing vicious cycles of the rich getting richer.

“There are people in Boston who deserve more than a million dollars in NIH money because that is the best use of those dollars,” said Dr. Barrett Rollins, chief scientific officer at Dana-Farber Cancer Institute, a top recipient of federal research funds. “Congress has a responsibility to spend taxpayer money in the best possible way, and to me, the most straightforward way to do that is to make sure the dollars are invested in the most meritorious work without regard to geographic distribution.”

Because the quality of science is not evenly distributed across the country, researchers should not expect federal dollars to be either, said Harry Orf, senior vice president for research at Massachusetts General Hospital, another top recipient of NIH grants.

“You have congressmen who can’t evaluate science sending money to places not rated for innovation,” Orf said. “As funds get more and more scarce, you want to make sure you’re betting on the best science.”

It is beyond asinine to pretend that the NIH grant money is distributed by geographic affirmative action to any extend that squeezes the elite coastal research institutions. The above numbers and any current search on RePORTER verifies that the kind of money that is being proposed to go into this geographical affirmative action is a drop in the bucket. One or two of the larger institutions funded by NIH (and keep in mind that a place such as "Harvard" is made up of multiple institutions which are named as independent awardees in the NIH records) account for the entire outlay in the the NIH’s Institutional Development Award program. Even if the increase to $310M goes through.

There is considerable debate about "the best science" and about the best way to hedge our scientific bets. The NIH works, haltingly, in a way by which the serendipity of chance discovery from a diversity of approaches is balanced against predictable brute-force progress from exceptionally well funded Universities, Medical Schools and research institutions. I find myself citing papers from the very biggest institutions, sure, but I have numerous critical findings that I cite in my work that have come from smaller research programs in smaller Universities and (gasp) Colleges. Don't you? If you do not, I question your scholarship. Seriously.

I suggest a purely self-interested goal, for those of you who are elite-coastal-University die hards. Every Congress Critter gets a more or less equal vote. The ones from Maine (Susan Collins, see above), from Alabama....

“It’s hard to compete against MIT or Harvard. . . . They’ve had their share. A lot of state colleges and universities all over the country, from Idaho to Maine, have some ideas too, and I think we should give these people from smaller schools in other states an opportunity,” said Senator Richard Shelby of Alabama, the top Republican on the powerful Senate Appropriations Committee. “It’s time to fix that.”

from West Virginia...

“The program stipulates that not everything goes to Harvard, Yale, and Stanford,” said Senator Jay Rockefeller, a West Virginia Democrat.

and from Oklahoma, among others.

Representative Tom Cole, a Republican from Oklahoma who serves on the House Appropriations Committee, said he’s simply interested in supporting research that occurs “outside the normal corridors of power.”

Rep Cole seems to understand why geographical affirmative action is necessary, doesn't he?


“There is a network where you tend to reward peers and people you know, and I think the distribution of funds, not intentionally, is skewed a bit toward places like Boston,” Cole said. “We just want to make sure that the playing field is fair.”

We need all these Critters to be on board if we expect Congress to listen to our pleas on behalf of the NIH.

It is politically stupid to fail to understand this.

52 responses so far

Guest Post: BUILDing Diversity with Dollars: Can Grants Change Culture

This guest post is from @iGrrrl, a grant writing consultant. I think I first ran across her in the comments over at writedit's place, you may have as well. She brings a slightly different, and highly valuable, perspective to the table.


iGrrlCartoonThere was an old New Yorker cartoon of two people at a party, and one tells the other, "I'm a fiction writer in the grant-proposal genre." I hate putting fiction in grant applications, especially the type that will be due shortly in response to the Ginther report.

For those who have been worrying about their own grant applications, the Ginther report detailed the relationship of race and ethnicity to NIH grant funding at the R01 level, and NIH has created a few initiatives to try to change the pattern of lower success rates for African American applicants. In early April, the applications are due for the Building Infrastructure Leading to Diversity (BUILD) Initiative, which would fund large-scale projects within individual institutions, and the NIH National Research Mentoring Network (NRMN), which is designed to build a network of mentors. In other words, diversity interventions writ large, with millions of dollars behind them.

The NSF ADVANCE program started in a similar way, and some of the early ADVANCE projects included programs with a limited evidence base. The successes of components of these programs were variable, and the recent RFA included a social science research component to complement the required evaluation of ADVANCE-funded activity. It started with good intentions, and eventually became clear that more than good intentions were needed. Part of the NRMN announcement calls for pilot programs, but I would argue that in the first year, the network should do the social science work to consolidate the anecdotes African Americans tell about their mentoring experiences into hard data, so that the pilot programs can be based upon addressing the needs identified by African Americans who have been through, or training right now in, the current system.

At the recent AAAS meeting in Chicago there were sessions on building diversity in science. At one I learned that explicit bias has reduced in the last 30 years, but implicit bias hasn't. We think we have made progress, and that our conscious intentions are enough. But they clearly are not. Expecting trainees to overcome biased behaviors (to which the actors are blind) places an undue burden on those who are discriminated against. There are studies showing that education about implicit bias helps to reduce such biased behavior, but education attempts can also be done badly and backfire. As pointed out in a recent piece in Science by Moss-Racusin, et al., there is an evidence base now for doing intervention well. If NIH is putting money into large-scale intervention, I hope the existing science will be part of the applications, and expected by the reviewers.

I've spent a lot of my professional life working on exactly the kind of large, infrastructure-based grant application represented by the BUILD and NRMN programs. It is easy for PIs to make assumptions that interventions that sound good on paper will actually have any impact. My concern is that what will be proposed by the applicants to BUILD and NRMN may miss the strong social science work that exists, and that still needs to be done. In fact, some of the best research on effective mentoring is the business literature, a place where few biomedical scientists would think to look.

Grant applications shouldn't be pure fiction, but based on solid evidence. Every grant application represents a possibility, a reality, that could come to pass if the funds are awarded. In the mentoring literature, practices that improve the success for African Americans are often shown to improve the climate for everyone. There is an opportunity here for those in biomedicine to learn from other fields, to consider an evidence base that is outside their usual ken, and to improve the entire biomedical enterprise by improving the overall environment. I hope that those applying for BUILD and the NRMN include the social sciences, and even more importantly, include the voices and ideas of the very people these programs are meant to serve.

NIH has a long history of using dollars to encourage cultural change, with mixed results, because applicants can have varying levels of commitment to the NIH vision while being happy to take NIH dollars. The ADVANCE program at NSF had some hiccups as they worked out what worked to improve the climate for women in STEM. The leadership teams for BUILD and NRMN should include people with a deep knowledge of the research and scholarship on bias and on mentoring, and who can do the rigorous analysis of the current state of affairs for African Americans in biomedical science. I hope I'm wrong here in worrying that such people won't be included, but I've seen fiction in grant applications a few too many times.

No responses yet

Berg requests your input on NIH data mining queries

This is important enough to elevate to an entry.

I had a recent post discussing some analysis Jeremy Berg posted at ASBMB Today ("The impact of the sequester: 1,000 fewer funded investigators") looking at some NIH data on the number of PIs who entered and exited the R-mech funded population across FY11-13.

 

He came by and left this comment:

I would welcome any suggestions about other longitudinal aspects of the NIH grantee pool that might be high priorities for analysis. Post here, at http://www.asbmb.org/asbmbtoday/201403/PresidentsMessage/ or email me at jberg@pitt.edu.

So if you can clearly specify some sort of examination of the extramural PI population go to it! He's apparently the guy who can actually make it happen.

 

 

No responses yet

Berg posts data on NIH Intramural funding

Berg2014IntramuralChartJeremy Berg has a new column up at ASBMB Today which examines the distribution of NIH intramural funding. Among other things, he notes that you can play along at home via searching RePORTER using the ZIA activity code (i.e., in place of R01, R21, etc). At first blush you might think "WOWZA!". The intramural lab is pretty dang flush. If you think about the direct costs of an extramural R01 grant - the full modular is only $250K per year. So you would need three awards (ok, the third one could be an R21) just to clear the first bin. But there are interesting caveats sprinkled throughout Berg's comments and in the first comment to the piece. Note the "Total Costs"? Well, apparently there is an indirect costs rate within the IRPs and Berg comments that it is so variable that it is hard to issue anything similar to a negotiated extramural IDC rate for the entire NIH Intramural program. The comment from an ex-IRP investigator points to more issues. There may be some shared costs inserted into a given PI's apparent budget that this PI has no control over. Whether this is part of the overhead or an overhead-like cost....or maybe a cost shard across one IC's IRP...who knows?

We also don't know what a given PI has to pay for out of his or her ZIA allocation. What are animal housing costs like? Are they subsidized for certain ICs' IRPs? For certain labs? Who is a PI and who is a staff scientist of some sort within the IRPs? Do these status' differ? Are they comparable to extramural lab operations? I know for certain sure that people who are more or less the equivalent of an extramural Assistant/Associate Professor in a soft money job category exist within the NIH IRPs without being considered a PI with their own ZIA allocation. So that means that a "PI" on the chart that Berg presents may in fact be equivalent to 2-3 PIs out here in extramural land. (And yes, I understand that some of the larger extramural labs similarly have several people who would otherwise be heading their own lab all subsumed within the grants awarded to one BigCheez PI.)

With that said, however, the IRP is supposed to be special. As Berg notes

The IRP mission statement asserts that the IRP should “conduct distinct, high-impact laboratory, clinical, and population-based research” and that it should support research that “cannot be readily funded or accomplished in traditional academia.”

So by one way of looking at it, we shouldn't be comparing the IRP scientists to ourselves. They should be different.

Even if we think of IRP investigators as not much different from ourselves, I'm having difficulty making any sense of these numbers. It is nice to see them, but it is really hard to compare to what is going on with extramural grant funding.

Perhaps of greater value is the analysis Berg presents for whether NIH's intramural research is feeling their fair share of the budgetary pain.

In 2003, when I became an NIH institute director, the overall NIH appropriation was $26.74 billion, while the overall intramural program consumed $2.56 billion, or 9.6 percent. In fiscal 2013, the overall NIH appropriation was $29.15 billion, and the intramural share had grown to $3.26 billion, or 11.2 percent.
 
Some of this growth is because of ongoing intramural activities, such as those involving the NIH Clinical Center, where, like at other hospitals, costs are very hard to contain below rates of inflation, or because of new activities, such as the NIH Chemical Genomics Center. The IRP is particularly expensive in terms of taxpayer dollars, because it is difficult to leverage the federal support to the IRP with funds from other sources as occurs in the extramural community.

So I guess that would be "no". No the IRP, in aggregate, is not sharing the pain of the flatlined budget. There is no doubt that some of the individual components of the various IRPs are. It is inevitable. Previously flush budgets no doubt being reduced. Senior folk being pushed out. Mid and lower level employees being cashiered. I'm sure there are counter examples. But as a whole, it is clear that the IRP is being protected, inevitably at the expense of R-mech extramural awards.

 

 

34 responses so far

New Grant Snooping

Feb 04 2014 Published by under NIH, NIH Budgets and Economics, NIH funding

As usual, I like to keep and eye on RePORTER and SILK to see what the various ICs of my own dearest interest are up to with regard to grants that were supposed to fund Dec 1, 2013. Per usual, there was no budget and the more conservative ICs wait around to do anything. Some of the less-conservative ones do tend to start funding new grant awards in December and Jan so there is always something to see on SILK.

I noticed something interesting. NIAID has 44 new R01s listed that were on the A1 revision and 19 that were funded on the "first" submission. RePORTER notes that 30 funded in Dec, 12 of these funded in Jan and  17 on or after 2/1/2014 (not sure if I miscounted totals on SILK or RePORTER hasn't caught up or what).

My ICs of dearest concern are still waiting, only a bare handful of new R01s are listed.

NCI has 36 new R01 apps funded on A1, 21 on the A0. DK is running 15/13.

Scanning down the rest of the list of ICs, it looks like DK is about as close to even as it gets and that a 2:1 ratio of A1 to A0 being funded is not too far off the mean.

 

I still think we'd be a lot better off if something like 2/3rd of grants were awarded on first submission and the A1s were only about a third.

11 responses so far

More thoughts on the dismal NIH response to Ginther

Jeremy Berg made a comment

If you look at the data in the Ginther report, the biggest difference for African-American applicants is the percentage of "not discussed" applications. For African-Americans, 691/1149 =60.0% of the applications were not discussed whereas for Whites, 23,437/58,124 =40% were not discussed (see supplementary material to the paper). The actual funding curves (funding probability as a function of priority score) are quite similar (Supplementary Figure S1). If applications are not discussed, program has very little ability to make a case for funding, even if this were to be deemed good policy.

that irritated me because it sounds like yet another version of the feigned-helpless response of the NIH on this topic. It also made me take a look at some numbers and bench race my proposal that the NIH should, right away, simply pick up enough applications from African American PIs to equalize success rates. Just as they have so clearly done, historically, for Early Stage Investigators and very likely done for woman PIs.

Here's the S1 figure from Ginther et al, 2011:
Ginther-S1

[In the below analysis I am eyeballing the probabilities for illustration's sake. If I'm off by a point or two this is immaterial to the the overall thrust of the argument.]

My knee jerk response to Berg's comment is that there are plenty of African-American PI's applications available for pickup. As in, far more than would be required to make up the aggregate success rate discrepancy (which was about 10% in award probability). So talking about the triage rate is a distraction (but see below for more on that).

There is a risk here of falling into the Privilege-Thinking, i.e. that we cannot possible countenance any redress of discrimination that, gasp, puts the previously underrepresented group above the well represented groups even by the smallest smidge. But looking at Supplementary Fig1 from Gither, and keeping in mind that the African American PI application number is only 2% of the White applications, we can figure out that a substantial effect on African American PI's award probability would cause only an imperceptible change in that for White PI applications. And there's an amazing sweetener....merit.

Looking at the award probability graph from S1 of Ginther, we note that there are some 15% of the African-American PI's grants scoring in the 175 bin (old scoring method, youngsters) that were not funded. About 55-56% of all ethnic/racial category grants in the next higher (worse) scoring bin were funded. So if Program picks up more of the better scoring applications from African American PIs (175 bin) at the expense of the worse scoring applications of White PIs (200 bin), we have actually ENHANCED MERIT of the total population of funded grants. Right? Win/Win.

So if we were to follow my suggestion, what would be the relative impact? Well thanks to the 2% ratio of African-American to White PI apps, it works like this:

Take the 175 scoring bin in which about 88% of white PIs and 85% of AA PIs were successful. Take a round number of 1,000 apps in that scoring bin (for didactic purposes, also ignoring the other ethnicities) and you get a 980/20 White/African-AmericanPI ratio of apps. In that 175 bin we'd need 3 more African-American PI apps funded to get to 100%. In the next higher (worse) scoring bin (200 score), about 56% of White PI apps were funded. Taking three from this bin and awarding three more AA PI awards in the next better scoring bin would plunge the White PI award probability from 56% to 55.7%. Whoa, belt up cowboy.

Moving down the curve with the same logic, we find in the 200 score bin that there are about 9 AA PI applications needed to put the 200 score bin to 100%. Looking down to the next worse scoring bin (225) and pulling these 9 apps from white PIs we end up changing the award probability for these apps from 22% to ..wait for it..... 20.8%.

And so on.

(And actually, the percentage changes would be smaller in reality because there is typically not a flat distribution across these bins and there are very likely more applications in each worse-scoring bin compared to the next better-scoring bin. I assumed 1,000 in each bin for my example.)

Another way to look at this issue is to take Berg's triage numbers from above. To move to 40% triage rate for the African-AmericanPI applications, we need to shift 20% (230 applications) into the discussed pile. This represents a whopping 0.4% of the White PI apps being shifted onto the triage pile to keep the numbers discussed the same.

These are entirely trivial numbers in terms of the "hit" to the chances of White PIs and yet you could easily equalize the success rate or award probability for African-American PIs.

It is even more astounding that this could be done by picking up African-American PI applications that scored better than the White PI applications that would go unfunded to make up the difference.

Tell me how this is not a no-brainer for the NIH?

34 responses so far

In hard times in NIH Grantlandia, guess who pays the steepest price?

A post over at Rock Talk blog describes some recent funding data from the NIH. The takeaway message is that every thing is down. Fewer grants awarded, fewer percentages of the applications being funded. Not exactly news to my audience. However, head over to the NIH data book for some interesting tidbits.

2013-FundingByCareerStageFirst up, my oldest soapbox, the new investigator. As you can see, up to FY2006 the PI who had not previously had any NIH funding faced a steeper hurdle to get a new grant (Type 1) funding compared to established investigators. This was despite the "New Investigator" checkbox at the top of the application and the fact that reviewers were instructed to give such applications a break. And they did in my experience....just not enough to actually get them funded. Study section discussion that ended with "...but this investigator is new and highly promising so that's why I'm giving it such a good score...[insert clearly unfundable post-discussion score]" were not uncommon during my term of appointed service. So round about FY2007 the prior NIH Director, Zerhouni, put in place an affirmative action system to fund newly-transitioned independent investigators. There's a great description in this Science news bit [PDF]. You can see the result below.

Interestingly, this will to maintain success rates of the inexperienced PIs at levels similar to the experienced PIs has evaporated for FY2011 and FY2013. See title.

2013-FundingBySexofPINext, the slightly more subtle case of women PIs. This will be a two-grapher. First, the overall Research Project Grant success rate broken down by PI sex. As you can see, up through FY2002 there was a disparity which disappeared in the subsequent years. Miracle? Hell no. I guarantee you there has been some placing of the affirmative action fingers on the scale for the sex disparity as well. Fortunately, the elastic hasn't snapped back in the past two FYs as it has for inexperienced investigators. But I'm keeping a suspicious eye on it, as should you. Notice how women trickle along juuuuust a little bit behind men? Interesting, isn't it, how the disparity is never actually reversed? You know, because if whomever was previously advantaged even slipped back to disadvantaged (instead of merely equal) the whole world would end.

2013-FundingBySexandTypeR01Moving along, we downshift to R01-equivalent grants so as to perform the analysis of new proposals versus competing continuation (aka, "renewal") applications. There are mechanisms included in the "RPG" grouping that cannot be continued so this is necessary. What we find is that the disparity for woman PIs in continuing their R01/equivalent grants has been maintained all along. New grants have been level in recent years. There is a halfway decent bet that this may be down to the graybeard factor. This hypothesis depends on the idea that the longer a given R01 has been continued, the higher the success rate for each subsequent renewal. These data also show that a goodly amount of the sex disparity up through FY2002 was addressed at the renewal stage. Not all of it. But clearly gains were made. This kind of selectivity suggests the heavy hand of affirmative action quota filling to me.

This is why I am pro-quota and totally in support of the heavy hand of Program in redressing study section biases, btw. Over time, it is the one thing that helps. Awareness, upgrading women's representation on study section (see the early 1970s)...New Investigator checkboxes and ESI initiatives* all fail. Quota-making works.

__
*In that Science bit I link it says:

Told about the quotas, study sections began “punishing the young investigators with bad scores,” says Zerhouni. That is, a previous slight gap in review scores for new grant applications from firsttime and seasoned investigators widened in 2007 and 2008, Berg says. It revealed a bias against new investigators, Zerhouni says.

26 responses so far

A Bold Proposal to Fix the NIH

Our longtime blog commenter dsks is always insightful. This time, the proposal is such a doozy that it is worth dragging up as a new post.

... just make it official and block all triaged applications from subsequent resubmission. Maybe then use the extra reviewer time and money to bring back the A2, perhaps restricting it to A1 proposals that come in under ~30%ile or something.

Hell, I think any proposal that consistently scores better than 20%ile should be allowed to be resubmitted ad infinitum until it gets funded. Having to completely restructure a proposal because it couldn't quite make the last yard over what is accepted to be a rather arbitrary pay-line is insane.

On first blush that first one sounds pretty good. Not so sure about the endless queuing of an above payline, below 20%ile grant, personally. (I mean, isn't this where Program steps in and just picks it up already?)

This reminds me of something, though. Unlike in times past, the applicant now has some information on just how strong the rejection really was because of the criterion scores. This gives some specific quantification in contrast to only being able to parse the language of the review.

One would hope that there would be some correlation between the criterion scores and the choice of the PI to resubmit. As in, if you get 4s and 5s on Approach or Significance, maybe it is worth it. 7s and 8s mean you really better not bother.

36 responses so far

Should you revise and resubmit a triaged NIH Grant application?

Jan 06 2014 Published by under NIH, NIH Budgets and Economics, NIH funding

A December 18 post on the Rock Talk blog issued an update on the funding rate situation for grant applications submitted to the NIH. The data provide

...an early snapshot on success rates for 2013 competing research project grant (RPG) applications and awards.

We received 49,581 competing RPG applications at NIH in fiscal year 2013, slightly declining compared to last year (51,313 applications in FY2012).

... In FY2013 we made 8,310 competing RPG awards, 722 fewer than in FY2012. This puts the overall research project grant (RPG) success rate at 16.8%, a decline from the 17.6% reported in FY2012. One might have expected a bigger drop in the success rates since we made about 8% fewer competing awards this year, but the reduction in the number of applications explains part of it.

emphasis added, as if I need to do so.

See this graph for a recent historical trend on success rates and application submission numbers. With respect to the latter, you can see that the small decrease to 49,581 is not hugely significant. We'll have to wait for a few more years to be convinced of any trend. Success rates are at an all-time low. This is rather unsurprising to anyone of you that has been paying attention to doing at the NIH and is a result of the long trend toward Defunding the NIH.

Of greater interest in the Rock Talk post was a comment made in response to a query about the fate of initially-triaged applications. A Deborah Frank wrote:

A few months ago, I emailed Rock Talk to ask the same question as Mr. Doherty’s question #3. My query was routed to the Freedom of Information Act Office, and a few months later I received a table of data covering A0 R01s received between FY 2010 and FY2012 (ARRA funds and solicited applications were excluded). Overall at NIH, 2.3% of new R01s that were “not scored” as A0s were funded as A1s (range at different ICs was 0.0% to 8.4%), and 8.7% of renewals that were unscored as A0s were funded as A1s (range 0.0% to 25.7%). These data have at least two limitations. First, funding decisions made in 2013 were not included, so the actual success rates are likely a bit higher. Second, the table does not indicate how many of the unscored A0s were resubmitted.

The NIH data miner / blog team then posted a link to an excel spreadsheet with the relevant numbers for ICs, divided by Type 1 (new) and Type 2 (renewal, aka competing continuation) applications. The spreadsheet notes that this analysis is for unsolicited (i.e., non-RFA) applications and that since the FY2013 funding data were not complete when these were generated (7/15/2013), it is possible that some A0 submitted in this interval may still be funded.

Now, this is not precisely the same as the usual success rate numbers because of

  • the aforementioned exclusions
  • the way A0 and A1 submitted in the same FY are counted as one application in success rate calculation
  • the fact that if an A1 is not submitted it isn't (cannot be) counted in success rate

Nevertheless, keeping these details in mind it is hard to escape noticing that one is facing steep odds to get a triaged A0 Type 1 proposal funded. On the face of it, anyway. And I have to tell you, Dear Reader, that this is consistent with my personal experience. I can't recall ever getting a triaged application to the funded level on the next submission. In fact I'm hard pressed to recall getting a triaged A0 funded as an A2 when that was still possible.

Yet I continue to revise them. Not entirely sure why, looking at these data.

Moving along, it is really disappointing that the NIH didn't go ahead and put all the relevant numbers in their spreadsheet. The thing that PIs really want to know is still terribly obscured by this selected analysis. NIDA, for example, lists 394 unscored Type 1 applications of which 11 (2.8%) were eventually funded. But unlike the now-disappeared CSR FY2004 databook analysis (see here, here for reference to it), they have failed to provide the number of applications that were initially triaged that the PI actually resubmitted as A1! If only half of the triaged applications were amended and resubmitted, then the odds go to 5.6%.

Is this difference relevant to PI decision making? I don't know for sure but I suspect it would be. It is also relevant to understanding the different success rate for initially-triaged Type 1 and Type 2 applications. The mean and selected ICs I checked tell the same tale, i.e., that Type 2 apps have a much better shot at getting funded after triage on the A0. NIDA is actually pretty extreme from what I can tell- 2.8% versus 15.2%. So if there is a difference in the A1 resubmission rate for Type 1 and Type 2 (and I bet Type 2 apps that get triaged on A0 are much more likely to be amended and resubmitted) apps, the above analysis doesn't move the relative disadvantage around all that much. However for NIAAA the Type 1 and Type 2 numbers are closer- 4.7% versus 9.8%. So for NIAAA supplicants, a halving of the resubmission rate for Type 1 might bring the odds for Type 1 and Type 2 much closer.

Do these data change my approach? They probably should. However, there is a factor of submission dates here. For any given round, new applications are submitted one month and then amended applications are due the next month. So if you are a few weeks away from the second deadline and considering whether to resubmit an application or not....there is no "new" application that you could submit right now. You have to wait for the next round. So if you are feeling grant pressure..what else are you going to do? Take the low odds or take the guarantee of zero odds?

Final note. I continue to believe, until NIH demonstrates my error very clearly, that considerable numbers of "A0" submissions are really a reworking of ideas that have been previously reviewed. I also believe that these "A0" submissions are disproportionally likely to be funded due to the prior submission/review rounds. Whether this is due to improved grant crafting, additional preliminary data, better approaches, gradual convincing of a study section or Program is not critical here, I'd say all these contribute. If I am correct, then there is value in continuing to work the steps by resubmitting a triaged A0.
__
Additional Reading:

NIH Historical Success Rates Explain Current Attitudes

More data on historical success rates for NIH grants

Old Boys’ Network Favors Men’s Continuing Grants?

35 responses so far

Repost: Estimating the Purchasing Power of the NIH Grant

A query on the Twitts today:

reminded me of this post. It originally went up 12 July, 2012.

This reality is echoed in personal anecdote. If I look across my grant submissions within a particular part of the lab over the years, I am more or less proposing the same scope of work in each R01. I started submitting grants within the first few years of the modular budgeting era and was matching my proposals to what could be accomplished within the $250K limit. Time marched on...but it took me a long time to cotton on to the purchasing power issue. I just squeezed and tried to compensate by proposing new projects. Because of the considerably reduced hit rate, I've taken to doing traditional budgeting lately. And, what do you know? It comes in at about $375K. Same scope as I used to fit within the $250 limit.


You are probably aware, DearReader, of the concept of inflation. This means that the amount of money that you pay today for a good or service is higher than the amount of money that you paid yesterday.

On average.

So for example, this US inflation calculator tells me that the purchasing power of $12,000 in 1972 has the purchasing power of $65,975.60 in 2012. This is a convenient set of figures if, for example, you are shooting the breeze with a senior faculty member* who started his or her Assistant Professor appointment in the early 70s. You may want to grapple with pay on even terms. Naturally, not every good or service has the same inflation rate and this is just one model/estimator. Jeans may cost less and houses may cost more. etc.

Moving along, we come to the discussion of NIH Grants. In the past I've posted the analysis that shows that the doubling of the NIH budget was rapidly un-doubled and fell back on the historical trend line. [see update suggesting we are now defunding the NIH] That analysis depended on the Biomedical Research and Development Price Index or BRDPI. This brings us to an interest in the purchasing power of the full modular R01. "Modular" refers to the specification of the budget for most NIH grant types in units of $25,000 in direct costs. These are the "modules".

There has been a cap of $250,000 per year in direct costs since the 6/1/1999 initiation of this structure, if I have that right. You can ask for more money per year but then you revert to a line-item type budget (called "traditional budgeting"). The modular cap has not changed and, I assert, this limit affects the vast majority of NIH R01 proposals since there is high motivation (or has been, I may have touched on reasons for future changes before) to adhere to the modular grant structure. Overall, I do like the notion of the modular budgeting procedures because it keeps reviewers from ticky-tacking a bunch of irrelevancies about grants when they should focus on the science.

However, the use of a limit like this brings up the unpleasant inevitability of inflation.

Comrade PhysioProf has been noting that the real purchasing power of the R01 has been dropping due to inflation in the context of postdoctoral fellow demands for ever increasing salaries. He's not alone in noticing. I offer today, a graphical depiction pulled from data provided by the NIH Office of Budget on the BRDPI.
I"ve taken their table of yearly adjustments and used those to calculate the increase necessary to keep pace with inflation (black bars) and the decrement in purchasing power (red bars). The starting point was the 2001 fiscal year (and the BRDPI spreadsheet is older so the 2011 BRDPI adjustment is predicted, rather than actual). As you can see, a full modular $250,000 year in 2011 has 69% of the purchasing power of that same award in 2001.

For those looking at the increasing numbers of applications being submitted presented in the prior post, you must include some understanding of this inflationary pressure in your thinking.

The second thing we've found here is the target number to restore spending parity.

In simple terms, we should now be advocating for an increase to $350,000 as the new modular cap.

__
*Particularly handy when said senior (or emeritized, retired) faculty members are members of one's own family. just sayin.

26 responses so far

Older posts »