Archive for the 'Ask DrugMonkey' category

Structural Aspects of Revising Your NIH Grant

A recent query to the blog emailbox echos things I occasionally see on the search traffic leading to the blog and that I deal with IRL. It asks about the more structural aspects to revising a NIH grant application that has been unsuccessful. In the most recent case it is coming from someone pretty junior who has been asked by the PI to learn how to accomplish this task. (I think that's weird, of course, this is the PI's job in my view but so it goes.)

One of the best starting places for getting in the proper mindset to revise a NIH grant is to think about the review process. There are a couple of key aspects that I didn't pick up on until I was actually doing reviews and a couple that I knew but didn't understand how to incorporate that information.

You are aware that in most cases your grant will go back to the same study section...but sometimes it will not. It will likely go back to at least one reviewer that has seen it before, but also to at least one reviewer new to the application. It is not impossible to get all three retreads or all three new, btw. The reviewers will see the summary statement from the prior review but not the application itself. This last is absolutely key.

You also have to keep in mind that any prior reviewers (and it is not impossible that members on the panel may have read your application closely even if not assigned as a reviewer) may of course remember your proposal. They may also have, illegally, retained a copy in their files to which they may refer.

With that as background, structurally speaking the revised application is duck soup. Here's what I recommend based on my own approaches and a distilled impression from reviewing grants that I am far from alone in this approach.

At present you have a one-page "Introduction to the Revised Application" to work with. Before the shortening of the R01 application, you had three. Space is most assuredly at a premium in the present era.

You should start off with a sentence to the effect of "This is a revision of IC031666 reviewed in Panel VWXYNot in Feb of 2011 where it received a priority score of 31 and a percentile rank of 26%".

Recall that while any reviewers who were present at the prior review know the post-discussion score range, they do not know how the mean of the panel went down, nor the all-important percentile rank. I think it a good idea to get this in their minds. Yes, yes, scores are not supposed to be benchmarked on the prior score but let me tell you this is a nearly inescapable psychology of some reviewers.

Next you will be inclined to polish the apple a little bit. Don't. You simply don't have room for that crap, nowadays.

The less-obvious no-no is that you will be inclined to reiterate some of the more glowing and approving comments made by the reviewers. I used to do this...

"We are gratified that Reviewer #2 found the PI's laboratory 'uniquely qualified' for the studies and Reviewer #1 thought the Approach was elegant and ideal for..."

...and got smacked down for it by a senior colleague who had study section experience. This is where your understanding that the reviewer has the summary statement right on her desk next to your revised application comes in handy. They can read the good bits and heck, they might have written those themselves. Cut to the chase.

I shouldn't have to mention this but also resist the urge to talk schmack about the prior review(ers). This doesn't go well.

The rest is, structurally speaking, quite simple. It should be a listing of the most-important and/or most-consistent criticisms, one by one, with your reply underneath. I like to set quoted material in italics and then answer in plain font. You can do this with a line in the margin (meh) or with font face (yuk) but I like italics better. I also edit this down to a few phrases that communicate the point and combine the same criticism from multiple reviewers if applicable- gotta save space. Remember, they have the summary statement.

Identifying which are the most critical comments is up to the Investigators and it is very hard to set general principles or advice here. Obviously you'd be best off if you can reply to anything that looks like a knock on your prior version but space is limited. Having more-experienced colleagues read your summary statement and draft Intro can be helpful here.

Likewise, the content of your response is going to be up to the criticism, your proposal and your situation in general. My generic advice is to give them something. Throw the reviewer a bone, even if you can't deliver the response they probably want to see. Never, ever totally stiff a comment by saying there is no way in hell you are going to do it. That is a surefire way to another crappy score.

Where possible it is nice to point the reviewer to where you made the discussed changes. A few parenthetical references to "see Innovation" or "Specific Aim 2" goes a long ways here.

Try as hard as you can not to blow off a criticism that seems important. If it shows up in the Resume of Discussion you'd damn well better have a response. Ditto if your conversation with your PO after the review revealed a major issue of discussion.

Then you end (or possibly end the first paragraph before you get into the point-by-point) with the comment that major changes in the proposal are indicated by a line in the margin (I find this the most readable) or italics or altered font or something. If there really is wholesale revision, you can say this and omit the indication of revised bits. But in most cases you are going to have a few key passages and design features, perhaps some new data, and you want to draw the reviewer's eye to what is new. Remember, that she does not have access to the prior version of the proposal...and may not have ever seen it before anyway.

These changed bits will hopefully correlate directly with the items you have listed in your point-by-point and indeed with other criticisms that have not made the cut for the one page Intro to the Revised Application.

The quality of your response to the prior criticism is a major factor in review. You do not want the reviewer in any doubt as to just what you have changed. Fortunately, the structural part is relatively easy.

It is only the content of your revision that should have you sweating bullets.

13 responses so far

Strike a balance, New Investigator

Apr 22 2011 Published by under Ask DrugMonkey, NIH Careerism, Science Publication

A question arose on the Twitts recently where a newish investigator wanted to know if it was wiser to push earlier for his/her first senior author paper in a ~3-5 IF journal or hold out for a ~9 IF journal submission. It emerged during the ensuing Twitting that the person had about 10 months of some degree of independent funding under the bridge so far.

I had a post awhile ago (quite awhile) that I thought would cover my main point but it only touches on the idea of balancing your attack. The opening comment sets the right tone though.

My suggestion is, if you expect to have a career you had better have a good idea of what the standards are. So do the research. Do compare your CV with those of other scientists. What are the minimum criteria for getting a job / grant / promotion / tenure in your area? What are you going to do about it? What can you do about it? Don't misunderstand me- nobody is going to hand you a job / grant / etc just because you hit the modal publication numbers. But it will be very easy for you to be pushed out of the running if you do NOT hit the expected values. So do what you can to keep your CV as competitive as possible.

And this, really, is the starting point to my answer to @chemstructbio. It is absolutely critical to understand within your own subfield, within the pool of investigators whom you consider to be your peers, what the standards are. Particularly when it comes to Impact Factor. In my fields of interest, the answer to this question is going to be quite simple. If you have zero senior author papers from your lab, and you have been at it with funding for coming up on a year or more, the priority for a 3-5 IF paper is absolutely acute*. If this is in reach and a ~9 IF is a stretch of more than a single review cycle, do the lower IF one. Now.

The reason being the idea of balance, and the difficulty your advocates have making a case for you when you give them nothing to work with. A society level journal publication is respectable for all but the snootiest of assclams. Respectable. An advocate can work with this. Published data can be argued on the merits. It is very hard to argue with nothing.

Yes, yes, yes. We all know what time it is on the street and how long it takes to get to that first ass-kicking paper. But when you are sitting there with the Biosketch.....evidence is the thing. Evidence of a published, peer-reviewed paper. Not in prep, not in submission. Accepted.

Now, if you have one already....then it is time to start balancing numbers against the IF against the real impact of the paper. Then you can afford it. But if you have none, my friends, it is time to get one.

__
*This is not saying that you are hosed if you do not have a senior author paper. Not at all. It's just that it makes things go better. So if one is in reach, make it happen. It should be a huge priority, particularly if you have a little bit (or a lot) of funding in hand already. Because the reviewer demands for evidence of independent productivity will ramp up...clock's aticking.

15 responses so far

More NIH Grant criterion score fun, writ small

As we were just discussing on the Sb blog, the Approach, Significance and Innovation criterion scores are the biggest drivers of Overall Impact Score. Approach remains the king. Or, at least the Approach score correlates best with the Overall Impact score voted for some 32 thousand research grant applications that made it to discussion for the 2010 Fiscal Year.

A good friend of the blog submits the following outcome of a recent grant review. The grant was triaged, thus no discussion and no overall score. However each of three reviewers issued a putatively non-triage score (2-3) for one of the three big criteria. (As per our aforementioned discussion, the Investigator and Environment criterion scores were 2s or better. I told you they matter very little to the outcome!) As you might anticipate, the reviewers also each bagged on (4-6s) the other two remaining important criterion scores.

But here's the funny part. The three reviewers each picked a different one of the Approach, Innovation and Significance criteria to laud.

So I should advise her to get Scarpa on the phone pronto to complain about the clearly erroneous review, right?

32 responses so far

On submitting a grant application from a University you plan to leave

A bit of confusion has arisen on the Twitts over who can serve as the PI of a grant application submitted to the NIH, who "owns" the award and what the implications are for moving the award to another University.

For a highly related topic I recommend you re-read my old post Routes to Independence: Beyond Ye Olde Skool Tenure Track Assistant Professorships (original).

To distill it to a few simple points for the current discussion:

  • The University (or Research Institution, company, etc) submits the grant to the NIH and receives the award from the NIH.
  • Anyone who the submitting institution deems to be a PI can serve as the PI. Job title or status is immaterial as far as the NIH is concerned.
  • Postdocs, Research Scientists, Staff Scientists, etc can be the listed PI on most broad NIH mechanisms (there may be the occasional special case like MD-required or something).
  • The submitting institutions, for the most part, permit anyone of tenure track professorial appointment to prepare NIH grants for them to submit but it gets highly variable (across institutions, across their respective non-professorial and/or non tenure track...and across time) after that.
  • The question of how study sections view applications submitted by those of other than tenure track professorial rank is a whole 'nother question, but you would be making a mistake to think there are hard and fast exclusive principles.

The second issue has to do with moving the award to another institution, given that a PI on an NIH award decides to go somewhere else. Although technically the University owns the award, in the vast majority of cases that institution will relinquish the award and permit it to travel with the PI. Likewise, in the vast majority of cases, the NIH will permit the move. In all cases I am aware of this move will occur at the anniversary of funding. That is because the award is in yearly increments (maximum of 5 unless you win a PECASE or MERIT extension* of the non-competing interval). Each progress report you submit? That's the "application" for the next year of funding. Noncompeting application, of course, because it does not go back to study section for review. At any rate it makes it less painful for all concerned to do the accounting if the move is at the anniversary.

Soooooo.....

Point being that if you are a postdoc or non tenure track scientist who wants to write and submit a grant, you need to start snooping around your local University about their policies. Sometimes they will only let you put in a R21 or R03 or some other nonrenewable mechanism. Sometimes they'll let you throw down the R01. Just depends. Most of the time it will require a letter of exception to be generated within the University- Chair or Dean level stuff. Which requires the approval of your current lab head or supervisor, generally. You need to start talking to all these people.

Since these types of deals are frequently case-by-case and the rules are unwritten, don't assume that everyone (i.e., your PI) knows about them. Snoop around on RePORTER for awards to your institution and see if anyone with non-TT professorial appointment has ever received an award from the NIH. Follow up on that rumour that Research Scientist Lee once had an award.

If you are really eager, be prepared to push the envelope and ask the Chair/Dean type person "Well why not? University of State1 and State University2 and IvyUni3 and Research Institute4 all permit it, why can't we?". This may require doing some background surveying of your best buddies spread around the country/world.

Final point:
Obviously I wouldn't be bringing up these theoretical possibilities if I hadn't seen it work, and with some frequency. As a reviewer on a study section I saw several applications come through from people who had the title of something below tenure track assistant professor. Instructor, Research Scientist and yes, even Postdoc. I myself submitted at least two R01 applications prior to being able to include the word "Professor" on my Biosketch. I have many peers that were in a similar circumstance at their early stage of grant writing/submitting and, yes, winning.

No, you will not be treated just like an Assistant Professor by the study sections. You will be beat up for Independence issues and with doubts about whether this is just the BigCheeze trying to evade perceptions of overfunding. You will have "helpful" reviewers busting on your appointment as evidence of a lack of institutional commitment that the reviewer really thinks will get the Dean or Chair to cough up a better title**.

In all of this however there is a chance. A chance that you will receive an award. This would have very good implications for your transition. (Assuming, of course, that you manage to get the grant written and submitted without too big of a hit to your scientific productivity, never forget that part.) And even if you do not manage to obtain a fundable score, I argue that you get valuable experience. In preparing and submitting a half-decent proposal. In getting some degree of study section feedback. In taking a shot across the bow of the study section that you have ideas and you plan to have them review them in the coming few years. In getting the PO familiar with your name. In wrangling local bureaucracy.

All of this without your own tenure clock running.
__
*there may be other extensions I am unaware of.

**One of the first questions I asked an experienced reviewer about after joining a study section. Sigh.

12 responses so far

Do you expose your trainees to the NIH Grant sausage making?

Jun 24 2010 Published by under Ask DrugMonkey, Careerism, Mentoring

Laws are like sausages, it is better not to see them being made. -Otto von Bismarck. (or maybe it was John Godfrey Saxe)

Our longtime commenter bsci recently asked:

DM, This brings up a suggestion for a potential future posts. What DO you do to train your mentees for academia? Do they get to read your grants? Comment? Write parts of an R01? I assume you have them submit NRSAs, but merely submitting isn't a training experience. Have you found ways to improve the educational utility of the process?

Let me answer this last question first. I have no idea if I am improving or impairing the "educational utility" of the training I provide. I just don't have the numbers. There are many differences in the motivations and desires of trainees, these motivations shift significantly from the beginning to the end of a typical training stint and if a job is the outcome measure, then we are all at the mercy of a varying job market.
The grant part, however, I can answer.

Continue Reading »

19 responses so far

You don't get to define your job. Is that a problem for you?

Jun 23 2010 Published by under Ask DrugMonkey, Careerism, Mentoring, Tribe of Science

You will probably have noticed by now, DearReader, that the NIH grant game is not exactly a distasteful part of my job. Don't get me wrong. I'd be much happier if I had landed in some hard-salary situation with exceptional institutional support, local funding sources procured by the philanthropy side of the institution and just generally had fewer concerns about actually funding my laboratory.

That didn't happen, however. I landed in a job which requires me to be at least minimally competent at acquiring major research funding. I was not particularly prepared for this.

Continue Reading »

15 responses so far

Ask DrugMonkey: Has the revision strategy changed for NIH Grants in the A1 era?

May 27 2010 Published by under Ask DrugMonkey, Careerism, Grant Review

Occasional commenter crystaldoc left a query on an early post.

Given recent changes in the NIH process (in particular only one chance for resubmission, and less information in the tea leaves of the new summary statements), when does it make sense to resubmit? When does it make more sense to change it up and put in a new submission? Any pragmatic advice or guidelines based on impact scores or percentiles? How often are A1's funded when the original submission was streamlined? Or at 40-50th percentile, or 30-40th percentile? Are these data available anywhere?

I offered up a prior post in which I posted some longitudinal data on the funding of NIH grants unrevised and at the A1 and A2 revision stages.
crystaldoc was not impressed.

Continue Reading »

No responses yet

#proofwaltdisneysmokedpot

Apr 30 2010 Published by under Ask DrugMonkey, Cannabis

I made the idle observation on Twitter yesterday that someone had found this blog by searching for the string "proof walt disney smoked pot". My initial read:

search words finding the blog: "proof walt disney smoked pot". Really people? Really?

Apparently some folks are amused by this concept.

No responses yet

Your academic society is working for (or against?) you under the NIH Grant waterline

A recent blog entry from Pascale H. Lane discusses her reasons for belonging to academic societies. Our good blog friend Dr. Isis is frequently found to be going all fangrrl about the APS (no, not the real APS, these Physiological pretenders who are well down the GoogJuice list). Pascale touched on one Golden Thought about what academic societies can do that is, or should be, of general interest to my readership:

The society maintains several grant programs for research funding, and it leads advocacy efforts to maintain adequate federal funding for kidney disease research and treatment.

Grant $$$! Wooot!

Continue Reading »

39 responses so far

Your Grant in Review: The outlier proves I need to appeal!!!!

Mar 10 2010 Published by under Ask DrugMonkey, Grant Review, Grantsmanship, NIH

I've been meaning to pick up on a comment made by a reader over at writedit's epic thread on NIH paylines, scores and whatall. (If you want to swap war stories and score/IC payline grumbling, that is the hot place in town.) The guy was ticked off about a recent review he received and had a question:

I am an establishe investigator. I subnitted a competing renewal ... I got a score of 40 (37 percentile). I was very shocked and dissapointed to find out that my application had a preliminary score of 2.7 (which would have been fundable) but it seems one negative reviewer carried the day, and convinced others to pull down the score. I have not yet seen the comments, but if the comments have factual errors, especially from the negative errors, can I appeal the review and request a re-review?

Recently, as luck would have it, a loyal reader of the blog submitted the following scores, received on the review of her R01 grant proposal. Under the new scoring procedures in place since last June, these are scores which each reviewer suggests for criteria of Significance, Investigator, Innovation, Approach and Environment. I may have slightly re-ordered specific scores for concealment purposes but this is essentially the flavor.
rev#1: 2,1,1,1,1
rev#2: 2,2,3,3,1
rev#3: 3,2,5,4,2
It really is always Reviewer #3, isn't it?

Continue Reading »

No responses yet

« Newer posts Older posts »