Archive for the 'Grant Review' category

GrantRant II

Jan 04 2013 Published by under Grant Review, Grantsmanship

You know when you are faced with using somebody's crappy bit of code but you could just write the whole thing from scratch? But then you'd have to 'splain to the boss man why you spent all that effort doing by a totally different way? And the client will be pissed....and the coding team will be pissed...and basically it all just sucks ass. But you can't bear to let the cluster borkage mess exist and still call yourself a professional?

Reviewing a revised NIH grant that you didn't review for the original submission is a little bit like that.

Share

28 responses so far

GrantRant

Jan 03 2013 Published by under Grant Review, Grantsmanship

If I do not have any idea, within about three sentences into your Specific Aims page, what model systems, subjects and broad experimental approaches are going to be in your proposal you are seriously screwing up your grantsmithing.

Share

26 responses so far

Sample R21 grants with Summary Statements from NIAID

Apr 26 2012 Published by under Grant Review, NIH, NIH Careerism, NIH funding

As I previously noted (somewhat critically) that the NIAID had posted sample R01 grants and the corresponding summary statements. Well, they've added some R21 applications to the page.

Again, I wonder how useful this really is for most applicants. First thing you notice is that it takes a perfect score to get funded. Three of the four received 10s and the fourth limped home with an 11. Remember, the study section score range starts at 1, which is then multiplied by 10 after the voting of the entire panel is averaged.

Then there's this (emphasis added):
From the Dow summary statement's resume of discussion: "Strengths of the application include the accomplished investigator and research team, strong preliminary data, the direct doable and logical set of experiments, and the likelihood of paradigm shifting insights into meliodosis"

From the resume on the Starnbach app: "Strengths of the application include the innovative use of the novel GPS strategy, compelling preliminary data, an investigator with a strong bacterial pathogenesis research track record, an excellent and appropriate set of collaborators, and a high degree of confidence that import results will emerge from these studies."

Weis, individual critique #2: "Strong and compelling preliminary data is presented that indicate a high likelihood of success"

Well, at least NIAID is telling it like it is with these examples.....

Share

17 responses so far

In NIH Grant Review, "Help" For the Applicant is Incidental

Apr 03 2012 Published by under Grant Review, NIH Careerism

This is my understanding, anyway. The job of the reviewer of the NIH grant application is to discuss the merits and weaknesses for the audience of the Program Staff within the ICs. One is supposed to be helping them to make funding decisions with respect to the current pool of applications.

The job is not to help the applicant improve her chances the next time. Nor is it to help the application be more attractive for the PO/IC.

This is my understanding, anyway.

See ProfLikeSubstance's comment.

Share

14 responses so far

This doesn't sound "mean" to me

Mar 27 2012 Published by under Grant Review, Grantsmanship, NIH Careerism, NIH funding

Some commenter at Rock Talk complained about a recent grant review:

I just received the most terrible of reviews, where the reviewer was not only biased but highly inflammatory, prejudicial and aggressive. I must say I was totally taken aback. When you say things like “…terribly convoluted approach”,…”PI has clearly no clue…” how something works, trashes my published work by saying these pubs “are a gross exaggeration”….the list goes on and on. Even as a relatively senior investigator, I was very shocked by the mean-spirited nature of the comments. I cannot imagine how it would destroy a new investigator.

I am having trouble seeing it. I mean sure "no clue" is directed at the applicant rather than the application, but it's pretty tame stuff. If a reviewer thinks your papers exaggerate? Presumably in wild speculative interpretation that runs beyond your data? Seems okay and even obligatory to express this. The "terribly convoluted approach" comment is a pretty inoffensive way to get to the heart of this common failing of grants as well...I'm not seeing how you could put it more "nicely".

Share

43 responses so far

NIH Advice for Noobs on the Twitts today

Mar 19 2012 Published by under Grant Review, Grantsmanship, NIH Careerism

Your roving reporter, @doc_becca (of the Fumbling Towards Tenure blog) has a few observations rolling under the #NIHsekrits hashtag on the Twitts today. Check it out.

Share

7 responses so far

Perfection

Mar 14 2012 Published by under Careerism, Grant Review, NIH Careerism, NIH funding

The notion that there is some perfect pedigree, some perfect CV that most applicants to the NIH (or other funding body, I assume) possess is untrue.

Oh, it can be comforting, I realize. To think that if you only had picked the right doctoral or post doc lab, if only you had been able to move across the country, if only that damn PI had written you a better letter and gotten you that job at HighFalutinU....

If only.

But for those slings and arrows of outrageous fortune, you too would get every application funded. Nary a triage and seldom a revision.

Hahahahahhahahahahaaaa!!!!

Get real.

Many, many funded PI's of my acquaintance have holes in their CVs that you can drive the ParanoidApplicantTrain through. The wrong doctoral Uni/program. Dismal productivity in a crucial stop. No post doc. Too many postdoc years. A less-than appointment. Seeming lack of independence. A mid career research drought. Low IF pubs. A scientific diversion. Too narrow a scientific focus. Too diffuse.

The thing is...the extramural, NIH-funded community is diverse on this axis. It permits a lot of room for Investigators who differ from the "ideal". To the degree that the "ideal" is more of a fantasy of the unsuccessful applicant than it is a reality.

My charge to you is this: Ignore your seeming deficits.

Ignore the inner voice telling you that easy street was back there on the path not taken. Or the path that was barred to you.

Focus instead on crafting the story of you, as a scientist and investigator. How did your experiences make you the independent scientist that you are?

Remember that you are not talking to your detractors but rather to your advocates.

Share

21 responses so far

Reviewing NIH Grants: Innocent until proven guilty?

Feb 24 2012 Published by under Grant Review, NIH Careerism, NIH funding

Commenter Physician Scientist notes on a prior post that an individual scientist under suspicion for several dubious papers has retained his NIH funding.

OMG

This grant was RENEWED!!!!

Not quite. Or maybe the timeline is not quite what it seems. This would appear to be the most recent competing award in 2009. The budget is listed as ending in 2010 but then it continues onward under the "7R01" code in the next year (indicating a change of University) until 2014.

So the objection may not be all that direct if the news of these alleged frauds, misdeeds and/or actual retractions and corrections hadn't been known when the grant was reviewed.

However, davebridges continues from my more general query about whether PIs should be viewed as innocent* until proven guilty of fraud.

innocent until proven guilty, but im sure reviewers sure can take into account historical accuracy of a lab. Better to renew a grant of a good lab with an instance sloppy record keeping (if thats the case) than a non-retraction lab whose data is never reproducible

This brings up a related, and more pernicious, issue. In my limited experience, "lab whose data is never reproducible" tends to be the stuff of rumor. Word around the campfire. Suspicion. Widespread far beyond those who might actually have tried seriously to replicate said data.

Correct or false, rarely is it a matter of well-explicated, scientific lack-of-evidence. Which, in itself, would still be problematic. There are many Nobel prizes and other fantastic scientific discoveries with a back story of "nobody believes his data". At least at first, but that could have continued for years or decades. But if it is only suspicion? even if there are a couple of retracted papers....

Should the grant reviewers bust on an application on this basis? If there was one retracted paper would you refuse to issue a fundable score, even if the application had little to do with the topic of the retracted work?

What if you've read some internet clown detailing the "obvious" duplications of figures in papers but they've yet to be retracted or corrected? Would you mark the present application from that PI downward?

The flip side is that nobody deserves NIH funding. It is a privilege that is getting rarer all the time, going by the success rates. Seemingly the proposals that make it over the bar are held to the highest standard. As we've noted repeatedly, there are LOTS of great applications which are not going to get funded.

So why should we (the system) tolerate even a whiff of impropriety? Why not apply the one-strike and yer out principle?

As you know, we had one major ass retraction in the substance abuse fields in recent memory. Major because of the profile and public interest rather than because it had broad influence on the other scientists. I mean sure, maybe people were trying to replicate and follow up but the retraction came out within a year. Not too much damage was done**.

As far as I can tell Ricaurte kept his grants and kept getting more of them. Never paid any obvious price. Was this right? Should he have been busted out of the business for something over which we still do not know, and will never know, the extent of culpability. Should the reviewers simply moved on to a less tainted individual?

I don't know. All I know is what I would do as a reviewer which is to try to be as fair as possible and to rely on my fellow panel members to reach consensus over how retractions or more suspicions should be viewed.

What would you do?

__
*from what I can tell in the chatter, this Chu case is limited to suspicion and a few retractions so far?

**yes, if you were the one wasting a year of work it sucked.

Share

15 responses so far

Why would you want to leave points on the table when your NIH Grant is reviewed[1]?

Jan 24 2012 Published by under Grant Review, Grantsmanship, NIH Careerism

It is well established that approximately 85.473% of the battle when it comes to NIH Grantsmithing is making it easy for the reviewers to grasp your point[2].

Really easy.

Many bloggers have described the fact that you need to think about your audience. The grant reviewer in my mind's eye is an overworked, grant-stressed, paper-decision-major-revision, lab disaster supervising PI who has finally cracked your grant open at 10 pm after putting his kids to bed, throwing some laundry in, running the dishes and making lunches for tomorrow's schoolday. . Plus, she may have an infant squirming and latching on poorly at the same time. Still licking wounds from the last disappointing summary statement from his or her own grant application.That's if you are lucky.

If you are unlucky she is getting serious about reviewing your grant while crammed in coach on the way to Bethesda for the actual meeting. And is planning on submitting the written critique the moment she hits the free wifi at the airport....

Then, should your application be so lucky as to be discussed, realize this, Dear Reader. The three assigned reviewers may have had the time, if they had the inclination, to pour over your application at leisure.

The rest of the panel? Not so much.

It is the rare reviewer who has read through the entire panel's worth of applications in detail. And if there appears to be a disagreement between the three assigned reviewers during discussion, these other reviewers have something on the order of 10-15 minutes to scan through your application to attempt to resolve matters in their own mind.

You want their job to be as easy as possible. This is why I hammer away at the creation of blank space, headings, clear figures and direct writing. You know this. What many people don't seem to realize is that your selection of citation style matters as well.

A lot.

Some folks seem to think that it doesn't matter[3] or that the NIH insists on numbered citation styles[4].

[As a bit of a sidebar, before I go too deep, there is absolutely nothing about grant review that is fixed in stone. So if you know people who always use the numbered citations and get funded[5], bully for them. But realize they got the award despite a lapse in grantsmithing[6], not because it is unimportant.]

Back to our story, it appears some of this confusion may be attributable to Endnote, as one authority claims that the Endnote style for "NIH" is numbered[7,8]. I have recently tested this on my Endnote X4 for Windows and I find this assertion to be untrue. My style for NIH (default installation) is Author, Date style, not numbered.

but still. If you have some version of Endnote that has a particular style for "NIH" don't you think you'd better check to see what the rules say? I mean, you could have a template grant from some idiot that uses Georgia font but you know to put it in the Arial, no? Or if your sample page is in 12pt font, you go and see what the rules actually say, right? So why would you take what some non-NIH entity says about the style for citation? I mean, you could ask the Twitts[8a]...but why not check the source [9].

Provide a bibliography of any references cited in the Project Narrative. Each reference must include the names of all authors (in the same sequence in which they appear in the publication), the article and journal title, book title, volume number, page numbers, and year of publication.

emphasis added- no reshuffling of "co-contribution" authors permitted!

also,

When citing articles that fall under the Public Access Policy, were authored or co-authored by the applicant and arose from NIH support, provide the NIH Manuscript Submission reference number (e.g., NIHMS97531) or the PubMed Central (PMC) reference number (e.g., PMCID234567) for each article. If the PMCID is not yet available because the Journal submits articles directly to PMC on behalf of their authors, indicate “PMC Journal – In Process.”

That's it though. No mention of a required citation style or of a bibliographic style, so long as the pertinent information is included.

Now, I know why you are tempted to use the gawdawful numbered citations. You think you are saving space and can squeeze yet more blabbedy-blah text in there to overhwhelm the reviewers.

Resist the urge.

It is hard to read academic text with numbered citations if you are actually using the citations to create the argument in your mind. Now maybe I am weird. But for me, especially in my subfields of interest, Name, Date for a paper citation is sufficient cueing to dredge up any number of details about the paper in question. With a mental image of the findings, interpretations, etc. Call me crazy, but your random schema for numbering the references doesn't do the job.

Even if I flip back and forth between my place in the text and the reference list, I've probably forgotten what the hell [254] refers to a paragraph later. There is a prayer that (Schmelmitz et al, 1968) will trigger me to remember the paper without any need to flip to the citation list at all. And I'll likely remember which one you mean, if I've flipped back to check the full citation, for longer than a few sentences.

Oh, and speaking of flipping to the Bibliography. Much easier to do with hardcopy grant applications. Much harder to do when reading the PDF on a computer. Maybe there are some fancy double pane views but I sure as heck don't use them...are you gambling your average reviewers knows some way to do this?

So numbered citations are a clearly inferior way to cite an academic work. In fact, when I am the boss of science, whomever came up with this or perpetuates it as a good way to cite papers is going to be in the second group up against the wall[10]. They make it hard for the reader to figure out what work you are citing. Let's face it, you aren't wasting the space to explain the prior finding, either, are you? You are using the mere citation itself in hopes that it makes the point for you! Why would you want to make it hard on the reviewers? It's only going to annoy them.

__
[1] Drugmonkey, 2012

[2]DrugMonkey, 2012

[3]Dr Zen, 2012

[4]Luminescer, 2012

[5]Luminescer, 2012

[6]Drugmonkey, 2010

[7]ericsuh, 2012

[8]ericsuh, 2012

[8a]Namnezia, 2012

[9]Application Guide for NIH and Other PHS Agencies

[10]Unless they are also a GlamourMag editor, then they go with their regular group[11].

[11]"First"

Share

59 responses so far

Nip this in the bud right friggin now, CSR!

Nov 09 2011 Published by under Grant Review, NIH

There was a rumor on the Twitts today about a study section experience in which the triage line was severe (35%, yikes)...but that isn't the bad part. In a 10% or worse payline environment, discussion of the top 35% seems appropriate.

What was absolutely horrifying was the blocking of the usual (IME) rule that any reviewer could lift any application out of triage and insist on discussion.

I believe this to be an absolutely fundamental safety valve to avoid the frequently bemoaned notion that "one biased reviewer torpedoed my grant".

I myself have found occasion to pull grants up for discussion. I doubt I ever got those particular versions funded but I know for sure of ones that got funded on a subsequent revision. I can't prove that my pulling it up for discussion led to the favorable outcome for the revision. But I think it safe to assume that if this path is replicated elsewhere in CSR then, statistically, this has an effect.

Don't get me wrong, one reviewer saves are going to be rare...but what if it were your grant?

Share

8 responses so far

« Newer posts Older posts »