Archive for the 'NIH' category

Grant Review Site Visits

...need to be ended.

They represent a huge risk for bias dependent on the personal characteristics of the investigators to rule the day.

Are you an older, white-haired, heteronormative appearanced, able-bodied picture of "Scientist and Professor"? Great!

Are you overweight? Do you stutter? Express unexpected gender presentation? Nonwhite? Female? Are your language skills less than native to the reviewer's ears? Too young? Too hot? Not hawt enough?

Should your grant proposal be affected strongly by the direct face to face impression of these characteristics?

No.

__
H/t: @jwoodgett

2 responses so far

Your Grant in Review: The Biosketch Research Support Section

Apr 21 2014 Published by under Grant Review, Grantsmanship, NIH, NIH funding

A question came up on the twitts about the Research Support section of the NIH Biosketch.

The answer is that no, you do not. I will note that I am not entirely sure if this changed over the years or if my understanding of this rule was incomplete at the start. However, the instructions on the Sample Biosketch [PDF] provided by the NIH are clear.

D. Research Support
List both selected ongoing and completed research projects for the past three years (Federal or non-Federally-supported). Begin with the projects that are most relevant to the research proposed in the application. Briefly indicate the overall goals of the projects and responsibilities of the key person identified on the Biographical Sketch. Do not include number of person months or direct costs.

The last bit is the key bit for Dr24Hour's question but I include the full description for a reason.

dr24Hours also asked:

and there was a followup to my initial negative response

Together, these questions seem to indicate a misunderstanding of what this section is for, and what it is trying to communicate.

Note the use of the term "selected" and "most relevant" in the above passage.

The Biosketch is, in total, devoted to convincing the reviewers that the PI and other Investigative staff have the chops to pull off the project under review. It is about bragging on how accomplished they all are. Technically, it is not even a full recitation of all the support one has secured in the past three years. This is similar to how the Peer-reviewed Publications section is limited to 15 items, regardless of how many total publications that you have.

Inclusion of items in the Research Support section is to show that the Investigators have run projects of similar scope with acceptable success. Yes, the definition of acceptable success is variable, but this concept is clear. The goal is to show off the Investigator's accomplishments to the best possible advantage.

The Research Support is not about demonstrating that the PI is successful at winning grants. It is not about demonstrating how big and well-funded the lab has been (no direct costs). It is not even about the reviewers trying to decide if the PI is spread too thinly (no listing of effort). This is not the point*.

In theory, one would just put forward a subset of the best elements on one's CV. The most relevant and most successful grant awards. If space is an issue (the Biosketch is limited to 4 pages) then the PI might have to make some selections. Obviously you'd want to start with NIH R01s (or equivalent) if the application is an R01. Presumably you would want to supply the reviewer with what you think are your most successful projects- in terms of papers, scientific advance, pizzaz of findings or whatever floats your boat.

You might also want to "selectively" omit any of your less-successful awards or even ones that seem like they have too much overlap with the present proposal.

Don't do this.

If it is an NIH award, you can be assured at least one of the reviewers will have looked you up on RePORTER and will notice the omission. If it is a non-NIH award, perhaps the odds are lower but you just never know. If the reviewer thinks you are hiding something...this is not good. If your award has been active in the past three years and is listed somewhere Internet-accessible, particularly on your University or lab website, then list it on the Biosketch.

Obviously this latter advice only applies to people who have a lot of different sources of support. The more of them you have, the tighter the space. Clearly, you are going to have to make some choices if your lab operates on a lot of different sources of support. Prioritize by what makes sense to you, sure, but make sure you pay attention to the communication you are trying to accomplish when making your selections. And beware of being viewed with a suspicion that you are trying to conceal something.
__
*Yes, in theory. I grasp that there will be reviewers using this information to argue that the PI is spread too thinly or has "too much" grant support.

10 responses so far

Thought of the Day: The NIH Can't Win

Apr 18 2014 Published by under Careerism, NIH, NIH Careerism, NIH funding

A comment over at Rock Talk made a fairly traditional complaint about the NIH funding system. Dan C stated that: "NIH is to be criticized that it funds “usual suspects.

Today, I find this funny. Because after all, most of the people complaining about the NIH system want to become one of the usual suspects!

Right? They want to get a grant, one. They want to have some reasonable stability of that grant funding in a program-like sustained career. Most of them don't want to have to struggle too hard to get that funding either....I doubt anyone would refuse the occasional Program pickup of their just-missed grant.

Once you cobble together a bit of success under the NIH extramural grant system, those who feel themselves to be on the outs call you a "usual suspect". For any number of reasons it is just obvious to them that you are a total Insider (and couldn't actually deserve what you've managed to accomplish, of course). This may be based on the mere fact that you've acquired a grant, because you work in a Department or University where a whole lot of other people are similarly successful. This may be because it appears that POs actually talk to the person in question. It may be because a FOA has appeared in a research domain that you work within.

Anyone sees the duck floating serenely on the water at a given point in time and it looks like this is one most usual suspect waterfowl indeed.

I used to be annoyed at my approximate lateral peers in science who appeared to be having an easier time of it than I did. I had my Insider attributes as a younger faculty member, make no mistake, but I also had considerable Outsider traits, considering where I was seeking funding and for what topics of research. Some of those folks, over there, well boy didn't they get an easy ride because of being such Insiders to the subpart of the NIH system!

I still have those thoughts. Even though I've seen many of the people I thought had it made in the shade go through their dry spells and funding down-cycles. Despite the fact that as each year goes by and my lab remains funded, I become more and more one of the "usual suspects".

I believe that if I ever feel like I am one of the usual suspects, if I feel like I deserve special treatment and stop fighting so hard to keep my lab going that this will be the end.

I advise you to try to retain the same feeling of "outsider" that you feel as a noob PI for as long as you can into your career.

Getting back to the point, however, the NIH simply cannot win with these criticisms. Those who are feeling unsuccessful will always carp about how the NIH just funds "their" people. And if the NIH does happen to fund one of these outsiders, this very act makes them a usual suspect to the next complainer.

The NIH can't win.

14 responses so far

NIH backs down on resubmitting unfunded A1 grant applications

Apr 17 2014 Published by under Grant Review, Grantsmanship, NIH, NIH Careerism, NIH funding

The rumors were true. NOT-OD-14-074says:

Effective immediately, for application due dates after April 16, 2014, following an unsuccessful resubmission (A1) application, applicants may submit the same idea as a new (A0) application for the next appropriate due date. The NIH and AHRQ will not assess the similarity of the science in the new (A0) application to any previously reviewed submission when accepting an application for review. Although a new (A0) application does not allow an introduction or responses to the previous reviews, the NIH and AHRQ encourage applicants to refine and strengthen all application submissions.

So, for all intents and purposes you can revise and resubmit your failed application endlessly. Maybe they will pick you up on the A6 or A7 attempt!

Sally Rockey has a blog entry up which gives a bit more background and rationale.

While the change in policy had the intended result of a greater number of applications being funded earlier,

I really wonder if she believes this or has to continue to parrot the company line for face saving reasons. There is no evidence this is true. Not until and unless she can show definitively that the supposed A0s being funded were not in fact re-workings of proposals that had been previously submitted. I continue to assert that a significant number of PIs were submitting "A0" applications that were directly and substantially benefited by having been previously reviewed in different guise.


As a result, we heard increasing concerns from the community about the impact of the policy on new investigators because finding new research directions can be quite difficult during this phase of their career.

If the true concern here was the ESI or NI, then they could have simply allowed them to pass the filter as a category.

The resubmission of an idea as new means the application will be considered without an association to a previous submission; the applicant will not provide an introduction to spell out how the application has changed or respond to previous reviews; and reviewers will be instructed to review it as a new idea even if they have seen it in prior cycles.

The only way this is remotely possible is to put it in a different study section and make sure there are no overlapping ad hocs. If they don't do this, then this idea is nonsense. Surely Dr. Rockey is aware you cannot expect "instruction" to stick and force reviewers to behave themselves. Not with perfect fidelity.

However, we will monitor this new policy closely.

HA! If they'd decided to allow endless amendments (and required related apps to be submitted as such) then they would have been able to monitor the policy. The way they did this, there is no way to assess the impact. They will never know how many supposed "A0" apps are really A2, A4, A6, nor how many "A1" apps are really A3, A5, A7...etc. So what on earth could they possibly monitor? The number of established PIs who call up complaining about the unfundable score they just received on their A1?

69 responses so far

Side thought on the NIH issuing project grants versus program grants

Apr 16 2014 Published by under NIH, NIH Careerism, NIH funding

I asked a poorly worded question on the Twitts

in which what I was trying to ask was this. From the perspective of awarding NIH grants, does it matter that a given proposal fits into a larger whole? If a brand new investigator, do we assume that he or she is applying for the first grant among many? For the greybeard for whom this might be a last-award, do we recognize that it is the capstone to a lengthy program? For the mid-career investigator do we assume this is only one of the many parts that will eventually form a large body of work?

Or is it all good if this is a singleton? One grant, awarded for 5 years and that is all.

The interesting thing is that nobody on the Twitts thought that I meant this. The answers went to various places- funding from non-NIH sources, relatively inexpensive research that didn't actually require an R01 to be vibrant, the idea of a single R01 that was continued beyond a mere 5 year interval. Many people assumed that what I was really talking about was assessing the merits and qualities of the PI.

After I got done kicking myself for not asking the question properly, a simple thought struck me.

Perhaps the very fact that people assumed I meant just about anything other than a single 5 year award, period, for a given PI was my answer. We do tend to expect that a R01 award fits into a larger research program. It does not stand alone as a single project.

19 responses so far

On 'Rescuing US biomedical research from its systemic flaws'

Apr 14 2014 Published by under Fixing the NIH, NIH

I was alerted on the Twitts

to a Perspective by Bruce Alberts (former Science EIC), Marc Kirschner (BSD), Shirley Tilghman (working tirelessly on "fix the NIH" committees) and Harold Varmus (former NIH Directior, current NCI Director). In Rescuing US biomedical research from its systemic flaws the authors recognize the current dismal state of affairs and issue several calls to action.

You will find nearly everything that has been put on the table here at this blog, at Rock Talking blog and, really, everywhere the discussions are serious about NIH-funded biomedical research. I find much to like in here and of course I disagree with several of their points and/or solutions.

Nevertheless, the overall goals are pretty good. They want to evolve to some more HHMI-like stability of funding, an equilibrium of participating scientists to the available budgets and to generally keep the grant process on improving scientific productivity rather than hampering it.

Overall there are good things being said here. But the specifics will be key. They argue for funding more graduate students on fellowships to give more control to the federal government. They phrase this in terms of "quality control" but they overlook using the bully pulpit to demand training programs actually reduce their output of PhDs! On the postdoctoral side of the over supply issue, they call for raising postdoctoral salaries until staff scientists are an attractive alternative. Then they call for increasing the ratio of staff scientist positions. I agree wholeheartedly but am not sure raising postdoc salaries is the way to go. There needs to be some additional mechanisms to provide a carrot to the expansion of the staff scientist category.

The Perspective also goes after the incentives local Universities, Med Schools and Research Institutes have to employ soft-money faculty. Proposals include going after soft-money job types, indirect cost recovery on faculty salary and some obscure but important stuff about including new construction costs in the IDC calculation.

Their recommendations for peer review changes are a mixed bag. They want to dismantle geographical diversity requirements for study section panels (O Rly?) and increase the number of oldsters on the panels. There is some of the usual blah-blah about risk taking, innovation, longer term outcome expectation and identifying the "strongest candidates for support".

What I don't like about these proposals is the stench of elitism that encircles them. As I have said repeatedly, one of the design virtues (even if imperfectly realized) of the NIH system is the respect for the democracy of ideas. In theory, anyone can propose a fundable study by coming up with a great idea that simply must be done. The alternative that is being proposed here is that we select good scientists at the beginnings of their careers and that is it. They get the money even if someone else has a better idea. Because that someone else may not have been selected in the beginning and will now never be able to get into the racket.

The racket will be dominated by pedigree, you better believe it. Not good.

There is one statement about peer review that I find hilarious:

Senior scientists with a wide appreciation for different fields can play important roles by counteracting the tendency of specialists to overvalue work in their own field.

...because "senior scientists" never "overvalue work" based on their biases, right? Please. I've said this many times in the context of grant review. Everyone has biases. EVERYONE! The only tried and true solution is the competition of biases which requires diversity. We need junior scientists in the mix. We need people from a diversity of institution types. We need people of diverse backgrounds and interests. We need a diversity of scientific approaches, orientations and interests. All competing on a more or less equal footing. It is the only way to minimize biases of review.

__
UPDATE/PostScript: I think these authors also fall into the usual trap of thinking that they know who the best scientists are and that if we could just get the money in their hands then the best science would result. It is the trap of thinking we can actually predict where the advances will come from. Read the history of optogenetics. Did it require Deisseroth? Would we have gotten there eventually? Would Deisseroth even have been funded under the New World Order envisioned by the Alberts et al Perspective piece? Has the infinitely more translatably useful DREADD technology been underdeveloped because of the shadow cast by optogenetics? In the New World Order might DREADD have been prevented entirely?

45 responses so far

On resubmitting unfunded A1 NIH grant applications

Apr 08 2014 Published by under Grant Review, Grantsmanship, NIH, NIH funding

Well, well, well.

The NIH limited applicants to a single revision ("amendment", hence -01A1 version) of an unfunded "new" grant submission (the -01 version, sometimes called "A0") in 2009.

This followed the action in 1997 to limit revisions to two (see RockTalk chart), which hurt PIs like Croce and Perrin. (Six revision? Wow, that is some serious persistence guys, my hat is off.)

I wasn't really paying attention to such matters in 1997 but there was some screaming in 2009, let me tell you.
Delusional Biomedical Researchers Seek Repeal Of Arithmetic
More on the new NIH policy on grant application revisions

Initial outcome of limiting NIH apps to a single revision?


NIH re-evaluating ‘two strikes’ rule – Updated

Crocodile tears from experienced NIH investigators over the discontinued A2 revision

I don't know how many people actually got stuck in the filter for submitting a A0 that was too similar to their prior, unfunded A1. I heard of a few, so it did happen. On the flip side of that, I've sure as heck been putting in more than two versions of a proposal which is designed to fund the same area of interest in my laboratory. I have not yet been flagged for it. My initial reaction that any PI who has an ounce of creativity ought to be able to come up with a credible alternative take on their project is still my current take.

Nevertheless, rumor has it that changes are in the wind.

Pinko Punko made an interesting comment on the blog:

DM, I heard the craziest thing today- the possibility of removing the "substantial revision" criterion for new A0 related to previous A1. Supposedly announcement soon- I was kind of surprised.

This was news to me but I have heard things from about five independent sources in the past few days that tend to confirm that changes are being considered.

The most consistent rumor is that new grants will no longer be checked for similarity to prior unfunded proposals. They will be called new grants, but there is no apparent reason for this. In all ways I can see, this is going to be a return to the days prior to 1997 where you could just endlessly revise until the study section relented.

The supposed benefit of reduced "time to award from original proposal" is now going totally by the wayside. I mean, the NIH will still be able to lie and say "look it was an A0!" if they want to but this is even less credible.

More dangerously, the will of study sections to endlessly queue applications will be released from whatever tepid effect the A1 limit has produced.

This is a very BadThing.

__
whoa. I found three A7 projects. All three are competing continuations. I can't EVEN....five and six year apparent funding gaps for two of them. The other I can't work out why there is no apparent gap in funding.

31 responses so far

The NIH Grant "Have" States Resist Sharing

Apr 04 2014 Published by under NIH, NIH Budgets and Economics, NIH funding

From the Boston Globe (of course):

Two dozen rural states stretching from Maine to Mississippi and Montana are clamoring to increase their share of federal research dollars now disproportionately awarded to Boston-area institutions and scientists.

Whaddaya mean, "disproportionately"? WE DESERVE IT!!!

“There’s a battle between merit and egalitarianism,” said Dr. David Page, director of the Whitehead Institute, a prestigious research institution in Cambridge affiliated with MIT.

Yeah, pure merit versus affirmative action quotas for lame ass science from Universities we've never heard of maaaang. There couldn't possibly be any bias in grant review and award that puts a finger on the scale could there?

In one of the efforts, Senator Susan Collins, a Maine Republican on the Appropriations Committee, is proposing that funding for the special program to benefit rural states, formally called the NIH’s Institutional Development Award, be raised to $310 million, up from the current $273 million. The current amount equals just 1 percent of the institute’s research grants — a drop in the bucket compared with what Boston researchers win each year.

Last time I checked Massachusetts Congressional District 8 for NIH funding (probably a number of FY ago), Brigham and Women's Hospital was pulling in $253,333,482 in NIH grants. MIT? $172,184,305. Harvard Medical School? $168,648,847. The list goes on in this single Congressional district.

and while the Globe has this scare passage near the top:

The coalition of states that benefits from the NIH special program for rural states doubled the amount of money it spent on lobbying in the last decade, to $590,000 in 2013 from $300,000 in 2003. That number does not include direct lobbying by universities in those states.

this is going to barely manage to tread water against the combined might of the richest of "have" Universities and institutions:

Representative Michael Capuano, whose district encompassing the Boston-area research hospitals wins more NIH money than any other congressional district, said the Massachusetts delegation is playing defense right now.

“The system works reasonably well but it’s under attack in a serious way,” Capuano said.

Massachusetts is mobilizing. Hospital executives, university presidents, and Washington lobbyists make routine trips to the Capitol. Their not-so-subtle message: Boston is on top because its elite institutions offer the best chances of big scientific breakthroughs.

then there is classic misdirection and the usual conceit that the NIH award process is purely about merit, uncontaminated by self-reinforcing vicious cycles of the rich getting richer.

“There are people in Boston who deserve more than a million dollars in NIH money because that is the best use of those dollars,” said Dr. Barrett Rollins, chief scientific officer at Dana-Farber Cancer Institute, a top recipient of federal research funds. “Congress has a responsibility to spend taxpayer money in the best possible way, and to me, the most straightforward way to do that is to make sure the dollars are invested in the most meritorious work without regard to geographic distribution.”

Because the quality of science is not evenly distributed across the country, researchers should not expect federal dollars to be either, said Harry Orf, senior vice president for research at Massachusetts General Hospital, another top recipient of NIH grants.

“You have congressmen who can’t evaluate science sending money to places not rated for innovation,” Orf said. “As funds get more and more scarce, you want to make sure you’re betting on the best science.”

It is beyond asinine to pretend that the NIH grant money is distributed by geographic affirmative action to any extend that squeezes the elite coastal research institutions. The above numbers and any current search on RePORTER verifies that the kind of money that is being proposed to go into this geographical affirmative action is a drop in the bucket. One or two of the larger institutions funded by NIH (and keep in mind that a place such as "Harvard" is made up of multiple institutions which are named as independent awardees in the NIH records) account for the entire outlay in the the NIH’s Institutional Development Award program. Even if the increase to $310M goes through.

There is considerable debate about "the best science" and about the best way to hedge our scientific bets. The NIH works, haltingly, in a way by which the serendipity of chance discovery from a diversity of approaches is balanced against predictable brute-force progress from exceptionally well funded Universities, Medical Schools and research institutions. I find myself citing papers from the very biggest institutions, sure, but I have numerous critical findings that I cite in my work that have come from smaller research programs in smaller Universities and (gasp) Colleges. Don't you? If you do not, I question your scholarship. Seriously.

I suggest a purely self-interested goal, for those of you who are elite-coastal-University die hards. Every Congress Critter gets a more or less equal vote. The ones from Maine (Susan Collins, see above), from Alabama....

“It’s hard to compete against MIT or Harvard. . . . They’ve had their share. A lot of state colleges and universities all over the country, from Idaho to Maine, have some ideas too, and I think we should give these people from smaller schools in other states an opportunity,” said Senator Richard Shelby of Alabama, the top Republican on the powerful Senate Appropriations Committee. “It’s time to fix that.”

from West Virginia...

“The program stipulates that not everything goes to Harvard, Yale, and Stanford,” said Senator Jay Rockefeller, a West Virginia Democrat.

and from Oklahoma, among others.

Representative Tom Cole, a Republican from Oklahoma who serves on the House Appropriations Committee, said he’s simply interested in supporting research that occurs “outside the normal corridors of power.”

Rep Cole seems to understand why geographical affirmative action is necessary, doesn't he?


“There is a network where you tend to reward peers and people you know, and I think the distribution of funds, not intentionally, is skewed a bit toward places like Boston,” Cole said. “We just want to make sure that the playing field is fair.”

We need all these Critters to be on board if we expect Congress to listen to our pleas on behalf of the NIH.

It is politically stupid to fail to understand this.

52 responses so far

Guest Post: Manage your career, folks!

Mar 27 2014 Published by under Academics, Careerism, NIH Careerism

This is another guest post from @iGrrrl, a grant writing consultant.


A few comments I've seen around, on top of my experience working with applicants for K-flavored and other career development grants, make it clear that they think the required career development parts are just window dressing. I hear complaints that they have to write a mentoring plan, and then they never do anything that is on it.

Is it the mentor's fault? The people who signed letters to be on the mentoring committee? No. (I'm going to switch voice now and talk at you K99/R00 or other K and F applicants/awardees.) And whose fault is it?

The fault is YOURS. No one cares about your career as much as you do, and even if it went in as fiction on paper, it is YOUR responsibility to make it reality. Otherwise you'll never know if it would have made a difference to tap into the brains on your mentoring committee, to impress them with your initiative and willingness to learn. Making someone feel smart and important to you (while also getting good advice) is a good way to increase their sponsorship of you--inviting you present at meetings, to small subdisciplinary meetings, talking positively about you.

I think it's easy for young people to underestimate the impact of the positive regard of more senior faculty, or for you young folks to know how that plays out in reality. No, they're not gossiping about you; they have better things to do. But that 'dream team' remembers that they signed letters for you and then never heard from you again.

11 responses so far

Should K99 awardees continue the R00 phase at the current Institution?

Mar 26 2014 Published by under NIH, NIH Careerism

The FOA for the K99/R00 says that a different institution is "encouraged (but not required)".

I will confess I thought this was somewhat stronger in tone but the original announcement was similar.

Applicants are encouraged to consider independent positions at departments and institutions different from where they conducted their mentored research. Should an awardee wish to activate the independent phase of the grant award at the same department or institution at which they conducted their mentored research, the individual must provide justification addressing the decision to remain at the same institution.

Still, I guess we can bench race what this would mean in our personal interpretation or recommendation. I would think this an exception for extreme circumstances... so more like 10%?

Well, a comment on a prior thread asserted that something around 25% of R00 awardees had remained at the same Institution in which they had been "training" under the K99 phase.

Naturally, I was interested in what my ICs of greatest interest have been up to. Of currently active R00 grants, one IC came in at 21% at the same institution as the K99 phase and another came in at either 29% or 35%. This latter one had a single award for which the K99 phase moved in the second year and then the R00 was continued in that new location.

Huh.

Huh.

Color me surprised.

I am no big fan of enforced academic nomadism. Were I the Boss of Science I may have omitted that little "encouraged but not required" clause, frankly. Maybe. But the clause is most assuredly in there and it gives an indication about intent. Nice to have exceptions for unusual circumstances, yes, but I would tend to interpret that statement as meaning rare.

At 25% of the awardees, clearly the NIH does not agree with me on this meaning.

Either that, or the reality of the (lack of) academic nomadism under the current system has essentially overcome the NIH's intent.

Here we have a fancy grant award that gives postdocs some negotiating room in where they would like to be. As we've been discussing, these are presumably some top-quality candidates, going by the usual academic seals of approval. And a quarter are disregarding the expectation of nomadic dispersal at the transition to faculty level independence.

To me this underlines the opinion I have on nomadism.

There is something seriously wrong with the expectation that academic scientists travel all over creation for their jobs, in the current era of dual-career families and "training" phases that extend well into the third decade of life.

Quite some time ago I started scrutinizing CVs of visiting speakers, grant applicants, etc to see their trajectories. I didn't formalize it but I came to the conclusion that a substantial (20-25% would probably have been my estimate prior to today's exercise) number had violated the "expectation". Yet it persists. Here we have fairly accomplished scientists (grant winners and invited speakers) violating the truthy truism of academic careers. Substantial numbers of them too. Yet the culture is to sneer at this as an intentional trajectory when it comes to advising junior scientists.

Shaking my head.

24 responses so far

Older posts »