Archive for the 'Careerism' category

Peer Review: Advocates and Detractors Redux

A comment on a recent post from Grumble is a bit of key advice for those seeking funding from the NIH.

It's probably impossible to eliminate all Stock Critique bait from an application. But you need to come close, because if you don't, even a reviewer who likes everything else about your application is going to say to herself, "there's no way I can defend this in front of the committee because the other reviewers are going to bring up all these annoying flaws." So she won't even bother trying. She'll hold her fire and go all out to promote/defend the one application that hits on most cylinders and proposes something she's really excited about.

This is something that I present as an "advocates and distractors" heuristic to improving your grant writing, surely, but it applies to paper writing/revising and general career management as well. I first posted comments on Peer Review: Friends and Enemies in 2007 and reposted in 2009.


The heuristic is this. In situations of scientific evaluation, whether this be manuscript peer-review, grant application review, job application or the tenure decision, one is going to have a set of advocates in favor of one's case and detractors who are against. The usual caveats apply to such a strict polarization. Sometimes you will have no advocates, in which case you are sunk anyway so that case isn't worth discussing. The same reviewer can simultaneously express pro and con views but as we'll discuss this is just a special case.

The next bit in my original phrasing is what Grumble is getting at in the referenced comment.


Give your advocates what they need to go to bat for you.

This is the biggie. In all things you have to give the advocate something to work with. It does not have to be overwhelming evidence, just something. Let's face it, how many times are you really in position in science to overwhelm objections with the stupendous power of your argument and data to the point where the most confirmed critic cries "Uncle". Right. Never happens.

The point here is that you need not put together a perfect grant, nor need you "wait" until you have X, Y or Z bit of Preliminary Data lined up. You just have to come up with something that your advocates can work with. As Grumble was pointing out, if you give your advocate a grant filled with StockCritique bait then this advocate realizes it is a sunk cause and abandons it. Why fight with both hands and legs trussed up like a Thanksgiving turkey?

Let's take some stock critiques as examples.

"Productivity". The goal here is not to somehow rush 8 first author papers into press. Not at all. Just give them one or two more papers, that's enough. Sometimes reiterating the difficulty of the model or the longitudinal nature of the study might be enough.

"Independence of untried PI with NonTenureTrackSoundin' title". Yes, you are still in the BigPIs lab, nothing to be done about that. But emphasize your role in supervising whole projects, running aspects of the program, etc. It doesn't have to be meticulously documented, just state it and show some sort of evidence. Like your string of first and second authorships on the papers from that part of the program.

"Not hypothesis driven". Sure, well sometimes we propose methodological experiments, sometimes the outcome is truly a matter of empirical description and sometimes the results will be useful no matter how it comes out so why bother with some bogus bet on a hypothesis? Because if you state one, this stock critique is de-fanged, it is much easier to argue the merits of a given hypothesis than it is the merits of the lack of a hypothesis.

Instead of railing against the dark of StockCriticism, light a tiny candle. I know. As a struggling newb it is really hard to trust the more-senior colleagues who insist that their experiences on various study sections has shown that reviewers often do go to bat for untried investigators. But....they do. Trust me.

There's a closely related reason to brush up your application to avoid as many obvious pitfalls as possible. Because it takes ammunition away from your detractors, which makes the advocates job easier.


Deny your detractors grist for their mill.

Should be simple, but isn't. Particularly when the critique is basically a reviewer trying to tell you to conduct the science the way s/he would if they were the PI. (An all to common and inappropriate approach in my view) If someone wants you to cut something minor out, for no apparent reason (like say the marginal cost of doing that particular experiment is low), just do it. Add that extra control condition. Respond to all of their critiques with something, even if it is not exactly what the reviewer is suggesting; again your ultimate audience is the advocate, not the detractor. Don't ignore anything major. This way, they can't say you "didn't respond to critique". They may not like the quality of the response you provide, but arguing about this is tougher in the face of your advocating reviewer.

This may actually be closest to the core of what Grumble was commenting on.

I made some other comments about the fact that a detractor can be converted to an advocate in the original post. The broader point is that an entire study section can be gradually converted. No joke that with enough applications from you, you can often turn the tide. Either because you have argued enough of them (different reviewers might be assigned over time to your many applications) into seeing science your way or because they just think you should be funded for something already. It happens. There is a "getting to know you" factor that comes into play. Guess what? The more credible apps you send to a study section, the more they get to know you.

Ok, there is a final bit for those of you who aren't even faculty yet. Yes, you. Things you do as a graduate student or as a postdoc will come in handy, or hurt you, when it comes time to apply for grants as faculty. This is why I say everyone needs to start thinking about the grant process early. This is why I say you need to start talking with NIH Program staff as a grad student or postdoc.


Plan ahead

Although the examples I use are from the grant review process, the application to paper review and job hunts are obvious with a little thought. This brings me to the use of this heuristic in advance to shape your choices.

Postdocs, for example, often feel they don't have to think about grant writing because they aren't allowed to at present, may never get that job and if they do they can deal with it later. This is an error. The advocate/detractor heuristic suggests that postdocs make choices to expend some effort in broad range of areas. It suggests that it is a bad idea to gamble on the BIG PAPER approach if this means that you are not going to publish anything else. An advocate on a job search committee can work much more easily with the dearth of Science papers than s/he can a dearth of any pubs whatsoever!

The heuristic suggests that going to the effort of teaching just one or two courses can pay off- you never know if you'll be seeking a primarily-teaching job after all. Nor when "some evidence of teaching ability" will be the difference between you and the next applicant for a job. Take on that series of time-depleting undergraduate interns in the lab so that you can later describe your supervisory roles in the laboratory.

This latter bit falls under the general category of managing your CV and what it will look like for future purposes.

Despite what we would like to be the case, despite what should be the case, despite what is still the case in some cozy corners of a biomedical science career....let us face some facts.

  • The essential currency for determining your worth and status as a scientist is your list of published, peer reviewed contributions to the scientific literature.
  • The argument over your qualities between advocates and detractors in your job search, promotions, grant review, etc is going to boil down to pseudo quantification of your CV at some point
  • Quantification means analyzing your first author / senior author /contributing author pub numbers. Determining the impact factor of the journals in which you publish. Examining the consistency of your output and looking for (bad) trends. Viewing the citation numbers for your papers.
  • You can argue to some extent for extenuating circumstances, the difficulty of the model, the bad PI, etc but it comes down to this: Nobody Cares.

My suggestion is, if you expect to have a career you had better have a good idea of what the standards are. So do the research. Do compare your CV with those of other scientists. What are the minimum criteria for getting a job / grant / promotion / tenure in your area? What are you going to do about it? What can you do about it?

This echos something Odyssey said on the Twitts today:

and

are true for your subfield stage as well as your University stage of performance.

2 responses so far

Thought of the day

Aug 17 2014 Published by under Careerism, Day in the life of DrugMonkey

Your failure to achieve exactly the career outcomes that you desire in academic science is 100% because the systems are broken and you are undermined by nefarious opponents using underhanded tricks to block you.

___
H/T: he knows who

8 responses so far

Replication costs money

I ran across a curious finding in a very Glamourous publication. Being that it was in a CNS journal, the behavior sucked. The data failed to back up the central claim about that behavior*. Which was kind of central to the actual scientific advance of the entire work.

So I contemplated an initial, very limited check on the behavior. A replication of the converging sort.

It's going to cost me about $15K to do it.

If it turns out negative, then where am I? Where am I going to publish a one figure tut-tut negative that flies in the face of a result published in CNS?

If it turns out positive, this is almost worse. It's a "yeah we already knew that from this CNS paper, dumbass" rejection waiting to happen.

Either way, if I expect to be able to publish in even a dump journal I'm gong to need to throw some more money at the topic. I'd say at least $50K.

At least.

Spent from grants that are not really related to this topic in any direct way.

If the NIH is serious about the alleged replication problem then it needs to be serious about the costs and risks involved.
__
*a typical problem with CNS pubs that involve behavioral studies.

34 responses so far

Thought of the Day

Jul 24 2014 Published by under Careerism, NIH, NIH Careerism, NIH funding

What fraction of the stuff proposed in funded grants actually gets done after feasibility and field movement come to play?

22 responses so far

A can't-miss inquiry to Editor following the initial review of your paper

Jul 23 2014 Published by under Careerism, Conduct of Science, Peer Review

Dear Editor Whitehare,

Do you really expect us to complete the additional experiments that Reviewer #3 insisted were necessary? You DO realize that if we did those experiments the paper would be upgraded enough that we sure as hell would be submitting it upstream of your raggedy ass publication, right?

Collegially,
The Authors

22 responses so far

Woodgett on modern science careers

Jul 14 2014 Published by under Careerism

Nailed it:

2 responses so far

Good and Bad Mentoring

This topic keeps coming up amongst the trainees and I have an area of confusion.

What does it mean that your mentoring has been bad? Is it all about *outcome*?

Is it about lab favoritism?

Does your mentor fail to advance the careers of everyone? Or is it just that you were not the favored one?

Are there specific things your mentor should and could have done for you that you can mention? Did you only recognize this is retrospect or was it frustrating at the time?

25 responses so far

Do you want a career or not?

May 12 2014 Published by under Careerism, Conduct of Science

This puts it...clearly:

A common misunderstanding among early-career scientists is the thought that their passion is their research focus. A more careful examination reveals that their passion is not so much the subject but rather, the promise of the life that academia might offer.

Painfully so, but clearly.

I've had to ask myself several times in my career if I was interested in TopicX More than I was with having a career. And occasionally the question forms itself in the other direction. Am I so focused on maintaining my career that the actual science isn't any fun anymore*?

At times I have felt as though I would walk away if I couldn't do TopicX. At other times, working on Topics Y and Z has (apparently) been sufficient*.

I am by no means done asking myself the questions, nor is it the case that I always have any choice, really.

I suggest you read the post...if for nothing else it may help you to think about how you make decisions about your career.

__
*In truth, perhaps one of the biggest surprises of my career arc is the degree to which I find I am interested by at least something in just about every project. As it "Yeah, the overall goals here are cool and all but..whoa! What the heck is UP with this thing over here? Wow. Let's get ON that for a few months.....[two years later]"

14 responses so far

Job ad for Assistant Professor position makes it explicit...

Apr 18 2014 Published by under Academics, Careerism

Drexel University College of Medicine is hiring! ....sortof.

The Department of Neurobiology and Anatomy at Drexel University College of Medicine invites applications for a tenure-track Assistant or Associate Professor. We seek a SYSTEMS/BEHAVIORAL NEUROSCIENTIST whose research utilizes contemporary molecular, physiological and/or imaging techniques to address fundamental questions related to monoamine networks, cognitive function and motivated behavior, or psychostimulant drug actions. Applicants should have a Ph.D. in Neuroscience or a related field, a record of excellence in neuroscience research and publication, and preferably extramural funding (e.g., K99/R00 grant).

emphasis added. Unnecessarily.

Very interesting to see this when the drumbeat against soft-money faculty hiring and Med schools lust for indirect costs is getting louder.

h/t: @markgbaxter

27 responses so far

Thought of the Day: The NIH Can't Win

Apr 18 2014 Published by under Careerism, NIH, NIH Careerism, NIH funding

A comment over at Rock Talk made a fairly traditional complaint about the NIH funding system. Dan C stated that: "NIH is to be criticized that it funds “usual suspects.

Today, I find this funny. Because after all, most of the people complaining about the NIH system want to become one of the usual suspects!

Right? They want to get a grant, one. They want to have some reasonable stability of that grant funding in a program-like sustained career. Most of them don't want to have to struggle too hard to get that funding either....I doubt anyone would refuse the occasional Program pickup of their just-missed grant.

Once you cobble together a bit of success under the NIH extramural grant system, those who feel themselves to be on the outs call you a "usual suspect". For any number of reasons it is just obvious to them that you are a total Insider (and couldn't actually deserve what you've managed to accomplish, of course). This may be based on the mere fact that you've acquired a grant, because you work in a Department or University where a whole lot of other people are similarly successful. This may be because it appears that POs actually talk to the person in question. It may be because a FOA has appeared in a research domain that you work within.

Anyone sees the duck floating serenely on the water at a given point in time and it looks like this is one most usual suspect waterfowl indeed.

I used to be annoyed at my approximate lateral peers in science who appeared to be having an easier time of it than I did. I had my Insider attributes as a younger faculty member, make no mistake, but I also had considerable Outsider traits, considering where I was seeking funding and for what topics of research. Some of those folks, over there, well boy didn't they get an easy ride because of being such Insiders to the subpart of the NIH system!

I still have those thoughts. Even though I've seen many of the people I thought had it made in the shade go through their dry spells and funding down-cycles. Despite the fact that as each year goes by and my lab remains funded, I become more and more one of the "usual suspects".

I believe that if I ever feel like I am one of the usual suspects, if I feel like I deserve special treatment and stop fighting so hard to keep my lab going that this will be the end.

I advise you to try to retain the same feeling of "outsider" that you feel as a noob PI for as long as you can into your career.

Getting back to the point, however, the NIH simply cannot win with these criticisms. Those who are feeling unsuccessful will always carp about how the NIH just funds "their" people. And if the NIH does happen to fund one of these outsiders, this very act makes them a usual suspect to the next complainer.

The NIH can't win.

14 responses so far

Older posts »