One of the biggest concerns I hear about NSF review is that reviews vary from one panel to the next. People who get good scores in one round and just miss funding scream bloody murder when their proposals doesn't score as well in the next round.

"Damn inconsistent panels! Last year they loved it and this year they don't know their ass from their elbow!"

But what if that's a feature, not a bug?

People always interpret their changing fortune in the light of the lower ranking being attributable to a panel that didn't get it. The alternate hypothesis, however, is that the higher ranking panel didn't get it and the second panel recognized a fatal flaw.

I've had a co-reviewed proposal that fell in exactly this situation, but it was the same proposal read by independent bodies, one in IOS and one in DEB. One of the panels loved it and the other was more cautious. It was ranked as "high priority" and "low priority", respectively and the POs involved decided to give us time to respond (i.e. it wasn't funded). I was upset at the time, because clearly the second panel was a group of fecal-tossing, vissionless, bucket-hat wearing ignorami.

So we went after the data they wanted. Hard. And we couldn't nail it down. Why? Because the key piece of data was flawed, and for a variety of reasons, we were not in a position to reveal that before the first submission.

Now, did we use the same data we produced to follow up something just as awesome? Of course. And it turned out incredibly well. But the foundation of the original proposal was flawed and one panel saw it and the other only saw the promise.

Something to chew on when considering anecdata about the variability of the NSF review process.

Terry, over at SmallPondScience, has a post up about the misuse of the term "deadwood" and how many who are labeled as such are still deadicated to their jobs, but have inadvertently drifted away from what we consider productive colleagues. These colleagues have become driftwood.

You might be driftwood if your teaching relies on concepts that date back to your grad school days more than what you’ve learned since then.

You might be driftwood if you have trouble publishing an article because no solid journal thinks that the topic is important.

You might be driftwood if you are uncomfortable telling your students “I don’t know” because you fear that you are supposed to know.

You might be driftwood if you’re avoiding a specific research agenda not because you lack the tools but because you lack the information.

You might be driftwood if you find yourself disagreeing with most of the junior faculty about research standards or contemporary teaching approaches.

You might be driftwood if you rely on skills you learned in grad school that aren’t being taught in grad school anymore.

I am not the type of person who thinks productivity is strictly based on publications and grant money. I think PIs who give up the grant game and take on other departmental responsibilities are critical to a well functioning department that values education. If one of my less research active colleagues steps up to teach a course that a well funded colleague needs to move out of to maintain their research load, I believe that's a good departmental model. But that's not always how it works, is it?

While I think Terry's assessment might be true for his experiences, I don't think it is universal. Nor are mine of course, but across several research-centric institutions I have witnessed some fairly blatant abuse of tenure privileges. I've certainly seen people who could be described as "mailing it in", but I doubted they would take the time to find a stamp*. To me, those deserving of the deadwood label put no effort into any aspect of their job and cause students to file complaints on a regular basis. While I agree with Terry that there are fewer of these people than legend would lead one to believe, they are not "rare".

So allow me to get the ball rolling:

You might be deadwood if you spend more time on your various litigious activities against the university than any scholarly pursuit.

You might be deadwood if your vacation schedule mirrors that of the undergrads.

You might be deadwood if you haven't updated a lecture, submitted a manuscript or proposal since 19somethingorother.

You might be deadwood if people wonder aloud whether your office is still occupied.

You might be deadwood if you fiercely defend your lab space that hasn't been entered in many years.

You might be deadwood if you haven't had money for research since the first Bush administration, but fight to take on graduate students every year.

You might be deadwood if you accept service responsibilities (under pressure from above) and then never perform that function.

I'm sure readers will have some other definitions.

*While "emailing it in" might be more appropriate to the effort level, it's not clear if these folks know how their email works or even check it if they did.

Today is the day there will be a collective groan across the land of NSF Bio. Roughly 80% of y'all who submitted preproposals to either DEB or IOS divisions will know by the end of the day (directly or indirectly) that you'll not be planning a full proposal for August, but a rewritten January 2014 preproposal. Many of you, myself included, already have that news.

I have to say that my attitude on the preproposals has changed as I have been more involved in the process. I'll admit right off the bat that having funding and not hitting the preproposal process as my introduction to NSF likely plays an enormous role in my reflections on the changes. I am sure I would be terrified as a new PI in the system or if I was facing a >year funding gap. No doubt I will down the road.

However, my experience has been that getting papers written has been incredibly difficult while churning 2-3 full grant applications twice a year. On top of teaching. On top of travel. On top of seeing my family once in a while. One thing the preproposals do is prevent people from sinking a ton of time into a proposal that is not ready for Prime Time. Little consolation, I know, but it's depressing to look at the trail of dead proposals I've left in my wake.

What does leave me uncomfortable is that every crucified proposal is a learning experience. Not only is it giving new PIs grantsmitting chops, it helps to learn How Things Work. For me, this was rather critical experience that I continue to learn from.

So, what is the solution for the new peeps? You're going to have to get feedback informally and from a bunch of people. Find someone who has been on the panel you're submitting to or at least a related panel. Swallow your pride and listen to the feedback you're getting. You may have a brilliant idea, but maybe you're writing to the wrong audience. Maybe the pitch is all wrong.

Oh, and get your ass on a panel.

More of a polling question today: How long does it take you to hire an employee on grant funds?

At my institution, hiring of different positions go through very different processes. Postdocs? Easy. Technicians? Good luck. Having started the process in March to hire someone in late May, I am facing the very distinct possibility of having to delay the hire because the position has not yet made it through the system. Surely other universities can manage to hire soft money limited positions in less than two months?

It always fascinates me to get different opinion of how this career works. It's part of my interest in blogging and why I find the NIH crowd worth watching. But one thing that unites scientists is the need to sell their ideas. Without being able to pitch your research plan to an audience (whoever has the money) and convince them that you have a worthy investment, you aren't going to be able to continue to do science. Whether you are going for crowd-sourced money, an institutional grant for $10,000 or an R01, your job depends on getting other people excited about what you are proposing to do.

That is part of why it has been so painful to watch the public journeyman science exploits of Ethan Perlstein. In a twitter exchange yesterday, it became clear that not everyone agrees with this. But whether your audience knows the same literature as you should not be the deciding factor as to whether or not your proposal is understandable. And no, this is not ancillary to the science, it's critical if you want to DO the science. These are fundamental aspects of grantsmithing and finding ways to keep a lab funded.

I hear people say, all the time, that so-and-so only got the money because they can sell their work. It is usually said with at least a subtle air of "my science is better, but they're smoother". The reality is that selling the ideas is critical to doing the science. Whether you are trying to get money from the federal government, industry or private donation, you still need to get people interested! Part of that is establishing feasibility based on what you have done, demonstrating you're on to something and making a case that the result is going to be AWESOME. If your response to any question of the work is either "well if YOU knew the literature like I do, you wouldn't ask such a stupid question!" or "Clearly you can't see the brilliance of my work because you're too vested in the current dogma." then you are biting the hand that feeds. Maybe BSD graybeards can pull that off, but it's no way to get established.

I'm not trying to pile on while there is an on-going discussion about the research involved, but if you can't sell an idea in plain language to your target audience you are not going to make a go of this. The sooner one recognizes the simplicity of this point, the better off they're going to be.

Oh, and it also doesn't help if you keep publicly bashing whole fields that review your work.

@drugmonkeyblog if pharmacologists actually knew the pre-80s literature this would be a non-issue. But they don't. Not my problem now

— Ethan O. Perlstein (@eperlste) May 8, 2013

@drugmonkeyblog not my responsibility if a field has lost its way and doesn't know its own past. In retrospect, made it a harder sell

— Ethan O. Perlstein (@eperlste) May 8, 2013

Exponentially increases the probability that one will be sent home from school/day care when you have a heap of shit to do.

Folks, if you haven't heard back on your DEB or IOS preproposal, it's not time to panic yet. Both divisions have set a target date of 5/15 for invites to be notified, as mentioned on the DEB blog. Yes, some people you know may have gotten word that they've been invited, but don't lose hope until May 16th. Then you can freak out.

One of the best things I've done as I've fumbled through learning how to teach, has been to routinely ask for student opinion. I've done this both anonymously and not, but the answers are always enlightening. I administer a mid-semester evaluation of the class to see how things are going, but I also find that asking students about what topic they liked the best over the last X time period can yield surprising results. On my last midterm exam I used this as a bonus question and found the the answers converged roughly on three topics. Two of them were side things I tossed in to the lecture because I thought the students would find it interesting.

Based on the feedback I may find ways to expand a bit on those topics in future years, but more than that, it gives me a bit more information on the types of subjects that engage the students. Biology is weird and fascinating and I like to make that clear. Sometimes I can grab the classes interest and sometimes they seem to glaze over. Asking them after the fact provides a lot of insight into what engaged them either in class or when they were studying later. Sometimes it's surprising what stood out to them.

Fresh on the heels of the Coburn amendment to the restoration of much of the NSF budget for 2012 that is likely to decimate political science funding in the near future, the Senate is trying to further limit NSF. Rep. Lamar Smith, Chair of the House of Representatives Committee on Science, Space, and Technology, is proposing to require all NSF funded research to justify how it will benefit the US population.

Set aside for a second that NSF funds fundamental science that is not required to have a direct application to human populations (I think we have a funding agency for that). The whole point of basic research is that it is foundational. In many cases the results of current studies funded by NSF may have massive human impacts down the road. For some, those impacts might be intentional or predicted, but for most they will not. The idea that we can easily predict the impact on society a priori is unicorn-riding fantasy.

But the bigger issue here is why we have SOPA-supporting, climate change skeptic tea party lawyer deciding how the NSF should decide on science funding priorities. Maybe it's an upgrade on his predecessor, Ralph Hall, but the absurdity of the Congressional Science Committee being populated by people who have zero background in science is disturbing, though nothing new. Maybe our situation isn't as bad as what is happening in Canada, but we need to be concerned when politicians think they know how to "improve" the way a science agency runs their business.

Smith's suggestion, made during the afternoon hearing, could signal yet another twist in the debate. It also suggests that his thinking had evolved in the 2 hours between hearings. Instead of confining himself to social science research, as he and his Republican colleagues had done during the morning hearing with Holdren, Smith focused on NSF's entire portfolio in his afternoon comments to acting NSF Director Cora Marrett and Dan Arvizu, chair of the National Science Board that oversees NSF.

"These questions are not easy," Smith said in his opening statement. "It requires recognition that we might be able to improve the process by which NSF makes its funding decisions."

Later in the hearing, Smith made the case for a new yardstick with which to measure an NSF grant that would focus on its likely contribution to "the national interest." Turning to Arvizu, he said, "If there's a way to improve the process by which NSF makes its awards, I assume that you'd support it."

If, in fact, the Science Committee splits NSF out of the COMPETES Act and into a separate bill, it'll make pretty clear that their intention is to tie the agencies hands as much as possible. This is not about making decisions about the national funding objective, it's about people who don't understand science trying to force money away from anything they fear. I think the president's Science Advisor, John Holdren, summed it up pretty well, when he said "I think it's a dangerous thing for Congress, or anybody else, to be trying to specify in detail what types of fundamental research NSF should be funding."