Negative scientist is negative

Mar 18 2013 Published by under [Education&Careers]

People have a lot of reasons they get into science as a career, but no one ever seem to go into it for the rejection. This is notable because if there one constant in a science career, it's that you're going to get a lot of rejections. And we only have ourselves to blame.

We're taught to be critical of everything all the time. Don't accept your results at face value. Critique pushes science forward (especially when it's of that other lab!) and keeps us thinking. But everyone has spent time in that journal club or lab meeting where it seems like the sole purpose is to tear anything to shreds. "Sure, you discovered the Higgs Boson, but you announced it in Comic Sans so is your work really any good?"

It's something I try to remember as a reviewer. I think my natural tendency is to focus on the things I don't like in a grant or paper. It's as though not being able to criticize something in a proposal is basically mailing in my review.

And I'm not alone, I've gotten my fair share of Excellent or Very Good rankings with a review including about a 4:1 hated it/loved it ratio. Critique is what we're good at. But I'm making a concerted effort this month to dedicate roughly the same amount of text to both the "Strengths" and "Weaknesses" sections of my reviews. Obviously it's not going to work out that way every time, but I'm reading a LOT of really good science. It's worth recognizing that in writing.

23 responses so far

  • This is a very nice blog post, addressing a real and critical issue in science. The title is appropriate (with the tautological overtone an enchanting touch), and the length reasonable. In particular, I commend the figure, utilizing "angry cat" to illustrate the greater complexities of the scientific community. Well donez. ;)

  • Jim Thomerson says:

    There are a fair number of capable people who do not become scientists because they do not want to face the criticism that anyone publishing science receives.

  • LD says:

    When I was in grad school one of the professors stated that there were worse things than having your paper ripped to shreds by a group of grad students in a class, the foremost among them being that they didn't even read your paper. I have always tried to keep that idea in mind - that being ignored can be considered worse than being criticized.

  • MediumPriority4Life says:

    Fitting blog for my day. I just received negative reviews for my C/N/S manuscript, the reviewers saw little to no merit in my lab's work. I even was threatened by having old vegetables thrown at me should I use SEM instead of SD for my stats in the future. Funny thing being, my last manuscript published in the same journal, I had to change from SD to SEM cause SEM was industry standard.

    Hopefully reviewers at my next stop are in the mood for some positive science. Any ways, I am off to go cry.

  • Whoosh says:

    This is a very good topic for a blog post. Often enough I get stuck with all the little things that annoy me about a paper even though there is great science behind it. I think I'll follow your idea and try to actively focus on the great science I have to review and it give it more of the attention it deserves.

  • theshortearedowl says:

    I sometimes wonder about this. Clearly if the purpose of science is to discover truth, then the more people you have fact-checking, asking "are you sure this is right?" and generally being attentive to detail, the better. That's constructive criticism, and it's necessary and good, however socially inept the criticiser is at conveying their point. But there are also people for whom it starts to become more about point-scoring. Once you're away from "is this right?" and into "I think it would be better like this" territory...

  • proflikesubstance says:

    You folks realize that I could write this any day of the year and it would be fitting for someone since there is always a rejection going out to someone in this biz? It's part of the deal but the good news is you get mostly jaded to it after a while.

  • Remember that if you cut yourself off from the critiques, it will be harder to enjoy the upsides.

    TSEO - the skill to do exactly this - constructive criticism - is something that can take a long time to learn (for me at least). Critique the work, not the person.

  • Comradde PhysioProffe says:

    This is stupid advice. If you lake woebegone every app, then you have discarded your opportunity as a reviewer to actually influence which apps get funded.

  • ecologist says:

    A big, big part of the problem is that we train students to be very critical of papers that they read, but largely ignore training them to be critical of the critiques. For example, "the authors failed to consider X" is perhaps the most common critique encountered on submitted papers and proposals, and certainly the most commonly encountered critique in graduate seminar courses. It is NOT a valid critique under almost any circumstances, in the absence of a reasoned and supported argument showing how the failure to consider X affects the conclusions.

  • MCA says:

    What works best for me is to do the specifics first, all the stuff ranging from "what about this statistical test?" down to "your axes aren't the same line thickness", then step back and think about the big stuff - is it good science? is this or that issue a fatal flaw? does the data support their conclusions sufficiently?

    IME, having that extra time to consider the big stuff gives you time to figure out why that "little stuff" item is nagging at you. More than once, a former "little stuff" item has, on sufficient consideration on my part, turned out to be a fatal flaw that was just very well hidden. On the other hand, forcing myself to step back and look at the paper as a whole after a thorough critique has sometimes led me to say "That's a long list, but all of these are just minor tweaks, and this is good science."

  • proflikesubstance says:

    This is stupid advice. If you lake woebegone every app, then you have discarded your opportunity as a reviewer to actually influence which apps get funded.

    You're missing the point. I have no intention to glorify everything, but simply to spend some time recognizing the good in even the "pretty good" proposals. Obviously there is going to be a tier of proposals where this is impossible, but I find I am often more critical than positive for even the proposals I like.

    What works best for me is to do the specifics first, all the stuff ranging from "what about this statistical test?" down to "your axes aren't the same line thickness", then step back and think about the big stuff - is it good science? is this or that issue a fatal flaw? does the data support their conclusions sufficiently?

    I try and use the summary section of the NSF review format to talk about the big picture and where I see this work fitting in. It's a useful overview that backs away from the more point-by-point of the intellectual merit portion of the review.

  • Dr. Strange says:

    You are on to something. While a critical eye is necessary for good science, I find that the majority of people reviewing papers and sitting on study sections have turned into professional "nattering nabobs of negativism" (to quote my least favorite vice-president. As I have survived many rejections, here in mid-career, I have noticed a striking lack of correlation between the quality of a given submission and the reception it receives. This is due to the culture of committees, where criticism is implicitly accepted as the currency of intelligence.

    Bottom line, I think people can be so smart about their criticism, they are idiots, and it slows things down. My motto is and shall remain, "I'll show them! I'll show them ALL!!"

  • Comradde PhysioProffe says:

    What's the purpose of wasting your time "recognizing the good" in applications that are surely non-fundable. Explain the weaknesses that doom the app and move on.

  • qaz says:

    Because criticism serves two functions. One is identifying why we gave the grant/paper/etc the score we did. But the other is to help the author do better next time. To help the author do better next time, it helps to have both positive and negative comments.

    And before DM and CPP yell about the arrogance of reviewers, I say that it is important as scientists to be humble enough to realize that we don't actually think of everything and sometimes our colleagues viewpoints are useful.

  • How do positive statements help a grant writer do better next time? The way to do better next time is to address the weaknesses.

  • eeke says:

    CPP - The NSF does not do triage, and all applications get discussed. I witnessed one application get pulled out of the "non-competitive" trash can up into high priority. I credit the reviewer who got the other nit-picky reviewers hung up on one stupid detail to see beyond their ignorance and how important, innovative, and outstanding the proposed science in the application really was. Sometimes saying positive things in a review (or in discussion) can be instrumental for changing the fate of an excellent application.

  • qaz says:

    CPP asks "How do positive comments help someone improve?"

    Ummm... isn't this criticism 101? And teaching 101? And job-assessment 101? Including a mix of positive and negative criticism produces better improvements than purely negative criticism?

    Pointing out what they did right helps them because they won't waste time trying to fix the things they got right. It helps them figure out how to make the things they did wrong more like the things they did right. It ensures that they won't throw the baby out with the bathwater. (I hate when a grant comes back taking out the one thing that everyone liked in the first round.)

    And, also, because people tend to react to pure negativity with defensive responses but with combinations of positive and negative by fixing the negative. Because this really is a f%$^ing care-bears party and we're all in this together. (See the great post on succeeding in graduate school by neuropolarbear: which applies to all steps in the academic career).

  • Sometimes saying positive things in a review (or in discussion) can be instrumental for changing the fate of an excellent application.

    This is not what we are talking about, so please try to pay attention. The question is how to review grants that are not excellent, but just decent.

    Obviously, for an excellent grant that is competitive for funding, it is essential for reviewers to "go to bat" for that grant by enumerating all the strengths and how strong they are. Sorry, but I don't have the time or inclination to go on in detail about the strengths of grants that are not excellent and not competitive for funding. And the stated purpose of written grant critique in the NIH system--which I agree is appropriate--is to provide the applicant with sufficient information to understand the strengths and weaknesses that drove the scoring.

    For decent, but not excellent, grants, there is no value to going further than that, other than making applicants "feel better".

  • proflikesubstance says:

    Part of the issue here, I think, is in how the proposals are selected for funding. At NIH the system is numbers based and the PO doesn't have nearly the latitude for "pick-ups" that NSF POs have. At NSF there is the portfolio-balancing aspect that keeps some proposals in play longer than they would be at NIH.

    Additionally, what is written in the review has a lot of weight at panel. Even if the discussion goes in a more positive direction than one's review, the PO will still ask you to justify a change in your assessment, based on what you wrote prior to the panel. Some positive text goes a long way towards making this easier should you need it.

    The other thing that I did not explicitly say is that I am reviewing preproposals right now, so the success rate at this stage will be much higher (~30%) than the funding rate. Anything that clears the bar for a full proposal will need feedback, both positive and negative, to shape the full proposal.

  • becca says:

    If it offers any illumination, there's some evidence on what role positive feedback actually plays. For a novice or even intermediate (depending in part on personality), positive comments are key- not only do people NOT always know what they did well on, but they also need the positive stuff to keep them going. Something even CPP should remember for e.g. fellowship applications.
    Of course, in a game theory sense, one could argue that the best thing you can do for your career is to discourage people who will be competing against you for grant money who will not help your career. Some of the negative comments, consciously or not, could be related to that.

    At the same time, it should be noted that one way experts get better is by seeking out critical feedback. At a certain level, when you know you're good, you filter out all the positive stuff because it's filler.

  • I second becca. For young people the positive and negative feedback are both really important, both for psychological reasons and knowing what they're doing right. Besides, the constant criticism, with no balancing by acknowledging what people are doing well, selects for certain personality types...My opinion is that this isn't good for science. We all have to learn to cope with the criticism - it's a necessary part of science. But it is not necessary that it is the only type of feedback we give.

  • newProf says:

    As a new, unfunded investigator, I'd learn fastest from both positive and negative feedback. I'd like to be able to tell the difference between "great idea with fatal errors" and "okay idea with fatal errors." I want to be able to triage my own efforts--should I improve my writing through more rounds of revision, or am I better off pouring all my energy into preliminary data? It would also be damn encouraging to know that with some adjustments, the suggested research might someday be interesting and meritorious to others.

Leave a Reply