Panel decisions for this year's preproposals are staring to come in. I got my first response last night and was lucky enough to get invited* for a full proposal for this one. Sure, I was in bed and turning off my phone when the email came in, but I had to go check out the reviews (not a recommended pre-bed activity, btw). A few things jumped out at me right away:

- Clearly there is lots of variation among panels with their approach to preproposals. The panel I applied to had considerably more applications than the one I served on. So many, in fact, that the had to deal with them in two panels. So, not only is there the complication of dealing with different audiences and expectations between the pre- and full proposal evaluation, but we also have multiple preproposal panels for the same application set. I don't really see that as a major issue, but I would be curious to hear from the POs whether there was consistency or differences.

- The invite rate was <20% for this panel. They were tougher than my experience, but we used high and low priority invite categories without knowing whether any of the proposals in the LPI group were going to get invited. If they don't, then the numbers will be close to what my panel did.

- Though there has been some debate about whether there should be a preliminary data requirement for the preproposals or if they should be just Big Ideas, I packed mine full of preliminary data and the reviewers responded favorably to it.

- Because all reviews were from panelists, they were short. This is not unusual and often a way you can tell which reviewers were panelists on your full proposals. If you reading 26 of these things, you're not spending a lot of time writing all your thoughts out when you can just discuss it at panel.

- Based on the reviews and my experience, I can tell two things:
1) My proposal went in decently rated (except one person who "didn't get it") but benefited from the discussion. I watched this happen to a couple of proposal in my panel, where the shortcomings seen by one person are explained away by another and soon the proposal looks better to everyone.
2) I had an advocate at the table. The shortest review seemed the most excited by the work and that person clearly influenced the discussion heavily. If you don't have someone like that in the room for your proposal, you're screwed. Thank you to whoever that was.

+++++++++++++++++++++++++++++++++++++++++++

So now the big questions: How much flexibility do we have to respond to the panel comments? How much should we?

Much was made of the possibility that someone could make major changes between their preproposal and their full proposal because there would be no one in common reviewing both to keep them honest. The POs in my panel said that they would be watching for that and some mechanisms were suggested to make sure this didn't happen, but no one asked the real question: why change the proposal that got you there?

In my case I am seeing some familiar criticisms in the preproposal panel summary that I feel I should address. My lab has been working hard towards dealing with these issues since we've heard them before. Now we can, but it means changing at least one aim substantially. Do we go for it, realizing that it may bring a new round of critique? The current proposal was enough to get us invited to the dance, is it worth trying to switch dates while there? Is it okay by the POs to make those kinds of changes or will they cry foul? All of this is up in the air.

So, I'm going to call my PO and see what they have to say. All of this is new territory for everyone, so I imagine that any reasonable modifications will not be denied. Interesting times ahead.

*Don't get all panicky if you haven't heard back yet, different panels are handling notification in different ways and times.

So here's a question for the blog peeps: what percentage of the students in classes you have taken or taught get A's?

In my classes I probably hand out somewhere between 10-25% A's depending on the class. Is it because I'm an asshole and expect unreasonable standards?

Maybe. I guess it all depends on the comparison. Friend of the blog, Namnezia, let it be known on twitter this evening that his 130 student mid-level class has a 65% rate of A's and 90% of the students "earn" a B or better.

To me, this is mind blowing. I don't know where Nam teaches, but perhaps it's the Xavier Institute for Higher Learning but I've been at a few universities in my time. I've even taught a few undergrads.

So my question to you is whether, in places I have not inhabited, it is common for 90% of the students to master 80%+ of the material? Or, could we be looking at a case of ridiculous grade inflation?

Perhaps I'm way off base here.

Is there a formal Space Policy at your institution or are changes are at the discretion of the Dean? If there is a policy, how long do investigators get before they are at risk of space contraction?

"Risky" is a double-edge sword in science that can cut both ways when it comes to funding agencies. In some contexts, such as the R21 mechanism at NIH and some seed programs, risky with a high up-side is a good thing. In more typical mechanisms, risky can be StockCritique (TM) for "we want to see proof before we give you money". Sometimes that criticism is warranted, other times it's just a bar-raiser for having most of the project done before you submit for funding.

Both NIH and NSF have certainly be criticized for shying away from risky projects and both have had trouble responding to those claims. Panels and Study Sections can be conservative with limited funds and unless the POs actively encourage the consideration of potentially transformative (in NSF-speak) work, a prove it or lose it mentality can take over.

But when was the last time you walked out of a really good talk and thought "that was some super cool data that came from a totally ordinary idea!"? Sure, it can happen, but...

When I started my lab I basically started three projects: One that was a slam dunk in terms of working, one that was a reach but I knew we could do and a related one that needed a LOT of ground work to be laid but would be worth it.

That first project we were able to get funded relatively easily (one resubmit), but the other two have taken some time. Part of the reason is that those two have had to be developed from scratch and that process takes longer (and more start-up $) than you think it will. In retrospect, it may have been wiser to have one more slam dunk project in the hopper earlier on before working on these but hindsight is always 20/20.

In any case, with project number two the gamble appears to be paying off. We had a lab meeting today where my grad student presented some compelling data that are demonstrating some of the things we had hoped to see. We have the data and a well thought out analysis plan that have borne some very exciting findings already with more undoubtedly behind them. The project is evolving from something that was risky into something that will be a slam dunk for funding* once we wade through the remaining data. Nearly fours years since starting the lab, with many adjustments along the way, we're seeing the project come together the way we hoped it would.

Yes, this could have blown up in our faces and we had several plan Bs ready to get something out of the data if it did, but if things keep going the way the are** this will be well worth the loss of sleep and gray hairs cultivated over the data and finances of the project.

Four years may seem like an eternity when trying to get the lab rolling (and it has), but seeing the fruit of that labor, the incredible collaborations that have come out of it and the development of the students who have made it happen, is all pretty sweet.

*Inasmuch as anything is....

**que lab disaster that smites me for hubris

More than other aspects, teaching has been one of the most difficult to learn on the job. There is no shortage of advice out there, both in person and on the intertoobs, but the gap between knowing and implementing can be wide at times. The first time I taught my main course I simply survived it. It was the most difficult semester I have had as a PI and there were times when I questioned whether I had made the right career choice. I was a terrible teacher at that point and made dozens of N00b mistakes, mainly because I spent most of my time on content and had nothing left when it came to effective communication.

In year two I was able to get my head around improving the organization and delivery. The third time around I overhauled several aspects of the course and gave it a central theme that wove through the semester. Now that this semester is winding up I think this has been largely effective, as have been several adjustments I have made to how I teach.

But every time I grow comfortable with the class, I realize it needs more. As I make up the exam for the class I am starting to realize that there are a significant number of slides I disregard when making test questions. Whereas some of them are important for context, I have to ask myself whether many of those slides add value or whether they just eat space. The need to fill the class certainly drove some of the content in the early stages of the class, but I feel like I'm past that now.

It has become abundantly clear to me that my students need about three main points from each lecture, and beyond that they get lost. And I struggle to balance my belief that they should be able to handle more if they put the time into the class with the repeated observation that these students do not effectively learn even core concepts without significant and repeated reinforcement. I can either fail a large number or adjust my teaching to the realities of the audience. It's easy to fault the students for not "getting" the material, but I'm not arrogant enough (yet!) to absolve myself of blame. Streamlining the lectures to demonstrate key points and engaging the students in discussion of those points are the goals that I am starting to focus on at this stage.

So, dear readers, what are the tricks you use to organize your lectures and deliver them effectively? How do you get students involved? Is something like clickers useful for a ~25 student class, or have you found different ways to get them engaged? I have tried a few novel (for me) strategies recently to get them talking in groups about certain concepts, but I'm not entirely happy with the outcome.

I feel like I am finally at a point with this class that I can experiment a bit and get out of the straight lecture routine. I make a lot of effort to get responses from the students during lecture, but at this point I realize the limitations of that style, as many students have never contributed a single time. I also think that mixing things up will keep the students involved and make the class more enjoyable for both them and I. I'm open for suggestions.

One of the things I like about reading grant proposals is that it makes me think. I'm asked to read and judge topics that can be far flung from my field and interests. But part of that process is, of course, the generation of ideas.

Some of the proposals are closer to my wheelhouse and I would be lying if they haven't made me ponder some alternative approaches to what we do. How could they not? One proposal, in particular, introduced me to a system that I was vaguely aware of, but not in any detail. While the proposal and questions were quite distinct from anything I do, the system piqued my interest as a vehicle to demonstrate principles we are working on in other systems.

Herein lies a dilemma with all reviewing - determining where the line is between stimulating independent thought and leaning on someone else's ideas. In science, we lean on the work of others all the time, but not in its nascent stages.

So, readers, how do you draw the line when reviewing?