I am hesitant to discuss much of this as I normally don't talk about my work. There are good reasons for that.
You see... I'm the enemy. I am a content specialist for a major producer of standardized tests. My specialty is (duh) science.
Now, I've been in this industry for 3 years now and let me tell you, it's not what you think it is. Unless you are in the industry or actively involved as a client, you can't imagine what it's like. I can share a few things with you.
It takes over 18 months for a question to go from an idea in someone's head to an operational item (that means it's scored and the score counts for whatever the test is for). Each question, depending on the project requirements, will be seen by 2-3 content specialists (usually each one more than twice), artists, copy editors, fact checkers, clients, client committees, and a bias/sensitivity expert before ever even seeing a test form. Then there is field testing, data review, final review by the client... then it MIGHT get on a test.
The first thing most people think of when they hear 'standardized testing' is the recently ended No Child Left Behind and maybe Obama's Race to the Top programs. I will say that it is my opinion that these standardized tests in these contexts are used for entirely incorrect purposes and at incorrect times. But those are client decisions and "him what pays, says". But there are a lot of tests that have to be standardized that you might not think about. Every industry that has some kind of certification exam has standardized tests... nurses, IT techs, aircraft mechanics, etc. etc. Those are generally used properly.
When I say properly, let me explain. What is the purpose of a test? To see if the tester knows something. Now, a well designed test question will not only tell you if the student knows the information, but can also tell you why the student got it wrong. That last bit is critically important and why much of the high school testing... isn't properly used. There's accountability with no chance at improvement. If the tester doesn't learn, then there' s very little point in doing it... if you don't learn, there's very little point in doing anything.
A properly designed test should have a diagnostic component. Which is a pre-test. What does the tester know now? It can identify areas of improvement and even (sometimes) over clues into the misconceptions the tester has so they may be taught correctly. Any assessment is a tool that students, teachers, parents, state officials can use to see what's going on with education at their level. Unfortunately, it's not being used this way (mainly because it is expensive). Again, there's a big difference between public school assessments (which are free for the students to take) and professional certification tests that are not free.
But why a standardized test? Well, that just means that over a given period or group, all the testers take the same test. Their are several reasons for this. One is so that scores can be compared between students, schools, classrooms, socioeconomic groups, gender, ethnicity, ect. And yes, we do compare every test question in every single one of these ways to check for issues. Because tests are standardized, they can even be compared year to year. Usually a group of questions are carried over from one year to the next and these form the basis of some extensive statistical analyses to determine how students compare year over year. It is truly staggering the amount of information that is developed from these tests.
It can go even further. A few of you may remember Obama's "Sputnick Moment". Well that's from another standardized test (PISA) that is given to students all over the world. The same questions given to students in 70+ countries. The US didn't do so well in the latest one, hence the "Sputnick Moment".
Another complaint that people often have about standardized testing is that it is too easy to guess. 99% of the time, the questions are 4 option multiple choice. Well, that is changing. A number of industry leading companies have a variety of new products out. Items that are hot spots, where a tester selects one or more portions of an image and the computer tabulates the location of each click to determine a score. Drag and drop, which is a glorified matching question, but often with some advanced features. There is even some significant research into computer scored essay questions. I've seen a demo and it is absolutely stunning. It is not a word count type of system. It is a learned relational database. It can tell the difference between a BS answer with lots of technical terms and one that has the exact same terms, but correct. I've seen it. It is truly amazing tech.
Sorry for the digression, but I hope that this has given you some insight into the industry. Like any industry, there is a lot of proprietary technology, processes, clients, etc. I can't get into that. If you have any questions, then I'll try to answer them if I can... the more general the better.
But standardized testing is here to stay.
Now, on to my project that I am epically excited out. This is really a pinnacle of the career type of thing. I am responsible for the development of the science standards for a MAJOR client. This isn't state wide or even national. We are likely to go multi-national with it. Now, I'm not doing this by myself. There is the client, various advisory committees, consultants, consultant groups, and a host of businesses all involved. But I'm the guy that is actually putting the words on paper. Which means, a lot of what I say will be incorporated into the science standards. I've made a number of changes and recommendations and the client seems to pleased.
My trip to New York, next week, will be the first of a series of committee reviews of these standards.
When I think about, which I try not to do, I am excited that I am working on such a major project. Then I get seriously nervous. What if I say something wrong, what if I didn't push hard enough to get something vitally important in or get something that ends up a waste of time out?
I actually started another draft document today and that's why I'm thinking about this now. We're talking about being an influence (however small it might be) on literally hundreds of thousands of students a year.
I can say for certain that evolution will be a major theme. The client unambiguously agrees with me and a consultation group that I assembled from experts in science education. We're not going to beat around the bush either. Common descent, speciation, selection, etc. will all be fair game. I am very happy about that. Even if students don't believe it, they still have to learn it and they will learn what evolution is really about instead of the misinformation that is promoted almost everywhere in the US.
I'm just babbling now... and as usual... I'm not sure where to stop. So, if I can answer any questions you might have, let me know.