January | 2012 | Whizbang

Archive for: January, 2012

It Must Be Measured: #Scio12 #Altmetrics

Jan 31 2012 Published by whizbang under [Information&Communication]

No, I am not referring to d00dly ruler tricks here. We all know that no one actually measures.

Fellow Scientopian DrugMonkey has blogged a perfect storm of a discussion on impact factor and glamour science. Click on over and read the comments (warning: your head may explode). This argument will sound familiar to most readers. Basically, everyone knows that the impact factor (IF) can be gamed by journals. IF reflects some sort of average citation rate for a journal; it says nothing about the quality of any given paper. Some people make the point that IF keeps the measurement of productivity from being solely a pub count. Others add that IF is imperfect, but it's "what we have."

Really?

At Science Online I attended a discussion of Alternative Metrics or altmetrics:

As the volume of academic literature explodes, scholars rely on filters to select the most relevant and significant sources from the rest. Unfortunately, scholarship’s three main filters for importance are failing:

Peer-review has served scholarship well, but is beginning to show its age. It is slow, encourages conventionality, and fails to hold reviewers accountable. Moreover, given that most papers are eventually published somewhere, peer-review fails to limit the volume of research.

Citation counting measures are useful, but not sufficient. Metrics like the h-index are even slower than peer-review: a work’s first citation can take years. Citation measures are narrow; influential work may remain uncited. These metrics are narrow; they neglect impact outside the academy, and also ignore the context and reasons for citation.

The JIF, which measures journals’ average citations per article, is often incorrectly used to assess the impact of individual articles. It’s troubling that the exact details of the JIF are a trade secret, and that significant gaming is relatively easy.

I hoped that the discussion would provide a gentle introduction to the concept of altmetrics. My hopes died, and I felt adrift during the session. I have played with some of the new measures on the altmetrics site. I get what these researchers want to do; I just have not figured out how each measure fits into a bigger picture. [I do appreciate more of the discussion now.]

For a kinder, gentler introduction to the topic, I recommend a piece in the current issue of The Chronicle of Higher Education that profiles Jason Priem, a graduate student in library sciences at the University of North Carolina at Chapel Hill. He helped develop Total-Impact, an altmetrics site that tracks information as it is discussed across the web. He discusses the general concept of the site as well as its current limitations (hey, it's still in alpha).

The internet disrupts traditional publishing; we no longer need to fit the scientific record to the dead-tree world of volumes and issues and page numbers. This shifting paradigm is dragging metrics along, potentially crushing IF in the process.

11 responses so far

Am I Science? Yes, #IamScience

Jan 30 2012 Published by whizbang under Women in Medicine, Women in Science, [LifeTrajectories]

Compared to the other stories posting via this meme, I feel almost traditional.

I do not remember a time when science was not part of my life. I recall fondly reading and re-reading All About Dinosaurs. I had a tiny kit containing most of the minerals in Moh's scale. Mom refused to complete my set with her jewelry, so I had to imagine the upper levels of hardness. Biology clearly won my heart, though. How things could be alive fascinated me to no end.

Unlike many scientists, I was not the outdoorsy type. I read fashion magazines, did a bit of modeling, and entered some teen-queen pageants. I often joke that a hotel without 24-hour room service is my idea of camping. I love air conditioning and indoor plumbing; I fail to see how doing without these conveniences constitutes "fun." This quirk effectively ruled-out a career in paleontology or biological field work. I do love people. Having a father in academia, and coming of age during the 1970's PhD glut, teachers suggested aiming for an MD which guaranteed employment.

Click for source

Admission to medical school was fiercely competitive in that era, and I aimed my sights on a relatively new program at the University of Missouri - Kansas City. The medical curriculum began on day 1 out of high school and went 11 months each year for 6 years. Getting in meant avoiding the cut-throat competition among pre-med students on many campus. Its goal when pitched to the legislature was producing primary care physicians for under-served areas of Missouri, not academic physician-scientists. My second year there, I got a work-study job as a lab assistant for a fresh-out-of-post-doc carcinogenesis researcher needing cheap labor. This year provided my first experience with real science as I worked with the doctoral student and another lab to set up our efforts. Everyone, including this 19-year-old part-timer, needed to generate data. I learned to do short-term lymphocyte cultures, murine surgery, and a number of assays. The principal investigators of these labs strongly suggested that I figure out a way to pick up a PhD to go with my MD, since I loved the science so much.

The next few years brought more intensive courses and clinical work with overnight call, making meaningful lab time improbable if not impossible. I love science, but another kind of love intervened, along with a big princess wedding. By the time I graduated with my BA and MD, the idea of getting another advanced degree sounded exhausting and unnecessary. I headed off to pediatric residency with the intention of becoming a hematologist-oncologist, building on my background in carcinogenesis. Of course, I met a whole bunch of nephrologists and their patients who convinced me to take my talents elsewhere. After all, urine is golden!

My first 6 months of fellowship were a gray blur. Post-partum depression plus a prolonged period of call without a break left me feeling bleak. January in Minnesota is not exactly rosy, but I entered a lab and felt alive again. More than 100 patients with diabetes of various stages had kidney biopsy material stored for study. I began to ask questions about diabetic kidney disease, learning to do electron microscopy along the way. I published papers, completed my training, and landed a faculty position. National funding followed, along with a better position in Omaha, a great place to live and raise our offspring.

Eventually, my science hit the wall. One project just would not work, no matter what we tried. Another project got shot down by reviewer 3 at the same time the NIH budget tanked. I realized that I could not write a better grant than what I had submitted. The probability of getting the funding expected at my professional level was incredibly close to zero. Even efforts with smaller agencies to get funding for pilot data failed, as these foundations cut back support to established investigators during the recession.

The kids left the nest, and my hubby had an amazing job offer in a warmer town. We moved on last year, and I am turning my problem solving skills back to the clinic and to research in faculty development. I still have a grad student back in Nebraska (who is proving reviewer 3 wrong; take that!), and I love the chance to talk science on a regular basis. I do not miss the grant pressure or knowing that several other people will be out of a job if I fail.

Am I still science? When I see a patient, I gather data through a history and physical exam. I create a hypothesis as to what I believe is wrong, and I test that diagnosis through laboratory studies or treatment. If I am wrong, I go back, readjust my hypothetical diagnosis, and test again. Sounds like the scientific method to me.

I may not have a full-time lab. I may not be a funded PI. I still believe that I am science - with incredible fashion sense, of course.

3 responses so far

Who Are They? More Musings from #scio12

Jan 27 2012 Published by whizbang under [Information&Communication]

Over at Take as Directed the always-marvelous David Kroll posted an example of a scientific author taking exception to what he said in a post. The author emailed him about "mythology and gross misstatements" in the original post, and he offered to chat by phone about the issues. David asked him to point out factual errors and make his case in the comments, which he refused to do.

The comments house an interesting discussion, but many respondents also feel that it is not worth their time to participate in the comments of a blog post:

I do relate to the author’s comment about hesitant about engaging in blog comments. We as scientists are often told to avoid the comments sections of posts, as they are a quagmire where the time and energy required for engagement vastly outweighs the effectiveness of participating. I am sympathetic with the author’s decision not to engage in that way.

As a moderator of the Science Online session on the resistance to scienceblogging by journals and other established authorities, I am curious about where this impression came from. Did someone actually tell this person not to engage in blog-based discussions? Is this merely a general impression? What is at the root of this resistance?

Please comment below or over at Take as Directed and let me know what you think. If this effort is worth my time, it's worth a bit of your day as well!

5 responses so far

Female Blogging Manifesto: #Scio12 In Action

Jan 25 2012 Published by whizbang under [Information&Communication]

The Science Online 2012 session on the perils of blogging female generated discussion, both at the conference and on the internet. Comments to female bloggers are not merely sexist. Many are viscious, some are threatening, and some cross the line into criminal intent. If you don't believe me, search the #mencallmethings hashtag on twitter for examples. Kate Clancy blogged about the need for a posse, a group that gets it and can fight off these, well, douchecanoes when they materialize.

A number of us gathered later that evening, expressing our frustration that the session continues to remain necessary. We cannot believe that we have not moved beyond these blatant displays of sexism and misogyny and hate. We are ready to move forward; why isn't the discussion?

Yup, it's pink.

The answer came at the banquet Friday evening, when Janet Stemwedel took to the stage in The Monti Storytelling event. (This story will eventually be available as a podcast here). In the fall of 2011 the blogosphere exploded with a discussion of "gendered" science kits - you know, pink girl kits for bubble bath and cosmetics, while the boys get microscopes and chemistry sets that look like something an actual scientist might have in the lab. These kits reinforce the overwhelming value of girls' femininity while supposedly encouraging scientific endeavors. Dr. Free-ride, her "nom de blog", related how she heard about this topic and thought, "Not again." She felt tired; she wanted to let someone else fight the battle this time.

Eventually, she sucked it up and posted.

Then, a miracle occurred. Someone at this scientific toy company saw the virtual shitstorm on the internet. Multiple blogs, opinions on Facebook, updates on Google+, and a flood from the Twitterverse were not ignored. The company announced that they would no longer sell gendered science kits. They would simply sell science kits.

VICTORY!

Now, I cannot say that without Janet's post that this would not have occurred. Was she the straw that broke the proverbial camel's back? We will never know what the minimal unit of rant is for any given change.

As I look back on our musings in the bar that evening, I realize that we must continue having these same sessions. The conversation and complaints must continue until the appropriate parties notice and act. Yes, we get tired of it. Yes, it is repetitive. Yes, it sucks. But it must be done. If not for us, for our daughters. The real daughters, whether they be tomboys or pretty-pink-princesses, and our daughters in society, those younger than us who want to inquire and write and express their thoughts on an equal footing with the menfolk.

So we will continue to complain and rant and fight and whine and even bitch. Get over it, boys - only then will it stop.

I am in this battle for the long haul. And so is my posse.

5 responses so far

Writing Tools: #scio12 Pays Off Already!

Jan 23 2012 Published by whizbang under [Information&Communication]

Science Online 2012 rocked! Of course, we all knew it would. Old friendships were renewed in real life. New friends were made. I finally graced the Duke Lemur Center with my presence!

One session I attended on the first day dealt with writing tools. While I write a lot for work, most of it has been in that highly structured scientific prescribed format. Standard word processing tools handle that work fine. As I have moved to other forms of written output, I find that these usual text editors often fall short. Based on less formal conversations at #scio11, I outfitted my computer with Devonthink, a Mac-only research tool with artificial intelligence engine, and Scrivener, a writing tool with text editor that allows you to organize thoughts, facts, and other musings, then rearrange them easily. Eventually you can output them to another program for final referencing and formatting. I love Scrivener already, and I have only touched the obvious capabilites of the program. Devonthink has a steep learning curve, and I have not yet mastered it.

Putting the "Prod" in Productivity

The latest addition to my repertoire is Dr. Wicked's Write or Die (this post is starting out on that program). This bargain software (desktop version is $10 and they encourage you to install it on all of your computers) has you fill in how many words you wish to write in a given time frame. You can set it for gentle cues: when you pause, your background color changes. If you fail to resume writing the color deepens, and eventually you get an alarm.

When this software was described at the unconference, the Kamikaze mode also got our attention. With this setting, if you pause too long it starts erasing your work. Yikes! I am not ready to go there yet! You can also set it to disable your backspace key so you cannot self-edit as you go. That will be my next step, since I probably do too much immediate editing (yeah, I have spelling issues).

I downloaded the program to my laptop Sunday while several of us awaited airport transportation in the Brownstone Doubletree Lobby. Usually I get distracted and converse when in a group, even in a group of all strangers. Those visual prods kept me on track, and I finished 800 words in about 30 minutes for a piece due today (OK, technically it was due last Friday, but no one will die if it goes in today). When the word goal hit, triumphant trumpets sounded, a congratulatory window popped up and the whole group turned my way.

I. Felt. So. Proud.

I cannot wait to hit my goal for this current post. I hope someone hears it just outside my office, so I can gloat.

If you have trouble with the "sit down, shut up, and write" command, Write or Die could make you way more productive. It continues to amaze me that even at this level of education and motivation, we still struggle with getting an initial draft written. We still need the carrot and the stick.

By the way, the first draft of this post (500 words) took just under 15 minutes with Write or Die watching. Add another 15 minutes to clean it up a bit and add the illustration and links after I finished my clinic. Oh, and both of the reports that were due today were submitted before I started blogging.

3 responses so far

All Packed for #Scio12

Jan 18 2012 Published by whizbang under [Information&Communication]

I have one really tightly packed roll-aboard suitcase and a purse that can hold wallet, MacBook Air, iPad and iPhone with some room to spare in case I need a lip gloss. The suitcase contains clothing for 4 days (including closed-toed flats for every outfit), chargers for all devices, toiletries and a nice tote bag. That way on the home trip I can check the suitcase and use the tote for extra swag.

I got the tour I wanted!

Unfortunately, I do not have an extra meeting lanyard. Anywhere. I moved last summer and divested my office and home of unnecessary crap, for lack of a better term. The hospital gift shop had some blingy ones, encrusted with various crystals; I never got by there to pick one up yesterday. Perhaps shopping opportunities will arise during the 2-hour layover in Houston.

Up next I have a Skype with my lab at my former employer. Then I get to do another conference call. Then I scoot to the airport for my visit to the mother ship.

See you in Raleigh, IRL or virtually!

No responses yet

Why Resist? #Scio12

Jan 10 2012 Published by whizbang under Uncategorized

I will be doing more than having fun in real life with my blogging buddies at Science Online next week, although fun will happen. On Saturday, from 10:45 - 11:45, Holly Bik and I will moderate a discussion on the resistance to science blogging.

Resistance occurs at two levels. Many in academia see online activities as a waste of time. They may admit to some value in outreach via the internet, but blogs are not something for your CV. Certainly, these social media activities could not provide any value to your career. So quit wasting your time and get off my lawn!

Traditional journals have generally had the same attitude. While most publish manuscripts online, it's the version that gets typesetting (pixel-setting?) that is the official version forever and ever, amen. Our publications still get referenced as if in print, even though many of us no longer deal with the dead-tree form of articles. When I changed institutions recently, I bought a high-speed scanner with a sheet feed and made into PDFs any articles I could not download. Not moving almost 3,000 articles is a good thing!

The most interesting example of internet-age post-publication review happened about a year ago. On 2 December 2010, Science published online an article suggesting that arsenate could replace phosphate in bacteria. Aided and abetted by a rumor mill leading up to a NASA press conference, this paper provoked immediate reactions. These ranged from disappointment (not really extraterrestrial life) to interest (growing with arsenate is interesting) to outrage (this science is flawed). The science blogosphere exploded with posts, many by reputable scientists who could have been peer-reviewers for the original article. Two nice lay summaries of these reactions were written by Carl Zimmer for Slate and Ed Yong for Discover. On 10 January 2012, a google search for "arsenic life nasa" yields approximately 6,000,000 hits.

The discussion ranged freely from serious scientific critiques to bashing NASA for over-hyping the study. It was messy, disorganized, unregulated, and fun. It reminded me of some places that hold journal clubs with wine.

Many of the microbiologists who blogged their problems with the paper turned their critiques into real comments submitted to Science. When the "Arsenic Life" paper actually hit the print version of Science on 3 June 2011, 5 of these formal commentaries also appeared in print (after online publication 27 May 2011). By then, the hurricane was downgraded, moving over land. The real discussion had taken place months before.

I decided to look more formally at the gap between print publication and correspondence in another weekly journal, New England Journal of Medicine. NEJM, now 200 years old, published 51 issues in 2011. I examined the Correspondence section of each issue for letters regarding previously published articles; case reports and corrections were not counted. The time from original print publication to the correspondence publication was determined for each of 517 letters to the editor.

Each issue contained, on average, 10 letters (range 5-20). Two letters regarding each article were published in an issue, on average (range 1-7). The average time from print article to letter publication was 14 weeks. Three articles had correspondence published simultaneously in the print edition a few weeks after online publication. One letter got in 41 weeks after the original publication. Only one letter showed a real "conversation": a July 2010 paper led to an October 2010 letter which led to a February 2011 letter.

The official correspondence proceeds at a stately pace, and NEJM requires that those authoring letters reveal potential conflict of interest, something that may not be obvious in the online world. This world is completely dignified and authorized, but slow. It also lacks the interaction of the net, where astute readers may alter one's perceptions within hours. Yes, the millions of voices fighting it out online can be overwhelming, and you do not have the validation of the journal. It still seems like there must be some way to capture that process, often more thoroughly written and referenced than letters (in which brevity rules) and make it an official post-publication review forum.

Even if we fail to make this chaotic interaction official, we need to encourage it so that all viewpoints can be examined. Too many journals do not allow any official discussion about their offerings. We bloggers can pick out an article, review it, and let the discussion proceed on our sites, but it would be more direct if the journal helped make this activity happen! Some editors argue that alternative opinions may lead to lawsuits, although I think they underestimate their liability for the science.

What do you think? Can we make this happen? Should we do this?

Attend the session in real life or virtually; I hope we have a good discussion and a great time.

3 responses so far

Advice for #Scio12 Noobs

Jan 09 2012 Published by whizbang under [Information&Communication]

Last year was my first to attend Science Online. Yes, I was a Noob! This year, as a seasoned veteran, I can provide some advice for those of you entering this wonderful group for the first time.

When it comes to electonic gizmos, more is more. This conference generates more bandwidth than anything else I attend. Find yourself in a session that's not what you expected? Pull up the live-feed of the others and change rooms. Some adventurous souls will try to live-blog sessions, while many of us settle for twitter-notes. Virtually everyone will be juggling laptops, tablets, and smart phones. That's who we are, for the most part. Don't have a smart phone? This is the place to get an overview (AKA knock-down, drag-out debate) on the relative merits of Android vs. iPhone platforms, especially if you get the right two people together in the bar. You have been warned.
You will feel welcomed. The group tends to be quite friendly, and your first task will be to hug Bora, the Godfather of Science Online (if you have had your flu shot, of course). I remember people running up to me, exclaiming how good it was to meet in real life! Since many bloggers use pseudonyms, I had NO IDEA who some of these folks were, at first. For some, I still don't know a real-life name, even though I feel like we have a great bond!
Try not to stare open-mouthed at your heroes. Meeting some of these writers can produce feelings of awe (yes, some of us are science groupies), but they are just as friendly and welcoming as the rest of the crowd (see #2). Want your books autographed? Bring them along!
Ocean bloggers are at least as welcoming as the rest of the crowd; however, they seem to have an alcohol tolerance well above the rest of us (and I have NEVER felt like a light-weight at any other meeting). Does this have something to do with time at sea? I don't know. Just be careful. You have been warned (and neuroscientists, don't get snitty; my experience is that the ocean crew can outdo you).
Be prepared to bring extra stuff home. Even with the new swag policy, I suspect we may need to check a bag and tote home an extra bag of new acquisitions. I still have my Sigma Life Science Magic 8 Ball on my desk. Yup, it made the cut for the move. Sometimes it provides the clearest, most logical solution to a daily conundrum.
Be prepared for an amazing experience. Science Online was like visiting the Mother Ship; nowhere else have I encountered this many people who, like me, love science, the written word, and online communication.

If you're a Science Online Veteran, feel free to leave your tips in the comments below. If you're a 2012 noob with a question, ask in the comments and you will get answers. I promise!

14 responses so far

Getting the Right Answers

Jan 06 2012 Published by whizbang under [Education&Careers]

I am still working like a fiend to get something done for a deadline later this month. It must be ready before I go back on the inpatient service next week and attend Science Online 2012 the following week. I do not mind last-minute tweaking, but I do not want to write the whole damn thing the day before.

This situation considerably limits my blogging time.

This morning, I browsed my LinkedIn headlines. If you haven't been over to "Facebook for Professionals" recently, you may not know about this feature. Based on your interests and groups, you receive 3 links to stories each day (sometimes more if you click over there throughout the day). One today, from Harvard Business Review Blogs, discussed The Art of Asking Questions.

As Ron Ashkenas points out, people rarely consider how well they ask questions:

From my experience, most managers don't think about this issue. After all, you don't usually find "the ability to ask questions" on any list of managerial competencies; nor is it an explicit part of the curriculum of business schools or executive education programs. But asking questions effectively is a major underlying part of a manager's job — which suggests that it might be worth giving this skill a little more focus.

His post provides some interesting views of this skill in the business arena, but he misses the most important point of all: Asking the right question! In science, we often ask questions, test a hypothesis, and get results that make no sense. The ability to take those crazy data and ask the next critical question is a key skill in science, often the difference between success and mediocrity.

In medicine, asking the right question becomes a matter of life and death (or prolonged sickness vs. speedy recovery). Asking the patient for the key piece of history or ordering the correct test are clearly crucial. If you don't measure the blood pressure, you will never diagnose hypertension!

When I assess students, I always comment on the questions they ask. Their questions show me how much basic physiology they have retained and how well they can use the information. Students who ask no questions in clinic are uninterested at best. Those who ask great questions are the stars.

No responses yet