Saturday, January 22, 2011

Terrible reviewing practices

As I mentioned in my last post, I got a paper rejection this week. That's nothing to write about in itself, but the journey that this particular paper has gone through is worth telling.

So let me start out by saying that this paper reports some seriously ground breaking results. I know it, my co-authors know it, everyone we have talked to about it knows it. And I personally have talked tens of other PIs in the field about it, and they agree. I've presented the results at conferences with much fanfare. So that's why it was initially submitted to Science. They said no. No shock there, the editors gave some half-assed response. So we sent it to Nature. Again a no, but this time the editors spent a few paragraphs summarizing the work and explaining their decision. I was seriously impressed with that. But their response boiled down to: we think these are great results, but it won't generate enough citations because it's not such a hot field. So next we sent the paper to the most appropriate Nature subjournal. It's a fairly "interdisciplinary" paper so it doesn't exactly fit into one of the Nature derivatives. They also stated fairly obviously as well that the paper was being rejected because it won't generate enough citations. In the meantime, we talked with more people about it and they all say that it's great stuff that will significantly affect the field--which to me sounds like it will be cited often. We tried the other Nature derivative that fit. Again no. Same reason. Never made it past the editors.

So finally we gave up and sent it to lower journal. Now, we were sure this would be a cake walk. Man, were we wrong. The paper had only two reviewers. The first had some well formulated comments and concerns, but recommended the paper for publication after the appropriate, relatively easy revisions. The second was a raving lunatic. They went on for almost a page. Almost no punctuation, spelling mistakes everywhere, no useful or insightful criticism. Outright recommendation for rejection. It really feels like some first year grad student jotted down comments on the paper after a bad day, perhaps after getting their own paper rejected, and the PI submitted the review without so much as looking at it. The editor must not have read the comments either, because it came back as a rejection based on the recommendations of the reviewers. The line has been crossed, this one is getting appealed. We'll see what happens.

Out of all this though, what bothers me the most is the raving lunatic review. Since when can reviewers be so unprofessional? I've gotten poorly written reviews in the past, at least once or twice that I can remember, but of course didn't mind too much because they were positive. Two or three phrases, not even sentences. Reviewers have at least a college degree and in theory the person getting the paper to review has a Ph.D. Is being insanely busy a valid excuse for not, at least, hitting spelling and grammar check in Word? Have these people never taken a professional writing class? Do they not realize this is wrong? This thought caused me to take action with my own trainees. I'm having them write up reviews for a paper and I'm going to critique what they submit to me next week. Maybe all new PIs should get a class in how to review a paper.

Do you folks (all 2 of you reading this) come across such poorly written reviews as well? Is there an editor out there that has ever returned a review back to the reviewers for doing a shitty job? Or at least told them that they should try harder in the future?

4 comments:

  1. Hi there, and welcome to the blogosphere!

    Regarding reviews, you can totally request another reviewer if one is being unreasonably hostile. Since you seem to be working in nanotechnology, I wonder where you sent it after Nature Progeny: NanoLett or PRL or something else?

    About poorly written reviews: they happen, don't take them personally. Be calm and appeal based on unwarranted hostility. About getting rejected in GlamourMag, I can tell you from experience, a lot of it has to do with a lead senior author who's not famous (I presume you are it?) For all the GalmourMag and PRL's I have had published, the review process has been extremely ugly, long, confrontational, and scarily arbitrary. And each time it helped that one of the co-authors was a BigWig senior prof who argued with the editors directly; whenever that was not the case, we didn't get in. So quality is a relative thing: young person's super paper does not equal BigWig's super paper.

    So brace yourself for battle, don't take this shit personally, and try to get into as high-impact joournal as you can without losing your sanity. Your TT has just started, so get the paper out and fast and move on. Best of luck!

    ReplyDelete
  2. Hi EngineeringProf,

    Heh heh, I am at least the second person commenting on this. I've also noticed a trend in engineering departments disregarding good quality work in disciplinary journals (ASME / IEEE) and increasingly looking for a Nature or Science publication instead. The problem is that these have absolutely dismal prospects for people working in many branches of engineering. You could do the best thing *evah* in your area and have Nature's gatekeepers turn in down saying it isn't of interest.

    Is this because engineers have got physics envy, just like physicists suffer from math envy?

    -Phdguy

    ReplyDelete
  3. To add to my previous post, I totally agree with you about the arbitrariness of the peer-review process in nominally "high-quality" journals.

    If I am to have a submission rejected, I'd at least like to have the satisfaction of knowing that it was reviewed by competent reviewer(s), not some obviously disgruntled grad-student whose celebrity prof can't be bothered to review papers entrusted to him by the journal editor.

    -Phdguy

    ReplyDelete
  4. Hi Phdguy,

    About the trend away from disciplinary journals...that's because we are all becoming slaves to the h-index and impact factors.

    Plus all of our research has to reach the broadest possible audience. It's not enough any more that research is appreciated by people in the field. Administrators would love to have a news team or reality show crew following faculty around, giving their university the ultimate exposure, even if what they report or show is completely useless or even wrong.

    ReplyDelete