IF discouraged THEN write good reviews

Ever get a Bad Review? I don't mean one where the reviewer gives constructive criticism and recommends rejection. I mean one that is really bad in the sense that it is unhelpful, off-topic, lacking in rigor, poorly written, pedantic or pompous. It takes a lot of energy to sort through these sorts of reviews, find the wheat discard the chaff and make sure that the experience doesn't drag you down to the point of derailing a potentially productive scientific endeavor. Bad Reviews sometimes even recommend acceptance. Acceptance leads, perhaps not to disappointment at the moment, but rather to more general scientific disheartenment: Is this really the level of intellectual standards that characterizes the field to which I have chosen to devote my career?

A surprisingly satisfying way to push back against Bad Reviews, is to engage in reflection upon one's own reviewing skills and strive to improve them. This course of action is not going to have an immediately measurable effect of improving the system as a whole, but it does restore a sense of balance. Especially if you interact with a lot of students, you have an amazing opportunity to teach them to review. There's something cheering about knowing that scientists that you have mentored are not going to be the ones generating the Bad Reviews of the next generation.

In order to be able to tell people quickly about my own reviewing values and my campaign to constantly improve my own reviews, I have packed the points I consider while reviewing into a scheme that I call IF THEN:

• I is for Issue: Does the paper motivate the issue that it addresses and then close the loop in the end, convincing the reader that it has accomplished what it set out to do?
• F is for Fit: Does the paper fit with the call for papers of the conference or scope of the journal to which it was submitted?
• T is for Technical soundness: Do the authors apply solid, state-of-the-art experimental and/or analytical method?
• H is for Historical context: Do the authors present the context of their work? (Including both the related work and the outlook onto future work.)
• E is for Exposition: Is the paper clearly written and a pleasure to read? Is the information it contains complete and comprehensive?
• N is for Novelty: Is the idea new in the field? Is it the sort of innovation that is destined to make an impact?

If you're doing IF THEN right it should be a bit uncomfortable. Particularly difficult is the self-reflection necessary to make an honest estimate of one's own expertise in some subjects. I didn't promise this wouldn't hurt, what I promised is that constant work on your own reviewing standards eases the pain (and and controls the damage) caused by receiving a Bad Review.

But what if you are already a world class reviewer? What then? Like musicians we must remember that excellence is not static. Moshe Vardi introduced a rule for reviewing an editorial in the current edition of Communications of the ACM. The rule reads, "Write a review as if you are writing it to yourself." He calls it The Golden Rule of Reviewing. Most people are still working on putting in to practiced the Golden Rule they learned in their childhoods. The Bad Reviews will keep on coming, and about the only thing we have control over is how we review back.