What constitutes a good academic review?

As an academic I spend a significant chunk of my time writing papers. While submitting my papers to publishing houses, I often keep my fingers crossed, hoping that the reviewers assigned to us will do a good job at assessing our work. The reason why this is an issue is because most academicians are trained to carry our research, and write papers. However amazingly little time or energy is spent to understand the process of peer review. In fact, even in graduate school, there is little or no training provided to students, so that they will go on to be great reviewers.

I must start with an honest confession: I have made several errors with the review assignments that were handed on to me. However I am not the only one who started out with a gross misunderstaning of the review process. I can say this with a fair degree of surety since I have also been at the receiving end of some really low quality reviews. I suspect the reason for this is that practising scientists put in very little effort or time into understanding the review process. And most researchers (even the really good ones), do not spend as much time thinking about what a good review means.

Given the circumstances, it seems like a few basic guidelines on academic peer reviewing will be helpful. As always, please remember that these are merely suggestions, and the list of guidelines provided here are not meant to be exhaustive.

Rule 1: If you feel unqualified for reviewing the paper, please get back to the editor immediately. The review process should not be delayed under any circumstance. Remember: the author's professional success might hinge on this work.

Rule 2: Anonymity does not imply lack of responsibility! Since academic peer reviewing is a process wherein the reviewer's identity remains undisclosed; there is a tendency to look at it as a process which does not have checks and balances. This is not true! Constructive criticism is always welcome, and publishing houses take note of reviewers who take their jobs seriously. More importantly, as academicians, it is our job to nudge our colleagues in the right direction, by offering sincere, honest, and constructive feedback.

Rule 3: While providing review, substantiate your claims, and ensure clarity. For example: if you have objection to a few statements made in the paper/book, explain the origin of your disapproval, and cite references appropriately. This helps generate a healthy academic dialogue. Vague or ambiguous comments must be avoided, since the authors would not know what they must do to improve their paper.

Rule 4: Be polite! Especially since peer review process is anonymous, people can be far less polite as reviewers than they are in person. Some conscious effort is essential to avoid this tendency under all circumstances. It is helpful to remember that authors are also people with real emotions. How you express your comment will influence them. You may disapprove; but do so politely, and provide clear reasons for disapproval. Denigrating any one cannot be right, no matter what.

Rule 5: Try not to get swayed by "dons"! Science is very dynastic (I may be shot down for saying this, but it is true!). There exist powerhouses in science, and if you are in and from one of those powerhouses, you have an intrinsic advantage for getting through the review process. This is because we are all human, and we tend to favor the big and powerful. However the review process is expected be fair towards every one: hence regardless of who authored the paper, please spend time to understand the work, and provide constructive criticism. It is helpful to avoid the tendency to be a "yes man" for the great and the mighty. Being "equally" just towards every one is one of cardinal virtues of a good academic.