For Reviewer
Scientific Paper: General Review Process
(Summarize from the lecture of prof. Kristin Sainani at Stanford University)
Attitude:
1. Avoid criticizing the authors, criticize the work
2. Point out specific errors
3. Using positive sentences
Process:
1. Scan the abstract
2. Jump to the data: review the tables and figures
a. Draw your own conclusions
b. Do the tables and fig stand on their own?
c. Are there any obvious statistical error?
d. Is there repetitive information?
3. Read the paper once through
a. Do the authors’ conclusions match their data?
b. Is the paper clearly written, or did you struggle to get through it?
c. Is the length of paper justified given the amount of new information that the data provide?
4. Read the introduction carefully
a. Is it sufficient succinct?
b. Does it roughly follow: known-> unknown-> research question/hypothesis?
c. Is there a clear statement of the hypotheses or aim of the study?
d. Is the detailed information about what was done that belongs in the method?
e. Is there information about what was found that belongs in results?
f. Is there distracting information about previous studies or mechanisms that are not relevant to the hypothesis being tested?
g. Do the authors tell you what gaps in the literatures they are trying to fill in
5. Read the methods carefully
a. Scan this section to find answers to your question about the data
b. Objectively, Subjectively
c. Read the statistics section carefully
6. Read the results carefully
a. Read this section with the tables and figures in front of you
b. Does each section roughly correspond to one table or figure?
c. Do the authors summarize the main trends and themes from the table, or do they just repeat what is in the tables?
d. If there are graphs, do the authors give precise numerical values in the text if it is not given in the graph?
e. Are the authors honest or do they try to draw your eye to what they want you t see?
f. Do the authors over-interpret statistical significance, by ignoring the fact that the magnitude is small or by ignoring the fact that they have done multiple subgroup analyses?
7. Look at each table and figure.
a. Did the authors choose the correct statistics?
b. Are there multiple tables or figures that tell the same story?
c. Is there evidence of cherry-picking or purposefully omitting data?
d. Are any graphs misleading?
e. Is the “treatment” group always compared with a proper group?
f. Are there inconsistencies in the data they present from one table to the next?
g. Did the authors make transcribing errors when going from the data in tables/results to the abstract?
8. Read the discussion carefully.
a. Does the first paragraph succinctly and clearly tell you what was found and what is new?
b. Are the authors’ conclusions justified or are they overreaching?
c. Do they clearly distinguish hypothesis-driven conclusions and exploratory conclusions?
d. Is the writing clear and to the point (active voice)?
e. References are correct?
f. Have they omitted the key reference?
9. Comments to authors:
a. Start with a one-paragraph “general overview”
i. State what you think is the major finding and importance of the work
ii. Give 2-3 positive, encouraging statements about the work
iii. State 1-2 major limitations (if there are any) to study design, writing presentation or conclusions.
iv. Do not tell the authors your overall recommendation (reject,…)
b. In a number list, give 5-15 specific criticisms, suggestion for revision
i. Point out specific mistakes
ii. List the issues that you found in your review
iii. Give specific comment
c. Focus on big-picture issues (do not spend your time nit-picking)
10. Comments to editors:
Following the form given by the journals