Paper Review Checklist

• General rules

  • Be polite and friendly even if the authors are not. This makes ourselves professional in the view of editors. 

  • Keep the reviewed material secret. 

• Usage of AI tools: 

  • Do not use AI tools to review papers. Doing that might lead to the leakage of secrets in the submissions prior to the publication of the papers. 

• Review comment format:

(Paragraph 1) Short summary of the paper, can be extracted and rephrased from (1) Abstract, and (2) the last paragraph of Section 1 (technical contributions).

(Paragraphs 2-(N-1)) List all issues, from the most critical issue to the minor issues.

(Paragraph N) Minor issues, e.g., typos, grammar errors, overuse of passive voice, inappropriate wording that does not affect semantics.

• The review standards for journal articles and conference papers:

  • Conference papers: 

    • Conference papers must be Accepted or Rejected. No revision opportunities are given. 

    • Conference papers are judged by scores. Typical range of scores: 1~5, 1~7, -2~+2, and -3~+3, etc. 

    • Check the historical acceptance ratio to determine the scoring standard. Occasionally, the target acceptance ratio will be announced by the Technical Program (Co)chair. 

    • For high-quality conferences, a negative score (e.g., 2 in 1~5) often leads to the rejection of the paper. Meanwhile, most high-quality conferences requires an average score of positive side (e.g., >= 0) to accept a paper. 

    • If possible, try not to give a neutral score as it often indicates "I don't know" or "I cannot judge that." 

    • Some conferences have rebuttal opportunities so that the authors can respond to the 1st-round review comments to clarify misunderstanding issues. However, rebuttal does not imply revision opportunities. 

    • Some conferences (e.g., ESWEEK) are co-published with journals. In this case, the papers can be viewed and judged as journal submissions; the only difference is that the revision period is much shorter (e.g., 3~4 weeks). 

  • Journal articles:

    • If at least one critical issue is found, in the 1st round of review, must give Major Revision or below. If a major issues remains in 2nd or later rounds, give a Resubmit or Reject.  

    • If multiple critical issues are found and they are unlikely to be fixed in the period of Major Revision (~2 months), give a Resubmit or Reject. 

    • If critical flaws are found in the problem definition or proposed solution, give a Reject or Definite Reject. 

• What to check:

  • For research-track papers:

    • Is the topic OR solution new? If NOT, what new key findings are proposed?

    • Is the background study rigorous (does not misunderstand the prior literature) and complete (miss any highly related work)? 

    • If all related works are outdated (>= 3 years, especially >= 5 years, for CS and AI papers), must defend the novelty of the work.

    • Are the story (the flow from background -> system architecture & problem definition -> motivations) of this work reasonable? 

    • Are the evaluation methods (mechanical/statistical analysis or experiments) reasonable? 

    • Is there any closely related work not compared in the evaluation? If not, must defend why this is reasonable. 

    • For data-driven simulation, are the used data reasonable? Are the data available to the public for reproducibility? 

    • If concrete absolute values are not provided (only relative values are provided), and this makes assesses difficult, request the concrete absolute values. 

    • Check for circular reasoning or other common logical flaws.

  • For survey papers: 

    • Does the topic deserve a survey paper? 

    • Are there any prior reviews with the same coverage (note: not necessarily with the same keywords!)? If YES, what are the differences? What are the "novelties" of the present submission? 

    • Is the survey method systematic? 

    • Are there any related works missing in the survey paper? 

    •