Reviewing Guidelines

Overview

Every student in the class will be assigned to one of two review panels and asked to provide written critiques of 2 of the projects assigned to that panel, which they will present and discuss during the project reviewing session to be held on Thursday, May 7 starting at 9 AM. Please use this template for completing your review.

Reviewers will give each project an integer overall score from 1 (best) to 9 (worst) as well as score each project on four different review criteria using the same scale. Written critiques will will also include the following: (1) a brief summary of the project, (2) a paragraph explaining their overall score, and (3) an evaluation of the strengths and weakness for each of the four project review criteria, which are discussed below.

Please consider the following scoring guidelines when assigning overall and criterion scores:

1 = Exceptional: no weakness and very strong strengths

2 = Outstanding; at most minor weaknesses and very strong strengths

3 = Excellent; very strong strengths with some weaknesses or strong strengths and no weaknesses

4 = Very Good: good strengths with minimal weaknesses; strong strengths that offset weaknesses

5 = Good: approximately equal strengths and weaknesses

6 = Satisfactory: weaknesses outweigh strengths, but there are some strengths

7 = Fair: weaknesses with and minimal strengths

8 = Marginal: strong weaknesses with weak strengths

9 = Poor: strong weaknesses with no strengths

We strongly encourage reviewers to use the full review scale in determining their scores. We are using the standard NIH scoring scale where 1 represents the best score and 9 the worst score in part to avoid the association between numbers and letter grades. NIH panel scores are reported as the average value of the overall score for the panel multiplied by 10, so the best score you can get is a 10 and the worst is a 90. Scores in the teens are really hard to get, and typical scores range between around 20 up to about 50. Scores higher than 50 are possible, but less common because of the practice of "triaging," which means not scoring, applications that are in the bottom half of the submissions.

The review process is meant to give honest and valuable feedback to everyone in the class and is only one of the determinants of the final project grades.

Reminder: All scores must be integer values!

Critique Submission

Critiques are due Wednesday, May 6. 2020 at 9 AM. It is critical that these critiques be posted on time to allow for distribution and review by other panel members prior to the review meetings.

When submitting your critique files, use the following filename: "Proj<number>_Rev<number>.doc", where <number> is an integer corrsponding to your assignment. This string should match the one in the email with your reviewing assignments. Please make sure there is no identifying information in the files you submit. These will be distributed to panel members and we want to maintain anonymity prior to the meeting. Submit as attached files by email to csm2020-instructors@googlegroups.com.

Review panels

Projects will be reviewed by review panels. Each proposal will be discussed and then following the discussion each panel member will assign an overall score . To avoid any conflicts, projects will be divided among panels so that none of the projects was performed by any of the panel members. For this year's class, there will be two panels, and each panel will be reviewing the projects of the students on the other panel.

During the meeting, the reviewers will be called upon at random to present summaries, figures, and preliminary critiques. All reviewers should come prepared to discuss the strengths and weaknesses for both overall and the individual review criteria.

It is absolutely critical that the no one discuss any of the reviews or the projects prior to the panel review meetings, and no one should attempt to "lobby" or otherwise influence other students about their project. Failure to maintain confidentiality of this review process will be considered a violation of academic integrity and subject to both academic penalty and disciplinary action.

Please keep in mind that although the reviews will affect the project grades, the course instructors will take the panels' scoring of projects as a recommendation and will make the final determination of grades. Thus, as a reviewer you should aim to provide an honest assessment of each project. We are hoping that this process provides everyone with critical but constructive feedback about their project, which will be more comprehensive than the instructors alone could provide. In addition, we are hoping that the process itself is a learning experience for all of us. The design of the panel review closely follows that used by NIH and NSF to evaluate scientific project proposals and fellowship applications, which many of you will encounter again in your academic careers. Learning to give and evaluate feedback is an essential component of your scientific training.

Project review criteria


Project evaluations should assess each of the individual components of the writeups. See the this template for a more complete description of how each component should be evaluated. Scores will be given for each of the following:

  1. Introduction and Related Work.
  2. Methods.
  3. Results.
  4. Discussion, Conclusions, and Outlook.

The scores on the specific components are used to determine the overall score. The exact weighting of the individual criteria scores is up to each reviewer, but for the purposes of the class a major goal is to demonstrate technical competence in biological modeling. Beyond that, the overall score should reflect your assessment of the overall scientific worth of the work that was performed. A high-scoring project would be one that with some additional effort could result in a publication in a journal (or conference) that publishes work on biological modeling for which novel experimental data is not required (many high-impact biology journals require such data).