Paper Critiques

Overview

You will need to submit 5 paper critiques from the following 7 papers from the schedule: [2], [5], [16], [21], [24], [27], [38]. You can select, which 5 papers to submit a critique for. Paper critiques will be due by 9:00 AM on the day of the class. You should upload the submissions through the "Assignments" section on Canvas.

Paper Critiques

Each paper will be evaluated along the following axes: (1) the selected research problem, (2) the technical novelty of the proposed approach, and (3) the empirical results. For some tips on how to read a research paper, check the following document by Philip Fong. Your critiques should concisely cover the following points: 

• What research problem does the paper address? 

• What is the motivation of the research work?

• What are the key technical challenges identified by the authors?

• How significant is the technical contribution of the paper in comparison to prior work?

• What are the strengths and weaknesses of the proposed approach?

• What are some of the key experimental results and what do they signify?

• Are there any weaknesses in the experimental section (i.e., unfair comparisons, missing ablations, etc)?

Question & Answer Prompts for a Detailed Paper Discussion

As part of your paper critique, you will also submit a paper discussion question and your answer to that question. We will be using your submitted discussion questions for detailed in-class paper discussions.

Below, I provided a list of general questions that could be useful for coming up with your own paper-specific discussion questions. Note that the questions below are not meant to be used directly in the paper discussions, i.e., they lack specific details that would make for a good paper discussion.

• Is there anything interesting/unique about the selected research problem that is worth discussing? 

• Is the proposed research well-motivated? If not, explain why, and formulate a question discussing this aspect.

• Is the technical contribution of the paper significant/unique? What previous papers (if any) does it borrow ideas from?

• Are there any particularly interesting strengths or weaknesses of the proposed approach that are worth discussing?

• Are there any interesting/surprising empirical results that might not be obvious to a regular reader?

• Are there any weaknesses in the experimental section (i.e., unfair comparisons, missing ablations, etc.) that would be interesting to discuss?

• Are there any claims in the paper that are not backed up by empirical studies? If so, formulate a question around them.

Additionally, here are some broader questions that could lead to a good discussion if your answer to these questions provides a perspective that is unique/sufficiently different from that of the majority of the students. ​​

• How does the paper fit into the broader modern research trends? Is it going to be relevant in the next 1,5,10 years? Why or why not?

• Is the paper useful to the research community now? Is it easily reproducible? Why or why not would other researchers build on this work?

• How does the paper fit in with the other publications of the same authors / their research agenda? Why is this interesting?

• Are there any positive/negative societal impacts that are not immediately obvious or that were not discussed in the paper?

• If you were a conference reviewer, would you accept or reject the paper? Why?

• Are there any good follow-up project ideas stemming from this work?

You should also provide an answer to your formulated question. Your answer should clearly state (1) your position on the question and (2) your arguments to justify that position. If I observe that you are not putting much effort into your discussion questions/answers, I will call on you to answer other students' questions. 

LaTeX Template

To generate your paper critiques, please use the following LaTeX template. Don't forget to compile the tex file into a PDF and submit the PDF in your final submission on Canvas.