A Quick Template for Writing Retrospectives

January 1, 2099

• Original Paper : An investigation into general relativity
• Paper written by : Alfred Einstein, Mary Curry
• Retrospective written by : Alfred Einstein

Paper TL;DR

Give a short description of the original paper. What claims did you make? What were the main results?

Overall Outlook

Looking back on your paper, what are your thoughts about it now? What’s been the impact of the paper in the community (if any)? Give an overview of the things you’ve learned about your paper since it’s been published.

Opportunities for Improvement

There are many things you can talk about in a retrospective. We list three of them below, but you can write about all or none of these. CURRENTLY ALL ML

1. Flaws or mistakes in the paper’s methodology. For example, you discovered a bias in your dataset, or there was accidentally some overlap between validation and test, or there’s a weird bug that you don’t know the cause of and didn’t really figure out. Maybe you didn’t run enough seeds, and when you run more seeds your method isn’t actually that much better. Basically, any way in which your scientific methodology was flawed or lacking.
2. Limitations in the applicability of the work. For example, you have reason to believe your approach doesn’t generalize well to other datasets, or your results are not robust to small changes in the experimental set-up. Maybe you tried switching from MLPs to LSTMs and everything broke. Maybe your classification model works better, but is less robust to adversarial attacks. Basically, any way in which your approach was limited that you didn’t realize or acknowledge at the time.
3. Changes in understanding or intuition. For example, you have a simpler way to motivate your idea, or your intuition for why your model or approach works has changed. Maybe there are new aspects of your idea that you didn’t explore in the paper, but want to talk about, or new visualizations that improve understanding. These can be accompanied by additional (informal) results. You can also point to new work by others that is relevant to the paper. Basically, anything that you’ve learned since writing the original paper that would be useful for others to know.”

This Opportunities for Improvement section is a way to combine your thoughts about points 1) and 2). What could have been improved about your paper’s methodology? What are some limitations of your work that you’ve learned since publishing? The more honest you can be about your thoughts, the more the community will benefit.

Subsection

This is a subsection where you could describe one of the opportunities for improvement. It’s nice to separate each opportunity for improvement by a different subsection, since this makes your retrospective more readable.

For example, I could have written some code like this, which would have made my approach more reproducible:

for thing in range(list_of_things):

    if thing > other_thing:

        print("I'm inside a print statement!")

New Perspectives

This is a section where you can talk about new perspectives that you have on your paper. What have you learned that would be helpful for other people to know? Is your paper’s approach worth pursuing in more detail, or was it more of an interesting curiosity? What are interesting directions for future work? Maybe some new papers have come out that have changed your view about what’s important about your paper.

If you want to be brief, you might decide to share your perspectives in bullet-point form.

• These are some bullet points.
• They are created using the * or - symbols.
  • You can also make nested lists using tabs.

Other sections

You can write other sections too, if you like! We don’t really have anything else to say here. You can write italics, bold, and bold italics in Markdown just by using a different number of asterisks. If you want more quick tips on using Markdown, see the page here.