Research

For a quick overview see DBLP, Google Scholar or Google research page.

Conference Papers

• N-Grammer: Augmenting Transformers with latent n-grams

  • Aurko Roy, Rohan Anil, Guangda Lai, Benjamin Lee, Jeffrey Zhao, Shuyuan Zhang, Shibo Wang, Ye Zhang, Shen Wu, Rigel Swavely, Tao (Alex)Yu, Phuong Dao, Christopher Fifty, Zhifeng Chen, Yonghui Wu

  • Preprint: [arxiv] [pdf]

• Hurdles to Progress in Long-form Question Answering

  • Kalpesh Krishna, Aurko Roy, Mohit Iyyer

  • Proceedings of NAACL 2021 [arxiv] [pdf] [code] [Google AI blog post]

• Efficient Content-based sparse attention with Routing Transformers

  • Aurko Roy, Mohammad Saffar, David Grangier, Ashish Vaswani

  • Proceedings of ACL 2021 [arxiv] [pdf] [Tf code] [Pytorch code] [slides] [samples]

• Unsupervised Paraphrasing without Translation

  • Aurko Roy, David Grangier

  • Proceedings of ACL 2019 [arxiv] [pdf] [poster] [code]

• Understanding and Improving Interpolation in Auto-encoders via an Adversarial Regularizer

  • David Berthelot, Colin Raffel, Aurko Roy, Ian Goodfellow

  • Proceedings of ICLR 2019 [arxiv] [pdf] [code]

• Towards a better understanding of Vector Quantized Auto-encoders

  • Aurko Roy, Ashish Vaswani, Niki Parmar, Arvind Neelakantan

  • Preprint [arxiv] [pdf]

• Fast Decoding in Sequence Models using Discrete Latents

  • Łukasz Kaiser, Aurko Roy, Ashish Vaswani, Niki Parmar, Samy Bengio, Jakob Uszkoreit, Noam Shazeer

  • Proceedings of ICML 2018 [arxiv]

• Adversarial Patch

  • Tom B. Brown, Dandelion Mané, Aurko Roy, Martín Abadi, Justin Gilmer

  • NIPS 2017 ML Security Workshop (spotlight) [arxiv]

• Thermometer Encoding: One Hot Way To Resist Adversarial Examples

  • Aurko Roy*, Jacob Buckman*, Colin Raffel, Ian Goodfellow

  • Proceedings of ICLR 2018. [openreview]

  • NIPS 2017 Workshop on Machine Deception (oral presentation) [pdf] [code]

  • Also at NIPS 2017 ML Security Workshop (spotlight)

• Reinforcement learning under model mismatch

  • Aurko Roy, Huan Xu, Sebastian Pokutta

  • Proceedings of NIPS 2017 [arxiv] [pdf]

• Learning to remember rare events

  • Łukasz Kaiser, Ofir Nachum, Aurko Roy, Samy Bengio

  • Proceedings of ICLR 2017. [arxiv] [pdf]

• Hierarchical clustering via spreading metrics

  • Aurko Roy, Sebastian Pokutta

  • Proceedings of NIPS 2016 (oral presentation). [arxiv] [pdf] [code] [slides]

• A noise tolerant stability-based algorithm for similarity clustering

  • Aurko Roy, Sadra Yazdanbod, Daniel Zink

  • Manuscript. [pdf] [code]

• Estimating anthropometric features from single depth images with very high accuracy

  • Aurko Roy, Sebastian Pokutta, Daniel Zink

  • ICS 2017

• Strong reductions for extended formulations

  • Aurko Roy, Gábor Braun, Sebastian Pokutta

  • Proceedings of IPCO 2016. [arxiv]

• The matching problem has no small symmetric SDP

  • Aurko Roy, Gábor Braun, Jonah Brown-Cohen, Arefin Huq, Sebastian Pokutta, Prasad Raghavendra, Ben Weitz, Daniel Zink

  • Proceedings of SODA 2016. [arxiv]

• Deterministic polynomial factoring under the assumption of the Extended Riemann Hypothesis.

  • Master's thesis 2012 [arxiv] [pdf]

Journal Papers

• Efficient Content-based sparse attention with Routing Transformers

  • Aurko Roy, Mohammad Saffar, David Grangier, Ashish Vaswani

  • Proceedings of TACL 2021

• Strong Reductions for Extended Formulations

  • Aurko Roy, Gábor Braun, Sebastian Pokutta

  • Proceedings of Mathematical Programming, Series B 2018

• Hierarchical clustering via spreading metrics

  • Aurko Roy, Sebastian Pokutta

  • Proceedings of JMLR

• The matching problem has no small symmetric SDP

  • Aurko Roy, Gábor Braun, Jonah Brown-Cohen, Arefin Huq, Sebastian Pokutta, Prasad Raghavendra, Ben Weitz, Daniel Zink

  • Proceedings of Mathematical Programming, Series A 2017

Service

• Program Committee: EMNLP 2022

• Reviewer for FOCS 2015, NIPS 2016, NIPS 2017, ICLR 2018, JACM 2018, ICML 2018, ICLR 2019, ACL 2019, EMNLP 2020, ICLR 2021, NAACL 2021, ACL 2021, EMNLP 2021, ACL 2022, EMNLP 2022

Talks

• International Symposium on Mathematical Programming (ISMP) 2015, Pittsburgh [slides]

• Semidefinite and Matrix Methods for Optimization and Communication 2016 (National University of Singapore) [slides]

• INFORMS Annual Meeting 2016, Nashville [slides]

• Ph.D. thesis proposal [slides]

Technical Reports

• Parallelism in Haskell (CS653, IIT Kanpur 2010) [report] [slides] [code]

• Some results in circuit lower-bounds (CS640, IIT Kanpur 2011) [slides]

• The Borsuk-Ulam theorem and its application to graph theory (CS697, IIT Kanpur 2011) [report] [slides]

• Random walks on directed graphs (CS698P, IIT Kanpur 2011) [report] [slides]