Readings

Paper presentation & scribe information

In the first class period, we have handed out paper cards to assign paper presentations to students for the entire semester. Here are a few important administrative notes:

• Please let us know your paper assignments: This google doc keeps track of which students hold which paper assignments. Please update this document with your assignments.
• If you are registered for the course and did NOT get paper assignments, please email the instructors at akbc-692a-organizers@iesl.cs.umass.edu
• If you are dropping the course and have paper assignments, please remove yourself from the assignment google doc and arrange a time to return the green paper cards by emailing akbc-692a-organizers@iesl.cs.umass.edu.
• Each student should be assigned 2-3 presentations (no more than two fifteen minute presentations)
• If you are trading paper assignments with another student, all that you need to do is update the google doc. Nothing further is required (e.g. emailing instructors).
• Each paper assignment card points to a particular paper assignment using a code of the format [week_id:paper_id]. Each lecture has its corresponding [week_id] in square braces. The assigned paper codes are listed in the schedule. PLEASE ensure you've located the dates of your presentation. For example, "[Type+Link:P2]" corresponds to "[Type+Link:P2] DeepType: Multilingual Entity Linking by Neural Type System Evolution. Jonathan Raiman, Olivier Raiman · AAAI · 2018" on the schedule page which is on 09.20.2019. Note that spotlight presentation cards indicate the week_id in square braces and are of the format, [Type+Link] Spotlight 1, [Type+Link] Spotlight2, [Type+Link] Spotlight 3 for the three spotlight presentations in the [Type+Link] week on 9.20.
• Please inform your classmates (and instructors) of your spotlight paper selection: by adding it to paper assignment google doc.
• Please ensure you don't select a spotlight paper that someone else has already selected.

As described in the first meeting, there are two kinds of presentations:

• 15-minute presentations -
  • A pair of students will work together to prepare and deliver a 15 minute presentation about a research paper or papers (in some cases two papers are assigned).
  • This presentation should assume that other students in the course have read the paper.
  • Interactive class exercises as part of presentation are encouraged.
  • The structure of the presentation is flexible, students should cover the research questions addressed by each paper, the contributions of the paper, methodologies introduced by the paper, empirical results, related work, future work, work impacted by the paper, etc.
• 3-minute spotlight presentations -
  • A single student will deliver a 3-minute presentations on the presenter's choice of paper from the suggested reading of the given week.
  • The spotlight presentation is meant to highlight contributions of the paper and inform other students about the main ideas of the work.
  • Check this paper assignment google doc to make sure you haven't selected paper that others have selected.

In addition to presenting papers, students will be assigned to scribe duties. Scribe notes are taken individually and are shared on the course website. These notes should provide a written summary of the assigned presentation. This google doc stores scribe assignments. Here are example scribe notes: example 1, example 2, example 3. Note that the topics of these examples may be more technical and broad than some of the topics discussed in this class. It is fine to have notes that are more brief than these examples if more detail is not required to convey the subject matter.

• Scribe notes are due within 1 week of the lecture and should be submitted here: https://docs.google.com/forms/d/e/1FAIpQLSc4N4vcebAMN6__jMoceKLNidlfgkgYiDsye6pOBzGmVZwCrA/viewform?usp=sf_link
• If you cannot use Overleaf to typeset your notes, please email the instructors.
• Link to latex template: https://drive.google.com/file/d/1aAoIrLSHTTDXk7tHKNrK5IlTC_8djbrV/view?usp=sharing

Additions? Corrections? Changes?

If you have a paper you would like to add to this list or have found a mistake in the information in this list, please email akbc-692a-organizers@iesl.cs.umass.edu

Suggested Reading

Knowledge Representations

1. The Vadalog System: Datalog-based Reasoning for Knowledge Graphs. Luigi Bellomarini, Emanuel Sallinger, Georg Gottlob. PVLDB2018
2. TextRunner: Open Information Extraction on the Web. Alexander Yates, Michele Banko, Matthew G Broadhead, Michael J. Cafarella, Oren Etzioni, Stephen Soderland · HLT-NAACL · 2007
3. DBpedia: A Nucleus for a Web of Open Data. Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, Zachary G. Ives · ISWC/ASWC · 2007
4. The Description Logic Handbook. Franz Baader, Diego Calvanese, Deborah L. McGuinness, Daniele Nardi, Peter F. Patel-Schneider · 2007
5. Yago: a core of semantic knowledge. Fabian M. Suchanek, Gjergji Kasneci, Gerhard Weikum. WWW 2007.
6. Freebase: A Shared Database of Structured General Human Knowledge. Kurt D. Bollacker, Robert P. Cook, Patrick Tufts · AAAI · 2007
7. WordNet and wordnets. Christiane Fellbaum. Encyclopedia of Language and Linguistics, Second Edition, Oxford: Elsevier, 665-670. 2005
8. Semantic Integration: A Survey Of Ontology-Based Approaches. Natalya Fridman Noy · SIGMOD Record · 2004
9. Web-scale information extraction in knowitall: (preliminary results). Oren Etzioni, Michael J. Cafarella, Doug Downey, Stanley Kok, Ana-Maria Popescu, Tal Shaked, Stephen Soderland, Daniel S. Weld, Alexander Yates. WWW. 2004
10. Methods for Domain-Independent Information Extraction from the Web: An Experimental Comparison. Oren Etzioni, Michael J. Cafarella, Doug Downey, Ana-Maria Popescu, Tal Shaked, Stephen Soderland, Daniel S. Weld, Alexander Yates. AAAI 2004.
11. The Knowledge Model of Protégé-2000: Combining Interoperability and Flexibility Natalya Fridman Noy, Ray W. Fergerson, Mark A. Musen · EKAW · 2000
12. CYC: A Large-Scale Investment in Knowledge Infrastructure. Douglas B. Lenat · Commun. ACM · 1995
13. What You Always Wanted to Know About Datalog (And Never Dared to Ask). Stefano Ceri, Georg Gottlob, and Letizia Tanca. Trans. on Knowledge & Data Engineering. 1989

Philosophy of Identity and Logic

1. The Emergence of First-Order Logic. Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/logic-firstorder-emergence/
2. Term logic. https://en.wikipedia.org/wiki/Term_logic
3. Paraconsistent Logic. https://plato.stanford.edu/entries/logic-paraconsistent/
4. Classical Logic. https://plato.stanford.edu/entries/logic-classical/
5. Modal Logic. https://plato.stanford.edu/entries/logic-modal/
6. Ship of Theseus. https://en.wikipedia.org/wiki/Ship_of_Theseus (See also in De Corpore, 2.11)
7. Heraclitus, "Into the same rivers we step and do not step, we are and are not." https://plato.stanford.edu/entries/heraclitus/

Suggested Reading

Shallow Parsing, Unsupervised Segmentation, Phrase Finding & Parsing

1. Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders. Andrew Drozdov, Patrick Verga, Mohit Yadav, Mohit Iyyer, Andrew McCallum · NAACL-HLT · 2019
2. Compound Probabilistic Context-Free Grammars for Grammar Induction. Yoon Kim, Chris Dyer, Alexander M. Rush · ACL · 2019
3. An Imitation Learning Approach to Unsupervised Parsing. Bowen Li, Lili Mou, Frank Keller · ACL · 2019
4. Unsupervised Recurrent Neural Network Grammars. Yoon Kim, Alexander M. Rush, Lei Yu, Adhiguna Kuncoro, Chris Dyer, Gábor Melis · NAACL-HLT · 2019
5. Contextual Dependencies in Unsupervised Word Segmentation. Sharon Goldwater, Thomas L. Griffiths, Mark Johnson · ACL · 2006
6. Shallow Semantic Parsing using Support Vector Machines. Sameer Pradhan, Wayne H. Ward, Kadri Hacioglu, James H. Martin, Daniel Jurafsky · HLT-NAACL · 2004
7. Shallow Parsing with Conditional Random Fields. Fei Sha, Fernando Pereira · HLT-NAACL · 2003

Named Entity Recognition

1. Towards Improving Neural Named Entity Recognition with Gazetteers. Tianyu Liu, Jin-Ge Yao, Chin-yew Lin · ACL · 2019
2. Fast and Accurate Entity Recognition with Iterated Dilated Convolutions. Emma Strubell, Patrick Verga, David Belanger, Andrew McCallum · EMNLP · 2017
3. Neural Architectures for Named Entity Recognition. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, Chris Dyer · HLT-NAACL · 2016
4. TaggerOne: joint named entity recognition and normalization with semi-Markov Models. Robert Leaman, Zhiyong Lu · Bioinformatics · 2016
5. Learning Dynamic Feature Selection for Fast Sequential Prediction. Emma Strubell, Luke Vilnis, Kate Silverstein, Andrew McCallum · ACL · 2015
6. Named Entity Recognition with Bidirectional LSTM-CNNs. Jason P. C. Chiu, Eric Nichols · Transactions of the Association for Computational Linguistics · 2015
7. Bidirectional LSTM-CRF Models for Sequence Tagging. Zhiheng Huang, Wei Xu, Kai Yu · ArXiv · 2015
8. Lexicon Infused Phrase Embeddings for Named Entity Resolution. Alexandre Passos, Vineet Kumar, Andrew McCallum. ACL 2014
9. POLYGLOT-NER: Massive Multilingual Named Entity Recognition. Rami Al-Rfou', Vivek Kulkarni, Bryan Perozzi, Steven Skiena · SDM · 2014
10. PubTator: a web-based text mining tool for assisting biocuration. Chih-Hsuan Wei, Hung-Yu Kao, Zhiyong Lu · Nucleic Acids Research · 2013
11. ChemSpot: a hybrid system for chemical named entity recognition. Tim Rocktäschel, Michael Weidlich, Ulf Leser · Bioinformatics · 2012
12. Design Challenges and Misconceptions in Named Entity Recognition. Lev-Arie Ratinov, Dan Roth · CoNLL · 2009
13. Named entity recognition in query. Jiafeng Guo, Gu Xu, Xueqi Cheng, Hang Li · SIGIR · 2009
14. Joint Parsing and Named Entity Recognition. Jenny Rose Finkel, Christopher D. Manning · HLT-NAACL · 2009
15. Nested Named Entity Recognition. Jenny Rose Finkel, Christopher D. Manning · EMNLP · 2009
16. Search-based structured prediction. Hal Daumé, John Langford, Daniel Marcu · Machine Learning · 2009
17. An Effective Two-Stage Model for Exploiting Non-Local Dependencies in Named Entity Recognition. Vijay Krishnan, Christopher D. Manning · ACL · 2006
18. Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling. Jenny Rose Finkel, Trond Grenager, Christopher D. Manning · ACL · 2005
19. Named Entity Recognition with Character-Level Models. Dan Klein, Joseph Smarr, Huy Nguyen, Christopher D. Manning · CoNLL · 2003
20. Named Entity Recognition through Classifier Combination. Radu Florian, Abraham Ittycheriah, Hongyan Jing, Tong Zhang · CoNLL · 2003
21. Named Entity Recognition with Long Short-Term Memory. James Hammerton · CoNLL · 2003
22. Named Entity Recognition using an HMM-based Chunk Tagger. Guodong Zhou, Jian Su · ACL · 2002
23. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. John D. Lafferty, Andrew McCallum, Fernando Pereira · ICML · 2001
24. Named Entity Recognition without Gazetteers. Andrei Mikheev, Marc Moens, Claire Grover · EACL · 1999

Fine-Grained Entity Typing

1. Improving Distantly-supervised Entity Typing with Compact Latent Space Clustering. Bo Chen, Xiaotao Gu, Yufeng Hu, Siliang Tang, Guoping Hu, Yueting Zhuang, Xiang Ren. NAACL-HLT 2019
2. Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking. Shikhar Murty, Patrick Verga, Luke Vilnis, Irena Radovanovic, Andrew McCallum. ACL 2018.
3. Neural Fine-Grained Entity Type Classification with Hierarchy-Aware Loss. Peng Xu, Denilson Barbosa. NAACL-HLT 2018
4. Universal schema for entity type prediction. Limin Yao, Sebastian Riedel, Andrew McCallum. AKBC 2013.
5. Fine-Grained Entity Recognition. Xiao Ling, Daniel S. Weld. AAAI 2012

Entity Linking

1. Zero-Shot Entity Linking by Reading Entity Descriptions. Lajanugen Logeswaran, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, Jacob Devlin, Honglak Lee · ACL · 2019
2. Distant Learning for Entity Linking with Automatic Noise Detection. Phong Le, Ivan Titov. ACL 2019
3. End-to-End Neural Entity Linking. Nikolaos Kolitsas, Octavian-Eugen Ganea, Thomas Hofmann · CoNLL · 2018
4. Joint Multilingual Supervision for Cross-lingual Entity Linking. Shyam Upadhyay, Nitish Gupta, Dan Roth · EMNLP · 2018
5. Neural Collective Entity Linking. Yixin Cao, Lei Hou, Juan-Zi Li, Zhiyuan Liu · COLING · 2018
6. Collective Entity Disambiguation with Structured Gradient Tree Boosting. Yi Yang, Ozan Irsoy, Kazi Shefaet Rahman · NAACL-HLT · 2018
7. Improving Entity Linking by Modeling Latent Relations between Mentions. Phong Le, Ivan Titov · ACL · 2018
8. DeepType: Multilingual Entity Linking by Neural Type System Evolution. Jonathan Raiman, Olivier Raiman · AAAI · 2018
9. Deep Joint Entity Disambiguation with Local Neural Attention. Octavian-Eugen Ganea, Thomas Hofmann · EMNLP · 2017
10. Entity Linking via Joint Encoding of Types, Descriptions, and Context. Nitish Gupta, Sameer Singh, Dan Roth · EMNLP · 2017
11. Collective Entity Resolution with Multi-Focal Attention. Amir Globerson, Nevena Lazic, Soumen Chakrabarti, Amarnag Subramanya, Michael Ringgaard, Fernando Pereira · ACL · 2016
12. Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation. Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji · CoNLL · 2016
13. Joint Learning of Local and Global Features for Entity Linking via Neural Networks. Thien Huu Nguyen, Nicolas R. Fauceglia, Mariano Rodriguez-Muro, Oktie Hassanzadeh, Alfio Massimiliano Gliozzo, Mohammad Sadoghi · COLING · 2016
14. Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks Matthew Francis-Landau, Greg Durrett and Dan Klein NAACL-HLT 2016
15. Plato: A Selective Context Model for Entity Resolution. Nevena Lazic, Amarnag Subramanya, Michael Ringgaard, Fernando Pereira · Transactions of the Association for Computational Linguistics · 2015
16. Design Challenges for Entity Linking. Xiao Ling, Sameer Singh, Daniel S. Weld · Transactions of the Association for Computational Linguistics · 2015
17. Probabilistic Bag-Of-Hyperlinks Model for Entity Linking. Octavian-Eugen Ganea, Marina Ganea, Aurélien Lucchi, Carsten Eickhoff, Thomas Hofmann · WWW · 2015
18. Entity Linking meets Word Sense Disambiguation: a Unified Approach. Andrea Moro, Alessandro Raganato, Roberto Navigli · Transactions of the Association for Computational Linguistics · 2014
19. Evaluating Entity Linking with Wikipedia. Ben Hachey, Will Radford, Joel Nothman, Matthew Honnibal, James R. Curran · Artif. Intell. · 2013
20. Relational Inference for Wikification. Xiao Cheng, Dan Roth · EMNLP · 2013
21. Linking Named Entities to Any Database. Avirup Sil, Ernest Cronin, Penghai Nie, Yinfei Yang, Ana-Maria Popescu, Alexander Yates · EMNLP-CoNLL · 2012
22. KORE: keyphrase overlap relatedness for entity disambiguation. Johannes Hoffart, Stephan Seufert, Dat Ba Nguyen, Martin Theobald, Gerhard Weikum · CIKM · 2012
23. Robust Disambiguation of Named Entities in Text. Johannes Hoffart, Mohamed Amir Yosef, , Ilaria Bordino, Hagen Fürstenau, Manfred Pinkal, Marc Spaniol, Bilyana Taneva, Stefan Thater, Gerhard Weikum · EMNLP · 2011
24. AIDA: An Online Tool for Accurate Disambiguation of Named Entities in Text and Tables. Mohamed Amir Yosef, Johannes Hoffart, Ilaria Bordino, Marc Spaniol, Gerhard Weikum · PVLDB · 2011
25. DBpedia spotlight: shedding light on the web of documents. Pablo N. Mendes, Max Jakob, Andrés García-Silva, Christian Bizer · I-SEMANTICS · 2011
26. Local and Global Algorithms for Disambiguation to Wikipedia. Lev-Arie Ratinov, Dan Roth, Doug Downey, Mike Anderson · ACL · 2011
27. See also: http://nlp.cs.rpi.edu/kbp/2014/elreading.html

Meeting 4 (9.27.2019)

[Clustering] Entity Resolution & Clustering

Assigned Reading

• [Clustering:P1] End-to-end Neural Coreference Resolution. Kenton Lee, Luheng He, Mike Lewis, Luke Zettlemoyer. EMNLP. 2017
• [Clustering:P2] Robust Entity Clustering via Phylogenetic Inference. Nicholas Andrews, Jason Eisner, Mark Dredze. ACL 2014
• [Clustering:P3] Affinity Clustering: Hierarchical Clustering at Scale. Mohammadhossein Bateni, Soheil Behnezhad, Mahsa Derakhshan, MohammadTaghi Hajiaghayi, Raimondas Kiveris, Silvio Lattanzi, Vahab Mirrokni. NeurIPS 2017.

Entity Resolution & Clustering

Within Document Coreference

1. End-to-end Deep Reinforcement Learning Based Coreference Resolution. Hongliang Fei, Xu Li, Dingcheng Li, Ping Li. ACL 2019.
2. Higher-order Coreference Resolution with Coarse-to-fine Inference. Kenton Lee, Luheng He, Luke S. Zettlemoyer. NAACL-HLT2018
3. Improving Coreference Resolution by Learning Entity-Level Distributed Representations. Kevin Clark and Christopher D. Manning, ACL 2016.
4. Learning Global Features for Coreference Resolution Sam Wiseman, Alexander M. Rush, Stuart M. Shieber. HLT-NAACL, 2016
5. Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution Sam Wiseman, Alexander M. Rush, Stuart M. Shieber, Jason Weston. ACL. 2015
6. Entity-Centric Coreference Resolution with Model Stacking. Kevin Clark, Christopher D. Manning. ACL. 2015
7. A Joint Framework for Coreference Resolution and Mention Head Detection. Haoruo Peng, Kai-Wei Chang, Dan Roth. CoNLL 2015
8. Latent Structures for Coreference Resolution. Sebastian Martschat, Michael Strube. Transactions of the Association for Computational Linguistics. 2015
9. Prune-and-Score: Learning for Greedy Coreference Resolution. Chao Ma, Janardhan Rao Doppa, John Walker Orr, Prashanth Mannem, Xiaoli Z. Fern, Thomas G. Dietterich, Prasad Tadepalli EMNLP. 2014
10. Easy Victories and Uphill Battles in Coreference Resolution. Greg Durrett, Dan Klein. EMNLP, 2013
11. Decentralized Entity-Level Modeling for Coreference Resolution. Greg Durrett, David Hall, Dan Klein. ACL2013
12. Easy-first Coreference Resolution. Veselin Stoyanov, Jason Eisner. COLING. 2012

Cross-Document Coreference

1. Revisiting Joint Modeling of Cross-document Entity and Event Coreference Resolution. Shany Barhom, Vered Shwartz, Alon Eirew, Michael Bugert, Nils Reimers, Ido Dagan. ACL 2019
2. Resolving Event Coreference with Supervised Representation Learning and Clustering-Oriented Regularization. Kian Kenyon-Dean, Jackie Chi Kit Cheung, Doina Precup *SEM. 2018
3. CESI: Canonicalizing Open Knowledge Bases using Embeddings and Side Information. Shikhar Vashishth, Prince Jain, Partha Talukdar. WWW 2018.
4. Revisiting the Evaluation for Cross Document Event Coreference. Shyam Upadhyay, Nitish Gupta, Christos Christodoulopoulos, Dan Roth. COLING2016
5. Event Detection and Co-reference with Minimal Supervision. Haoruo Peng, Yangqiu Song, Dan Roth. EMNLP. 2016
6. Twitter at the Grammys: A Social Media Corpus for Entity Linking and Disambiguation. Mark Dredze, Nicholas Andrews, Jay DeYoungPublished in SocialNLP@EMNLP 2016
7. Cross-document Event Coreference Resolution based on Cross-media Features. Tongtao Zhang, Hongzhi Li, Heng Ji, Shih-Fu Chang. EMNLP 2015
8. A Hierarchical Distance-dependent Bayesian Model for Event Coreference Resolution. Bishan Yang, Claire Cardie, Peter I. Frazier. TACL. 2015
9. Cross-Document Co-Reference Resolution using Sample-Based Clustering with Knowledge Enrichment. Sourav Dutta, Gerhard Weikum. TACL. 2015
10. Robust Entity Clustering via Phylogenetic Inference. Nicholas Andrews, Jason Eisner, Mark Dredze. ACL 2014
11. Cross-Document Coreference Resolution Using Latent Features. Axel-Cyrille Ngonga Ngomo, Michael Röder, Ricardo Usbeck. LD4IE@ISWC. 2014
12. Entity Clustering Across Languages. Spence Green, Nicholas Andrews, Matthew R. Gormley, Mark Dredze, Christopher D. Manning. HLT-NAACL. 2012
13. Joint Entity and Event Coreference Resolution across Documents. Heeyoung Lee, Marta Recasens, Angel X. Chang, Mihai Surdeanu, Daniel Jurafsky. EMNLP-CoNLL. 2012
14. A Discriminative Hierarchical Model for Fast Coreference at Large Scale. Michael L. Wick, Sameer Singh, Andrew McCallum. ACL. 2012
15. Large-Scale Cross-Document Coreference Using Distributed Inference and Hierarchical Models. Sameer Singh, Amarnag Subramanya, Fernando Pereira, Andrew McCallum. ACL 2011
16. Streaming Cross Document Entity Coreference Resolution Delip Rao, Paul McNamee, Mark Dredze COLING 2010
17. Author Disambiguation using Error-driven Machine Learning with a Ranking Loss Function. Aron Culotta, Pallika Kanani, Robert Hall, Michael Wick, Andrew McCallum. 2007
18. Weakly supervised learning for cross-document person name disambiguation supported by information extraction. Niu, C.; Li, W.; and Srihari, R. K. ACL. 2004
19. Unsupervised personal name disambiguation. Mann and Yarowsky NAACL. 2003;
20. Entity-Based Cross-Document Coreferencing Using the Vector Space Model Amit Bagga, Breck Baldwin. COLING-ACL 1998

Clustering Methodology

1. Analysis of Ward's Method. Anna Großwendt, Heiko Röglin, Melanie Schmidt. SODA. 2019
2. DBSCAN++: Towards fast and scalable density clustering. Jennifer Jang · Heinrich Jiang. ICML 2019.
3. Scalable Hierarchical Clustering with Tree Grafting. Nicholas Monath, Ari Kobren, Akshay Krishnamurthy, Michael R. Glass, Andrew McCallum. KDD 2019
4. Gradient-based Hierarchical Clustering using Continuous Representations of Trees in Hyperbolic Space. Nicholas Monath, Manzil Zaheer, Daniel Silva, Andrew McCallum, Amr Ahmed. KDD 2019
5. Hierarchical Clustering better than Average-Linkage. Moses Charikar, Vaggos Chatziafratis, Rad Niazadeh. SODA. 2018
6. Hierarchical Clustering with Structural Constraints. Vaggos Chatziafratis, Rad Niazadeh, Moses Charikar. ICML 2018
7. Hierarchical Clustering: Objective Functions and Algorithms. Vincent Cohen-Addad, Varun Kanade, Frederik Mallmann-Trenn, Claire Mathieu. SODA 2018
8. Canopy Fast Sampling with Cover Trees. Manzil Zaheer, Satwik Kottur, Amr Ahmed, José M. F. Moura, Alexander J. Smola. ICML 2017
9. A Hierarchical Algorithm for Extreme Clustering. Ari Kobren, Nicholas Monath, Akshay Krishnamurthy, Andrew McCallum. KDD 2017
10. Can clustering scale sublinearly with its clusters? A variational EM acceleration of GMMs and k-means Dennis Forster, Jörg Lücke. AISTATS. 2017
11. One-Shot Coresets: The Case of k-Clustering. Olivier Bachem, Mario Lucic, Silvio Lattanzi AISTATS 2017
12. Hierarchical Clustering Beyond the Worst-Case. Vincent Cohen-Addad, Varun Kanade, Frederik Mallmann-Trenn NIPS. 2017
13. A Dual-Tree Algorithm for Fast k-means Clustering With Large k. Ryan R. Curtin. SDM 2017.
14. Training Gaussian Mixture Models at Scale via Coresets. Mario Lucic, Matthew Faulkner, Andreas Krause, Dan Feldman J. Mach. Learn. Res.2017
15. Exponential Stochastic Cellular Automata for Massively Parallel Inference. Manzil Zaheer, Michael Wick, Jean-Baptiste Tristan, Alexander J. Smola, Guy L. Steele AISTATS 2016
16. Approximate K-Means++ in Sublinear Time. Olivier Bachem, Mario Lucic, S. Hamed Hassani, Andreas Krause. AAAI. 2016
17. Fast and Provably Good Seedings for k-Means. Olivier Bachem, Mario Lucic, Seyed Hamed Hassani, Andreas Krause. NIPS. 2016
18. A cost function for similarity-based hierarchical clustering. Sanjoy Dasgupta. STOC. 2016
19. Finding Planted Partitions in Nearly Linear Time using Arrested Spectral Clustering. Nader H. Bshouty, Philip M. Long. ICML 2010
20. Web-scale k-means clustering D. Sculley WWW 2010
21. A discriminative framework for clustering via similarity functions. Maria-Florina Balcan, Avrim Blum, Santosh S. Vempala STOC 2008
22. BIRCH: A New Data Clustering Algorithm and Its Applications. Tian Zhang, Raghu Ramakrishnan, Miron Livny. Data Mining and Knowledge Discovery 1997

Suggested Reading

Binary Relation Extraction

1. Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network. Sunil Kumar Sahu, Fenia Christopoulou, Makoto Miwa, Sophia Ananiadou. ACL 2019
2. Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction. Christoph Alt, Marc Hübner, Leonhard Hennig. ACL 2019
3. Attention Guided Graph Convolutional Networks for Relation Extraction. Zhijiang Guo, Yan Zhang, Wei Lu ACL 2019
4. A Richer-but-Smarter Shortest Dependency Path with Attentive Augmentation for Relation Extraction. Duy-Cat Can, Hoang-Quynh Le, Quang-Thuy Ha, Nigel Collier NAACL-HLT 2019
5. Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction. Patrick Verga, Emma Strubell, Andrew McCallum. NAACL-HLT. 2018
6. Cross-Sentence N-ary Relation Extraction with Graph LSTMs. Nanyun Peng, Hoifung Poon, Chris Quirk, Kristina Toutanova, Wen-tau Yih. Transactions of the Association for Computational Linguistics. 2017
7. Relation Classification via Multi-Level Attention CNNs. Linlin Wang, Zhu Cao, Gerard de Melo, Zhiyuan Liu. ACL 2016
8. Learning Entity and Relation Embeddings for Knowledge Graph Completion. Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, Xuan Zhu AAAI 2015
9. Representing Text for Joint Embedding of Text and Knowledge Bases. Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, Michael Gamon. EMNLP. 2015
10. Multilingual Relation Extraction using Compositional Universal Schema. Patrick Verga, David Belanger, Emma Strubell, Benjamin Roth, Andrew McCallum. HLT-NAACL 2015
11. Translating Embeddings for Modeling Multi-relational Data. Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, Oksana Yakhnenko. NIPS 2013
12. A latent factor model for highly multi-relational data. Rodolphe Jenatton, Nicolas Le Roux, Antoine Bordes, Guillaume Obozinski NIPS 2012
13. A Three-Way Model for Collective Learning on Multi-Relational Data. Maximilian Nickel, Volker Tresp, Hans-Peter Kriegel. ICML 2011
14. Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks. Rajarshi Das, Arvind Neelakantan, David Belanger, Andrew McCallum. EACL 2016
15. Compositional Learning of Embeddings for Relation Paths in Knowledge Base and Text. Kristina Toutanova, Victoria Lin, Wen-tau Yih, Hoifung Poon, Chris Quirk. ACL, 2016
16. Efficient and Expressive Knowledge Base Completion Using Subgraph Feature Extraction. Matt Gardner, Tom M. Mitchell. EMNLP, 2015
17. Compositional Vector Space Models for Knowledge Base Completion Arvind Neelakantan, Benjamin Roth, Andrew McCallum. ACL 2015
18. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, Li Deng ICLR. 2014
19. Random Walk Inference and Learning in A Large Scale Knowledge Base. Ni Lao, Tom M. Mitchell, William W. Cohen. EMNLP, 2011

N-ary Relations

1. Cross-Sentence N-ary Relation Extraction with Graph LSTMs Nanyun Peng, Hoifung Poon, Chris Quirk, Kristina Toutanova, Wen-tau Yih · Transactions of the Association for Computational Linguistics · 2017
2. Simple Algorithms for Complex Relation Extraction with Applications to Biomedical IE. Ryan T. McDonald, Fernando Pereira, Seth Kulick, R. Scott Winters, Yang Jin, Peter S. White · ACL · 2005

Semantic Frames

1. The State of the Art in Semantic Representation. Omri Abend, Ari Rappoport · ACL · 2017
2. Abstract Meaning Representation – A Survey. Melanie Tosik · 2015
3. A Frames Approach to Semantic Analysis. Charles J. Fillmore, Collin F. Baker · 2009
4. The Proposition Bank: An Annotated Corpus of Semantic Roles. Martha Palmer, Paul Kingsbury, Daniel Gildea · Computational Linguistics · 2005

SRL & Supervised Semantic Parsing:

1. AMR Parsing as Graph Prediction with Latent Alignment. Chunchuan Lyu, Ivan Titov · ACL · 2018
2. A Joint Sequential and Relational Model for Frame-Semantic Parsing. Bishan Yang, Tom M. Mitchell · EMNLP · 2017
3. Deep Semantic Role Labeling: What Works and What's Next. Luheng He, Kenton Lee, Mike Lewis, Luke S. Zettlemoyer · ACL · 2017
4. Frame-Semantic Parsing. Dipanjan Das, Desai Chen, André F. T. Martins, Nathan Schneider, Noah A. Smith · Computational Linguistics · 2014
5. A Discriminative Graph-Based Parser for the Abstract Meaning Representation. Jeffrey Flanigan, Sam Thomson, Jaime G. Carbonell, Chris Dyer, Noah A. Smith · ACL · 2014

SRL and Unsupervised Parsing

1. Probabilistic Frame Induction. Jackie Chi Kit Cheung, Hoifung Poon, Lucy Vanderwende · HLT-NAACL · 2013
2. Unsupervised Induction of Semantic Roles within a Reconstruction-Error Minimization Framework. Ivan Titov, Ehsan Khoddam · HLT-NAACL · 2014

Suggested Reading

1. Smoothing the Geometry of Probabilistic Box Embeddings. Xiang Li, Luke Vilnis, Dongxu Zhang, Michael Boratko, Andrew McCallum · ICLR · 2019
2. Inferring Concept Hierarchies from Text Corpora via Hyperbolic Embeddings. Matt Le, Stephen Roller, Laetitia Papaxanthos, Douwe Kiela, Maximilian Nickel ACL 2019
3. Learning Mixed-Curvature Representations in Product Spaces. Albert Gu, Frederic Sala, Beliz Gunel, Christopher Ré. ICLR 2019
4. Hyperbolic Disk Embeddings for Directed Acyclic Graphs. Ryota Suzuki, Ryusuke Takahama, Shun Onoda ICML 2019
5. Hierarchical Density Order Embeddings (On Modeling Hierarchical Data via Probabilistic Order Embeddings). Ben Athiwaratkun, Andrew Gordon Wilson · ICLR · 2018
6. Generalizing Point Embeddings using the Wasserstein Space of Elliptical Distributions. Boris Muzellec, Marco Cuturi · NeurIPS · 2018
7. Hyperbolic Neural Networks. Octavian-Eugen Ganea, Gary Bécigneul, Thomas Hofmann · NeurIPS · 2018
8. Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures. Luke Vilnis, Xiang Li, Shikhar Murty, Andrew McCallum · ACL · 2018
9. Hyperbolic Entailment Cones for Learning Hierarchical Embeddings. Octavian-Eugen Ganea, Gary Bécigneul, Thomas Hofmann · ICML · 2018
10. Representation Tradeoffs for Hyperbolic Embeddings. Christopher De Sa, Albert Gu, Christopher Ré, Frederic Sala. ICML 2018
11. Poincaré Embeddings for Learning Hierarchical Representations. Maximilian Nickel, Douwe Kiela · NIPS · 2017
12. Multimodal Word Distributions. Ben Athiwaratkun, Andrew Gordon Wilson · ACL · 2017
13. Order-Embeddings of Images and Language. Ivan Vendrov, Ryan Kiros, Sanja Fidler, Raquel Urtasun · ICLR · 2016
14. Complex Embeddings for Simple Link Prediction. Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard ICML 2016
15. Word Representations via Gaussian Embedding Luke Vilnis, Andrew McCallum · ICLR · 2015
16. Learning to Represent Knowledge Graphs with Gaussian Embedding. Shizhu He, Kang Liu, Guoliang Ji, Jun Zhao · CIKM · 2015
17. Holographic Embeddings of Knowledge Graphs. Maximilian Nickel, Lorenzo Rosasco, Tomaso A. Poggio AAAI 2015

Meeting 7 (10.18.2019)

Midpoint Project Presentations

No Readings

Meeting 8 (10.25.2019)

Midpoint Project Presentations

No Readings

Meeting 9 (11.01.2019)

[GNN+Index] Graph Neural Networks & Learned Index Structures

Assigned Reading

• [GNN+Index:P1] The Case for Learned Index Structures. Tim Kraska, Alex Beutel, Ed Huai-hsin Chi, Jeffrey Dean, Neoklis Polyzotis in SIGMOD Conference 2017
• [GNN+Index:P2] Graph convolution over pruned dependency trees improves relation extraction. Yuhao Zhang, Peng Qi, Christopher D. Manning. EMNLP. 2018 and Attention Guided Graph Convolutional Networks for Relation Extraction. Zhijiang Guo, Yan Zhang, Wei Lu. ACL 2019.
• [GNN+Index:P3] Learning to Route in Similarity Graphs. Dmitry Baranchuk · Dmitry Persiyanov · Anton Sinitsin · Artem Babenko. ICML 2019

Suggested Reading

Learned Index Structures

1. Meta-Learning Neural Bloom Filters. Jack W. Rae, Sergey Bartunov, Timothy P. Lillicrap. ICML 2019
2. Learning to Route in Similarity Graphs. Dmitry Baranchuk · Dmitry Persiyanov · Anton Sinitsin · Artem Babenko. ICML 2019
3. Learning to Index for Nearest Neighbor Search. Chih-Yi Chiu, Amorntip Prayoonwong, Yin-Chih Liao. IEEE transactions on pattern analysis and machine intelligence. 2018
4. A Model for Learned Bloom Filters, and Optimizing by Sandwiching. Michael Mitzenmacher. NeurIPS 2018.
5. The Case for Learned Index Structures. Tim Kraska, Alex Beutel, Ed Huai-hsin Chi, Jeffrey Dean, Neoklis Polyzotis lessPublished in SIGMOD Conference 2017
6. Adaptive Cuckoo Filters Michael Mitzenmacher, Salvatore Pontarelli, Pedro Reviriego ALENEX. 2017
7. Cuckoo Filter: Practically Better Than Bloom. Bin Fan, David G. Andersen, Michael Kaminsky, Michael Mitzenmacher CoNEXT 2014

Graph Neural Networks

1. Inductive Representation Learning on Large Graphs. William L. Hamilton, Zhitao Ying, Jure Leskovec. NIPS. 2017
2. Semi-Supervised Classification with Graph Convolutional Networks. Thomas N. Kipf, Max Welling. ICLR. 2016
3. Graph U-Nets. Hongyang Gao, Shuiwang Ji. ICML 2019
4. Self-Attention Graph Pooling Junhyun Lee, Inyeop Lee, Jaewoo Kang. ICML. 2019
5. Hierarchical Graph Representation Learning with Differentiable Pooling. Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, William L. Hamilton, Jure Leskovec. NeurIPS. 2018
6. Attention Guided Graph Convolutional Networks for Relation Extraction. Zhijiang Guo, Yan Zhang, Wei Lu. ACL 2019.
7. Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling. Diego Marcheggiani and Ivan Titov. EMNLP. 2017
8. Dating Documents using Graph Convolution Networks. Shikhar Vashishth, Shib Sankar Dasgupta, Swayambhu Nath Ray, Partha Talukdar. ACL. 2018
9. Graph Convolutional Networks With Argument-Aware Pooling for Event Detection. Thien Huu Nguyen, Ralph Grishman. AAAI2018
10. Graph Convolutional Encoders for Syntax-aware Neural Machine Translation. Joost Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, Khalil Sima'an EMNLP 2017
11. Graph convolution over pruned dependency trees improves relation extraction. Yuhao Zhang, Peng Qi, Christopher D. Manning. EMNLP. 2018

Meeting 10 (11.08.2019)

[QA] Question Answering, Reasoning, & Pretrained language models

Assigned Reading

1. Learning to Compose Neural Networks for Question Answering. Jacob Andreas, Marcus Rohrback, Trevor Darrell and Dan Klein. , NAACL 2016

2. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, NAACL 2018

3. End-to-End Differentiable Proving. Tim Rocktäschel, Sebastian Riedel, Neurips 2017

4. Language Models as Knowledge Bases. Fabio Petroni, Tim Rocktaschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu. Alexander H. Miller, Sebastian Riedel. EMNLP 2019

Suggested Reading

1. Large Memory Layers with Product Keys. Guillaume Lample, Alexandre Sablayrolles, Marc'Aurelio Ranzato, Ludovic Denoyer, Hervé Jégou. NeurIPS 2019
2. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G. Carbonell, Quoc V. Le, Ruslan Salakhutdinov. ACL 2019
3. Dynamic Evaluation of Transformer Language Models. Ben Krause, Emmanuel Kahembwe, Iain Murray, Steve Renals. ArXiv 2019
4. Augmenting Self-attention with Persistent Memory Sainbayar Sukhbaatar, Edouard Grave, Guillaume Lample, Hervé Jégou, Armand Joulin. ArXiv. 2019
5. Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning. Pradeep Dasigi, Nelson F. Liu, Ana Marasovic, Noah A. Smith, Matt Gardner IJCNLP 2019
6. Multi-Granularity Hierarchical Attention Fusion Networks for Reading Comprehension and Question Answering Wei Wang, Ming Yan, Chen Wu ACL 2018
7. SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference.Rowan Zellers, Yonatan Bisk, Roy Schwartz, Yejin Choi. EMNLP 2018
8. Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning. Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi. IJCNLP 2019

Meeting 11 (11.15.2019)

[Fair] Fairness & Adversarial Attacks

Assigned Reading

• [Fair:P1] Compositional Fairness Constraints for Graph Embeddings. Avishek Joey Bose, William L. Hamilton ICML 2019
• [Fair:P2] Fairness in Relational Domains. Golnoosh Farnadi, Behrouz Babaki, Lise Getoor AIES 2018
• [Fair:P3] Mitigating Gender Bias in Natural Language Processing: Literature Review. Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth M. Belding-Royer, Kai-Wei Chang, William Yang Wang ACL 2019

Suggested Reading

1. Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings. Thomas Manzini, Yao Chong Lim, Alan W. Black, Yulia Tsvetkov NAACL-HLT 2019
2. Understanding Undesirable Word Embedding Associations. Kawin Ethayarajh, David Duvenaud, Graeme Hirst ACL 2019
3. Interventional Fairness: Causal Database Repair for Algorithmic Fairness. Babak Salimi, Luke Rodriguez, Bill Howe, Dan Suciu SIGMOD Conference 2019
4. Capuchin: Causal Database Repair for Algorithmic Fairness. Babak Salimi, Luke Rodriguez, Bill Howe, Dan Suciu ArXiv 2019
5. Operationalizing Individual Fairness with Pairwise Fair Representations. Preethi Lahoti, Krishna P. Gummadi, Gerhard Weikum ArXiv 2019
6. Bias in OLAP Queries: Detection, Explanation, and Removal. Babak Salimi, Johannes Gehrke, Dan Suciu SIGMOD Conference 2018
7. Flexibly Fair Representation Learning by Disentanglement Elliot Creager, David Madras, Jörn-Henrik Jacobsen, Marissa A. Weis, Kevin Swersky, Toniann Pitassi, Richard S. Zemel ICML. 2019
8. Attenuating Bias in Word vectors. Sunipa Dev, Jeff M. Phillips AISTATS 2019
9. Learning Adversarially Fair and Transferable Representations. David Madras, Elliot Creager, Toniann Pitassi, Richard S. Zemel. ICML 2018
10. Adversarial Contrastive Estimation. Avishek Joey Bose, Huan Ling, Yanshuai Cao ACL 2018
11. When Worlds Collide: Integrating Different Counterfactual Assumptions in Fairness. Chris Russell, Matt J. Kusner, Joshua R. Loftus, Ricardo Silva NIPS 2017
12. Censoring Representations with an Adversary. Harrison A Edwards, Amos J. Storkey ICLR 2015
13. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. NIPS 2016.

Meeting 12 (11.22.2019)

Guest Lecture - Marco Serafini

Meeting 13 (12.06.2019)

Final Project Poster Presentations

No Readings