The first Workshop on Natural Legal Language Processing (NLLP) was co-located with NAACL 2019 in Minneapolis, Minnesota, USA.

The workshop took place at the Hyatt Regency Hotel Minneapolis on 7 June 2019 (room: Lakeshore A).

The schedule, papers and slides are available below.

The full workshop proceedings are available here.

Thank you everyone for supporting the NLLP 2019 workshop.

Program

09:00 - 09:10 Workshop opening [paper] [slides]

09:10 - 10:10 Invited Talk: Law as Data: The Promise and Challenges of Natural Language Processing for Legal Research - Arthur Dyevre [slides]

10:10 - 10:30 Session 1: Bias

10:10 - 10:30 Implicit Bias in the Judiciary - Daniel Chen, Elliott Ash, Arianna Ornaghi [slides]

10:30 - 11:00 Coffee Break

11:00 - 12:20 Session 2: NLP Applications

11:00 - 11:20 Plain English Summarization of Contracts - Laura Manor, Junyi Jessy Li [paper] [slides]

11:20 - 11:40 Question Answering for Privacy Policies: Combining Computational and Legal Perspectives - Abhilasha Ravichander, Alan W Black, Thomas Norton, Shomir Wilson, Norman Sadeh [slides]

11:40 - 12:00 Scalable Methods for Annotating Legal-Decision Corpora - Lisa Ferro, John Aberdeen, Karl Branting, Craig Pfeifer, Alexander Yeh, Amartya Chakraborty [paper] [slides]

12:00 - 12:20 The Extent of Repetition in Contract Language - Dan Simonson, Daniel Broderick, Jonathan Herr [paper] [slides]

12:20 - 14:00 Lunch

14:00 - 15:00 Invited Talk: NLP & Law {Past, Present + Future} - Daniel M. Katz

15:00 - 15:30 Session 3: Short Papers

15:00 - 15:15 Sentence Boundary Detection in Legal Text - George Sanchez [paper] [slides]

15:15 - 15:30 Legal Linking: Citation Resolution and Suggestion in Constitutional Law - Robert Shaffer, Stephen Mayhew [paper] [slides]

15:30 - 16:00 Coffee Break

16:00 - 17:00 Session 4: Demos

16:00 - 16:20 Litigation Analytics: Case Outcomes Extracted from US Federal Court Dockets - Thomas Vacek, Ronald Teo, Dezhao Song, Timothy Nugent, Conner Cowling, Frank Schilder [paper]

16:20 - 16:40 Developing and Orchestrating a Portfolio of Natural Legal Language Processing and Document Curation Services - Georg Rehm, Julian Moreno-Schneider, Jorge Gracia, Artem Revenko, Victor Mireles, Maria Khvalchik, Ilan Kernerman, Andis Lagzdins, Marcis Pinnis, Artus Vasilevskis, Elena Leitner, Jan Milde, Pia Weißenhorn [slides] [paper]

16:40 - 17:00 Transparent Linguistic Models for Contract Understanding and Comparison - Arvind Agarwal, Laura Chiticariu, Poornima Chozhiyath Raman, Marina Danilevsky, Diman Ghazi, Ankush Gupta, Shanmukh Guttula, Yannis Katsis, Rajasekar Krishnamurthy, Yunyao Li, Shubham Mudgal, Vitobha Munigala, Nicholas Phan, Dhaval Sonawane, Sneha Srinivasan, Sudarshan Thitte, Shivakumar Vaithyanathan, Mitesh Vasa, Ramiya Venkatachalam, Vinitha Yaski, Huaiyu Zhu [slides]

17:00 - 17:10 Break

17:10 - 17:50 Session 5: Prediction

17:10 - 17:30 Legal Area Classification: A Comparative Study of Text Classifiers on Singapore Supreme Court Judgments - Jerrold Soh, How Khang Lim, Ian Ernst Chai [paper] [slides]

17:30 - 17:50 Extreme Multi-Label Legal Text Classification: A Case Study in EU Legislation - Ilias Chalkidis, Emmanouil Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, Ion Androutsopoulos [paper] [slides]

17:50 - 18:00 Workshop Closing

Motivation

As electronic information becomes increasingly available around the world, automated tools for processing that information have grown apace. These tools can be especially effective and time-saving on text where information can be distilled in interesting ways including auto-summarization, named-entity extraction, machine translation, sentiment analysis, topic classification and others. As a result, natural language processing (NLP) applications are popular in important commercial contexts such as finance and healthcare.

The Legal domain however is still largely underrepresented in the NLP literature despite its enormous potential for generating interesting research problems on a par with other important commercial areas. In fact the US Legal Services market alone is valued at 211 billion according to US government price indices.

The accessibility of legal texts in the US in particular was an issue in the past preventing some researchers from working on legal NLP problems. Over the last few years however, more legal corpora have come online at low- or no-cost including the BYU Corpus, the Free Law Project and the expansion of resources published by the Library of Congress through Law.gov. A variety of growing electronic legal resources already exist free of charge for countries in Europe and Asia. Thus we feel that the timing is excellent to bring together researchers from around the world to focus on NLP problems in this area.

This workshop adheres to the ACL Anti-Harassment Policy.

Goal

The aim of the workshop is to bring together researchers and practitioners from Natural Language Processing (NLP) and the legal domain to:

• Foster collaboration between the two fields;
• Discuss applications of NLP technologies in the legal domain;
• Formulate targets and priorities for NLP approaches to legal text;
• Identify challenges specific to the legal domain.

We consider “legal text” to include litigation-related corpora such as dockets, opinions and court transcripts but also corpora based on patents, briefs, public financial filings, civil code, local ordinances, privacy policies, law enforcement records, congressional records and speeches.

Topics

We welcome all submissions describing original work with one or more of the following contribution types:

• Applications of NLP to legal tasks including:
  • Legal Citation Resolution
  • Case Outcome Analysis and Prediction
  • Models of Legal Reasoning
  • E-Discovery
  • Lexical Resources for the Legal Domain
  • Bias and Privacy
• Experimental results using and adapting NLP methods in legal documents including:
  • Text Classification
  • Information Retrieval
  • Entity Recognition
  • Entity Disambiguation
  • Training and Using Embeddings
  • Parsing
  • Dialogue and Discourse Analysis
  • Knowledge Graph population
  • Text Summarization
  • Relation Extraction
  • Event Extraction
  • Anaphora Resolution
• Tasks
  • Description of new legal tasks for NLP
  • Structured overviews of a specific task with the goal of identifying new areas for research
  • Position papers presenting new visions, challenges and changes to existing research practices
• Resources
  • Creation of curated and/or annotated data sets that can be publicly released and used by the community to advance the field
• Demos
  • Descriptions of systems which use NLP technologies for legal text
  • Industrial applications, e.g. papers describing research on proprietary data

Submission

We accept papers reporting original (unpublished) research of two types:

• Long papers (8 pages + references)
• Short papers (4 pages + references)

All submissions should be anonymized to facilitate double blind reviewing.

Submissions that do not adhere to the author guidelines or ACL policies will be rejected without review.

To submit a paper, please access the submission link: http://www.softconf.com/naacl2019/nllp/

Non-archival option

The authors have the option of submitting previously unpublished research as non-archival, meaning that only the abstract will be published in the conference proceedings. We expect these submissions to describe the same quality of work as archival submissions. These will be reviewed following the same procedure as archival submissions.

This option accommodates publication of the work or a superset at a later date in a conference or journal which does not allow previously archived work and to encourage presentation and feedback on mature, yet unpublished work.

Dual Submission Policy

Papers that have been or will be submitted to workshops, conferences or journals during the review period must indicate so at submission time. Authors of papers accepted for presentation at the NLLP workshop 2019 must notify the organizers by the camera-ready deadline as to whether the paper will be presented or withdrawn.

Exception: Submissions with the non-archival option are excepted from this requirement.

Submission Style & Format Guidelines

Submissions to the workshop must be in PDF format and should follow the NAACL-HLT 2019 style templates:

• LaTeX
• Microsoft Word
• Overleaf

Important Dates

Submission deadline ― 11 March 2019

Notification ― 29 March 2019

Camera ready ― 5 April 2019

Workshop ― 6 or 7 June 2019

All deadlines are 11.59pm UTC -12h

Presentation

• Presentation format and schedule will be announced before the camera-ready deadline.
• At least one author of each accepted paper must register for the NLLP workshop 2019 by the early registration deadline in order for the submission to be published in the proceedings.

Organizing Committee [e-mail]

• Nikolaos Aletras ― University of Sheffield
• Elliott Ash ― ETH Zurich
• Leslie Barrett ― Bloomberg Law
• Daniel Chen ― Toulouse School of Economics
• Adam Meyers ― New York University
• Daniel Preotiuc-Pietro ― Bloomberg LP
• David Rosenberg ― Bloomberg LP
• Amanda Stent ― Bloomberg LP

Program Committee

• Tomaso Agnoloni ― Institute of Legal Information Theory and Technologies
• Ion Androutsopoulos ― Athens University of Economics and Business
• Joan Bachenko ― Linguistech LLC
• Claire Cardie ― Cornell University
• Ilias Chalkidis ― Athens University of Economics and Business
• Rajarathnam Chandramouli ― Stevens Institute of Technology
• Laura Chiticariu - IBM Research
• Walter Daelemans ― University of Antwerp
• Marina Danilevsky ― IBM Research
• Stefania Degaetano-Ortlieb ― Saarland University
• Luigi Di Caro ― University of Turin
• Liviu P. Dinu ― University of Bucharest
• Eileen Fitzpatrick ― Montclair State University
• Enrico Francesconi ― Institute of Legal Information Theory and Technologies
• Frank Giaoui ― Columbia Law School
• Matthias Grabmair ― Carnegie Mellon University
• Ilan Kernerman ― K Dictionaries
• Seth Kulick ― University of Pennsylvania
• Vasileios Lampos ― University College London
• Mark Liberman ― University of Pennsylvania
• Shervin Malmasi ― Harvard Medical School
• Elena Montiel ― Universidad Politécnica de Madrid
• Paulo Quaresma ― University of Evora
• Georg Rehm ― DFKI
• Victor Rodríguez-Doncel ― Universidad Politécnica de Madrid
• Victoria Rubin ― University of Western Ontario
• Eugene Santos ― Thayer School of Engineering at Dartmouth
• Maosong Sun ― Tsinghua University
• Dimitrios Tsarapatsanis ― University of York
• Mihaela Vela ― Saarland University
• Marc B. Vilain ― MITRE Corp
• Jianqian Wang ― SUNY Buffalo
• Adam Wyner ― Swansea University
• Diyi Yang ― Georgia Tech
• Marcos Zampieri ― University of Wolverhapton
• Lina Zhou ― University of Maryland, Baltimore County

Venue

The workshop will be collocated with NAACL 2019 in Minneapolis, MN, USA.

Registration

Registration is available through the NAACL website. Early registration end on April 28.

Participants may register only for the workshop or - at a reduced rate - together with the conference registration.

Local Information

About Minneapolis

Transport

Accommodation

Visa information

Childcare

The NLLP workshop 2019 adheres to the ACL Anti-Harassment Policy. Any participant who experiences harassment or hostile behavior may contact any current member of the ACL Executive Committee or contact Priscilla Rasmussen, who is usually available at the registration desk of the conference. Please be assured that if you approach us, your concerns will be kept in strict confidence, and we will consult with you on any actions taken.