Data:

The data consists of the following files and folders:

1. Query_doc.txt -- contains the 50 queries, i.e., description of legal situations
2. Object_casedocs -- contains 2,914 prior case documents, some of which are relevant to the queries (to be used for Task 1).
3. Object_statutes -- contains the title and textual description of 197 statutes (to be used for Task 2)
4. A README.txt file that specifies the formats of the other files / folders

Download the dataset here

The dataset is an encrypted zip file. Please fill the registration form to receive the decryption key.

Evaluation plan:

Standard performance measures such as Precision, Recall, MAP, and F-score will be used to evaluate the runs.

For both tasks, it is expected that, for each query, a run will give a set of statutes / prior cases that are judged to be relevant to the query. Higher credit will be given to runs that identify more number of actually relevant statutes / prior cases (according to our gold standard) and less number of non-relevant ones.