RDoC Task 2019
Information Retrieval and Sentence Extraction on Mental Health using Research Domain Criteria
RDOC Task 2019 is closed now. However, the datasets are available here.
RDoC task is a combination of two subtasks. Participants can choose to participate in one or both tasks. Task 1 (RDoC-IR) is on retrieving PubMed Abstracts related to RDoC constructs, while Task 2 (RDoC-SE) is on extracting the most relevant sentences for an RDoC construct from a relevant abstract.
Task 1 (RDoC-IR)
In this subtask participants are given a set of PubMed abstracts and they are required to retrieve abstracts that are relevant for various RDoC constructs.
The training data will have approx. 250 abstracts each annotated with one or more RDoC constructs.
Test data will contain a list of approx. 200 abstracts and the goal is to submit a ranked lists of relevant articles for each RDoC construct.
We plan to compute the Average Precision (AP) for each construct independently and average across the constructs to compute the Mean Average Precision (MAP).
Detailed instructions will be provided to the registered task participants. You can register for the RDoC task in the Registration page.
Task 2 (RDoC-SE)
In this subtask, participants are given a set of PubMed abstracts relevant for RDoC constructs, and they are required to extract the most relevant sentence from each abstract for the corresponding RDoC construct.
The training data will have approx. 250 abstracts each annotated with one RDoC construct and the most relevant sentence.
Test data will contain a list of approx. 50 abstracts relevant for RDoC constructs and the goal is to submit a list of predicted most relevant sentence for each abstract.
We plan to compute the Accuracy (percentage of correctly predicted sentences).
Detailed instructions will be provided to the registered task participants. You can register for the RDoC task in the Registration page.
Example annotation for Task 1
Example annotation for Task 2
You can find the annotation guidelines here.