Looking for information on the 2012 track? Go here.
Task 1: Assessment
Crowd judgments submitted by participating teams (worker IDs have been anonymized)
Test topics and Data
Task 2: Aggregation
Test Set (August 19, 2011)
Evaluation Software (by Hyun Joon Jung, University of Texas at Austin)
Submissions and Results (restricted to participating teams)
Thanks to our sponsors for helping make the 2011 track a great success!
CrowdFlower (see their 4/26/11 blog post): $100 credit per participant + support. CrowdFlower will provide technical support and resources to assist teams with an introduction to the CrowdFlower self-service platform. Email Joseph Childress (joseph at crowdflower dot com) with Subject "TREC 2011 Crowdsourcing Track credit request" to receive your credit and inquire regarding technical support.
Amazon: $300 reimbursement per team using Mechanical Turk in submitted results