Looking for information on the 2012 track? Go here. 2011 Track Other Years Organizers
Task 1: Assessment Crowd judgments submitted by participating teams (worker IDs have been anonymized) Test topics and Data Development Data
Task 2: Aggregation
Test Set (August 19, 2011) Development Set Evaluation Software (by Hyun Joon Jung, University of Texas at Austin)
Track Sponsors Thanks to our sponsors for helping make the 2011 track a great success! CrowdFlower (see their 4/26/11 blog post): $100 credit per participant + support. CrowdFlower will provide technical support and resources to assist teams with an introduction to the CrowdFlower self-service platform. Email Joseph Childress (joseph at crowdflower dot com) with Subject "TREC 2011 Crowdsourcing Track credit request" to receive your credit and inquire regarding technical support. Amazon: $300 reimbursement per team using Mechanical Turk in submitted results |