TREC 2013

As part of the National Institute of Standards and Technology (NIST) Text REtrieval Conference (TREC), the Contextual Suggestion Track deals with complex information needs which are highly dependent on context and user interests.

Track Organizers

  • Charles L A Clarke, Waterloo
  • Adriel Dean-Hall, Waterloo
  • Jaap Kamps, Amsterdam
  • Paul Thomas, CSIRO

TREC 2013 Contextual Suggestion Track Guidelines

ClueWeb12 Contextual suggestion Subcollection

A subset of ClueWeb12 has been created that can be used for this task.

Validation, Conversion, and Evaluation Scripts

Scripts for validating submission files and for converting submissions files from JSON to CSV are available. The evaluation script is also available.

Metrics used at TREC 2013

A short description of the three metrics used to evaluate runs.

Presentations Given at TREC 2013

Slides from some of the presentations given at TREC 2013.

Contexts, Profiles, and Examples

The contexts, profiles, and examples for the TREC 2013 contextual suggestion track have been released and are available below. The seed city for TREC 2013 is Philadelphia, PA. For an overview of the format of these files and the required submission file format see the guidelines.

The contexts were updated on June 7, 2013 to include an id for each context. The new file also had "latitude" changed to "lat" and "longitude" changed to "long" to match the guidelines.

ċ
contexts2013.csv
(2k)
Adriel Dean-Hall,
Jun 7, 2013, 1:15 PM
ċ
contexts2013.json
(6k)
Adriel Dean-Hall,
Jun 7, 2013, 1:15 PM
ċ
examples2013.csv
(15k)
Adriel Dean-Hall,
Jun 7, 2013, 1:15 PM
ċ
examples2013.json
(18k)
Adriel Dean-Hall,
Jun 7, 2013, 1:15 PM
ċ
profiles2013.csv
(327k)
Adriel Dean-Hall,
Jun 7, 2013, 1:16 PM
ċ
profiles2013.json
(3084k)
Adriel Dean-Hall,
Jun 7, 2013, 1:16 PM