Metrics used at TREC 2013
Time-Biased Gain (TBG)
D - Exponential decay function with a Half-Life of 224 seconds
T - Time to reach rank k, i.e., the time it took to read the number of descriptions and documents before rank k
Every description is read
If the description is found to be interesting or neutral the document is read
A(k) = 1 if the user likes or is neutral about the description and likes the geographically appropriate document at rank k, otherwise it is 0 (i.e., did the user like the suggestion)
Z(k) = 1Â if the user doesn't like either the description or the document at rank k or if the document at rank k is not geographically appropriate, otherwise it is 0 (i.e., did the user dislike the suggestion)
More information about this metric can be found in the Evaluation Contextual Suggestion paper here: http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings10/EVIA/abstract_evia.html#EVIA09
Precision at Rank 5 (P@5)
The number of suggestions where the user liked both the description and the geographically appropriate document, divided by 5.
Mean Reciprocal Rank (MRR)
1 over the rank of the first suggestion where the user liked both the description and the geographically appropriate document.