COCO-Search18
COCO-Search18 is a laboratory-quality dataset of goal-directed behavior large enough to train deep-network models. It consists of the eye gaze behavior from 10 people searching for each of 18 target-object categories in 6202 natural-scene images, yielding ~300,000 search fixations. COCO-Search18 is now part of the MIT/Tuebingen Saliency Benchmark, previously the MIT Saliency Benchmark but renamed to reflect the new group that will be managing the competition. As part of this re-organization, the benchmark will broaden its scope to go beyond purely spatial fixation prediction and even beyond the free-viewing task. COCO-Search18 is partly responsible for this broadening, and represents a significant expansion of the benchmark into goal-directed search behavior. The training, validation, and test images in COCO-Search18 are already freely available as part of COCO. Researchers are also free to see and use COCO-Search18’s training and validation search fixations, but the fixations on the test images are withheld. As part of a separate benchmark track, it will be possible to upload predictions and have them evaluated on the test dataset. In this initial stage of release, only fixations made on target-present search trials are available at this time (stay tuned for release of the target-absent fixations). We hope you enjoy using COCO-Search18!
Resource
Download
In this initial stage of release, only fixations made on target-present (TP) search trials are available at this time.
👏 We are taking down the testing data in order to set up an online benchmark, stay tuned!
Stay tuned for release of the target-absent fixations.
COCO-Search18 is now part of the MIT/Tuebingen Saliency Benchmark.
COCO-Search18 TP Dataset (1.1 GB) contains :
Image Stimuli: 3101 target-present (TP) images (size: 1680x1050) [Download]
Eye Fixation: fixations on target-present search trials
Training and validation set [Download]
Testing set [Download]
Readme file: details on the data format [Download]
COCO-Search18 TA Dataset (1.1 GB) contains :
Image Stimuli: 3101 target-absent (TA) images (size: 1680x1050) [Download]
Eye Fixation: fixations on target-absent search trials
Training and validation set [Download]
Testing set [Download]
Readme file: details on the data format [Download]
Supplemental:
Saccade data: saccade amplitude on target-present (TP) search trials (trainval set) [Download]
Excluded images: image ids and category detection scores for excluded images [Download]
Code on Github
A related dataset: MCS Dataset
Paper
If you use COCO-Search18, please cite:
Chen, Y., Yang, Z., Ahn, S., Samaras, D., Hoai, M., & Zelinsky, G. (2021). COCO-Search18 Fixation Dataset for Predicting Goal-directed Attention Control. Scientific Reports, 11 (1), 1-11, 2021. https://www.nature.com/articles/s41598-021-87715-9
Yang, Z., Huang, L., Chen, Y., Wei, Z., Ahn, S., Zelinsky, G., Samaras, D., & Hoai, M. (2020). Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 193-202).
@article{chen2021coco,
title={COCO-Search18 fixation dataset for predicting goal-directed attention control},
author={Chen, Yupei and Yang, Zhibo and Ahn, Seoyoung and Samaras, Dimitris and Hoai, Minh and Zelinsky, Gregory},
journal={Scientific reports},
volume={11},
number={1},
pages={1--11},
year={2021},
publisher={Nature Publishing Group}
}
@InProceedings{Yang_2020_CVPR,
author = {Yang, Zhibo and Huang, Lihan and Chen, Yupei and Wei, Zijun and Ahn, Seoyoung and Zelinsky, Gregory and Samaras, Dimitris and Hoai, Minh},
title = {Predicting Goal-Directed Human Attention Using Inverse Reinforcement Learning},
booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}
Examples
Notes
You agree not to reproduce, duplicate, copy, sell, trade, resell or exploit for any commercial purposes, any portion of the images and any portion of derived data.
You agree not to further copy, publish or distribute any portion of the COCO-Search18 and COCO-FreeView dataset. Except, for internal use at a single site within the same organization it is allowed to make copies of the database.
The EyeCogLab and/or CVLab@Stony Brook University reserves the right to terminate your access to the database at any time.