Project Ideas
Eye Tracking
Nilavra Bhattacharya, Somnath Rakshit, Jacek Gwizdka, and Paul Kogut. 2020. Relevance Prediction from Eye-movements Using Semi-interpretable Convolutional Neural Networks. In Proceedings of CHIIR '20. ACM, New York, NY, USA, 223–233. https://doi.org/10.1145/3343413.3377960
If it is possible to get the eye-tracking dataset:
Better comparison with CNN methods & traditional methods (SVM/Random Forest)
Trying cross-validation with traditional methods, see if we can get better results than they presented
Using more semantic or higher-level information about the document in the images.
Canary - We have eye-tracking data for users reading the text part in the Canary project,
We applied VTNet to Cookie Theft task but have not yet tried with the Reading task
Would be interesting to implement the codification of scanpaths in the paper as well (different icons to represent different-length fixations)
MSNSV - The text in the MSNV documents contain relevant and non-relevant sentences (for which we have AOIs), and the users further reported their levels of interest in the documents. Thus there is potential for analysis of the eye-tracking data over the text, for instance to predict user interest or to mine behaviors over the relevant vs. non relevant sentences.
Interpretability
Chen et al. This Looks Like That: Deep Learning for Interpretable Image Recognition Chen, C., Li, O., Tao, C., Barnett, A. J., Su, J., & Rudin, C. (2018). This looks like that: deep learning for interpretable image recognition. arXiv preprint arXiv:1806.10574.
It would be interesting to apply the same approach proposed in the paper to other datasets
The paper targets the object detection task, but it would be interesting to extend the method to other vision tasks
A related work applies a similar approach on sequential data. Though the code is not available, it would be interesting to reimplement that method and applying it to some other datasets (e.g., eye-tracking)