Publications

Conference Papers

  • Qiang Zhang, Emine Yilmaz and Shangsong Liang: Ranking-based Method for News Stance Detection. In Proceedings of The Web Conference (WWW) 2018, Companion Volume, April 2018.

  • Rishabh Mehrotra, Ahmed Hassan Awadallah, Emine Yilmaz. LearnIR: WSDM 2018 Workshop on Learning from User Interactions. In Proceedings of WSDM 2018, February 2018.

  • Manisha Verma, Emine Yilmaz, Nick Craswell: Study of Relevance and Effort across Devices. In Proceedings of ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR), March 2018.

  • Task Embeddings: Learning Query Embeddings using Task Context. Rishabh Mehrotra and Emine Yilmaz. In Proceedings of ACM CIKM 2017 Conference on Information and Knowledge Management.

  • Deep Sequential Models for Task Satisfaction Prediction. Rishabh Mehrotra, Ahmet Hassan Awadallah, Milad Shokouhi, Emine Yilmaz, Imed Zitouni, Ahmed El Kholy, Madian Khabsa. In Proceedings of ACM CIKM 2017 Conference on Information and Knowledge Management.

  • Extracting Hierarchies of Search Tasks & Subtasks via a Bayesian Nonparametric Approach:. Rishabh Mehrotra, Emine Yilmaz. In Proceedings of ACM SIGIR 2017: 285-294.

  • Search Costs vs. User Satisfaction on Mobile. Manisha Verma and Emine Yilmaz. In Proceedings of the European Conference on Information Retrieval, ECIR 2017.

  • Auditing Search Engines for Differential Performance Across Demographics. Rishabh Mehrotra, Ashton Anderson, Fernando Diaz, Amit Sharma, Hanna Wallach and Emine Yilmaz. In Proceedings of the 26th International World Wide Web Conference (WWW), April 2017.

  • A Concept Language Model for Ad-hoc Retrieval: Bin Zou, Vasileios Lampos, Shangsong Liang, Zhaochun Ren, Emine Yilmaz, Ingemar J. Cox. In Proceedings of the 26th International World Wide Web Conference (WWW), Companion Volume, April 2017.

  • User Behaviour and Task Characteristics: a Field Study of Daily Information Behaviour. Jiyin He and Emine Yilmaz. In Proceedings of ACM SIGIR Conference on Human Information Interaction and Retrieval, March 2017. [BEST PAPER AWARD WINNER].

  • Collaborative User Clustering for Short Text Streams. Shangsong Liang, Zhaochun Ren, Emine Yilmaz and Evangelos Kanoulas. In Proceedings of AAAI 2017, February 2017.

  • Dynamic Clustering of Streaming Short Documents. In Proceedings of KDD 2016, August 2016

  • Exploiting Distributional Representations with Distance Dependent CRPs for Extracting Subtasks. Rishabh Mehrotra, Prasanta Bhattacharya and Emine Yilmaz. In Proceedings of NAACL HLT 2016. June 2016.

  • Explainable User Clustering in Short Text Streams. Yukun Zhao, Shangsong Liang, Zhaochun Ren, Jun Ma, Emine Yilmaz and Maarten de Rijke. In Proceedings of ACM SIGIR 2016, July 2016.

  • Sessions, Tasks and Topics – Uncovering Behavioral Heterogeneities in Online Search Behavior. Rishabh Mehrotra, Prasanta Bhattacharya and Emine Yilmaz. In Proceedings of ACM SIGIR 2016, July 2016.

  • Bayesian Performance Comparison of Text Classifiers. Dell Zhang, Jun Wang, Emine Yilmaz, Xiaoling Wang and Yuxin Zhou. In Proceedings of ACM SIGIR 2016, July 2016.

  • A Task Based Perspective to Information Retrieval. Emine Yilmaz. In Proceedings of the European Conference on Information Retrieval (ECIR), Padova, Italy, 20 – 23 March 2016.

  • Characterizing Relevance on Mobile and Desktop. Manisha Verma and Emine Yilmaz. In Proceedings of the European Conference on Information Retrieval (ECIR), Padova, Italy, 20 – 23 March 2016.

  • Characterizing User’s Multi-Tasking Behavior in Web Search. Rishabh Mehrotra and Emine Yilmaz. In Proceedings of the ACM SIGIR 2016 Conference on Human Information Interaction and Retrieval, Chapel Hill, North Carolina, 13-17 March 2016.

  • Extracting Category Oriented Tasks. Manisha Verma and Emine Yilmaz. In Proceedings of the ACM SIGIR 2016 Conference on Human Information Interaction and Retrieval, Chapel Hill, North Carolina, 13-17 March 2016.

  • On obtaining effort based judgments for Information retrieval. Manisha Verma, Emine Yilmaz and Nick Craswell. In Proceedings of the ACM WSDM Conference on Web Search and Data Mining, February 2016.

  • Overview of the TREC 2015 Tasks Track. Emine Yilmaz, Evangelos Kanoulas, Manisha Verma, Nick Craswell and Rishabh Mehrotra. In Proceedings of The Twenty-Fourth Text REtrieval Conference, TREC 2015, Gaithersburg, Maryland, USA, November 17-20, 2015.

  • Terms, Topics and Tasks: Enhanced User Modelling for Better Personalization. Rishabh Mehrotra and Emine Yilmaz. In Proceedings of the ACM SIGIR Conference on Theories of Information Retrieval, September 2015.

  • Actively Selecting Representative & Informative Queries via Submodular Functions. Rishabh Mehrotra and Emine Yilmaz. In Proceedings of the 37th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 2015.

  • Anchoring and Adjustment in Relevance Estimation. Milad Shokouhi, Ryen White and Emine Yilmaz. In Proceedings of the 37th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 2015.

  • Towards Hierarchies of Search Tasks & Subtasks. Rishabh Mehrotra and Emine Yilmaz. In Proceedings of the 24th International World Wide Web Conference (WWW), May 2015.

  • Effect of Intent Descriptions on Retrieval Evaluation. Emine Yilmaz, Evangelos Kanoulas and Nick Craswell. In Proceedings of the 23rd International ACM Conference on Information and Knowledge Management (CIKM), November 2014.

  • Relevance and Effort: An Analysis of Document Utility. Emine Yilmaz, Manisha Verma, Nick Craswell, Filip Radlinski and Peter Bailey. In Proceedings of the 23rd International ACM Conference on Information and Knowledge Management (CIKM), November 2014.

  • Entity Oriented Task Extraction from Query Logs. Manisha Verma and Emine Yilmaz. In Proceedings of the 23rd International ACM Conference on Information and Knowledge Management (CIKM), November 2014.

  • Task-Based User Modelling for Personalization via Probabilistic Matrix Factorization. Rishabh Mehrotra, Emine Yilmaz and Manisha Verma: RecSys Posters. October 2014.

  • User intent and assessor disagreement in web search evaluation. Gabriella Kazai, Emine Yilmaz, Nick Craswell and Seyed M. M. Tahaghoghi. In Proceedings of the 22nd International ACM Conference on Information and Knowledge Management (CIKM), October 2013.

  • Modelling Score Distributions without Actual Scores. Stephen Robertson, Evangelos Kanoulas, Emine Yilmaz. In Proceedings of the 4th International Conference on the Theory of Information Retrieval (ICTIR), October 2013.

  • Incorporating Variability in User Behavior into Systems based Evaluation. Ben Carterette, Evangelos Kanoulas, Emine Yilmaz. In Proceedings of the 21stInternational ACM Conference on Information and Knowledge Management (CIKM), October 2012.

  • An Analysis of Systematic Judging Errors in Information Retrieval. Gabriella Kazai, Nick Craswell, Emine Yilmaz, Seyed M. M. Tahaghoghi. In Proceedings of the 21stInternational ACM Conference on Information and Knowledge Management (CIKM), October 2012.

  • On Judgments Obtained from a Commercial Search Engine. Emine Yilmaz, Gabriella Kazai, Nick Craswell, Seyed M. M. Tahaghoghi: In Proceedings of the 34thAnnual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 2012.

  • An Uncertainty-aware Query Selection Model for Evaluation of IR Systems. Mehdi Hosseini, Ingemar J. Cox, Natasa Milic-Frayling, Milad Shokouhi, Emine Yilmaz. In Proceedings of the 34th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 2012.

  • Semi-supervised Learning to Rank with Similarity-based Regularization. Martin Szummer and Emine Yilmaz. In Proceedings of the 20th International ACM Conference on Information and Knowledge Management (CIKM), October 2011.

  • Simulating Simple User Behavior for System Effectiveness Evaluation. Ben Carterette, Evangelos Kanoulas, Emine Yilmaz. In Proceedings of the 20thInternational ACM Conference on Information and Knowledge Management (CIKM), October 2011.

  • Relevance Feedback Exploiting Query-Specific Document Manifolds. Chang Wang, Emine Yilmaz, Martin Szummer. In Proceedings of the 20th International ACM Conference on Information and Knowledge Management (CIKM), October 2011.

  • Inferring and Using Location Metadata to Personalize Web Search. Paul N. Bennett, Filip Radlinski, Ryen W. White, Emine Yilmaz. In Proceedings of the 33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 2011.

  • Detecting Duplicate Web Documents Using Clickthrough Data. Filip Radlinski, Paul N. Bennett, and Emine Yilmaz. In Proceedings of the 4th International ACM Conference on Web Search and Data Mining (WSDM), February 2011. [Best paper award candidate based on votes of the participants]

  • Expected Browsing Utility for Web Search Evaluation. Emine Yilmaz, Milad Shokouhi, Nick Craswell, Stephen Robertson. In Proceedings of the 19thInternational ACM Conference on Information and Knowledge Management (CIKM), October 2010.

  • Low Cost Evaluation in Information Retrieval. Ben Carterette, Evangelos Kanoulas, Emine Yilmaz. In Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 2010.

  • Extending Average Precision to Graded Relevance Judgments. Stephen Robertson, Evangelos Kanoulas and Emine Yilmaz. In Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 2010. [BEST PAPER AWARD CANDIDATE]

  • Microsoft Research at TREC 2009. Nick Craswell, Dennis Fetterly, Marc Najork, Stephen Robertson and Emine Yilmaz. In Proceedings of the 18th Text REtrieval Conference (TREC), November 2009.

  • Deep versus Shallow Judgments in Learning to Rank. Emine Yilmaz and Stephen Robertson. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 2009.

  • Document Selection Methodologies for Efficient and Effective Learning-to-Rank. Javed A. Aslam, Evangelos Kanoulas, Virgil Pavlu, Stefan Savev, Emine Yilmaz. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 2009. [BEST PAPER AWARD CANDIDATE]

  • Relevance Assessment: Are Judges Exchangeable and Does it Matter? Peter Bailey, Nick Craswell, Ian Soboroff, Paul Thomas, Arjen P. de Vries, Emine Yilmaz. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 2008.

  • A New Rank Correlation Coefficient for Information Retrieval. Emine Yilmaz, Javed A. Aslam and Stephen Robertson. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 2008.

  • A Simple and Efficient Sampling Method for Estimating AP and NDCG. Emine Yilmaz, Evangelos Kanoulas, Javed A. Aslam. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 2008.

  • Inferring Document Relevance from Incomplete Information. Javed A. Aslam and Emine Yilmaz. In Proceedings of the 16th International ACM Conference on Information and Knowledge Management (CIKM), November 2007.

  • Estimating Average Precision with Incomplete and Imperfect Information. Emine Yilmaz and Javed A. Aslam. In Proceedings of the 15th International ACM Conference on Information and Knowledge Management (CIKM), October 2006. Invited to Appear in International Journal of Knowledge and Information Systems.

  • A Statistical Method for System Evaluation Using Incomplete Judgments. Javed A. Aslam, Virgil Pavlu and Emine Yilmaz. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 2006.

  • Inferring Document Relevance via Average Precision. Javed A. Aslam and Emine Yilmaz. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 2006.

  • A Geometric Interpretation and Analysis of R-Precision. Javed A. Aslam and Emine Yilmaz. In Proceedings of the 14th International ACM Conference on Information and Knowledge Management (CIKM), October 2005.

  • The Maximum Entropy Method for Analysing Retrieval Measures. Javed A. Aslam, Emine Yilmaz and Virgil Pavlu. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 2005.

  • A Geometric Interpretation of R-precision and Its Correlation with Average Precision. Javed A. Aslam, Emine Yilmaz and Virgil Pavlu. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 2005.

  • Measure-based Metasearch. Javed A. Aslam, Virgil Pavlu and Emine Yilmaz. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 2005.

Journal Articles

  • Collaboratively Tracking Interests for User Clustering in Streams of Short Texts. Shangson Liang, Evangelos Kanoulas and Emine Yilmaz. IEEE Transactions on Knowledge and Data Engineering (TKDE), May 2018.

  • Report on the 2017 ACM SIGIR International Conference Theory of Information Retrieval (ICTIR'17). Hui Fang, Jaap Kamps, Evangelos Kanoulas, Maarten de Rijke, Emine Yilmaz. In ACM SIGIR Forum 51(3): 78-87 (2017).

  • Search Result Diversification in Short Text Streams. Shangsong Liang, Emine Yilmaz, Hong Shen, Maarten de Rijke, W. Bruce Croft: ACM Transactions on Information Systems (TOIS) 36(1): 8:1-8:35 (2017).

  • Report on the Second Workshop on Supporting Complex Search Tasks. Marijn Koolen, Jaap Kamps, Toine Bogers, Nicholas J. Belkin, Diane Kelly, Emine Yilmaz: ACM SIGIR Forum 51(1): 58-66 (2017).

  • Inferring Dynamic User Interests in Streams of Short Texts for User Clustering. Shangson Liang, Zhaochun Ren, Yukun Zhao, Jun Ma, Emine Yilmaz, Maarten de Rijke: ACM Transactions on Information Systems (TOIS) 36(1): 10:1-10:37 (2017).

  • EVIA 2016: The Seventh International Workshop on Evaluating Information Access. Charles L. A. Clarke, Emine Yilmaz. ACM SIGIR Forum 50(2): 44-46, 2016.

  • Matthew Lease, Emine Yilmaz: Crowdsourcing for Information Retrieval: Introduction to the Special Issue. Information Retrieval Journal 16(2): 91-100, 2013.

  • Report on the SIGIR 2013 Workshop on Modeling User Behavior for Information Retrieval Evaluation. Charles L.A. Clarke, Luanne Freund, Mark D. Smucker and Emine Yilmaz. In ACM SIGIR Forum 47(2): 84-95, 2013.

  • Crowdsourcing interactions: Capturing query sessions through crowdsourcing. Guido Zuccon, Teerapong Leelanupab, Stewart Whiting, Emine Yilmaz, Joemon M. Jose, Leif Azzopardi. Information Retrieval Journal, Special Issue on Crowdsourcing for Information Retrieval, 2013.

  • Crowdsourcing for Information Retrieval. Matthew Lease and Emine Yilmaz. In ACM SIGIR Forum, 45 (2), December 2011.

  • Crowdsourcing for Search and Data Mining. Matthew Lease, Vitor R. Carvalho, Emine Yilmaz. In ACM SIGIR Forum, 45(1), June 2011.

  • On the Choice of Effectiveness Measures for Learning to Rank. Emine Yilmaz and Stephen Robertson. Special Issue on Learning to Rank for IR, Information Retrieval Journal, 2010.

  • Crowdsourcing for Search Evaluation. Vitor Carvalho, Matthew Lease, and Emine Yilmaz. In ACM SIGIR Forum, 44(2), December 2010.

  • Estimating Average Precision When Judgments are Incomplete. Emine Yilmaz and Javed A. Aslam. International Journal of Knowledge and Information Systems (KAIS), 2008.