Workshop program

 9:00 - 9:10

 Opening remarks

 9:10 - 9:40
 CONTRIBUTED TALK: Evgeniy Gabrilovich, Google
  • Title: "From information needs to action needs: towards contextual app search and recommendation"
  • [Download slides (PDF)]
  • Abstract: Classical IR is centered on the concept of an information need, which can be satisfied by (reading) one or more documents. Yet people rarely search merely to satisfy their curiosity; rather, they search the Web to get things done. Imagine a search engine that instead of offering a multitude of Web pages returns applications (apps) that directly help the users accomplish their goals. Inspired by the notion of an information need, we formulate the concept of an action need, which can be satisfied by a single app or a sequence thereof. Examples of action needs include reserving a restaurant table, booking a flight or an entire vacation, and comparison shopping. Although related to collaborative filtering, recommending apps in Web search is a more powerful paradigm as it considers not only users’ past activity and similarity to other users, but also their current context and action need. Contextual recommendation of applications is particularly useful in mobile Web search, where the limited form factor of the device leads the users to appreciate any assistance they can get.
While there exist literally hundreds of thousands of apps, actually finding an app relevant to the user’s current action need is far from trivial. Designing app search systems calls for principled information retrieval techniques, such as query analysis and rewriting, sophisticated app indexing, learning to rank, and location awareness. Recommending relevant apps (both paid and free) in Web search also leads to interesting monetization opportunities. As opposed to sponsored search ads, however, pro bono apps can co-exist with paid ones, as long as they help the user satisfy his action needs (and thus help the search engine keep the user satisfied). Finally, observing users’ interaction with apps enables better characterization of users’ interests. This talk will introduce the emerging field of contextual app search and recommendation, and discuss its technical challenges and promising research directions.

This work has been done while the author was with Yahoo! Research, in collaboration with Zhaohui Zheng and many other colleagues in Yahoo! Labs.
  • Bio: Evgeniy Gabrilovich is a Senior Staff Research Scientist at Google. Prior to that, he was a Director of Research and Head of the NLP & IR Group at Yahoo! Research. He authored over 50 publications in top international venues, and filed over 20 patents. Evgeniy is a recipient of the 2010 Karen Sparck Jones Award for his contributions to natural language processing and information retrieval. He served as a Senior PC member or Area Chair at SIGIR, WWW, WSDM, AAAI, IJCAI, ACL, EMNLP, CIKM, ICDM, and ICWSM, and served on the program committees of virtually every major conference in the field. He organized a number of workshops and taught multiple tutorials at SIGIR, ACL, WWW, WSDM, IJCAI, AAAI, CIKM, and EC. Evgeniy earned his PhD degree in Computer Science from the Technion - Israel Institute of Technology.
9:40 - 10:30
 INVITED TALK: Patrick Pantel, Microsoft Research
  • Abstract:  We introduce an entity-centric search experience, called active objects, in which entity-bearing information requests are paired with actions that can be performed on the entities, thus enabling the brokering of web pages and applications on the Web that can satisfy the intended action.

    In this vision, the broker is aware of all entities and actions of interest to its users, understands the intent of the user, ranks all providers of actions, and provides direct actionable results through APIs with the providers. For example, consider a user who queries for "jetbeam rrt-0," a flashlight. The broker would recognize the particular entity mentioned in the query, would return a personalized ranked list of actions to the user, would allow the user to save clicks and time to accomplish his intended action, and even would allow the user to sometimes discover new actions to help them toward their goals. New revenue streams open up from paid action placement, lead generation, and on-site commercial transactions.

    In an annotation study conducted over a random sample of user query sessions, we found that a large proportion of queries in query logs involve actions on entities, calling for an automatic approach to identifying relevant actions for entity-bearing queries. We pose the problem of finding actions that can be performed on entities as the problem of doing probabilistic inference in a graphical model that captures how an entity bearing query is generated. We design models of increasing complexity that capture latent factors such as entity type and intended actions that determine how a user writes a query in a search box, and the URL that they click on. Given a large collection of real-world queries and clicks from a commercial search engine, the models are learned efficiently through maximum likelihood estimation using an EM algorithm. Given a new query, probabilistic inference enables recommendation of a set of pertinent actions and hosts. We propose an evaluation methodology for measuring the relevance of our recommended actions, and show empirical evidence of the quality and the diversity of the discovered actions. We further show how the models can be used to enhance entity repositories such as Freebase with type distributions over an application domain such as web search. We end by showing how the proposed models can be cast in a wider framework providing a generalized entity-centric natural language user interface, where experiences with entities can be rendered consistent across information devices.
  • Bio:  Patrick Pantel is a Senior Researcher at MSR, conducting research in large-scale natural language processing, text mining, web search, and knowledge acquisition. Prior he served as a Senior Research Manager at Yahoo! Labs, and as a Research Assistant Professor at the USC Information Sciences Institute. In 2003, he received a Ph.D. in Computing Science from the University of Alberta in Edmonton, Canada.
10:30 - 11:00


11:00 - 11:50

 INVITED TALK: James G. Shanahan, Independent Consultant, Advisor to Quixey
  • Title: "Bridging the app gap: from web search to app search via functional search"
  • [Download slides (PDF)]
  • Abstract: An app is a piece of computer software designed to help a user perform specific tasks. Apps were originally intended for productivity (email, calendar and contact databases), but consumer and business demand has caused rapid expansion into other areas such as games, factory automation, GPS and location-based services, banking, order-tracking, and ticket purchases. In short, apps are transforming how we work, rest and play, in terms of how information and services are generated, consumed and produced. This has lead to the creation of millions of apps that can be deployed on a wide array of platforms including, smartphones, tablet computers, desktop PCs, automobiles, gaming consoles, and refrigerators. A big challenge posed by this paradigm shift is how to match apps with users’ needs. Web search engines have attempted to accomplish this, albeit in a limited fashion, through augmenting search engine results pages (SERPs) with apps in response to task-centric queries (e.g., flight-search app). However, app search, like other forms of search, such as local search or multimedia search, is non-trivial and requires a fundamentally different approach to search.  This talk will briefly review the field of search while focusing on a new paradigm, Functional Search™, as a key means to fulfilling app search. An example of a Functional Search™ engine, Quixey, will be discussed. In addition, this talk will discuss some of the challenges and promising research directions in app search.
  • Bio: Jimi has spent the last 23 years developing and researching cutting-edge information management systems. During the summer of 2007, he started a boutique consultancy (Church and Duncan Group Inc.,  in San Francisco) whose major goal is to help companies leverage their vast repositories of data using statistics, machine learning, optimization theory and data mining for big data applications in areas such as web search, local and mobile search, and online advertising. Church and Duncan Group’s clients include Adobe, AT&T Interactive,, eBay,,,, and In addition, Jimi has been affiliated with the University of California at Santa Cruz since 2009 where he teaches a sequence of courses on big data analytics and stochastic optimization (ISM 209, ISM 250 and ISM251). He advises several high-tech startups (including Quixey)  in the Silicon Valley Area and is executive VP of science and technology at Irish Innovation Center (IIC). He has served as a fact and expert witness.
Prior to starting Church and Duncan Group Inc., Jimi was Chief Scientist and executive team member at Turn Inc. (an online ad network that has recently morphed to a demand side platform).  Prior to joining Turn, Jimi was Principal Research Scientist at Clairvoyance Corporation where he led the “Knowledge Discovery from Text” Group. In the late 1990s he was a Research Scientist at Xerox Research Center Europe (XRCE) where he co-founded Document Souls, an anticipatory information system. In the early 90s, he worked on the AI Team within the Mitsubishi Group in Tokyo.  He has published six books, over 50 research publications, and 15 patents  in the areas of machine learning and information processing. Jimi chaired CIKM 2008 (Napa Valley), co-chaired International Conference in Weblog and Social Media (ICWSM) 2011 in Barcelona, and will be the PC co-chair of ICWSM 2012 (Dublin). He co-chaired the ISSDM Workshop on Knowledge Management: Analytics and Big Data at UC Santa Cruz.  He has organized several workshops in digital advertising as part of SIGIR, KDD and NIPS. Jimi received his Ph.D. in engineering mathematics from the University of Bristol, U.K. and holds a bachelor of science degree from the University of Limerick, Ireland. He is a Marie Curie fellow. In 2011 he was selected as a member of the Silicon Valley 50 (Top 50 Irish Americans in Technology).

 11:50 - 12:30

 Anindya Ghose (Stern School of Business, NYU and Wharton School, University of Pennsylvania) and Sang Pil Han (College of Business, City University of Hong Kong)
  • Abstract: A fundamental change brought forth by the advent of the mobile internet has been the widespread adoption of mobile phone based applications (apps). Mobile apps are now being sued worldwide to perform a variety of tasks - access social networks, read ebooks, play games, listen to music, watch videos and so on. In 2010 more than 300,000 applications were downloaded 10.9 billion times and in 2014 global downloads are projected to reach 76.9 billion downloads worth approximately US$35 billion. As consumers increasingly use mobile apps, it is important to estimate consumer demand for mobile apps and quantify the value created by these new apps and services. We build a structural model of user demand for mobile applications and estimate the change in consumer surplus from the usage of mobile apps. We use a panel dataset consisting of top 300 ranked applications’ sales rank, prices, and characteristics data from the two leading app stores – Apple’s App Store and Google’s Android Market. We address the price endogeneity issue intrinsic in demand estimation by building a random coefficient logit demand model in a similar vein to the BLP (1995) method. Our results show that demand increases with the file size of apps and the length of description, but decreases with the age of apps (time since release) and the number of screenshots included. Compared to utility apps, education, gaming, and multimedia apps have a positive effect on demand. We incorporate consumer heterogeneity in the model and find that older and male consumers tend to be less sensitive to the prices of apps than younger and female consumers, respectively. Using the estimated demand function, we find that the top 300 ranked applications in both app stores enhanced consumer surplus by approximately $935 million over the time period of our study.
  • Bio: Sang Pil Han is an assistant professor at Information Systems department in City University of Hong Kong. He was a postdoctoral researcher at Stern School of Business of New York University. He received his doctoral degree from Korea Advanced Institute of Science and Technology Business School. His research interests include economics of mobile technology, economic value from content in spaces mediated by social media, consumer behavior in social networks, and economic aspects of ecommerce. His papers have been published or accepted in Management Science, Management Information Systems Quarterly (MISQ), Telecommunications Policy among others.
   12:30 - 14:00


 14:00 - 14:50
 CONTRIBUTED TALK: Marcin Rudolf, CTO, Xyologic
  • Abstract: How should perfect app search engine work? What is the “essence” of the app and how does it relate to books, music and web pages as informational objects? What are specific properties of the app we can use to find them?

    Our first observation is that apps are strongly grouped into classes (called by us ‘topics’) which are much more granular than native app categories used in app stores. When a user looks for an app she typically wants the best app from a particular topic (for example: the best app for making notes or the best arcade game with birds). The way we look at it app search engines face a following set of problems (1) classify all apps into topics (2) guess the intent of the user (which topic?) (3) find the best app in topic. The first problem can be solved with many existing classification algorithms and is the least app specific. Attempts to solve the second problem led us to our next observation: keyword search is not capturing user’s intent optimally. Did you know that 80% of all search queries are what we consider generic app search queries like „best app”, „games”, „music”, „cars”? For example the “sleep” keyword will return you alarm clocks, binaural music generators, hypnosis or meditation helpers and apps with “sleep mode”. A proper search engine should at least disambiguate the results and guide the user to a proper topic. When we add other problems with keywords search on top, we often think that search box is not an answer to apps’ search at all. The third problem is as important as the second. We solve it by computing various popularity and quality signals from social interactions in the app store. Above all, we analyze downloads trends on a country level. On top of that we add many other signals including qualitative ones like positive and negative sentiments we discover in the users' comments.

    The bottom line is that in our opinion a perfect app discovery platform should merge search and recommendation engines into one experience.

  • Bio: Thanks to the first 8bit machines that appeared at the time Marcin Rudolf started attending primary school, he has been hacking and programing computers for his entire life, including at least 15 years of doing it professionally. From the very start of his career he was dealing with mobile technologies, starting from the first WAP portal in Poland and a large SMS/MMS messaging system and then proceeding to more advanced projects like designing a whole mobile DRM system for a big German company.  During his studies at Warsaw University of Technology he got interested in data visualization, physical process modeling and machine learning and which become his passion and are often a part of his projects. In the last 1,5 years Marcin has been leading technology development at Xyologic, where he builds a perfect search engine for apps. Xyologic is what he considers a perfect project for him as it blends his experience in mobile technologies with text mining and advanced machine learning techniques.  Marcin got his Master of Science in IT from Warsaw University of Technology and also finished sociology at Warsaw Univeristy. He has published in both fields.

 14:50 - 15:30
 CONTRIBUTED PAPER:  Venu Vasudevan, Silviu Chiricescu, Jehan Wickramasuriya, Gilles Drieu, and Sriram Yadavali (Motorola Mobility)  
  • Abstract: TV as a medium is undergoing two notable trends - dispersion and atomization. Dispersion refers to the evolution of TV into a distributed experience across multiple screens, and atomization to the componentization of parts of a TV experience into modular and separate apps. While there is marked enthusiasm around the emerging area of TV Apps, we anticipate that complexity of experience will be a barrier to mainstream adoption of Appified TV. To maintain the efortless experience associated with TV viewing while supporting the new TV apps and addressing the aforementioned trends requires a new approach to both application search & discovery and multi-app interactivity.
The proposed search & discovery approach combines elements of organic and sponsored search adapted for the unique challenges of the TV experience to provide timely, relevant and most importantly, effortless discovery of TV-centric applications to the user. Our approach to multi-app interactivity minimizes the effort in (concurrently) stateful interactions within apps that are used as companion to TV programs that are watched repeatedly. A framework that enables the user to concurrently interact with multiple apps while providing auto-adaptive interfaces that can deal with the specific context and modality needs of dual screen viewing in living room contexts is presented.
  • Bio: Venu Vasudevan is a manager at Motorola Technology, and manages Betaworks - a lab exploring commercializable opportunities in mobile and interactive media. In past lives, Venu has managed a research group on Pervasive Computing, been an architect on a rather large distributed system, and been involved in a couple of startups. Over the past few years, his work has led to 40'ish technical papers, over 25 filed patents, Motorola's first business with a major international carrier in 15 years, two emerging media products, new initiatives around targeted advertising, and the definition and execution of a business around dual-screen TV. Venu has a PhD from the Ohio State University, is an Adjunct Professor at Rice University ECE, and a member of Motorola's Science Advisory Board.
 15:30 - 16:00

 16:00 - 17:00
 Concluding remarks and informal discussion