Commonsense Integration for Video Browsing

Introduction

When we browse media items on the web to gather information of an event, there are great difficulties in having a comprehensive understanding of this event using the current media browsing interfaces. Most of the interfaces provide only single view to list media items. Storied Navigation [Shen et al. 2009] is an interface agent with commonsense knowledge (see figure 1). It acts as an assistant to organize the media items using the context of an event, rather than as a conventional tool to list content.

Figure 1. Storied Navigation suggests video when user types “a leader says...”

In order to make conceptual relationship between videos to assist user to browse media items, we should provide context of a specific domain for interface agent to model the media content and users. These contextual information is used not only in media browsing but also in context-aware and storytelling interfaces. Spreading activation is the most common contextual reasoning method to apply on commonsense semantic networks. Although it is convenient to use APIs to retrieve the spreading activation results, coverage of knowledge bases and correctness of reasoning results are two major problems in using the retrieved contextual information to generate assistance to users. If the coverage/correctness is low, the interface agent may always fail to give helps to users and lower the satisfaction of users.

Instead of exploiting heuristics to improve the quality of contextual reasoning results, we use reasoning composition, an approach to combining multiple reasoning methods, to find an exact inference chain to the goals. We formalize the profile of reasoning methods and synthesize them through a planning procedure. With reasoning composition, it is also possible to reason with multiple commonsense KBs at the same time to enhance the ability of Storied Navigation.

Demo Video