MediaSync2015

Media Synchronization Workshop (MediaSync) 2015

in conjunction with ACM TVX 2015

June 3, 2015, Brussels, Belgium

Crowne Plaza Brussels (Rue Gineste 3)

Proceedings: ISBN/EAN: 978-90-5968-463-4

The workshop is sponsored by the following projects:

Some Pictures

Technical Program Committee

Cyril Concolato, (Telecom ParisTech),

Davy Van Deursen (EVS),

Luiz Fernando Gomes Soares (PUC-RIO),

Jack Jansen (CWI),

Tim Stevens (BT),

Christian Timmerer (Klagenfurt University),

Ishan Vaishnavi (Huawei).

Keynote

Matt Hammond: Senior Research Engineer, BBC R&D

Matt Hammond is a Senior Research Engineer at the BBC's Research and Development department in London, UK where he has worked since 2001. In recent years his research work has focused on the interaction between companion devices, televisions and broadcast services; developing and looking at both the underlying technologies and what new user experiences and accessibility improvements can be achieved. He has contributed significantly to the development of the new DVB CSS and HbbTV 2.0 specifications to enable a range of companion and 2nd screen capabilities including media synchronisation.

Synchronisation standardised for connected TVs

Work over the past two years in industry standards groups (DVB and HbbTV) makes it likely that we will see support for high-accuracy media synchronisation between phones or tablets and the next generation of connected TVs in Europe. This talk will give an introduction to the features, technologies and protocols that have been defined and speculate on areas of relevant research that follow from it.

Papers

"Multi-device Linear Composition on the Web, Enabling Multi-device Linear Media with HTMLTimingObject and Shared Motion"

by Ingar M. Arntzen, Njål T. Borch, François Daoust, Dominique Hazael-Massieux

Composition is a hallmark of the Web, yet it does not fully extend to linear media. This paper defines linear composition as the ability to form linear media by coordinated playback of independent linear components. We argue that native Web support for linear composition is a key enabler for Web-based multi-device linear media, and that precise multi-device timing is the main technical challenge. This paper proposes the introduction of an HTMLTimingObject as basis for linear composition in the single-device scenario. Linear composition in the multi-device scenario is ensured as HTMLTimingObjects may integrate with Shared Motion, a generic timing mechanism for the Web. By connecting HTMLMediaElements and HTMLTrackElements with a multi-device timing mechanism, a powerful programming model for multi-device linear media is unlocked

"Time-Awareness for Media-Synchronisation– Opportunities & Challenges"

by Hugh Melvin, Kevin Stanton, Sundeep Chandoke

Although the scope of Media Synchronisation is quite broad, a common thread relates to a sense of time and/or frequency as being a key enabler. The term ‘timing’ as applied in this paper, is used as an umbrella term that represents all aspects – whether time, frequency, or phase. In this paper, we firstly examine a range of multimedia scenarios where time and/or frequency synchronisation are critical requirements to ensure good Quality of Service/Quality-of-Experience(QoS/QoE) for end-users - these scenarios include RealTime Communications (RTC) applications, Hybrid Broadcast/Broadband (Hbb) applications and HTTP Adaptive Streaming (HAS). We then step back from the applications and examine the notion of Time-Awareness and its key role in Media Synchronisation. In particular we discuss the growing realisation of the importance of Time Awareness across the full end-to-end infrastructure over which Multimedia applications operate, and in this context, introduce the work of the recently formed Interest Group (IG) termed Time Aware Applications, Computers and Communications Systems (TAACCS). Though the scope of TAACCS extends to the so-called Internet of Everything (IoE), the Media Synchronisation community, although already acutely aware of timing issues , community can learn a lot from its approach.

"Review of Media Sync Reference Models: Advances, Inconsistencies and Open Issues"

by Mario Montagud, Jack Jansen, Fernando Boronat, Pablo Cesar

The advances on multimedia systems have brought new challenges and requirements for media sync. Over the years, many media sync solutions have been devised. Due to this variety, several studies have surveyed the existing solutions and proposed classification schemes or reference models for media sync. This paper claims the relevance of media sync reference models to systematically structure and synthesize this research area. Accordingly, a review of the existing reference models is provided, by examining the involved features, components and layers in each one of them. Likewise, some inconsistencies, open issues and missing components in existing reference models have been identified. Accordingly, this study reflects the need for a new modular and extensible theoretical framework or reference model to efficiently comprehend the overall media sync research area.

"New Approaches to Improving the Quality of Media Accessibility Services on HbbTV"

by Andreas Jesina, Juan Martínez, Robin Nachtrab-Ribback, Marko Nalis, Gion Linder, Pablo Romero-Fresco

This paper describes the efforts carried out by the authors to improve the quality of currently existing media accessibility services within the Interpreter Telepresence System by VerbaVoice, focusing on subtitle quality using the NERstar Editor as well as accessibility of TV programming for hearing impaired people via the VerbaVoice HbbTV App. Specifically, it describes algorithms currently used to improve subtitle readability and synchronicity, as well as issues and solution approaches towards synchronizing source material, sign language video and subtitles within a live broadcasting context for HbbTV.

"An Automatic Media Synchronizing Mechanism with TV Programs"

by Yuanyuan Wang, Yukiko Kawai, Kazutoshi Sumiya

Almost all TV programs provide a high affinity geographic data, such as tourist spots, historical places. However, current services cannot present geographic contents synchronized with the TV programs, and viewers (e.g., tourists) difficult to grasp the surroundings of the geographic data, how the locations are related, and distances between them during the TV programs. Therefore, we have developed a system based on the concept of the second screen service, which presents supplementary information synchronized with a TV program on a big display or smartphones, this enables the users to easily understand the geographic data of the TV programs. When one scene first introduces New York, next, touring Manhattan; the system simultaneously presents a map of both, and moving on the route between them. For this, the system first extracts geographical geometry, i.e., locations, geographical relationships, and semantic structure, i.e., temporal meaning, intentions, by extracting place names and their appearance time from closed captions of video streams. Based on them, the system presents geographic contents (i.e., maps, Street View, etc.) with seamless effects.

"Leveraging Audio Fingerprinting for Audio Content Synchronization and Replacement"

by Paolo Casagranda, Maria Luisa Sapino, K. Selcuk Candan

In this paper we describe an innovative synchronization technique based on audio fingerprinting to help create a time base common to source content and receivers. We also introduce, as a relevant use case, the concept of hybrid content radio, a new radio service allowing the enrichment of broadcast radio with personalized audio content. Hybrid content radio is a novel radio service relying on audio replacement, and needing a precise synchronization between the main audio content and the enriching audio. We then discuss a variation of an existing audio fingerprinting algorithm, used to assess the proposed technique using real audio content

"A Test Bed for Hybrid Broadcast Broadband Services"

by Jean Le Feuvre, Viet-Than-Trung NGuyen, Wassim Hammidouche, Patrick Marchal, Raoul Monnier, Pascal Dupain

The users’ demand for more content with more features have made broadcaster worldwide investigate how to enhance their broadcast services while not overloading their bandwidth capacities. This paper presents various use cases and a technical framework for experimenting with hybrid broadcast broadband delivery. This test bed can be reproduced from scratch using open source tools.

"Synchronization and Customization of Subtitles in Web-based Main-Screen and Multi-Screen Scenarios"

by Zuzanna Parcheta, Mario Montagud, Jordi Belda, Fernando Boronat

This paper presents a web-based platform that enables the customization and synchronization of subtitles when using both a shared main screen and multiple secondary screens. This platform enables the dynamic adaptation of subtitles’ format (font family, size, color...) and position according to the users’ preferences and/or needs. Likewise, the users can dynamically select the subtitles’ language and adjust the number of lines to be displayed. Moreover, a delay offset to the subtitles can be applied. This platform can also be used in multi-screen scenarios to enable personalized media experiences. By using a secondary device, each user can enable the consumption of subtitles in two different languages and navigate between them, being able to set the position of the video by clicking on a specific subtitle line. Synchronization between subtitles and other media is achieved thanks to standardized HTML5 elements and Javascript. Synchronization across devices is achieved by using Javascript components (Node.js and Socket.IO) and a designed virtual clock synchronization mechanism.

"10 things that ruin video delays, and how to figure them out"

by Jack Jansen

MediaSync 2015

The third edition of the Media Synchronization Workshop will happen in conjunction with ACM TVX 2015. This workshop series brings together an active community around the topic of media synchronization, attracting many relevant researchers in this area. The objective of this workshop is to further built this community and set the research agenda on the topic of media synchronization. We will do this by sharing our current research in short presentations, and by having an active afternoon. We will be working in subgroups on key problem areas, present our work to the whole group as a starting point for an active discussion on the most relevant research to be carried out in the coming years.

Theme

Media synchronization has been a challenge in A/V transmission for quite some time. Over the years, many techniques to achieve intra- and inter-media synchronization in various network conditions have been developed. In many cases, synchronization between different media is crucial for a satisfying perceived Quality of Experience (QoE). Currently, many new media synchronization issues arise. On the one hand, novel media technologies such as HTTP streaming protocols, media encoders and HDTV often require new synchronization techniques. On the other hand, new patterns in media consumption often introduce specific synchronization issues. For example, internet applications evolving around broadcast TV content may need synchronization between the application and the broadband stream. Synchronization between different TV receivers may be needed in Social TV (inter-destination media synchronization). In some interactive TV cases synchronization between handheld devices and the TV screen may be needed. Moreover, novel 3D technologies for TV broadcasting and tele-presence (e.g., 3D tele-immersion) may require the adoption of several of these synchronization techniques to achieve a satisfying quality of user experience (QoE). Additionally, the deployment of applications that integrate, and synchronize inputs across all senses including the tactile, olfaction, and gustatory senses, apart from inputs for audio and visual senses, is currently a reality. The integration of such multiple sensorial effects into multimedia applications, or mulsemedia (multiple sensorial media) applications, also requires research on new media synchronization techniques, aligning rich data from multiple human senses, to provide a truly enriched and immersive experience. This workshop proposal addresses all the above synchronization issues and requests contributions from different perspectives.

Scope and Goal

The scope of the workshop is media synchronization in all its facets: architecture, protocols, algorithms, simulations, implementations, standardization, user experience studies, business models, etc. Depending on the submissions we receive, certain topics may receive more attention at the workshop than others.

The goal of the workshop is to determine the most relevant research aspects of media synchronization in the coming years. Much research has been done in this area, as said, but many researchers involved in media synchronization have the feeling that they have not finished their job just yet. In this workshop, we want to concretely identify the most relevant research topics in the area of media synchronization that will require our effort in the coming years.

Event Format

The workshop will be highly interactive, as we will use a subgroup working structure. From the submissions, we will determine the 3 most relevant topics for future research. On these topics, we will define short cases. These cases may be use cases, they may be major technical problem areas, they may be user experience issues, etc. We will divide the participants into 3 subgroups, and assign each of the subgroups one of the cases. The subgroups will be asked to work out the case, and provide a solution direction, or a clear insight into the problem. We will ask the subgroups to make 1 poster each on their work, which they will present to the whole group afterwards. These posters will be the input for a group discussion on the future of media synchronization research.

The idea is to have short presentations on the submissions by the participants in the morning, followed by the presentation of the 3 topics we have chosen for the interactive afternoon session. We will also form the subgroups at this point, so they can have lunch together if they want. In the afternoon, we will reiterate the topics, and then the people will work in the 3 subgroups. After that, each subgroup will present their work to the whole group, followed by a final discussion.

We will also invite demonstrations. Depending on the number of demonstration submissions we receive, we will either have a demonstration session at the workshop, or we may request people to submit their demonstration proposal to the TVX demonstration session.

Submission Guidelines

Authors are requested to submit a short paper (max 4 pages A4) for work in progress or a full paper (max 8 pages A4) on their work, including a limited number of references. Note that there are no minimum pages restrictions, and also note that demonstrations are invited. Please indicate if you submit a paper if you can also show a demonstration.

Authors should use the SIGCHI Extended Abstract template in PDF format. Templates for paper submission are available here. Articles should be sent by e-mail to the organizing committee