Ok. This time we will talk about track events and cue events.
So first, let's start by a small demonstration.
If I play this video, the video is going on and I can listen to events like 'cuenter' and 'cueexit'.
Each time a new cue is entered, we will display it here, and each time it is exited, we will display it here.
We saw that we can display them in sync with the video now.
So each time a cue is reached, it means that the current time entered a new time segment defined by the starting and ending time of a cue.
Let's have a look again at one of the VTT files.
Each cue holds a start time and a end time so when the time enters the 15th second, we've got a cueenter event and we can get this content and show it on the HTML page.
When we go out of this time period, when we go further than 18 seconds, we exit this cue and we enter this cue.
How are these events handled in the JavaScript code? Everything is done in the readContent method that we saw earlier.
This time instead of iterating on different cues of the TextTrack, we will just, for each cue individualy, we will iterate on the cues and add a listener on that cue, an exist listener and an enter listener.
So what do we do? We iterate in the cues called addCueListenners for the current cue.
This method, addCueListeners, it will define two listeners: the cue enter listener and the cue exit event listener.
On the cue enter listener we just create a string "entered cue id=" and "text=" that will correspond to the text displayed when a cue is just reached.
We display the id and we display the text.
The same thing is done when we exit, when we exit we just display the id "exited cue id=".
This is how we can have individual enter and exit listeners for each cue.
It will enable us to highlight the current cue in a transcript while the video is playing.
The only problem is that as of December 2015,
FireFox still does not recognize this sort of listeners.
The implementation is not done yet, so you can use a fallback.
You can use a listener on the track that will listen to the 'cuechange' event.
So if I just comment the function addCueListeners and I uncomment this piece of code that has a cueChange listener on the track itself, then instead of knowing that we entered of exited and individual cue, we can get, for every new time segment, the list of the cues that are triggered, that should be activated and displayed for this time segment.
As the different cues can overlap, the time segments can overlap -it is not often the case but it may occur- what the callback function from this listenner gives is a list of active cues.
Most of the time you have got only one.
Anyway you can just work with the list of the cues.
Here we just take the first active cue because we are assuming that the cues are not overlapping.
I added a small test here because sometimes we've got some strange ghost cues that are active and not defined.
I do not exactly what was the problem when I test it but I added this test here to avoid some error messages...
We have got the first active cue and we just display it.
We get the id, we get the text and this is all.
In that case, if I run the application again, instead of having enter and exit, I will just have cue change events.
It starts at 15 seconds.
I had a small bug here, it is not this.id but cue.id.... we can start again.
This is a fallback for FireFox if you want to display the cues in sync with the video.
The next video will show how to display a transcript here with the current cue highlighted and you can click on them in order to jump to the right place in the video.
After that I think we will have seen the most useful properties, methods, events you can use with tracks and cues...
Instead of reading the whole content of a track at once, like in the previous example, it might be interesting to process the track content cue by cue, while the video is being played. For example, you choose which track you want - say, German subtitles - and you want to display the subtitles in sync with the video, below the video, with your own style and animations... Or you display the entire set of subtitles to the side of the video and you want to highlight the current one... For this, you can listen for different sorts of events.
The two types of cue event are:
enter and exit events fired for cues.
cuechange events fired for TextTrack objects (good support).
// track is a loaded TextTrack
track.addEventListener("cuechange", function(e) {
var cue = this.activeCues[0];
console.log("cue change");
// do something with the current cue
});
In the above example, let's assume that we have no overlapping cues for the current time segment. The above code listens for cue change events: when the video is being played, the time counter increases. And when this time counter value reaches time segments defined by one or more cues, the callback is called. The list of cues that are in the current time segments are in this.activeCues; where this represents the track that fired the event.
In the following lessons, we show how to deal with overlapping cues (cases where we have more than one active cue).
// iterate on all cues of the current track
var cues = track.cues;
for(var i=0, len = cues.length; i < len; i++) {
// current cue, also add enter and exit listeners to it
var cue = cues[i];
addCueListeners(cue);
...
}
function addCueListeners(cue) {
cue.onenter = function(){
console.log('enter cue id=' + this.id);
// do something
};
cue.onexit = function(){
console.log('exit cue id=' + cue.id);
// do something else
};
} // end of addCueListeners...
HTML code:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Using HTML views of tracks</title>
</head>
<body>
<video id="myVideo" preload="metadata" controls crossOrigin="anonymous">
<source src="https://mainline.i3s.unice.fr/mooc/elephants-dream-medium.mp4" type="video/mp4">
<source src="https://mainline.i3s.unice.fr/mooc/elephants-dream-medium.webm" type="video/webm">
<track label="English subtitles" kind="subtitles" srclang="en"
src="https://mainline.i3s.unice.fr/mooc/elephants-dream-subtitles-en.vtt" >
<track label="Deutsch subtitles" kind="subtitles" srclang="de"
src="https://mainline.i3s.unice.fr/mooc/elephants-dream-subtitles-de.vtt" default>
<track label="English chapters" kind="chapters" srclang="en"
src="https://mainline.i3s.unice.fr/mooc/elephants-dream-chapters-en.vtt">
</video>
<h3>HTML track descriptions</h3>
<button id="buttonLoadFirstTrack" onclick="forceLoadTrack(0);" disabled>Start listening to cuechange events for track 0 (English subs)</button> <p>
<button id="buttonLoadThirdTrack" onclick="forceLoadTrack(1);" disabled>Start listening to cuechange events for track 1 (German subs)</button><p>
<div id="trackStatusesDiv">
</div>
</body>
</html>
CSS code:
#trackStatusesDiv {
border:1px solid;
height:auto;
padding: 20px;
}
JS code:
let video, htmlTracks;
let trackStatusesDiv;
let buttonLoadFirstTrack, buttonLoadThirdTrack;
window.onload = () => {
// called when the page has been loaded
video = document.querySelector("#myVideo");
trackStatusesDiv = document.querySelector("#trackStatusesDiv");
buttonLoadFirstTrack = document.querySelector("#buttonLoadFirstTrack");
buttonLoadFirstTrack.disabled=false;
buttonLoadThirdTrack = document.querySelector("#buttonLoadThirdTrack");
buttonLoadThirdTrack.disabled=false;
// Get the tracks as HTML elements
htmlTracks = document.querySelectorAll("track");
// display their status in a div under the video
displayTrackStatuses(htmlTracks);
};
function displayTrackStatuses(htmlTracks) {
trackStatusesDiv.innerHTML = "";
// display track info
for(let i = 0; i < htmlTracks.length; i++) {
let currentHtmlTrack = htmlTracks[i];
let currentTextTrack = currentHtmlTrack.track;
let label = "<li>label = " + currentHtmlTrack.label + "</li>";
let kind = "<li>kind = " + currentHtmlTrack.kind + "</li>";
let lang = "<li>lang = " + currentHtmlTrack.srclang + "</li>";
let readyState = "<li>readyState = " + currentHtmlTrack.readyState + "</li>";
let mode = "<li>mode = " + currentTextTrack.mode + "</li>";
trackStatusesDiv.innerHTML += "<li><b>Track:" + i + ":</b></li>" + "<ul>" + label + kind + lang + readyState + mode + "</ul>";
}
}
function readContent(track) {
console.log("adding cue change listener to loaded track...");
trackStatusesDiv.innerHTML = "";
track.addEventListener("cuechange", (e) => {
let cue = e.target.activeCues[0];
if(cue !== undefined)
trackStatusesDiv.innerHTML += "cue change: text = " + cue.text + "<br>";
});
video.play();
}
function getTrack(htmlTrack, callback) {
let textTrack = htmlTrack.track;
if(htmlTrack.readyState === 2) {
console.log("text track already loaded");
callback(textTrack);
} else {
// will force the track to be loaded
console.log("Forcing the text track to be loaded");
textTrack.mode = "hidden";
htmlTrack.addEventListener('load', function(e) {
callback(textTrack);
});
}
}
function forceLoadTrack(n) {
getTrack(htmlTracks[n], readContent);
}
Source code extract:
function readContent(track) {
console.log("adding cue change listener to loaded track...");
trackStatusesDiv.innerHTML = "";
// add a cue change listener to the TextTrack
track.addEventListener("cuechange", function(e) {
var cue = this.activeCues[0];
if(cue !== undefined)
trackStatusesDiv.innerHTML += "cue change: text = " + cue.text + "<br>";
});
video.play();
}
Source: https://codepen.io/w3devcampus/pen/PobgdgL
Source code extract:
function readContent(track) {
console.log("adding enter and exit listeners to all cues of this track");
trackStatusesDiv.innerHTML = "";
// get the list of cues for that track
var cues = track.cues;
// iterate on them
for(var i=0; i < cues.length; i++) {
// current cue
var cue = cues[i];
addCueListeners(cue);
}
video.play();
}
function addCueListeners(cue) {
cue.onenter = function(){
trackStatusesDiv.innerHTML += 'entered cue id=' + this.id + " "
+ this.text + "<br>";
};
cue.onexit = function(){
trackStatusesDiv.innerHTML += 'exited cue id=' + this.id + "<br>";
};
} // end of addCueListeners...