Online Amnestic Dynamic Time Warping to Allow Real-Time Golden Batch Monitoring
Chin-Chia Michael Yeh, Yan Zhu, Hoang Anh Dau, Amirali Darvishzadeh, Mikhail Noskov, Eamonn Keogh
Embedded Files
Introduction
Introduction
This is the supporting page for paper Online Amnestic Dynamic Time Warping to Allow Real-Time Golden Batch Monitoring. This page contains materials like video, source code, and datasets.
Expanded version of the paper: link
Normalization
Normalization
Normalization is crucial in the case of multidimensional time series when each dimension is measured by different types of instruments (e.g., thermometer, pressure gauge, or flow-rates gauge) [1][2]. Because the measurements from different instruments carry different units (e.g., kelvin, pascal, or cubic meters per second), numbers from different dimensions are not commeasure. If we sum the error from different dimensions without normalization, the result localized error profile would be dominated by the dimension with largest range. For example, in the Wafer database [3], the range of the six dimensions are {9, 234, 10, 4, 1, 61}; the first, third, fourth, and fifth dimension have little to no effect to the localized error profile compared to the second dimension. Such problem is usually addressed by applying z-normalization [4] or 0-1 normalization to each dimension of the multidimensional time series.
[1] C.-C. M. Yeh et al. “Time series joins, motifs, discords and shapelets: a unifying view that exploits the matrix profile.” DMKD 32, no. 1 (2018): 83-123. [2] Y. Zhu et al.. “Exploiting a novel algorithm and GPUs to break the ten quadrillion pairwise comparisons barrier for time series motifs and joins.” KIS (2018): 1-34. [3] R. T. Olszewski. Generalized feature extraction for structural pattern recognition in time-series data. No. CMU-CS-01-108. Carnegie Mellon University, Pittsburgh, PA. 2001. [4] F. Petitjean et al. “Dynamic time warping averaging of time series allows faster and more accurate classification.” In ICDM on, pp. 470-479. IEEE, 2014.
Human Performance Coaching: Music
Human Performance Coaching: Music
Real-time performance feedback is important for learning musical instruments [5]. For example, if a musician plays the wrong note in the first chorus during practice, it is very likely that she will make the same mistake in the second chorus of the same playthrough because she may misread the music script in the same fashion. However, if the golden batch system notifies the musician in the first chorus about the misplayed note, it is possible the same mistake can be avoided in the second chorus.
We use the piano arrangement of Stairway to Heaven by Led Zeppelin. The musical script was downloaded in MIDI format from [6][7]. We use the downloaded MIDI as the golden batch, then generate the batch being monitored with the following errors, which were suggested by a music teacher as being common mistakes by amateurs:
We vary the start time and end time of each piano keystroke with a lead/lag of seconds.
Each piano keystroke is given a 0.5% chance of being wrong.
A wrong key is generated by randomly ascending/descending the pitch. Figure 1 shows a snippet of the modified midi visualized as piano roll. The wrongly played key (circled in red) can be higher or lower in pitch comparing to the golden batch.
Figure 1: A ten second snippet piano roll of the modified MIDI starting from the 5th seconds of the song. The musician made two mistakes, which are circled in red. The correct key is shown in blue and the incorrect key in orange.
The outputted DCM of our golden batch algorithm of the same ten seconds snippet is shown in Figure 2.bottom. The DCM raised prominently when the wrong note is played. Although our method only shows the temporal location of the wrongly played note, it is trivial to identify the specific key which the musician wrongly played using an XOR operation given the temporal alignment (which is part of the gold batch’s output). A demo video, which shows the DCM being computed just-in-time as the musician plays the instrument, can be found in [8].
Figure 2: top) The golden batch for the same section of the song. middle) The same snippet shown in Figure 1. bottom) The DCM computed by our method. The red lines indicate the alignment between the current batch and the golden batch.
[5] S. Giraldo et al. “Automatic assessment of violin performance using dynamic time warping classification.” In 2018 26th Signal Processing and Communications Applications Conference (SIU). IEEE, 2018.. [6] https://musescore.com/user/7639766/scores/2847181. [7] https://www.dropbox.com/s/lgte3oq9967fcue/music_data.zip?dl=1[8] https://vimeo.com/295305382
Source Code
Source Code
MATLAB: link
Google Sites
Report abuse