kartikeyasharma

Live vs. Recorded Music

(Kartikeya Sharma)

Listeners of music enjoy a preference for either live or recorded music. Each type of music has its own unique properties that make one better suited than the other, for entertainment as well as therapeutic purposes. This study presents an analysis of the spectral and temporal variations of vocal features using recordings of live as well as studio-recorded music, in an attempt to identify the components of a song’s vocals that possibly affect the preference for one type of music over the other, and how they influence a listener's perception.

Literature Review

There is a unique interaction between the human auditory system and music, and it is an essential element in decoding the underlying processes of how the human cognition interprets sound stimuli. This is not limited only to music cognition, but it also allows us to understand the way in which the neural, psychological, psychoanalytical and cognitive systems interact with each other, and what role each system plays in auditory perception. Recent research has established music to have therapeutic properties which make it pleasurable to listen to. These therapeutic properties are useful in providing healthcare to patients struggling with medical problems. The kind of music to be administered to a patient depends on a patient's type of ailment or discomfort. For example, classical music did not prove to be as effective as popular music for patients exhibiting symptoms of psychosis (Silverman, 2003). Numerous studies have also been performed to evaluate whether live music or recorded music is generally better suited to treat a patient, and both have their respective beneficial properties. Live music tends to be more expressive due to its human element (Bailey, 1983), whereas recorded music might often be preferable due to its familiarity and refined audio quality. 

According to Bailey (1983), music is known to provide both physical and emotional comfort to patients with a troubled mental state. Music also acts as a reliever of anxiety, a property which makes it suitable for anxious patients undergoing MRI scans (Walworth, 2010). Similarly, music is also helpful for post-traumatic amnesia patients, because it enhances orientation and reduces agitation (Baker, 2001) in patients. However, the use of live music is sometimes encouraged over recorded music (Groene, 2001), because of a significant improvement in the patients’ health. For non-responsive patients, live music is more effective in eliciting an alert behavior state than recorded music (Segall, 2007). Same is the case for hospitalised cancer patients, where live music results in significantly less tension-anxiety, more vigor, better mood and less physical discomfort (Bailey, 1983). However, in certain cases there is less or no difference between the results of using live and recorded music. Similar results for live and recorded music can be seen in patients suffering from psychosis (Baker, 2001) and post-traumatic amnesia (Silverman, 2003), with live music yielding better results only marginally but not significantly.

The perception of sound by the human auditory system involves complex interactions and communication between multiple cognitive processes of the brain. It is simply not possible to assume that a particular component of a piece, be it the temporal properties (tempo, beat, pitch, timbre, etc.), the choice of instrument, expressivity of the vocals or its familiarity, can be identified as being responsible for the desirable (or undesirable) effect. For instance, Garunkstiene et al. (2014) found that singing live lullabies to preterm infants resulted in a deeper sleep state than the recorded lullabies. Therefore, this study aims to perform a comparison between the vocal components of live as well as recorded music samples, and if present, the amount of influence of vocals on the overall therapeutic effect of popular music. 

Dataset

The dataset used in this study is a corpus of sixteen 20-second excerpts taken from tracks chosen out of popular music genres, and the vocals of each excerpt are then extracted. The tracks are chosen such that for each studio-recorded track, there exists a corresponding recording of its live performance. The 20-second clip sections are chosen in a way that contain minimum bleed from the background elements of the track. Out of the 16 tracks, 8 are the vocals of the live versions of the tracks, and 8 are the corresponding vocals of the recorded versions of the same tracks. The tracks are chosen from diverse popular music genres such as rock, hip-hop, alternative, jazz, etc. The test data does not contain tracks from the classical genre, as the study focuses on music with a certain metrical structure and repeating patterns. This is because therapeutic applications of music are seen to yield better results by making use of popular music, perhaps due to the ease of interpretation and higher expectation-reward for the patients. The excerpts are chosen to be of 20-second length so that the data obtained from the tracks is comparable amongst each other. 

The tracks used in the dataset (one live and one recorded version for each) are as follows:

• • Sultans of Swing (Mark Knopfler).

  • Heartbreak Hotel (Elvis Presley).

  • Rolling in the Deep (Adele).

  • Fortunate Son (John Fogerty).

  • DNA. (Kendrick Lamar).

  • How Blue Can You Get? (B.B. King).

  • Hallelujah (Jeff Buckley).

  • Fly Me to the Moon (Frank Sinatra).

Procedure

• Retrieval of the Dataset: 

To be able to analyse the influence of vocals on perception, the vocals of artists in the excerpts are isolated and extracted for all the tracks. This is done using centre channel extraction, which allows you to isolate the centre channel in a stereo recording, and attenuate the left and right channel signals. It allows the user to specify a range of frequencies that they wish to preserve, thus allowing the user to tune out very high or very low frequencies. This is done because most of the vocals are recorded in mono, and are centered in a stereo track. Since the centre channel also might contain some instruments that are centered, the resulting audio still contained some bleed from other sound sources. To further attenuate particular unwanted frequencies, the audio clips were further de-noised using EQ in Logic Pro X, and noise reduction in Audacity.

• Analysis Methods:

Once the vocals from every track have been retrieved, the dataset is ready to be tested upon. To be able to find notable differences between the live excerpts and the recorded ones, an analysis of spectral and temporal variations between the two needs to be carried out. For the purpose of this study, certain features of audio that possibly affect how it is perceived by a listener are numerically extracted, such as the loudness, the tempo, and the pitch & duration of each word spoken by the performer.

1. Loudness: Loudness is a descriptor of the perceptual level of sound pressure that a listener can hear. The values of loudness will vary for the live and the recorded version, which can help to determine which clip has a higher average loudness. If one of the two were to be analysed for preference based on to their energising factor, often a live performer will exert more energy than that in a recorded version. Thus, an analysis of the loudness of vocals can help understand the difference between how louder and softer sounds are perceived. The loudness values for the tracks are extracted in Sonic Visualiser using a loudness extraction plugin.

Fig. 1: Extracted loudness values in Sonic Visualiser

1. Pitch: Pitch is used to define the frequency of the sound as perceived by a listener. Thus it is important to understand how variations in pitch can affect the way each performers' vocals are perceived by listeners. The pitch values are extracted for each spoken word by the artist, and a comparison is drawn for the word-by-word pitch variations between the two forms of presentation of music. This is done by annotating each word in every sound excerpt, and extracting the values of their fundamental frequencies. The F0 values are taken from the middle point in time for every spoken word. The manual process of word annotations and the extraction of fundamental frequency values is carried out using Praat. The pitch values for sections of silences in the track are omitted from the final data to obtain definite and smooth pitch contours.

2. Tempo: The tempo of a track describes the speed and timing at which it progresses in time. Every performance progresses at a certain tempo, which is responsible for the induction of a beat in the listener. It is possible that the tempo for a live performance may deviate from a recorded version's tempo. By analysing how the tempo of a live performance differs from the recorded's, it is possible to analyse subtle variations in the metrical structure that can affect the way a listener interprets it. The tempo is calculated for each excerpt by annotating their beats using Sonic Visualiser, and the beat data is then used to calculate the tempo of the sound clip.

3. Word Duration: It is the duration of a word as uttered by a speaker. Each word spoken will have a different duration during a performance, and this variation in word lengths introduced by a speaker may also contribute to how a collection of words is heard and perceived. Thus, the lengths of spoken words by performers can be interpreted as being analogous to expressive timing in instruments. The word lengths are determined using the word annotations from Praat, and these annotations save the timestamp values of the onsets and offsets of each spoken word. All other data except the start, mid and end points (in milliseconds) and the duration of the words are removed from the final data. The durations of silences are ignored and removed, to be able to visualise only the word durations in a better way. The difference in the number of spoken words is not adjusted for in the final results, to get a true shape of the final word length plot for the whole audio excerpt.

Fig. 2: Word annotations in Praat

• Refining the Extracted Data: 

The data that defines the values of the stated audio features, is then retrieved by using the above methods of data extraction. The analysis data is exported in the form of .csv files, so that it is easily compatible for access via MATLAB. However, some of the .csv files were not in a MATLAB-readable format. Numbers (Apple Inc.) was used to manipulate the incorrectly formatted data in the files to make them compatible. The loudness values and the beat annotation data are exported directly into .csv files through Sonic Visualiser. However, the word annotations that are done using Praat result in a .TextGrid file containing the onset-offset times and durations for each word, and the .TextGrid file is then converted into a .csv file using an online TextGrid to CSV conversion tool by Daffyd Gibbon. To retrieve the values of fundamental frequencies for each word, a Praat script was written that extracts the F0 data at the midpoint of every word's duration using the audio and its corresponding textgrid file, and exports the pitch data directly into a .csv file. The pitch values and word length data for silences (pauses) was removed from the .csv files so as to obtain smoother contours for both the audio descriptors, which are more informative for understanding their continuous influence.

Results

To draw a comparison between the live and recorded versions, the results for this study are presented for two kinds of variations that can occur: intra-track and inter-track variations. The intra-track variations evaluate the within-track effects for both versions for each single track, which tells us what are the subtle variations that make one version so different from the other in terms of their perception. The inter-track variations between the two versions helps us understand the global differences between the values of each audio feature, for both the live and recorded form of presentation of music.

• Loudness:  

The loudness values of the live version of a track are seen to be different than the loudness values for the recorded version, with the loudness of live performances being either equal or higher than the recorded performance loudness for most of the recordings. There is no fixed pattern in the variation of loudness values within the tracks, but the loudness values maintain a specific range of values depending on the amplitudes of the vocals in the performances.

Fig. 3: Loudness value plots for each track.

The mean loudness values for live performances of the tracks are seen to be higher than the recorded mean loudness values, for 62.5% of the live clips. The increase in loudness for live performances, is much higher than the increase in loudness in case of recorded music performances.

Fig. 4: Average loudness values over the entire excerpts.

• Pitch:

The effect of pitch within each track can be noticed in the plots from Fig. 5, and it is evident that the pitch contours for the live and recorded versions of the same track are different from each other. The values of pitch have been extracted per word, and it is seen that the F0 values for each word also vary for both the versions of a single track, and this is true for each track. This is possibly due to an improvised use of words in the live performances which are not there in the recorded versions, which results in a difference in the number of words and in the words themselves, for each track.

Fig. 5: Fundamental frequency contours for each audio excerpt.

The mean F0 values for live performances of the tracks are seen to be higher than the recorded mean F0 values, for 62.5% of the live clips. The increase in F0 for live performances, is not much higher than the increase in F0 in case of recorded music performances. The mean values for each track, for each version, are seen to be different from each other.

Fig. 6: Average fundamental frequency values for the entire sound clips.

• Tempo:

The tempo varies within each track, with both the versions of the track having a slight deviation from each other. The tempo fluctuates for each beat within the track, but for certain tracks the tempo is completely different for the corresponding live and recorded versions.

Fig. 7: Tempo plots for each audio excerpt.

The median values of the tempo are calculated for all the tracks. The median is chosen as a measure since the tempo does not drastically vary within a piece, and the median is immune to any skewing of data due to outliers, as at the start and end of the audio excerpts. The median tempo values vary among the tracks: for 37.5% of the tracks, the live performances have a higher tempo. For the other tracks, the values of tempo for recorded performance tracks are either almost equal (25%) to or greater than (37.5%) the live performance tempo values.

Fig. 8: Median tempo values of the tracks.

• Word Duration:

There is a visible variation in the word durations for each and every word within the live and recorded versions of a single track. Each word is pronounced uniquely, and there are differences in the lengths of the same word being spoken in both versions of a track. Also for some cases, the number of words for the live and recorded performances are different, due to the unique addition of certain words by the artists in the live versions of the tracks. This may seem ambiguous in mathematical terms, but practically it is an essential descriptor of the level of expressivity introduced into a performance by the performer. 

Fig. 9: Word durations for each vocal track.

A mean of the word lengths for all the sound clips are calculated. The mean values of word lengths across all the excerpts are seen to be varying non-deterministically for the live and recorded clips, with 50% of the live tracks having a greater mean word length and 50% of the recorded tracks with a greater mean word length.

Fig. 10: Average word durations in every excerpt's vocals.

Discussion

As per the results of the study, an analysis of the variations in loudness, tempo, pitch and word duration indicate that these values differ for a live performance and a recorded one. This suggests that the subtleties introduced by the vocals in a fully-mixed track are capable of influencing how it is processed by the auditory system, along with other influential elements of a track such as its metrical structure, degree of syncopation, rhythm, the expressive timing with instruments, etc. However, the interpretation by a listener involves an interaction of all these elements of music for it to be perceived by a listener. The variations in voice introduced by a performer for the audio features in question, are consequential and they certainly affect the way live and recorded music are perceived. Due to such variations, listeners often prefer one form of presentation over the other, with most people finding live music to elicit more positive behaviour than recorded music.

The loudness values for within each track suggest that most of the live music recordings have a higher amplitude and thus greater perceived loudness. This may be due to the fact that a performer exerts more energy during a live performance to reach the crowd, as compared to while recording it. Additionally, the average loudness values for the bulk of live recordings are greater than those of the studio-recorded music excerpts. The preference of live music can be explained by their higher loudness values, which may be perpetrated by the gradual increase in music loudness over the last few decades.

The values of fundamental frequency of each spoken word are used to compare pitch variations between live and recorded music. As evident from the pitch contour results in Fig. 5, the word-by-word F0 values are different for each corresponding word in both the live and the recorded performances, within each single track. The pitch contour however, follows a somewhat similar shape for both the versions of a track. This suggests that the pitch contour is what affects if pitch is perceived as pleasant, and not just the absolute value of pitch. The mean of fundamental frequency values of a track are seen to be higher for the live track recordings. However, one frequency of pitch cannot be categorised as better than the other. Thus, there is no effect of the absolute value of pitch in determining which method of presentation of music is "better". 

The tempo of a track influences the process of beat induction in a listener. The variations in tempo within both the versions of each track indicate that each version of a track introduces its own unique metrical variation in tempo during that performance. The median values of tempi for the tracks show no effect of the type of presentation, with a balanced number of live and recorded performances being either higher, lower or equal in tempo. The variation in tempo between the two versions, and the greater appeal of live music, is interpreted as the live performance subtly violating the metrical expectations that are formed by the recorded tracks, which leads to a higher degree of satisfaction in listening for live music.

The duration of every spoken word can be interpreted as being analogous to a performer's expressive timing while playing an instrument. There is a lot of variation in the number and length of words between both the versions of every track, suggesting that each word is expressed in a unique way by the artist for each performance which directly affects its perception. The mean value of word length over all tracks is not descriptive of any perceptual aspect of the sound, and only serves to tell us that there exists a variation in the lengths of spoken words between live and recorded clips. Variations in words and their lengths yields a completely new rendition of a performance and results in a unique expression of each word by the performer.

The perception of a fully-mixed track involves a complex interaction of the neural, psychological, psychoanalytical and cognitive systems. The constituent elements of the mix such as vocals, the backing instruments, their timbre, the meter, etc., always undergo the hearing process as a single collective of all the elements via complex pathways. There is no single element or feature of music that can be identified that fully quantises the factors that determine whether one form of presentation is better than the other.

Future Work

The study opens up avenues to understand the relationship that exists between the vocal components and the backing instruments in a mix, and how they are collectively perceived as a single entity by the human auditory system. The interpretation of music involves the interaction of multiple cognitive processes carried out by not one, but many systems of the body.

This study can be extended by evaluating the results on a larger dataset, which will help generalise the results better and more conclusively. A large enough dataset can also be helpful for training a machine learning based model that can identify the key differences between tracks of the two forms of music presentation being studied. Furthermore, an extensive analysis of other significant spectral features of track vocals and the role they play, may help to better identify the role of vocals in creating a preference between live and recorded music within a listener.

References

1. Aid, F. (2013). B.B. King - How Blue Can You Get (Live at Farm Aid 1985). YouTube. Retrieved March 21, 2019, from https://www.youtube.com/watch?v=P27tTNP7SFc

2. Bailey, L.M. (1983). The Effects of Live Music versus Tape-Recorded Music on Hospitalized Cancer Patients. Music Therapy, 3(1), 17-28. doi: 10.1093/mt/3.1.17

3. Baker, F. (2001). The Effects of Live, Taped, and No Music on People Experiencing Posttraumatic Amnesia. Journal of Music Therapy, 38(3), 170–192. doi: 10.1093/jmt/38.3.170

4. Baron, R. (2018). Elvis Presley Heartbreak Hotel 1968 Remaster (Hd). YouTube. Retrieved February 23, 2019, from https://www.youtube.com/watch?v=9E6aRc_QGLQ

5. Garunkstiene, R., Buinauskiene, J., Uloziene, I. & Markuniene, E. (2014). Controlled trial of live versus recorded lullabies in preterm infants. Nordic Journal of Music Therapy, 23:1, 71-88. doi: 10.1080/08098131.2013.809783

6. Groene, R. (2001). The Effect of Presentation and Accompaniment Styles on Attentional and Responsive Behaviors of Participants with Dementia Diagnoses. Journal of Music Therapy, 38(1), 36-50. doi: 10.1093/jmt/38.1.36

7. KendrickLamarVEVO (2017). Kendrick Lamar - DNA./HUMBLE.. YouTube. Retrieved February 23, 2019, from https://www.youtube.com/watch?v=bhCfQ8BGow8

8. Music, J.B. (2014). Jeff Buckley - Hallelujah (Live). YouTube. Retrieved March 22, 2019, from https://www.youtube.com/watch?v=2YjbJTS5C_I

9. Segall, L.E. (2007). The Effect of Patient Preferred Live versus Recorded Music on Non-Responsive Patients in the Hospice Setting as Evidenced by Physiological and Behavioral States (Unpublished master’s thesis). Florida State University College of Music, Florida, USA.

10. Silverman, M.J. (2003). The Influence of Music on the Symptoms of Psychosis: A Meta-Analysis. Journal of Music Therapy, 40(1), 27–40. doi: 10.1093/jmt/40.1.27

11. Sinatra, F. (2018). Frank Sinatra - Fly Me To The Moon (Live At The Kiel Opera House, St. Louis, MO/1965). YouTube. Retrieved March 18, 2019, from https://www.youtube.com/watch?v=Y2rDb4Ur2dw

12. vancel2005 (2012). Adele - Rolling in the deep (Live Royal Albert Hall). YouTube. Retrieved March 23, 2019, from https://www.youtube.com/watch?v=PaEhQ51BvSw

13. Walworth, D.D. (2010). Effect of Live Music Therapy for Patients Undergoing Magnetic Resonance Imaging. Journal of Music Therapy, 47(4), 335–350. doi: 10.1093/jmt/47.4.335

14. Zarastro1040 (2009). Bruce Springsteen w. John Fogerty - Fortunate Son - Madison Square Garden, NYC - 2009/10/29&30. YouTube. Retrieved March 20, 2019, from https://www.youtube.com/watch?v=iQqHzPzQsMg