Psychoacoustics

• Explore the anatomy of the human ear.

• Learn how our ears’ biological functions translate sound waves into what we hear. 

• Learn about pitch and volume as perceptual equivalents to frequency and amplitude.

• Understand how we use Interaural Time Difference (ITD) and Interaural Level Difference (ILD) for sound localisation.

• Discover different psychoacoustic phenomena: the missing fundamental, Shepard tones, the tritone paradox, and auditory masking.

It might surprise you to learn that all mammals have this same auditory system, consisting of an eardrum, a three-bone middle ear, and a fluid-filled spiral cochlea. This auditory system is actually one of  the defining characteristics of mammals, despite there being a noticeable variation of the structures and effects between different species. For example, as mentioned humans can hear from 20Hz to 20,000Hz, whereas dogs can hear sounds between 40Hz and 60,000Hz and a bat might be able to hear from 700Hz up to 200,000Hz depending on the species.

We’ve covered how sound travels to our brains, but what about how we process that sound and understand it? You might have noticed that so far we have talked about frequency (how high or low sounds are) but we have been very careful not to use the word pitch. This is because in the world of audio and acoustics, frequency and pitch are different. 

Frequency vs. Pitch

Frequency is an intrinsic property of a sound wave, which describes the number of cycles per second (Hertz, or Hz) of the wave. A 5Hz sound goes through 5 cycles per second. You can see a representation of this in the graph below.

Pitch is the perceptual equivalent of frequency, but what does that mean? 

In general, a higher frequency sound will sound like a higher pitch sound. This is certainly true for a pure tone- a sound that consists of only one frequency. The demonstration of the range of human hearing above was made using a pure tone, known as a sine wave. However, most sounds that we hear consist of many frequencies at once, and these frequencies are referred to as harmonics. 

Whilst most sound is made up of multiple harmonics, often we perceive one pitch at the most dominant frequency, known as the fundamental frequency. The frequencies above (and sometimes below) the fundamental frequency provide a lot of what is referred to as the timbre of a sound. Think of timbre as the quality which distinguishes different sounds, it is the reason we can tell the difference between a guitar and a piano playing the same note, or why you can tell the voice of a friend apart from the voice of a stranger. 

Another thing to consider when thinking about how we hear frequencies is octaves. If you are a musician, you will be familiar with the term octave. In western music, our note system is based on octaves. 

Scientifically, an octave refers to a doubling of Hz. For example, the jump from 100Hz to 200Hz is one octave, just as the jump from 800Hz to 1600Hz is one octave. What is fascinating is how we perceive the interval between these notes. Even though the second example covers a much larger physical range of frequencies (800Hz vs. only 100Hz), our brains hear the interval between pitches as identical. This proves that human hearing is logarithmic: we hear in relative ratios rather than a fixed number of Hertz. Because of this, our ears are far more sensitive to fine details at the bottom of the frequency spectrum. In the low end, a tiny shift of just 2 or 3Hz can make a bass note sound painfully out of tune. However, at the high end, we require a massive change in Hertz to even notice a difference.

In summary: pitch is a subjective sensation. It is how your brain 'labels a sound so you can understand it.

Amplitude vs. Loudness

The amplitude of a sound wave is the maximum displacement or distance moved by a point on a wave measured from its undisturbed (rest) position. Just as we learnt that pitch is the perceptual equivalent of frequency, loudness is the perceptual equivalent of amplitude.

Loudness is frequency-dependent. We don’t hear all frequencies equally and are more sensitive to some frequencies than others. Whilst 20 to 20,000Hz forms the extremes of our hearing range, we are most sensitive to frequencies in the 2000-5000 Hz range. The figure below shows an equal loudness contour. The curve represents the level at which most people hear an equal volume across frequencies. 

Try out an interactive version of this graph which lets you create your own equal loudness contour. When you click on the graph, it will place a sine tone at the frequency (horizontal axis) and volume (vertical axis) that you have selected. Notice that if you place sine tones along the equal loudness contour, the tones will sound as if they are the same volume, despite being at different positions on the vertical axis.

Audio Warning: Clicking near the top of the graph plays very loud sound. Please be very careful.  Click here to go to the interactive graph.

Earlier, we learnt about the structure of the ear. The pinna (outer ear) has a shape which helps us to figure out where sound is coming from. However, this is not the only technique humans have developed to identify the locations of sounds. 

The Interaural Time Difference (ITD) is the difference in time it takes for a sound to travel to each ear. For example, the sound from a radio which is sitting to your right, will reach your right ear before your left ear. The additional time it takes for the sound to reach your left ear is used by the brain to determine where the sound source is horizontally.

Another similar technique we make use of is the Interaural Level Difference (ILD). This works in a similar way to the ITD except instead of looking at the difference in time between sound sources reaching each ear, we look at the amplitude. Considering the same example again of a radio placed to your right, the sound that reaches the right ear will be louder than the sound reaching the left ear as the sound waves will lose energy as they cover the extra distance to get to your left ear.

Through a combination of Interaural Time Difference, and Interaural Level Difference, the brain is capable of locating a sound across the horizontal (left to right) plane. In order to identify if a sound is in front of you or behind you, the pinna is required.

The unique shape of the outer ear filters sound in a particular way depending on the direction it is coming from. Our brain learns these differences and combined with the ITD and ILD, allows us to localise sounds all around us in 3D space.

Sometimes, the sound that we perceive is different from the actual sound that is being played due to the way that our hearing system receives and analyses sound.

The Royal Institution via YouTube. Used via YouTube Standard Embed Terms.

The missing fundamental phenomenon can be utilised in audio mixing. Have you ever wondered how you can hear a deep bass line through tiny smartphone speakers or earbuds? The speakers are physically too small to produce those low frequencies, which is where the missing fundamental phenomenon comes in handy. As long as the harmonics are included in the mix, our brains can recognise the low notes without needing to actually hear the fundamental frequency.

This was also useful in the early days of telephone technology, where the hardware couldn’t transmit frequencies below 300Hz. Thanks to the trick of the missing fundamental, it was still possible to recognise the voices of family and friends with low voices.

Shepard Tones

Another auditory illusion that tampers with our perception of pitch is known as a Shepard-Risset-glissando. It is created using scales made up of Shepard tones. A Shepard tone is formed of several sine waves which are separated by octaves. You can listen to a single Shepard tone below.

The Shepard-Risset-glissando is a version of the Shepard scale where the tones glide upwards continuously, creating an effect where the pitch appears to rise continuously (when in reality, the same octave is being played over and over again). The video below shows the spectrogram of a Shepard-Risset-glissando.

Tritone Paradox

Individual Shepard tones can be used to create another illusion, known as the Tritone Paradox.

Not all humans perceive sounds in the same way. The Tritone Paradox illustrates how two different individuals can perceive something differently when listening to the same audio clip. It was discovered in 1986 by the psychologist Diana Deutsch. 

A tritone is formed of two notes that are separated by half an octave. For instance, you could play a C followed by an F#, or a G# followed by a D. 

Shepard tones contain one note with several octaves, so when a tritone sequence is created using Shepard tones, it is ambiguous whether the notes descend or ascend. Some people hear the first note higher than the second because they identify a higher octave in the first note and a lower octave in the second, and some people perceive it the other way around.

Try this for yourself! Ask your friends or family to listen and compare their judgments for each pattern. Some people hear the pattern ascending, and other people hear the descending pattern descending. Listen to the examples of Diana Deutsch's Musical Illusions - The Tritone Paradox.

Auditory Masking

Before we get into auditory masking, listen to the following audio examples of a pure-tone and noise. Learning what these sound like will help you understand the next section.

As we explored previously, most sounds we hear are more complex than a single, pure-tone sine wave. Normally, sounds contain multiple sine tones and noise. The presence of a tone with other tones or noise can lead to a difficulty perceiving the tone. This is what is meant by masking.

As an example, imagine you are having a conversation with a friend. As a car goes past, you may not be able to hear what your friend is saying and after the car passes, you can understand them again. In this Masking example, there is a maskee (the friend) and a masker (the car). 

In its simplest case, the maskee is a tone that is masked by a masker which can be a tone or noise. This makes the maskee difficult to hear in the presence of the masker.

In each of the following examples of the different kinds of auditory masking, you will hear a maskee and masker. You should aim to follow along and see how long you can hear the maskee as the volume or timings of the masker changes.

Haas Effect/Precedent Effect

In addition to illusions involving pitch and volume, there are psychoacoustic phenomena that impact the direction that we think a sound has come from. One such illusion is the Haas effect (also known as the precedence effect).

If a sound arrives at your ears from one direction, and reflection (i.e. identical copy) of the sound arrives at your ears from a different direction within ~40ms, your brain perceives there to be only one sound, and it perceives the sound to come from the direction of the first sound. In other words, your brain gives precedence to the first sound that arrives as long as the reflection arrives within ~40ms.

This is an important effect to consider in the design of concert halls, recording studios and music venues, because it is desirable to design a space where the first reflections of a sound arrive at the listener within ~40ms so that the listener perceives a single sound, rather than hearing an echo.

It is also widely used in audio mixing. Single tracks (mono audio) can be panned between the left and right speakers in a stereo (two speaker) mix so that the audio sounds like it is coming from a particular direction. But, it can be desirable to make the location of the audio more ambiguous for the listener, creating the impression that it may have come from a wider range of locations. This is known as ‘widening’ the mix.

This can be achieved using the Haas effect by making a copy of the mono track and panning one track to the left and the other to the right. One of the tracks can then be delayed by up to ~40ms to create the impression of a single (wider) audio source.

In the following example, the first extract is the original audio, and the second is the audio with the effect applied. This effect will only work if you are wearing headphones.

Congratulations! You have reached the end of this module 🎉

You have learnt about how human ears receive sound, how we perceive pitch and loudness and how this can differ from the actual frequency and amplitude of the audio signal. You explored how sound can create illusions (psychoacoustic phenomena), and how these illusions can be used in audio technology and sound mixing for music, film and video games.

There are many more illusions that we have not had time to explore in this module, but if you are interested in learning more you can explore Diana Deutsch's website which provides more information about the tritone paradox that we discussed earlier, and further illusions identified by the Diana.