Bird Species Identification Through Audio Analysis Using MATLAB

Abstract

The research question asks how effective the use of power spectra and spectrograms are in identifying bird species from recorded bird calls. This was investigated through the recording of audio signals from various birds found around Somerville, Massachusetts. To identify potential birds for each recording, the bird identification app Merlin was used to record the bird calls as well. In addition to bird calls recorded in the real world, recordings of clean versions of each potential bird were made using samples from the Merlin database. Both the experimental data and sample data were transformed into the frequency space to visualize the power spectra associated with each sample and each bird species, in order to make comparisons between the two. Spectrogram plots were also utilized to visually compare the rhythm and frequency of the different songs. At the conclusion of the investigation, it was determined that MATLAB analysis of bird songs is a possible tool for species identification, but more controlled recording conditions and additional testing would be needed to ensure that results are accurate. Recordings taken on a typical urban block are subject to additional noise, and a much more controlled environment would be needed for definite identification.

Introduction and Background

This topic was chosen because one team member is participating in what is called a “big year” of birdwatching. The goal of a big year is to see as many different species of bird as possible over the course of one year. This big year has included a significant amount of use of a bird identification app called Merlin. Merlin is an app created by the Cornell Ornithology Lab, which helps users identify birds by recording their calls and songs. It is beloved by birdwatchers worldwide, and widely considered one of the best bird-identification softwares available. Merlin shows a spectrogram of the audio on a user’s phone as it is recorded live, along with the most likely bird the user is hearing based on location and the audio signal. The goal of this experiment is to see if it is feasible to use a microphone, and MATLAB-based audio processing to create a system which can identify bird calls. This would allow for exploration into alternative methods for bird call identification, which could be more or less effective than existing software.

Merlin’s “Sound ID” was introduced in June 2021, and is advertised to provide live predictions of the bird or birds a user can hear. The audio is converted into a spectrogram, which plots the audio frequencies as a function of time, as it is recorded through a phone microphone. This allows a user to view audio signals visually, with intensities of each frequency indicated by the relative darkness or lightness of the graph. Sound ID then sends the spectrogram into a computer vision model–essentially an AI program which analyzes given inputs, such as audio recordings. This model is built on a network which can “learn” and predict results. This network was trained on hundreds of hours of bird audio, identified directly by birdsong experts, as well as many common background noises, such as cars, footsteps, or wind. These audio files are made into master spectrograms of known bird calls. The model transforms its generated spectrogram of a user’s input audio, cycling through millions of small adjustments to make any recorded spectrogram match the master spectrograms. The model retains the information on which adjustment led to the closest match, which enables it to improve its speed and accuracy over time [1].

Merlin’s Sound ID is not the first program to make audio-based bird identification widely available. Cornell Lab of Ornithology collaborated with the K. Lisa Yang Center for Conservation Bioacoustics at Cornell and Chemnitz University of Technology in Chemnitz, Germany to create BirdNet, prior to the creation of Merlin Sound ID. BirdNet uses the same architecture and process as Sound ID. BirdNet’s models are open source, so users can integrate them into their own projects [2]. BirdVox was another audio recognition program created by the Cornell Lab of Ornithology, this time in collaboration with the Music and Audio Research Laboratory at NYU [3]. Finally, the Cornell Lab of Ornithology’s Center for Conservation Bioacoustics has historically held competitions to create programs which can identify bird calls from audio. BirdCLEF 2021 and Birdcall Identification 2020 were two such competitions [4] [5]. While Merlin’s Sound ID is absolutely built upon the work of these earlier models, none have come close to being as widely available nor used as Sound ID. Merlin is currently considered to be the gold standard of accessible acoustic identification within the birding world.

Audio identification is also useful for tracking migration progress of birds and bird populations in different areas, but it also allows researchers to keep tabs on population numbers of threatened species. This acoustic detection can give conservators an idea of how many different species inhabit an area and when they are most active [6]. It can also alert researchers to unusual sounds, such as habitat destruction or poaching, giving local authorities the location of the sound to intervene [6]. 

Some of the major issues birdsong identification currently faces includes interfering background noises, multiple birds calling at once, birds having many variants of calls and songs, and bird calls being too faint to pick up correctly on the microphone. A paper in the Journal of Avian Biology points out that the inaccessibility of identification tools is a major problem for ecologists [7]. The software takes a long time to calibrate and requires a significant amount of technical expertise which these scientists might not have. While many birdsong recognition systems boast high accuracy rates, none are yet good enough to use completely unattended in the field. More exploration is needed into new methods or improving existing ones for better usability and higher accuracy.

While the current methods of audio-based identification work very well, there is still room for improvement. No major identifiers use a system other than spectrogram analysis, and there is the possibility that other strategies may be more accurate, faster, or more efficient than the current one. This project investigates the viability of an alternative method of birdcall identification with the use of readily available software and without any artificial-intelligence-based models. This means that any success of a system created through these methods is significantly more accessible as an open-source, readily available project to all.

Experimental Methods and Calibration

Audio data was collected from quiet side streets around the Tufts University campus in Somerville, Massachusetts. Both Merlin and the iMM-6 microphone were used to record birds when they were heard chirping. The microphone used is a Dayton Audio iMM-6 microphone, serial number 99-61304. The microphone uses a TRRS to TRS adapter so the audio can be received and saved by the model A12 digital voice recorder from Tonfarb (see Fig. 2). An iPhone recorded the bird calls using Merlin.

One goal of data collection was to record a variety of bird species. At the end of data collection, ten different possible bird species had been recorded, according to preliminary analysis conducted using Merlin. The birds observed in the samples were the northern cardinal (NC), house sparrow (HS), carolina wren (CW) and white-breasted nuthatch (WBN). Abbreviations of bird names as initials were used in code and referred to throughout the report for succinctness. The audio files were uploaded to computers and trimmed to span the length of one bird call, which lasted a few seconds each. This was done because MATLAB has a restriction on file size, and it would be easier to monitor frequencies within a smaller dataset. The clips were compared to Merlin’s database of bird calls to determine potential species they may belong to. The two bird calls from the database that were most similar to the experimental audio clips were recorded using the same microphone, played from the speaker of a phone.

The audio data needed to be calibrated to correct the signal so that the relative loudness of the frequency bounds were correct. The first step was to read in data from the audio file, to return the sample data and sampling frequency using  the audioread() function in MATLAB. A calibration file provided by the manufacturer was found on Dayton Audio’s website upon inputting the microphone and serial number used for recording. Then, the frequency and dB calibration vectors could be established using the two columns from the calibration file. The first two rows needed to be deleted from the original calibration text file, because the first row created NaN values and the second row was empty. The next step was to calculate the FFT shift of the data using the fftshift() and fft() functions, followed by finding the corresponding frequency vector. Then, linear interpolation with the interp1() function was used with the calibration data to estimate decibel values at the desired frequency. Next, the interpolated decibel values were converted to a linear scale and applied to the fourier transform of the original data. Using the ifft() and ifftshift() functions, the signal was transformed back into the time domain where it was normalized and the real part of the signal was taken to be the final signal. Finally, the corrected signal was played, to ensure it sounded similar to the original signal. Refer to Appendix A for the commented code used for this calibration.

Data was analyzed using spectrograms and power spectra plots of the audio samples recorded using an experimental setup and the clean audios recorded from Merlin.  Patterns in spectrograms and peaks in power spectra were the two main modes of analysis.

Results and Analysis

The first step of analysis was creating spectrograms with MATLAB [8]. This allowed for visualization of the audio signal, showing the strengths of different frequencies against time. Appendix B contains spectrograms for all the audio samples which were utilized in this experiment. The purpose of this step was to recreate a simplified version of the processing done by major audio identification softwares. While one can visually compare generated spectrograms, direct spectrogram comparison is not the best possible method of analysis, since it takes significantly larger systems to generate quantitative results in the comparison. Experienced birders can recognize bird species by the plots of their spectrograms alone, and visual parallels can be drawn between samples collected in the field for this experiment and isolated audio clips from Merlin. 

Table 1 shows similarities between spectrogram graphs of two audio samples. The two swooping frequencies up and the staccato calls afterwards parallel each other, indicating that Sample 8 is likely a northern cardinal.  Mimic birds, like mockingbirds, can make these types of identification more difficult, but any program will struggle to tell the difference between a mimic and a true species. 

Birds with less distinctive calls are also far harder to identify from a spectrogram. Merlin identifies Sample 10 as a white-breasted nuthatch, known for its low pitched “wha-wha-wha” song or its loud “yank” calls [9].  Table 2 compares the spectrogram of the sample with the spectrogram of two other bird calls. It is impossible to give a confident identification of the nuthatch by comparing it to the other spectrograms, as none have particularly distinctive characteristics. Both bird species have short, quick calls in rapid succession, followed by a pause of varying length. Background noises in Sample 10 may also make it appear more similar to the other recordings, by adding frequencies which were not in the bird calls or by muddying the actual songs themselves.

To visualize the power spectra of the audio files from experimental data collection, pre-calibrated  audio signals were imported into MATLAB and  transformed into the frequency domain, where the corresponding power spectrum for each recording was derived. Since the recordings produced different frequency vectors, one dataset was linearly interpolated to produce a common frequency vector shared between all the power spectra, to allow for direct comparison between trials. The power spectra were then averaged pointwise to obtain a representative spectrum for the samples. Since the recordings of the bird calls from the database were so clear, only one sample was used to generate the power spectra for the known bird species (refer to Appendix C). 

The uncertainty in measurements due to random variation in the tone and duration of bird calls was determined to be much more significant than the uncertainty from the microphone alone, which was deemed to be negligible in this case. To calculate this uncertainty, the power spectrum was calculated and averaged for multiple recordings of the same bird call. Then, the standard deviation at each frequency was used to calculate the variability across trials to provide a measure of uncertainty at each frequency. From there, the peaks in the power spectrum were investigated, and the values of the peaks at specific frequencies were calculated with the frequency-specific uncertainty. Refer to Table 3 to see the values that were obtained:

Across multiple samples, peak frequencies in the experimental recordings often aligned roughly with peaks from known bird recordings, but the power values showed large differences and uncertainties. Sample 1 and Sample 3 both showed partial alignment with house sparrow recordings, yet their peak power differences (over 60%) and high uncertainties made species identification unreliable. Sample 6 showed closer agreement with a white-breasted nuthatch in both frequency and power (16% difference), but this identification was ultimately incorrect, further highlighting limitations in the method. Samples 8 and 10 also loosely matched white-breasted nuthatch peaks, but very large power differences (up to 95%) and uncertainty again prevented confident identification. Overall, while some frequency similarities were observed, the variability in power spectra and high uncertainty indicate that this method alone is insufficient for accurate bird species identification.

Conclusion and Future Work

With present samples and reasonable amounts of MATLAB computing, it becomes apparent that audio processing systems are not able to produce confident predictions of bird species based solely on audio recordings. Spectrograms generated had some similarities in audio patterns, but this did not apply to all bird species tested, only the ones with more distinctive calls, such as a northern cardinal. Spectrograms do not, however, provide a quantitative similarity measure between two samples, and require a person to visually compare a given spectrogram to pre-identified sample spectrograms. Power spectra provided less conclusive results. Neither the locations nor the amplitudes of the peaks of high amplitude frequencies corresponded to the graphs of other recordings of the same bird.

The findings of this project highlight the usefulness of iterative computer models. Spectrograms are difficult to match since birds constantly vary their calls, singing louder or softer, changing the timing of their calls, or singing in a different pitch. Without the ability to change these factors in an efficient way, it is very difficult to use spectrograms for identification. This is due to the random variation in the measured quantities, specifically recorded birdsong as well as its surroundings. Power spectra, on the other hand, theoretically allow for more wiggle room. Since they compare the powers of different frequencies, timing of the call becomes irrelevant. This is helpful in removing the issue of timing differences, but timing of calls is also often a large part of how bird calls are identified, for example a drawn out “coo” may be a mourning dove, while a series of quick chirps might be a chickadee. In perfect conditions, power spectra should be able to give a solid bird identification foundation, even if it is not completely comprehensive. 

While programs for identification which use less computing power are still something to work towards, simple MATLAB code cannot identify birds from their calls in the wild. There are too many outside factors such as weather, distance, and call variation, all of which compound to make computer audio analysis less accurate. Open source code which does not use neural networks would make modifying or adapting the code for other projects much easier, therefore making it more accessible to anyone in the world with a project.

The lack of secondary peaks in the power spectrum graphs may be due to wind and other outside factors. Wind attenuates higher frequencies faster than lower ones, and the real world samples for this experiment were taken on a day with moderate wind. The materials between the birds and the microphone also may have absorbed more of the higher frequencies of sound. The birds were often in the middle of a bush or between two houses while being recorded, which will change the acoustics of their calls. While recordings were taken at the quietest possible moments, there was also constant background noise which included music, cars, construction, and other birds.

In a perfect world, a controlled area with one bird a set distance from a microphone and no background noise would be ideal, but those conditions are hard to come across, especially when working with birds. This is also not a position any user of a bird identification software would wind up in. For a future iteration of this experiment, finding a very quiet area on a calm day would be a strong starting point for more useful samples. After recording those samples, the same analysis procedure would be repeated, and the graphs would be compared to the clean Merlin recordings.

Finally, it is possible that Merlin compresses or stores their bird audio samples in a way which changes their frequencies. This is unlikely, but since the processing system for the audio files is completely unknown, it is still possible.

Appendix B

Spectrograms

Appendix C

Power Spectra