HARKBird

updated on 06/11/2024

HARKBird is a set of scripts that allows us to conduct the following processes using HARK and a microphone array with a simple GUI interface:

• Released HARKBird3.0d, supporting 2D localization with the annotation data of two microphone arrays. 

• HARKBird3.0a is released, which is based on PySimpleGUI and have some additional functions.

• We released a software for a Raspberry Pi node for field recording. 

• When using different transfer function file, there could be inconsistency between MUSIC spectrum and the DOA of localized sounds. When using "tamago_rectf.zip" or "tamago_geotf.zip" comment out the next line in the recordSeparation.py.
            self.SPECTRUM = list(self.SPECTRUM[len(self.SPECTRUM)//2:])+list(self.SPECTRUM[:len(self.SPECTRUM)//2])

• For better presentation of spectrogram and MUSIC spectrum in annotation tool, you can tweak the representation in the corresponding functions in Tools.py

• In order to meet with the current version of HARK, we slightly modified the scripts (ver. 2.02).

• HARK has now updated to the version 3.0, which assume the use of python 3.x. However, the current HARKBird use the python 2.7x. 

• HARKBird 2.0 includes an interactive classification tool of separated sounds based on t-SNE. See the manual file in the zip file for the detail. Further information will be provided in near future.

• Runs on both Windows and Ubuntu Linux (under the assumption that HARK, HARK-python and other related packages and software are installed correctly).

• records and play a track with a microphone array TAMAGO (System in frontier Inc.). Recording with multiple arrays (connected to a single PC with USB cables) were supported (Ubuntu only).

• performs a preliminary localization (with custom settings of some

• parameters for birdsong recordings)

• outputs the localization results with PDF, JSON, and tab separated text formats for further analyses.

• provides a simple tool for editing and annotating (adding species label) the results.

Selected papers (updated 06/11/2024)

• Shiho Matsubayashi, Hideki Osaka, Reiji Suzuki, Kazuhiro Nakadai, Hiroshi G. Okuno: Monitoring the courtship flight trajectory of Latham's snipe (Gallinago hardwickii) using microphone arrays, Ecology and Evolution, 13 (4), e9938 (2023/04). https://doi.org/10.1002/ece3.9938

• Reiji Suzuki, Koichiro Hayashi, Hideki Osaka, Shiho Matsubayashi, Takaya Arita, Kazuhiro Nakadai and Hiroshi G. Okuno: Estimating the Soundscape Structure and Dynamics of Forest Bird Vocalizations in an Azimuth-Elevation Space Using a Microphone Array, Applied Sciences, 13(6), 3607 (2023/03). https://doi.org/10.3390/app13063607

• Hao Zhao, Reiji Suzuki , Shinji Sumitani , Shiho Matsubayashi , Takaya Arita , Kazuhiro Nakadai , Hiroshi G. Okuno: Visualization and quantification of activities of animal vocalizations in forest species using robot audition techniques, Journal of Ecoacoustics, Journal of Ecoacoustics, 7(1), 2 (2023/12). http://dx.doi.org/10.35995/jea7010002

• Shiho Matsubayashi, Kazuhiro Nakadai, Reiji Suzuki, Tatsuya Ura, Makoto Hasebe and Hiroshi G. Okuno: Auditory Survey of Endangered Eurasian Bittern Using Microphone Arrays and Robot Audition, Frontiers in Robotics and AI, 9: 854572. doi: 10.3389/frobt.2022.854572 (2022/04). 

• Shinji Sumitani, Reiji Suzuki, Takaya Arita, Kazuhiro Nakadai, Hiroshi G. Okuno: Non-invasive monitoring of the spatio-temporal dynamics of vocalizations among songbirds in a semi free-flight environment using robot audition techniques, Birds 2021, 2(2), 158-172 (2021). https://doi.org/10.3390/birds2020012

• Shinji Sumitani, Reiji Suzuki, Shiho Matsubayashi, Takaya Arita, Kazuhiro Nakadai, Hiroshi G. Okuno, Fine-scale observations of spatio-spectro-temporal dynamics of bird vocalizations using robot audition techniques, Remote Sensing in Ecology and Conservation (Special Issue: Ecoacoustics and Biodiversity Monitoring ) (2020/09) https://doi.org/10.1002/rse2.152 .

• Reiji Suzuki and Martin L. Cody: Complex systems approaches to temporal soundspace partitioning in bird communities as a self-organizing phenomenon based on behavioral plasticity, Artificial Life and Robotics, 6 pages (invited paper), 24, 439-444 (2019/12). https://doi.org/10.1007/s10015-019-00553-x.

• Shinji Sumitani, Reiji Suzuki, Naoaki Chiba, Shiho Matsubayashi, Takaya Arita, Kazuhiro Nakadai, Hiroshi G. Okuno: An Integrated Framework for Field Recording, Localization, Classification and Annotation of Birdsongs Using Robot Audition Techniques — Harkbird 2.0, Proc. of 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP2019), pp. 8246-8250 (2019/05/17). [PDF]

• Reiji Suzuki, Shiho Matsubayashi, Fumiyuki Saito, Tatsuyoshi Murate, Tomohisa Masuda, Koichi Yamamoto, Ryosuke Kojima, Kazuhiro Nakadai, Hiroshi G. Okuno: A Spatiotemporal Analysis of Acoustic Interactions between Great Reed Warblers (Acrocephalus arundinaceus) Using Microphone Arrays and Robot Audition Software HARK, Ecology and Evolution, 8(1): 812-825 (2018/01). doi: 10.1002/ece3.3645

• Reiji Suzuki, Shinji Sumitani, Shiho Matsubayashi, Takaya Arita, Kazuhiro Nakadai, Hiroshi G. Okuno: Field observations of ecoacoustic dynamics of a Japanese bush warbler using an open-source software for robot audition HARK, Journal of Ecoacoustics, 2: #EYAJ46 (11 pages) (2018/06).  [PDF]

• Reiji Suzuki, Shiho Matsubayashi, R. Hedley, Kazuhiro Nakadai, Hiroshi G. Okuno: HARKBird: Exploring acoustic interactions in bird communities using a microphone array. Journal of Robotics and Mechatronics, 27 (1): 213-223 (2017/02). [preprint]

• Reiji Suzuki, Charles E. Taylor and Martin L. Cody: "Soundscape Partitioning to Increase Communication Efficiency in Bird Communities", Artificial Life and Robotics, 17(1), 30-34 (2012/10). 

Download

A zip file can be downloaded from, here [3.0d]

Tutorial pdf: [Japanese], [English (translated with DeepL)]

Previous versions: [3.0a],  [2.091, 2.08, 2.05, 2.02, 2.0].

Installation and operation

For 3.0a+:

• The largest update of the version 3 is that the GUI is based on PySipleGUI, which enabled the usability of the system easier.  The following commands need to run it in addition to the standard installation of HARK and HARK-python3

  • > sudo apt install xterm sox audacity python3-tk python-is-python3 python2

  • (On the HARKBird3.0x directory) > pip install -r requirements.txt 

  • The system can start up with the following command: (On the HARKBird3.0x directory) > python hb_main.py

  • 
    

For 2.08 and 2.091:

• The latest version 2.08 or 2.09 is based on python3 (+PySide2). When installing this version on Ubuntu 20.04 or 22.04 (2.09), the following commands would be required (in addition to the setup for HARK and HARK-python3):

  • sudo apt install python3-pip 

  • sudo apt install xterm

  • sudo apt install libxcb-xinerama0

  • sudo apt install  hark-python3

  • sudo apt install sox

  • pip3 install scipy pandas matplotlib numpy pyside2 soundfile pyqt5

For 2.0.2:

- We recommend to use the HARKBird on Ubuntu Linux 18.04, and assume that it is based on this. 

- Install the latest version of HARK and hark-python3

- Install the necessary packages for python2 and python3 (PySide, pandas, scipy, numpy, soundfile, etc.)

(You can use pip2 and pip3 for installation of packages for python 2 and 3)

(install pyside using "sudo apt-get install python-pyside")

- Install sox and xterm

- extract this zipped folder

- move to "harkbird" folder

- run "./harkbird" or "python ./harkbird"

Follow the installation (2.0) and operation manual here (English, Japanese).

HBPi (Raspi node for HARKBird)

• Download a set of files here. 

• Install the Raspberry Pi OS on a microSD card.

• Initialize a USB memory stick for storing recorded data with exfat (you can also put a set of files here).

• Start the Raspberry Pi and connect it to an available network.

• Move the file set to /home/pi

• Make install.sh and install_raspap.sh as executable files (from the right-click properties). 

• Open a terminal and type in . /install.sh, and run it (you will be asked y/n, then press y for all).

• After rebooting, open a browser and access raspberrypi.local:1880 (access node-red)

• Open the top-right menu→manage palette and search for node-red-dashboard in install section, then install it.

• Open the top-right menu→import, load flows.json in the set of files, and deploy. (You will be able to connect to ----:1880/ui.)

• Open a terminal and type . /install_raspap.sh, and execute it (y is fine for this, too).

• Open a browser, then enter raspberrypi.local, and access to RaspAP (user:admin password:secret).

• Enter a desired SSID in the Basic Hotspot field and a desired password in the Security field, and set the access point.

• Connect to the configured access point and connect to raspberrypi.local:1880/ui to operate the recording unit.

About HARK, HARKBird and HBPi

• HARK (Honda Research Institute Japan Audition for Robots with Kyoto University) is an open source software for robot audition, which enables us to localize, separate sound sources in real time.

• HARKBird is a set of scripts that use HARK for analyzing acoustic interactions among species and their surrounding acoustic environment.

• HBPi is a software for field recording with a raspberry pi for HARKBird.

License

• HARKBird and HBPi is developed by the research group of Reiji Suzuki, Graduate School of Information Science, Nagoya University. Currently, it is maintained by Reiji Suzuki and Shinji Sumitani.

• HARKBird and HBPi is released under the Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0) license (https://creativecommons.org/licenses/by-sa/3.0/).

• We do not assume any liability or responsibility for any problems arising from use of the scripts. 

• Note that HARKBird itself does not contain any packages or files of HARK (only scripts that use HARK). Also note that HARK is free for an academic use. 

Acknowledgement

• We thank the HARK developer team for many supports with regard to construction of the scripts the use of HARK.

Contact

• Reiji Suzuki, Graduate School of Informatics, Nagoya University: reiji@nagoya-u.jp

Snapshot of English manual