SoundSpace

Introduction

The premise of this project is to introduce a novel method for perceiving a 2 dimensional grayscale image, with astronomical images the primary type of image that will be converted. Here you will find links to the program as well as instructions to install it and other stuff.

Getting It

The project can be found here: https://github.com/rdawis/SoundSpace

It is actively being maintained by Raki Dawis (raphael.dawis@gmail.com) as of September 2017.

Installation

Prerequisites:

• Python 2.7.12 or above (as of now, NOT any version of Python 3)
• Pip (make sure to update Pip)

Downloading:

To download the files, press the button that says "Clone or Download" and click "Download ZIP". Alternatively, if you're using Git, you can use the command

git clone https://github.com/rdawis/SoundSpace.git

Preparing for first runtime:

When you first download the program, you may or may not have the libraries necessary to run it, or they're outdated. To ensure up-to-dateness, take a terminal into the SoundSpace folder and run the following command:

pip install -r requirements.txt

It should install all the requirements. If you have problems with this, make sure Pip is in your environment path (especially in Windows) and ensure that you're running the Pip for Python 2 (in Arch Linux, Python 2 and 3 can coexist, as can their respective Pips as pip2 and pip3).

Running the program:

Go into the SoundSpace folder from a terminal and run the command:

python main.py <FITS Image>

where FITS Image is replaced by the path to the FITS Image. Alternatively, you can omit a FITS Image, and use the selector to pick a file.

Altering the program:

NOTE: This is 100% permitted by the GPL, just don't try to make my code less accessible than it was to you, I.E. don't make people pay to see the work I've done (or if you do, donate half of it to my Twitch stream) or don't try to use it in something that requires an NDA I think. I'm not a lawyer though, so be sure to read up on the GPL if you're not sure.

To change how the program works, you'll want to just alter the main.py portion of it, but if you have any suggestions for the classes I created for the program, you can submit a pull request to me, and I'll check it within a week.

Scripting with my Libraries:

Importing these functions is simple:

import img_deconstruct

import sound_reconstruct

or

from img_deconstruct import *

from sound_reconstruct import *

Here's the general syntax for the two classes:

img_deconstruct.py:

Image(<file path>, 

      [ <length of desired x axis>,

        <length of desired y axis>, 

        <threshold for a black pixel between 0-100> ]

     )

Image.lower_res([ <pixel sensitivity between 0-100> ])

you can always run Image.lower_res() whenever you want - be sure to normalize after the fact (I should fix this maybe)

sound_reconstruct.py:

SoundSpace(<Image.reduction>, 

           <sound length>,

           [ <number of notes>, 

             <spacing between notes> ]

          )

Note that things in SQUARE brackets are optional and can be called via keyword.

Things in ANGLE brackets are plainspeak descriptors of what everything is.

How to interpret the results:

You'll hear a bunch of things, lemme break it down for y'all:

• Pitch: Vertical positioning.
  • If you hear a higher pitch, it's further up on the page; conversely, lower pitches are lower on the page.
  • If you hear no pitch, there is nothing in that column.
  • If you hear multiple pitches, there are multiple points of light in that column.
    • If the multiple pitches are really muddled, the points of light are close together.
    • If you can hear distinct pitches, the points of light are further apart.
• Directional Audio: Horizontal positioning.
  • The "column scanner" will run from left to right. You will hear the first few pitches start from your left ear, the next few over to the middle (in front of you/ really in your head), then the last few pitches over to your right ear.
• Ticking sounds/dings: Indexes for horizontal pitches.
  • before a column is played, a tick or a ding will sound. Dings occur every 5 ticks. If there is no sound between ticks/dings, there is nothing in that column. If there is sound in between ticks/dings, there is something in the column.
• Volume (not well implemented): brightness of pixel.
  • If you hear a louder pitch, that pixel is brighter. If you hear a softer pitch, that pixel is dimmer.

Ideas for Projects:

• Use a better HRTF library/use OpenALSoft (Python library stuff for OpenALSoft is wack, swhy I didn't use)
• Make the image scannable via mouse/touchscreen.
  • Easier: implement scrobbling of my sound via mouseover on the image
  • Harder: make each pixel scannable by mouse.
  • Hardest: Do it with a computer vision image.
• Implement in SEO Pipeline (ask Marc Berthoud)
• Implement documentation in haikus again???
• Implement JPG/PNG support: Only FITS works right now
  • Also: Implement color support
• Implement sonic overexposure detection (may be its own project)
• Implement in Afterglow tool
• Design a teaching program in order to allow new users to learn how to use this method of perceiving astronomical images quickly.
• Add on more options:
  • Option to omit tick/ding sounds
  • Option to change tick/ding sounds
  • Ask Marc for more :D

Questions:

Shoot me an email at raphael.dawis@gmail.com with the header [Yerkes] SoundSpace Question, and I will reply within about a week. If I don't, tell Marc/Kara to tell me to check my emails for a question :)

GLHF

- Raki

You can catch me playing video games online, and if you like the tool, think about donating lol :)