If you get bored, you could make a fun guitar hero style game with this
To begin with, 2 blueprints are needed, one which analyses the sub-mix, and another, that just plays your sound. Realistically you can do it with just the 1, but keeping things separate is easier and keeps things more organised.
The sound source just plays the selected audio asset on begin play (as this is a runtime audio analysis):
The sound that is play is a simple chord progression, that you can use here if you want to recreate this for yourself Chords are: Gmin7, GMaj7, Emin7, DMaj7. The 2nd half are the same chords with a bit of flair, being the added notes of: A5, B5, F5#, D5
The metasound is just a mono waveform player, to keep it simple.
The real magic is a step by step process that happens within the second blueprint, the submix analyser.
To begin with, I mapped out the 4th octave notes on a keyboard, using the frequencies of each one, to each integer I needed
I use that to define what frequencies I’m looking for on begin play
I also set a timer to check and print the maximum magnitude of each frequency that is found on a looping 0.2s delay, so I can see what the results are
Using the frequencies to analyse, I prefill the required number of arrays with 0, because if I don’t do this, unreal just returns 0s no matter what, so it needs the space, or something
Directly after fillng the magnitude values with 0, I start analyzing the output.
The settings for the output analyzer are incredibly important, there is no “one size fits all” setup for this, sadly, and so depending on what sound you’re analysing, you’ll need to fiddle around.
The settings I used for this demonstration were:
- Submix to Analyze: None (this means it analyzes the master submix, which is all sounds, you can change this to music, or input, whatever you need)
- FFTSize: Max
- Interpolation Method: Nearest Neighbour
- Window Type: Blackman (I also found hann worked very well, as they both minimize spectral leakage, which is helpful for some muddier sounds)
- Hop Size: 0.0 (this is how many samples It can move forwards before doing another analysis, the higher this number, the less performative, but less accurate, 0.0 is every single sample which is heavy performance cost! ideally, you can know what the distance in your frequencies you’re looking for is, and change based on that, to sample a closer range to specifically the things you *need*)
- Magnitude spectrum (strength of the frequencies in magnitudes, can work with Db or power spectrum, but they just change the resulting values from the float print. They will all be the same relative to eachother.)
After setting up the analysis, you need to retrigger the actual magnitude check, this can be done better, but for the sake of testing, I did it on tick.
If you are doing this for a game, you’d be better off putting it on a timer by function like with the print from the begin play.
The Get Max Magnitude:
This works by going through every magnitude, holding the index of that specific magnitude, and seeing if its higher than the previous value that it had, if it is, then it sets the new magnitude as the highest, and adds it to the result. (locally)
When it is finished doing the above calculations, at the completed section of the for each, it sets the new locally highest, as the max magnitudes.
This means that when this is finished, it has the highest magnitudes stored as an array, 0-11.
Finally, looking back at the timed function, the print max magnitudes. It works by using the max magnitudes, printing them based on their integer to what frequencies I had previously set (the c-b range of octave 4)
now by placing both in the world, you should have something that works, printing you values like this, where you can see what the strongest values are that have been played.
Its worth noting, since the max magnitude is never cleared, those values will remain as high as they can be, you’d need another timer that goes off to reset the values back to 0 if you want this to continually work.
I will note, this is a very loose, unhelpful and relatively useless method. But when combined with other things (volume detection, like magnitudes again) it can become helpful.
Once you have numbers, you get to do the fun creative part of trying to figure out how to use the numbers.