research diary

Not bibliography; it's a list of articles, tutorials and surveys etc. I looked at during the course of the project with some writing detailing it's purpose or the impact it had on the product.

22/1/19

Trying to find ways to convert the rhythm from some audio into a file of timestamps


https://github.com/MonsieurV/py-findpeaks

https://arstechnica.com/gaming/2008/03/catching-waveforms-audiosurf-creator-dylan-speaks/

http://librosa.github.io/librosa/tutorial.html

https://jams.readthedocs.io/en/stable/jams.html


Tool I initially thought of using turned out to give beats rather than rhythm, but I was able to use it to generate a sample file I can use for testing the game in the meantime.

23/1/19

http://librosa.github.io/librosa/tutorial.html

What I was looking for turned out to be in another section of this website, used this to write a script to generate the song files.

25/1/19

Looking for inspiration on unconventional/less obvious UI from feedback on my pitch.


https://www.platinumgames.com/official-blog/article/9624

https://gamedevelopment.tutsplus.com/tutorials/game-ui-by-example-a-crash-course-in-the-good-and-the-bad--gamedev-3943

https://medium.com/ironequal/the-only-advice-you-will-need-to-make-a-great-game-ui-ux-74a0db8de642


Were helpful as they gave me things to think about but it wasn’t very specific to my audience, just general tips like don’t make it too complicated. The first one had some interesting points about consistency and symbolism, and I quite liked the faded colour scheme they had used.

28/1/19

https://www.templaza.com/blog/best-fonts-for-flat-ui-design

https://www.mockplus.com/blog/post/flat-mobile-ui-design

https://flatuicolors.com/palette/defo


Kerry suggested some kind of flat ui so I started looking into and experimenting with those for a lesson.

29/1/19

30/1/19

Tried to incorporate several scenes into my game file to prepare for making a functional menu.


https://forum.unity.com/threads/directory-getfiles-order-by-date.387275/

https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnTriggerEnter2D.html

https://stackoverflow.com/questions/45691706/unity-resources-load-is-not-working

https://docs.unity3d.com/ScriptReference/Application.Quit.html

https://answers.unity.com/questions/1109497/unity-53-how-to-load-current-level.html

https://answers.unity.com/questions/989004/how-can-i-design-for-1920x1080-in-unity.html

https://docs.unity3d.com/ScriptReference/Resources.Load.html

https://docs.unity3d.com/Manual/class-TextAsset.html

https://answers.unity.com/questions/917138/text-file-not-loading-after-building-the-game.html

https://gamedev.stackexchange.com/questions/110958/unity-5-what-is-the-proper-way-to-handle-data-between-scenes


Turns out it’s quite complicated. Also did more research into UI’s as my flat ui attempts all looked too boring.


https://www.photoshoptutorials.ws/photoshop-tutorials/special-effects/quick-tip-create-triangle-pixelation-effect-photoshop/?doing_wp_cron=1549280117.3133440017700195312500

https://www.youtube.com/watch?v=JfiJ7hz734U

https://www.youtube.com/watch?v=rnucMIXVcTo

http://www.gamasutra.com/view/feature/4286/game_ui_discoveries_what_players_.php?print=1

https://www.reddit.com/r/gamedev/comments/4inddf/how_do_you_design_main_menus/

https://www.gamasutra.com/blogs/MaxPears/20160421/270579/Setting_the_Tone_Main_Menus_are_the_Game.php

http://www.theappguruz.com/blog/game-main-menu-design

https://kotaku.com/the-ten-commandments-of-video-game-menus-sinners-and-s-5960230


Most articles had advice about not breaking from the norm in terms of control schemes and placement of things, i.e. make the B button on a controller go back not A, and people seemed to favour more minimalistic UIs over complicated ones. After a google search for “game main menus” the majority just have a small list of options floating in front of some art from the game.\


https://gameuipatterns.com/gameui/main-menu/


I probably won’t be able to make anything particularly impressive but I could do some kind of mosaic art or something with shapes in the background of the menu.


I also asked some friends close to my target audience for feedback on the playability of the game.

From what they said I’ve learnt that the control scheme takes a while to get used to/might be too awkward, the notes are a bit too fast, and that the scores reading doesn’t work properly, and the folder where the scores are kept is in the right place on windows but not on linux.

5/2/19

6/2/19

8/2/19

Asked ben for feedback on ui design, was going to get more into it but had to go and help other people. Also looked into some graphic design trends in 2018 but none really stuck out to me. He said he likes the font, I should be consistent with my use of caps and lower case, he doesn't like the white outlines on the boxes and to try and make it flatter.


https://www.youtube.com/watch?v=dcc1VyGvaYk

https://www.youtube.com/watch?v=-l2g4adWWwo

11/2/19

Did some research into colour theory in video games to try and help me pick a colour palette. I also looked into data storage methods and found out you can use sql in unity.


https://bishopgames.com/importance-colors-game-design/

http://howtonotsuckatgamedesign.com/2014/11/color-theory-game-design-1-fundamentals/

https://www.reddit.com/r/gamedev/comments/8oh09v/color_theory_in_games_an_overview/

https://www.gamasutra.com/blogs/DougStewart/20170411/295806/Color_in_Video_Games_How_to_Choose_a_Palette.php

https://github.com/robertohuertasm/SQLite4Unity3d

https://www.reddit.com/r/gamedev/comments/56z5dj/want_to_make_a_sql_database_for_my_game_in_unity/

https://youtu.be/baCOANi2gXc

Considering fully going towards zen/focus kind of game so researching into relaxing games to see their UIs.

https://itunes.apple.com/es/app/zentone-relaxing-music-game-for-meditation-rhythm-noise/id867637409?l=en&mt=8

https://www.lifehack.org/articles/lifestyle/10-relaxing-online-games-help-chill-you-out.html


Akira mentioned an oriental feel from the colour palette which goes with what i was thinking about taking the game in a zen/focus kind of direction. Did some research on visual styles of Japan with relaxing connotations etc., found lots of minimal art of temples, especially since that’s where zen gardens usually are. Another thing associated with rhythm in Japan and temples is the taiko drum. After finding it I realised the way the drum is played is quite similar to my control scheme in the game. New idea for main menu/ui is gate with taiko drums set up facing the player that say play etc., and when playing the notes are falling down onto a taiko drum.

15/2/19

Tried to find more justification for me going towards the relaxing style of game and the visual aspects I’ve chosen to find.


https://terebess.hu/zen/szoto/hangszersz.html

^ taiko drums are common at zen/buddhist temples


https://www.strawpoll.me/17433531

^ strawpoll I made on why people play rhythm games, posted to several discord servers where people play rhythm games


https://www.theguardian.com/lifeandstyle/2018/oct/14/the-lost-art-of-concentration-being-distracted-in-a-digital-world

https://medium.com/the-mission/how-practical-minimalism-helps-me-focus-8d204d60ae63

^ articles on benefits of minimalism in real life


https://mformaratz.wordpress.com/2013/03/03/super-hexagon-transcendence-through-minimalism/

^ article on how the minimalism draws you in on a game like super hexagon which is sort of similar


https://en.wikipedia.org/wiki/Zen

https://en.wikipedia.org/wiki/Japanese_Zen

^ research on what zen actually is

19/2/19

Got feedback from Akira - justifications good but too stereotypical and logical which is bringing down the creativity. Less stereotypes/trying to squeeze the culture into it, try to be more modern while still being minimal.


Thin, sans serif font

Gradients, warm colour palette

Subtle textures in the background/minimal icons


https://www.toptal.com/designers/subtlepatterns/


Colour intensity increases with difficulty?


https://www.vandelaydesign.com/thin-fonts/

https://99designs.co.uk/blog/trends/color-trends-2018/


Reaction to new UI

“Modern and sleek looking” someone who sits next to me

Seems like it’s achieving what I wanted now.

27/2/19

The IDD exhibition was on today and Ben let me set up my game on one of the tables. I started listening and watching people interact with it to try and get some feedback indirectly.


Of the maybe 15 people I saw playing the game I only remember seeing four who got above 80% accuracy on a song, and they were all people in my target audience. The only guy who got above 90% kept coming back to play again and was someone with lots of experience with rhythm games so it was nice to see someone enjoying it. I overestimated the lowest difficulties and also the amount of time people would be willing to spend playing if they found it difficult, as two of the three songs I had set up were over two minutes long, and the there was no indication of length on my temporary main menu, so a few people walked away before it was done but they were mostly adults, probably parents of other students. I also saw some people having an issue with the main menu because there was no feedback when you clicked on something interact-able but that shouldn’t be an issue with what the main menu should actually look like. Feedback on the UI included “very sleek”, “modern and clean”, and “quite soothing” so I think I got the visual style implemented well. Someone complained about the timing being dodgy so maybe I should add some kind of offset but I haven’t had any issues with it and most of the misses they had were probably a result of their lack of experience reading.


Overall things I’ve taken from this are:

  • Have easier, shorter and more songs.
  • Let you change the keybinds in the options in case people find it uncomfortable.
  • The game is attractive to people in my target audience.
  • Visual style achieves what I wanted it to.
  • Need a wider range of songs (although this wouldn’t be an issue in an actual release because they would be able to add their own songs)

5/3/19

When I started trying to work on it over the holiday none of the inputs were working in Unity, so I left it to deal with later. Today I started trying to fix it.


https://forum.unity.com/threads/keyboard-input-and-gui-buttons-not-working-properly.116505/

https://answers.unity.com/questions/957904/ui-button-is-not-detecting-mouse-click.html

https://answers.unity.com/questions/425478/onmousedown-not-firing.html

https://forum.unity.com/threads/updating-unity-may-require-re-import.457144/

https://forum.unity.com/threads/solved-ui-button-not-working-at-all.427750/


These are a few sources I found with issues similar to what I was experiencing. It turned out to be an issue with re-calibrating the project as my version of Unity was slightly newer than the one we were using at school, so I needed to re-import everything.


Next I started trying to recreate the mockup gameplay screen I had shown Akira before the holidays.


https://answers.unity.com/questions/1187755/how-to-make-a-gif-in-canvas.html

https://answers.unity.com/questions/931917/animate-image-ui-with-sprite-sheet.html


I started looking into animations in order to make the hit-circles fade when you hit and the lines flash red on a miss but it seemed overly complicated for what I was trying to do.


https://answers.unity.com/questions/1230671/how-to-fade-out-a-game-object-using-c.html

https://answers.unity.com/questions/1337625/change-image-on-keypress.html

https://docs.unity3d.com/ScriptReference/Transform-localScale.html

https://docs.unity3d.com/ScriptReference/Time-deltaTime.html


I found these people’s ideas which suggested slowly changing the alpha channel of the object to 0 to make it fade out, which made sense but I still wasn’t sure how to implement it. I also found out about localScale which let me make the note objects slightly larger over half a second before disappearing.


https://answers.unity.com/questions/1010494/having-trouble-trying-to-fade-out-a-sprite-after-5.html

I found this example that did exactly what I wanted to do, after reworking it slightly. I tried to do something similar with the miss lines but there was an issue of it playing the animation over itself several times if you missed consecutively which looked quite weird, as well as me not knowing how to properly fade from red to white made it look odd too.


https://forum.unity.com/threads/smooth-continuous-color-change-over-the-game.228223/

https://docs.unity3d.com/ScriptReference/SpriteRenderer-color.html

https://answers.unity.com/questions/459622/new-color-not-working-properly.html


I found a function called Color.Lerp where you give it two vectors of colours and a number between 0 and 1 and it uses linear interpretation to calculate the colour at that percentage travelled through the vector. It was exactly what I needed as I could just define two colours at the start, but it turned out there were two types of colours, Color and Color32 which use 1,1,1 and 255,255,255 respectively, and I was using the wrong one which made the colours way too bright. After fixing that it worked as intended.

6/3/19

Early in the morning I started trying to think of how to implement health bars, and how health should work using hp drain. I was talking about it to someone who’s fairly good at maths just so I could get an outside opinion and point out any flaws in logic before i tried to implement it.


Here I was trying to wrap my head around how to use the number given in the difficulty calculation in the actual calculations of health and he pointed out that my particular system didn’t make any sense so I started again.


Here he made a good point where again I was overestimating the lower difficulties and reminding me to think from the perspective of a new player. Eventually decided on takes away 10/100 health per miss and adds on 10*1-hpdrain per hit, so at 0 you can miss up to half of the notes, and at 1 you can only miss 10.

Next I had to actually implement this in the game. It was fairly easy to just add a health variable that is added to and subtracted from every hit and miss, but representing it proved harder than I expected. I couldn’t figure out how to scale a rectangle representing the health bar from one side (ironically I realised how to a few days later while researching ui elements, I needed to change where the pivot was in the unity editor).


https://answers.unity.com/questions/617970/scale-from-one-side.html

https://docs.unity3d.com/ScriptReference/Rect.html

https://docs.unity3d.com/ScriptReference/RectTransform-pivot.html


I wasn’t able to solve it using any of these, I then realised the game looked more balanced and symmetrical when the health bar was shrinking towards the middle rather than going from left to right, which fits the feeling of the game much better.


Next I started trying to implement my new difficulty stats on a wider range of songs. I tried looking for how the difficulty rating is weighted in other games where I know they are calculated automatically, as well as how it actually works.

https://osu.ppy.sh/community/forums/topics/763358


However my situation was quite different as the health is based solely on hits and misses rather than slowly draining over time. Next I looked into how to use python to do these so I could add it to my song generation script.


https://stackoverflow.com/questions/47040728/get-average-difference-between-all-numbers-in-a-list-python

https://stackoverflow.com/questions/25169297/numpy-logarithm-with-base-n


I tried it with 20 songs and got difficulties ranging from around 20 to around 70, which I thought was a good range.


Next I started trying to recreate the song selection menu I had a mockup of.


https://www.youtube.com/watch?v=3q4gt3OhWuw

https://answers.unity.com/questions/753755/can-you-animate-main-menu-title.html

https://answers.unity.com/questions/848115/how-to-set-ui-rect-scroll-for-dynamically-growing.html

https://unitycoder.com/blog/2016/02/23/ui-scroll-view-automatic-content-height/



I watched a bit of this tutorial but this guy was trying to make a sort of cutscene style animated menu, and what I wanted was constant animation on input. The second link made me realise I should be trying to animate it similarly to how I animated the notes and miss-lines in the gameplay scene, by manually changing the numbers of the properties over time. Unity has a built in scroller, but as I wanted the songs to move around the edge of the screen on a curve it wasn’t suitable.


https://docs.unity3d.com/ScriptReference/Input-mouseScrollDelta.html

https://forum.unity.com/threads/setting-top-and-bottom-on-a-recttransform.265415/

https://answers.unity.com/questions/888257/access-left-right-top-and-bottom-of-recttransform.html


Using unity’s mousescrolldelta function to get the distance the scroll wheel had moved since the last frame, I made a test on some rectangles that got them to move up and down as you scrolled. There was an issue when I tried to get it to go forwards and backwards depending on how high it was.

Using an equation made far more sense than what I was doing, and so I started looking into how to find the curve I needed.


https://www.physicsforums.com/threads/finding-equation-of-a-curve-through-5-points.765806/


When mapping out the selection boxes to get the points I wanted to find the curve I realised it would be fine to just do a linear equation and force it to be always positive.


I started adding the song boxes manually but I realised it would be a pain to make them all again when I inevitably had to redo all the songs, so I tried to make a modular system that generates all the songboxes in the correct positions with the right information when the scene loads.


https://osu.ppy.sh/help/wiki/Difficulties

^ to try and find inspiration for difficulty names as I realised a number between 1 and 100 wasn’t a very user friendly way of expressing the difficulty.


https://answers.unity.com/questions/866047/changing-unityuipanel-color.html

https://docs.unity3d.com/Manual/HOWTO-UICreateFromScripting.html

https://forum.unity.com/threads/what-is-a-good-way-to-initialise-or-refresh-gui-panels.266760/

https://gamedev.stackexchange.com/questions/103718/what-are-the-steps-to-instantiate-a-button-in-unity

https://gamedev.stackexchange.com/questions/141489/unity-how-to-make-instantiated-gameobject-to-have-recttransform-of-the-canvas


After researching the best ways to generate the buttons I decided on doing what I did when generating the note objects in the gameplay scene where it creates the rectangle from a prefab and then on that prefab is a script that gets all the relevant information from the song file with the same name. X position is always based on the equation and the y position is just increased by 200 for each song, so they cascade nicely when you scroll.


https://forum.unity.com/threads/sovled-nullreferenceexception-on-raycasts.427478/

https://answers.unity.com/questions/464954/raycast-tag-null-reference-exception.html

https://gamedev.stackexchange.com/questions/148735/why-does-this-raycast-code-give-me-a-nullreferenceexception


I tried to make it so you could click on them to load the songs but it kept erroring, it turns out I forgot to add a component to the canvas.


https://forum.unity.com/threads/audio-not-playing-in-unity-at-all-solved.414941/


Finally I spent 20 minutes trying to figure out why none of them were making any noise when I played them and it turned out I had muted my computer by accident.

12/3/19

Started adding album covers representing the songs when you hovered over the entries. It mostly used knowledge I already had so wasn’t too difficult to add.


https://forum.unity.com/threads/changing-panel-source-image-when-a-button-is-clicked.286287/


Next I started looking into what kind of sounds I should use, and whether I should use hitsounds at all. Since I didn’t have a good environment to record noises on my own I figured I should just find a sound I thought fit and use that.


https://www.reddit.com/r/osugame/comments/1gwkqr/hitsounds_are_actually_important/

https://www.pond5.com/sound-effects/1/rhythm-game.html

https://www.pond5.com/sound-effect/79273788/video-game-rhythm-gallup.html

https://www.reddit.com/r/osugame/comments/3co3tr/best_hitsounds_for_osu/

https://www.youtube.com/watch?v=cJfARi50Q7k


At first I thought I should just not use hitsounds but people seemed to say it was helpful for being able to identify your own rhythm and whether you need to correct when you’re tapping. I ended up using a hitsound from a rhythm game modification skin I had on my computer, as the sound was short and also slightly sounded like one of the instruments associated with zen things. When I’ve got an options menu made I could probably include several choices players could pick from.


https://support.unity3d.com/hc/en-us/articles/206116316-I-am-getting-a-lot-of-sound-latency-when-developing-my-game-for-Android-How-can-I-get-my-sounds-to-play-immediately-


There was some latency issues when I was adding them but they turned out to be from the offset I had added in the gameplay, after removing that the sounds lined up properly with the song.

13/3/19

I started trying to recreate the original main menu mockup I had made with the spinning circles, and add a fade in/out animation to indicate when it’s changing scenes as someone commented it was annoying that it lagged at the transitions but you couldn’t tell.


https://answers.unity.com/questions/1354294/how-to-animate-a-ui-panel.html

https://unity3d.com/learn/tutorials/topics/user-interface-ui/ui-transitions

https://forum.unity.com/threads/making-panels-opaque.522240/

https://forum.unity.com/threads/how-should-i-fade-a-panel.329836/

https://www.youtube.com/watch?v=MkoIZTFUego&app=desktop

https://docs.unity3d.com/ScriptReference/UI.Graphic.CrossFadeAlpha.html

https://forum.unity.com/threads/simple-fade-image-effect-vs-fading-using-ui-panel-performance.505443/

https://forum.unity.com/threads/having-some-serious-trouble-changing-image-alpha.516159/

https://stackoverflow.com/questions/40956407/how-to-fade-in-and-out-a-panel-in-c

https://forum.unity.com/threads/simple-ui-animation-fade-in-fade-out-c.439825/

https://answers.unity.com/questions/863729/ui-panel-bringtofront.html


For fading I ended up using the same function I had made for the hitcircles except it had an inverse too.


https://docs.unity3d.com/Manual/QuaternionAndEulerRotationsInUnity.html

https://docs.unity3d.com/ScriptReference/Transform-rotation.html


I found an example for passively rotating shapes in Unity which worked perfectly for the animation on the main menu.


https://answers.unity.com/questions/993437/rotating-around-a-point-in-a-circular-position-dep.html

https://forum.unity.com/threads/changing-panel-source-image-when-a-button-is-clicked.286287/

https://answers.unity.com/questions/914094/how-to-get-a-certain-child-from-a-canvas.html

https://docs.unity3d.com/ScriptReference/Vector3.RotateTowards.html

https://docs.unity3d.com/ScriptReference/Quaternion.LookRotation.html

https://docs.unity3d.com/ScriptReference/Quaternion.Euler.html


I ended up using some knowledge from maths for the little part of the circle that follows the mouse around, as the examples I found weren’t very similar to what I was looking for. Handily we did the exact method I needed in maths less than a month ago (angle = arctan(y/x)) which gave me the angle I needed to put into the quaternion.euler. Someone commented that it felt like it was moving too slowly so I adjusted the modifier a little bit to make it feel more active too.

20-21/3/19

Looking into the effects of gradients on people to justify using it in a calming/minimalist game, as well as researching what colours would fit as people commented on wanting alternate colour schemes.


https://www.stylus.com/dwmtkb


A hospital charity has started using colour gradients in their “de-escalation rooms” to help soothe and calm patients, showing the psychological effects of gradients.


https://99designs.co.uk/blog/trends/gradient-design-trend/


Gradients are currently trendy in the design industry which helps the game look modern.


http://www.colour-affects.co.uk/psychological-properties-of-colours


Psychological properties of colours - good colours from this list would be:

  • Blue: associated with “reflection, calm”
  • Green: associated with “balance, restoration, peace”
  • White: associated with “simplicity, purity, efficiency”


http://www.arttherapyblog.com/online/color-psychology-psychologica-effects-of-colors/#coolcolors


Blue is said to “decrease respiration and lower blood pressure”, and green can be “less strainful on your eye muscles” because of the way your eye focuses on it, showing physical links to these colours having a calming effect on people.


https://uigradients.com/


I started looking through this site and selecting gradients involving the “cool” colours I had selected before for my colour palette options.


I also started researching calming music in order to find good music and sound effects for the game.


https://www.unr.edu/counseling/virtual-relaxation-room/releasing-stress-through-the-power-of-music


Apparently music “around 60 bpm” synchronises with the brain to relax it.


https://knowingneurons.com/2017/12/21/binaural-beats/


Another method of relaxing music is “binaural beats”, where the left and right ear is listening to difference frequencies causing you to hear the difference which can help you go into a restful state, however there isn’t much definitive evidence on the effects it can have, and there are also warnings like “Please consult your doctor before listening to binaural beats if you suffer from any neurological disease or have had a stroke.” which makes it sound a bit risky, so I probably won’t use these.


https://music.stackexchange.com/questions/22489/the-concept-behind-relaxing-music


It looks like the speed of the music as well as the long droning synth kind of sound makes up a big part of how relaxing it is, so I will try and find music that fits all these kinds of things.


Similarly to the music that’s playable, as I’m not releasing the game I’ll use whatever music is suitable. In an actual game studio I would be able to pay for royalty free music/hire a composer in order to use sounds legally.


https://youtu.be/c1gOBgLVgZo

https://www.youtube.com/watch?v=oINR5JC6Tas


Some examples of ambient slow synth music that fit the requirements.


I also looked into good practices for the survey I’m preparing to ask for feedback with the game.

https://www.gamasutra.com/blogs/NeferDoane/20150316/238780/How_to_Design_a_Survey_for_User_Feedback.php

25/3/19

https://goo.gl/forms/s50bBsi9O6SlMCRo1

Google form I made to send out to people for feedback. Trying to send it to people with a range of rhythm game experience. It’s split into questions about thoughts on the UI and Gameplay that can be put into easy to read graphs, and then a section at the end for unique feedback for more specific things players want.


I sent it to someone highly ranked in the rhythm game osu! I was able to contact. Didn’t do the form possibly because of a language barrier but he sent me this.

https://gameanalytics.com/blog/popular-mobile-game-ad-formats.html

Looking into adverts since the aesthetic of my game is kinda similar to a mobile game.

3/4/19