(Side note: The game is meant to be played either standing or sitting, we were kneeling during the filming of demonstration in order to prevent blocking the screens. The pedal for missile launching was slightly damaged by kids during the IMA Show, so it wasn't working 100% properly during filming.)

After learning Processing in the second half of the semester, I was intrigued by the graphical capabilities of the language, and how it could be paired with the Arduino to invoke a whole new level of interaction. So for our final project, Jingjing and I wanted to create something more fun rather than a static or plain art piece, so we decided on making a game. 

During the brainstorming phase, we thought off all sorts of games, including a dance game inspired by Dance Maniax, a cooking game inspired by Papa's Pizzeria, a double player version of Gold Miner and much more. Our concept finally solidified after we discussed our ideas with Andy, who suggested that we could make a 2-player game which relied on communication.

Credit: Toronto Metropolitan University

Introducing communication as a form of interaction seemed innovative to us. Having an aspect of interaction depend of the chemistry between the two participants seemed risky, but it also made the interaction feel a lot more organic and natural. After further discussion, my partner and I decided that we wanted our game to be more intense and instantaneous compared to similar games in the genre, such as Codenames, and ended up settling down with a side-scroller game. Since we needed multiple forms of input for each player, we also aimed to make the controls be more immersive through the combination of digital fabrication and sensors.

Game Concept drawn by Jingjing Wang

The concept of Space Hunters is that Player A can control the vertical movement of the avatar of both players and see enemies, while Player B can shoot projectiles to defeat the enemies and see obstacles, which the players must avoid. (We also implemented features, such as enemies randomly moving a few titles up or down, as well as obstacles having randomized move mechanics, and stamina and ammo restrictions, meaning that the Player A would have to refill their stamina when it runs out to move, and Player B would have to reload to shoot more projectiles.)

Before finding the assets that we needed to style the game, we used simple shapes in Processing, such as rect(), to test rendering and collision, and used the PApplet Library in order to have multiple windows.

For the first two days, Jingjing drew the background image for the game and worked on the movement mechanics of the game and the Arduino code for sensors (we used a Joystick module, a foot pedal and a rotary encoder),  while I worked on implementing collision and projectile logic in Processing, as well as code for the flow sensor we were initially using. Player A was supposed to control movement using the Joystick and regain stamina by breathing (blowing into the flow sensor), while Player B fired projectiles using the foot pedal and reloaded with the rotary encoder.

Over the course of the next few days, we started replacing the simple shapes with assets we had found on itch.io and freesound.org

Compared to our midterm project, our game felt a bit more polished during the user testing phase, but a big problem was that it was time consuming to relaunch the game and drag the window to the extended display every time the game needed to be restarted. 

Aside from the problem of restarting the game, several of the testers voiced concerns on whether or not using a flow sensor was sanitary, since they felt since it was so hard to trigger, it would be full of everyone's saliva by the end of the IMA Show. I also noticed that the controls and concept of the game were a bit too hard for some of the users to grasp. So my partner suggested that we implement a tutorial level before the main game.

Since the Final Presentation and User Testing Session were just 9 days apart, I mainly focused on allowing players to restart the game without restarting the whole program, designing a simple tutorial level, as well as designing a pickup system (this ended up being scrapped) which I had always wanted to implement, while Jingjing focused on digital fabrication since she had much more experience than me on the topic. I helped Jingjing look for 3d models online that we could use, while she started brainstorming on what we could use to replace the gross flow sensor. In the end, we decided on using a tilt sensor and putting it in a 3d printed water bottle to give the sense of a person drinking water.

During this process, I also modified an old version of our code for collision testing, since it was hard to work out once you introduced sprites and other types of assets to the game.

However, a few days into the week, Andy stated that he thought the input methods, especially the foot pedal, weren't immersive enough, since the character in our game used a bow and arrow rather than anything to do with a foot. I thought of how we could replicate the feeling of using a bow with fabrication and sensors, including using an FSR Sensor to simulate someone pulling back the string, and distance sensors for drawing arrows from the quiver. But since we already had artifacts that were half-way through printing and had already spent so much time gluing our components together, I proposed to Andy that we just change the in-game theme to fit the input, to which he approved.

We decided to shift our theme to space (Jingjing drew a new background), since it felt more reasonable for someone to be able to shoot a rocket with their foot if they were in a sci-fi spaceship. I took the changing of assets as an opportunity to animate the sprites, so I asked our Fellow Kevin on advice on iterating through sprite sheets since he has experience in the game engine Unity. Since Processing has no functions to iterate through sprite sheets automatically, he suggested that I just crop the sprite sheets into single sprites and swap the image being displayed every few milliseconds.

When setting up the controls for testing, I felt that the small components were hard to carry around, so we designed a tray for faster setup.

Shortly after this, Jingjing sent me 3 tutorial videos she had drawn for the players to watch before each game. I tried implementing them, but the videos would lag the game, even after converting the files to H.264 format, and the game would freeze if the video had finished playing. 

I asked Fellow Angy to help me out with this problem, and due to the spaghetti code from deadlines and crunch, it ended up taking an hour even though the solution was just changing an else if statement, to an if, and moving a video.loop() call from draw() to setup(). Professor Viola (who was passing by), suggested that I put the logic to toggle on and off each Game/Tutorial State at the top of the draw() function rather than at the end of each game state. So I decided to reformat my code from scratch using object oriented programming before adding any other features, so that it wouldn't break as easily. I also took this as a chance to implement a feature where the obstacles would move in a random direction when approaching the player, as well as life steal when the player killed 3 aliens.

However, pulling an all-nighter reformatting the code, I realized that sprites couldn't be rendered onto the second window. I reached out to Kevin for debugging, and the verdict was that I needed the argument PApplet parent to render everything in the second window, otherwise the objects of each class would render in the window that they were declared in.

Finishing my code, I helped my partner make a plastic cup holder to hold the 3d printed cup and NeoPixel lights, which he had planned on to glow different colors depending on how much stamina the player had, but we couldn't realize this since we couldn't figure out how to send signals both ways at the same time (Processing to Arduino & Arduino to Processing).