Keeping Track

Keeping Track - A Train Game

Keeping Track is a voice-controlled game where users play as a co-conductor of the MTA. Using their co-conductor (voice bot) for audio clues of which stop the train is arriving, the user has to guide passengers off the train correctly by broadcasting their voice commands.

Github: https://github.com/PaulineOC/ITP-HelloComputer-HW4

Ideation

Originally, for my final I had two ideas:

  1. A game similar to The Sims where the user commands an avatar, but eventually the avatar gets fed up and doesn't obey

  2. A math quiz game where users can either type out the answers to math questions (but would have to spell it out so "3" would be "t-h-r-e-e-") or verbally speak them.

I realized quickly afterwards that I hated these ideas because while I liked the messages that they were exploring (a voice bot that has its own patience threshold, shouldn't be taken for granted versus a circumstance where voice technology is actually faster than typing), the actual system/experience seemed severely lacking. The system of the first idea was too general, and I had trouble thinking of a concept that made sense where the computer could get tired in a new way (Sims already get upset after a while, so this didn't feel satisfying enough to me). In the second idea, I found the concept too small and got quickly bored with it.

Then on my way home, I was inspired by what would become my final project idea - how inaudible train conductors are on the MTA and how difficult it must be for passengers not familiar with train system to navigate successfully if depending solely on the conductor's announcements.

V1 - Rough Gameplay

The first iteration of Keeping Track, I just implemented the voice commands and the basic game system.

I set-up the text-to-speech, and speech-to-text, as well as DialogFlow to track voice entities.

I also created the preliminary game structure where the game would have different phases: train is between stations, train is at a station, and train is at the end of its line. The game would flip between these stations based on two timers for the "train is at the station" and "train is between stations" states. I created a global object that stored the list of stations and how much time it would take to get to each stop.

In this prototype, there were no visuals, just text feedback. I also set up the system where if the user says a passenger's name correctly, they would "leave the train." The names of each passenger I pulled from a free api and I filtered out any non English characters to make the game slightly easier.

When setting up the assets, I decided to use Pixi.js for my framework since it was more lightweight for my gaming than Phaser.io (which included physics).

V2 - Incorporating Assets

When i was thinking about my idea 1 for the final (where the avatar fights back), I came across these adorable sprites that I thought could be fun to incorporate: https://shubibubi.itch.io/cozy-people. I can't draw, so I bit the bullet and purchased the asset pack.

I needed to isolate each sprite from the sprite sheet, and that took a lot longer than I expected. Since I wanted to generate random characters per each game by randomizing each characteristic (skin type, hair, top, bottom), I needed to isolate the sprites for each variant of the characteristic I wanted.

First, I used the Sprite Sheet Splitter feature of the software TexturePacker. I knew I didn't want to use animation just yet, so I had to isolate the sprites I wanted from the first frame of the walking animation.

Then I was left with tons of files because each type of sprite (like the blue pigtails hair) would have 8 frames per animation, and each animation type had 4 orientations (front, back, left, right). Thus there were a total of at least 32 sprites per animation type! I only wanted the sprites for the first frame of the front, left, and right walking animation (though later I realized I would only use the front orientation), so I ended up writing a script to help optimize this process. The split sprite sheets had their files indexed programmatically by TexturePacker, so I knew I could use this to my advantage. My script essentially read in the directory with the raw sprites and would delete any whose filename was not a multiple of 8. The reason I did this was because I knew any file whose filename was a multiple of 8 would be the first frame of the animation.

I realize afterwards that I could've refined the script to delete all files BUT those whose filenames contained 0, 8, or 24 since those numbers correspond to the first frame of the walking-front, walking-left, and walking-right animations. However, I didn't realize that until it much later...but oh well.

For the background, I ended up splicing together two images I found online:

Once my sprite assets were ready, I ended up spending a lot longer than I wanted on the loading of the sprites into my game system because I had to change how the assets were loaded into the system.

Sprite Sheet Splitter Feature of TexturePacker

My Cleaning Python script

Subway interior

Subway station

After much grueling effort, I managed to successfully import one naked person...

V3 - Another Prototype

For the next iteration of the game, that I presented in class, I implemented other features:

  • Place the passenger sprites in the correct position with their name and destination written above their head

  • Use only first names for the passengers to make saying names easier

  • Created a basic scoring mechanism

  • Cleaned up the "train is at the end of the line" state and the game end state

  • Cleaned up text captions for what is being said and scrambled what the conductor is saying

  • Added a Levenshtein distance function to handle the recognition of names. For example, if the computer detected "Billy" was spoken by the player, and a passenger's name was "Billy," the passenger should still leave the train

  • Cleaned up the actual HTML and CSS of the page

  • Implemented rough draft of the instructions

The actual game lives here: https://itp-hello-computer-final.herokuapp.com/.

However, since Google recently sent an email flagging my Dialogflow project, my project has been weirdly suspended? I'm not sure if it impacts the live demo, so just in case I included video documentation above.

V4 - Future Ideas for Later Iterations

Future ideas I had for the project:

  • Service announcements that affect stops and fully flesh out the local vs. express system feature (right now those flags are used to determine how many passengers have a certain stop for its destination)

  • Ensure that for every stop, there is a passenger with said stop as their destination

  • Add animations and emojis of passengers - ie show visual feedback that they have missed their stop

  • Fix scoring system

  • Perhaps a feature that allows you to choose which train line you'd like to be the conductor of

Feedback I'd definitely like to incorporate also in future iterations:

  • Other commands that really simulate the experience of being an MTA conductor: "Stand clear of the closing doors please"

  • Sounds for when the train is leaving/entering the station