Automatic Pose Analysis

One year I decided to look into machine learning and went through a course on it to see how I could apply it to analyze the animations of the game. I thought it would be cool to have automatic analysis that would be able to spot bugs automatically.

I spent the week learning about the various tools that many other people use. It's a course that Google put together that was quite nice because it had workbooks integrated with the course material so you could do tests right there with the text.

Machine Learning Crash Course

To feed machine learning with data you can package it up as chunks of data and having a standard set is key. But essentially what you do is identify the core joints in your skeleton and output them as poses. You want to tell the algorithm what is a good and what is a bad pose.

It's easy to feed it with good poses as that's just all the source data so you can fill that quite easily. What's not as easy is to tell it what is a bad pose and you can do that manually but you really need to feed it with a lot of bad data to let it know what bad looks like.

So you automate that process as well. I wrote code that finds the bad poses in the game based on some heuristics, finding joints that are twisted in the wrong direction. Finding where some joints have abnormal acceleration or velocity.

You can also utilize physics and find poses where the ragdoll is colliding with itself.

So I added all of those and you can easily figure out some other rules for the types of errors your game characters can run into.

So then I was going to train the machine learning to find these poses and it wasn't really working and then as an additional step I would have to deploy and run this while the game was running.

Or.

I could just run the functions that was feeding the machine learning with bad poses instead at runtime. It's fast code. It's easy to understand. It was already working. It wouldn't have any false positives which can be a danger with machine learning.

The last step in automatic routines is how do you output that in a format that is accessible by everyone and if you're lucky you might already have the analysis tools for presenting logging information for performance or memory profiling.

So what I did was to add another table where I output each type of bad pose cause. This is great, we can know when the character is in a bad pose.

Nice.

Well not that nice, sure we can know it was a bad pose but we can't output the pose or the entire state that caused that. What we can do is to output the animation transitions that have happened on another track.

So I started down one path, learned something. Tried to apply it and found a more optimal approach in solving my problem and could deploy it in a way that would help me find problems.

This is maybe just me but I do think we need to think about why we're doing something and if we actually need to do that thing we initially thought of. What if during that process we found a simpler and better way?