Games needed a way to organize animations and to control the playback of these animations in a designed fashion.
Animation State Machines or Animation Graphs became the solution for many game engines and middleware.
We attempted to add one to UE3.
UE3 had an animation tree for character animation, with an animation tree you can set up layered blending and animations that are activated contextually. With a tree it is hard to set up sequencing and manage conditions so generally what you want on top of that tree is a state machine.
The engine also had a graph editor interface for various editors, one being used for their visual scripting language Kismet. The precursor to Blueprint.
When you work on games, you never have enough time to do things in the best way possible. You are expected to deliver things continually and while tools are very important to make games they are generally not given as much as time as they need and they need to be continually developed.
So you take shortcuts.
We needed a way to design the state machine and one of the easiest editors in Unreal to extend was Kismet but we didn’t want to expose all the logic of Kismet to the animation state machine. Similar to blueprint code in UE4 a lot of that code is not safe to be used by an animation update.
So Kismet became the container of data that was built into a state machine. The connections between nodes defined the available transitions between animations, the parameters on the node defined animations to play and how.
The small extensions to Kismet that this required for easier use by animators became improvements that also helped the general kismet editor. Improved search and find functionality and specific animation state preview functionality.
It was a relatively small investment of time for functionality that let the animators describe states with animations and lists of animations.
With Unreal you have garbage collection on objects so you can easily allocate and throw away objects any way you like. This is not a good idea for anything that requires performance or stable memory usage which is the case for all games.
So you allocate pools in the systems that need it and you generally try to avoid dynamically resizing arrays.
The animation state machine needed a way to talk to the animation tree and we could achieve that with a simple communication that told the animation nodes in the tree to change their selected child.
This meant that we had a way to control flow from outside the animation tree and we could also preview and visualize our custom paths in the tree for the different contexts.
However we also wanted a way to play the different gameplay animation states that didn’t require custom paths in the animation tree and we achieved that by adding a buffer of available children to the blend nodes. These children supported simple things like adding a new animation sequence or some simple blends. Also instead of being thrown away they were simply reused after their blending was finished and another animation needed to be played.
Since this was part of the tree it also behaved in ways that was better than the montages and slots that Unreal now uses to solve a similar problem. We could set up layering inside the node, montages in UE4 only support playing one animation at a time.
Because we did not use Kismet code for the graph we had cut off the logic part of an animation graph. This is something that current Animation Graphs do not do, they rely on being able to set up logic in the graph, except if you care about performance and thread safety they do not allow you to execute any complex code and this is true for several animation engines that I know of.
Instead the conditions in these engines are just basic variables.
Our graph did not have that and instead the gameplay system had a request system where code could send requests for states in the graph and with that request they could send in the context for the state. This context included variables for blend children, alignment targets and so on.
The request also had callbacks for every step that it took, starting animations and activating transitions and reaching the end. The manager for the state machine would then do a search for how to get from the current state to the target.
Having worked on these more advanced animation systems later I’ve come to miss that contained context for the request. Our requests had a lifetime that lasted until they had reached the target state so any transitions to the target had the same variables and information as the target and the information remained until the target started playing. This meant that a blend parameter would carry through within the transitions and the target animation.
An Animation Graph is a hierarchical state machine, it allows you to create state machines that have inner state machines and it allows you to create a tree connecting these.
Having seen animators and designers use animation graphs they can easily go down a rabbit hole and never come up again. They design themselves into a corner and become afraid to attempt to alter their hierarchy.
We initially had set this up so that we had two state machines, one for the full body and one for layered animations. This was hard coded and designed for shooters, first person and third person. We knew what we needed from the beginning so we took another shortcut. The layered animations could be of different types so we could control the per bone blending.
However, later on we also wanted to improve the code and add support for a hierarchy. It was easily possible to inject states into a hierarchy but we found that the complexity of the editor that wasn’t designed for this became messy for users so we dropped the feature.
Instead for the use case where we actually wanted a state machine inside itself we created a way to define a complex node that generated states.
Both solutions were flawed and caused animators to avoid using the feature and as time was limited we had to rely on me to set up those few cases that required it.
Looking back I think we had a system that gave us the functionality we needed and we had the knowledge of how to use it, modify it and fix the problems that arose. That’s generally the benefit of tech that you write on a team but that additional tech comes with a maintenance cost that can become expensive.
We handled it because key people stayed on the project for the duration, the less risky approach would have been to go with a middleware solution. As I’ve mentioned in other posts middleware solutions come with their own problems and with the limitation that the people using the middleware will not know all the ins and outs of it and are thus unlikely to get the best results unless they spend the time.
Sometimes the custom solutions can yield better results than the middleware as it is not as heavily constrained and the tech knowledge is on the team. It is just a question of where do you want to spend your time.
After all the goal is a good end result and you achieve that with good tools and people.