AAI

Actions that AGAI can utilize are limited to actions it learned from its trainer. Human players can experiment with new moves and add a move to their arsenal if the move is observed to be effective. Sticking too close to the trainer’s style can also limit the AI’s performance. Therefore a different approach from AGAI is required. 

Ghost AI was modified to create Automatic Adaptive AI (AAI). AAI combines Ghost AI and AI that explores new/unused actions. Learned actions are chosen to be executed according to their performance scores. No action frequency from its trainer is considered because scores are more directly mapped to performance. AAI therefore does not preserve its trainer’s style but still learns actions from the trainer. 

AAI can also explore new actions. In a game state, the AI can choose to execute an action that has never been used by its trainer/opponent. If the newly experimented action successfully damages an opponent, the action gains points and its probability value goes up. If the experimented action is guarded, it gets fewer points. If the action gets counterattacked, the points received become negative.

With this method, new actions are added to the AI knowledge base and the gameplay strategy of the AI can be extended.

Pictures below illustrate how imitation works for AAI. The character on the left is controlled by the AI and the character on the right is controlled by a human player. When the AI tries to execute a punch, it is counterattacked with a high kick. The player gains points and the AI remembers this situation. Later on, when the player tries to punch, the AI can choose to counterattack with high kick because it knows it will gain points. The frequency of high kicks the player has used to counter punches does not matter. However, some high kicks that somehow do not land successfully (it is possible due to some random timing in the game) will reduce the overall points.

At the same time that the AI learns to copy the player’s counterattack, it also learns that when it attempts to throw a punch, the player will most likely counterattack with a high kick (because it rewards points). When throwing a punch, the AI can choose to try to change its move to counter/defend against the player’s move, by executing a new action. In the pictures below, when throwing the punch, the AI quickly detects its vulnerable state and immediately performs another action (which happens to be a low kick) to get out of its risky position. This low kick turns out to be effective since the AI manages to avoid damages from the player and scores against him at the same time. The AI will then record the situation and give a good probability value for the low kick.

Source code of AAI is available in download & instruction section.