A neural network is inspired by how neurons in the human brain process information. It is structured by interconnected artificial neurons, each taking weighted inputs, applying a nonlinear activation, and passing the result forward. By tuning the weights of each connection, the network learns useful patterns and behaviors.
Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is an evolutionary optimization method inspired by natural traits of evolution such as selection, mutation and adaptation. It works by sampling candidate solutions from a multivariate normal distribution, evaluating their performance, and adjusting the multivariate normal distribution’s mean and covariance to favor more successful regions of the search space. By continually updating these parameters, CMA-ES can efficiently explore high-dimensional search spaces and converge toward high-performing solutions.
In our approach, we utilize CMA-ES to evolve the weights of a neural network controller for Planet Wars. The neural network receives the current game state as input and outputs a target planet for sending ships. The network also has an option to not do any action and an output to decide the ratio of ships to send. Each individual in the CMA-ES population corresponds to a full set of network weights and biases, and evaluated by playing 100 games against the baseline greedy heuristic agent. The fitness of an individual is the proportion of games won, which allows CMA-ES to favor individuals that perform better.