Learning Techniques

Machine Learning

Learning techniques, also known as Machine Learning (ML), is a way of teaching machines to learn by itself. Machine learning strategies involve implementing an algorithm by giving it a set of guidelines that it must follow. Machine learning techniques have only become so popular in games within the last few years, but they are a better focus for games that have more complex AI as “most game AI that currently exists is hand coded, consisting of decision-trees with sometimes up to thousands of rules. All of which must be maintained by hand, and thoroughly tested.” (Juliani, 2017). This is highly inefficient from a programming point of view -having to code the behaviour manually of each individual AI. ML allows AI to be maintained by adding in generic guidelines of behaviour, and only telling the AI that good result should be done again, whereas poor result require further solutions.

Q Learning and Artificial Neural Networks

The Q-learning algorithm is “The Q-learning algorithm is a reinforcement learning algorithm. Reinforcement learning algorithms are a set of machine learning algorithms inspired by behavioral psychology. The basic premise is that you teach the algorithm to take certain actions based on prior experience by rewarding or punishing actions. Similar to teaching a dog to sit by giving it treats for good behaviour.” (Soren D, 2017). It’s essentially a trial and error algorithm which is able to learn from past situations.

In an interview with (Greenawalt, 2013), (Orland, 2013) asks him about the driving game Forza and how the AI is adapting its behaviour based upon player data. Greenawalt explains that the AI is gathering data not just from the player’s point of view, but from all cars on the track. When explaining an overtaking manoeuvre, he explains: "Why would you dodge all the way to the outside of the tracks? Is it because you're a bad driver or because you're going around somebody? It's just data, but we over-record the data because it's possible that any of those other pieces of information, like where the other cars are, will be important to the actual data itself." All of these data points are crucial in defining the behaviour of the AI. They can analyse how players use the track in order to performing overtaking manoeuvres, and that the AI can learn from these trends and use them to block the manoeuvre.

The way in which the Forza AI is learning is called an Artificial Neural Network (ANN). According to (Rouse, Neural Network, 2016) the definition of a neural network is “a system of hardware and/or software patterned after the operation of neurons in the human brain. Neural networks -- also called artificial neural networks -- are a variety of deep learning technologies”. These ANNs are used in many applications and have recently made an appearance in the gaming industry.