References

Bevilacqua, F. (2013, October 24). Finite-State Machines: Theory and Implementation. Retrieved from: https://gamedevelopment.tutsplus.com/tutorials/finite-state-machines-theory-and-implementation--gamedev-11867

Bevilacqua, F. (2013, October 24). Finite-State Machines: Theory and Implementation. [Online Image]. Accessed: [1 November, 2017]. Available from: <https://gamedevelopment.tutsplus.com/tutorials/finite-state-machines-theory-and-implementation--gamedev-11867>

Champandard, A. (2007, September 2). Strategies for Implementing Cover Behaviors. Retrieved from Aigamedev.com: http://aigamedev.com/open/article/cover-strategies/

Computerphile (2017). A* (A Star) Search Algorithm - Computerphile. [Online Image]. 4 January 2017. Available from <https://www.youtube.com/watch?v=ySN5Wnu88nE> [Accessed: 1 Novermber 2017]

Computerphile (2017). A* (A Star) Search Algorithm - Computerphile. [Online Video]. 4 January 2017. Available from <https://www.youtube.com/watch?v=ySN5Wnu88nE> [Accessed: 1 Novermber 2017]

Computerphile (2017). Dijkstra's Algorithm - Computerphile. [Online Image]. 4 January 2017. Available from <https://www.yout.ube.com/watch?v=GazC3A4OQTE> [Accessed: 1 Novermber 2017]

Computerphile (2017). Dijkstra's Algorithm - Computerphile. [Online Video]. 4 January 2017. Available from <https://www.yout.ube.com/watch?v=GazC3A4OQTE> [Accessed: 1 Novermber 2017]

Deangelis, S. F. (2017). Artificial Intelligence: How Algorithms Make Systems Smart. Retrieved from Wired.com: https://www.wired.com/insights/2014/09/artificial-intelligence-algorithms-2/

Dennis. (2008, June 2nd). The Difference Between Dijkstra’s Algorithm and A*. Retrieved from Allmybrain.com: http://allmybrain.com/2008/06/02/the-difference-between-dijkstras-algorithm-and-a/

Estrada, M. (2014, February 23). Finite State Machine vs Behavior Tree, A True Story. Retrieved from Coffeebraingames.com: https://coffeebraingames.wordpress.com/2014/02/23/finite-state-machine-vs-behaviour-tree-a-true-story/

Freiberger, M. (2012, October 22). The travelling salesman. Retrieved from Plus: https://plus.maths.org/content/travelling-salesman

Greenawalt, D. (2013, October 25). How Forza 5 and the Xbox One use the cloud to drive machine-learning AI. (K. Orland, Interviewer)

Headlessprofessor (2010). Heuristic vs. Algorithm. [Online Image]. 16 July 2010. Available from <https://www.youtube.com/watch?v=jJ3CV3yhajM> [Accessed: 1 Novermber 2017]

Headlessprofessor (2010). Heuristic vs. Algorithm. [Online Video]. 16 July 2010. Available from <https://www.youtube.com/watch?v=jJ3CV3yhajM> [Accessed: 1 Novermber 2017]

Juliani, A. (2017, June 26). Unity AI-themed Blog Entries. Retrieved from Unity: https://blogs.unity3d.com/2017/06/26/unity-ai-themed-blog-entries/

Kehoe, D. (2015, January 1). Designing Artificial Intelligence for Games (Part 1). Retrieved from Intel.com: https://software.intel.com/en-us/articles/designing-artificial-intelligence-for-games-part-1

Linneman, J. (2014, June 1). Do higher frame-rates always mean better gameplay? Retrieved from Eurogamer.net: http://www.eurogamer.net/articles/digitalfoundry-2014-frame-rate-vs-frame-pacing

Makuch, E. (2016, September 23). Civilization 6 PC System Requirements Announced. Retrieved from Gamespot: https://www.gamespot.com/articles/civilization-6-pc-system-requirements-announced/1100-6443866/

Makuch, E. (2016, September 23). Civilization 6 PC System Requirements Announced. [Online Image]. Accessed: [1 November, 2017]. Available from: <https://www.gamespot.com/articles/civilization-6-pc-system-requirements-announced/1100-6443866/>

Mark Brown. (2017). What makes Good AI? [Online Video]. 31 May 2017. Available from <https://www.youtube.com/watch?v=9bbhJi0NBkk> [Accessed: 1 Novermber 2017]

Mick maus (2017). How do Boids Work? A Flocking Simulation. [Online Video]. 30 August 2015. Available from: <https://www.youtube.com/watch?v=QbUPfMXXQIY. [Accessed: 5 November 2017].

Moore, K., & Ross, E. (2017, November 5). Greedy Algorithms. Retrieved from Brilliant.org: https://brilliant.org/wiki/greedy-algorithm/

Moore, K., & Ross, E. (2017, November 5). Greedy Algortithms. [Online Image]. [1 November, 2017]. Available from: < https://brilliant.org/wiki/greedy-algorithm/>

Orland, K. (2013, October 25). How Forza 5 and the Xbox One use the cloud to drive machine-learning AI. Retrieved from ars technica: https://arstechnica.com/gaming/2013/10/how-forza-5-and-the-xbox-one-use-the-cloud-to-drive-machine-learning-ai/

Pearl, J. (1984). Heuristics: Intelligent Search Strategies for Computer Problem Solving. In J. Pearl, Heuristics: Intelligent Search Strategies for Computer Problem Solving (p. 3). Los Angeles: Addison-Wesley.

Reynolds, C. (2001, September 6). Boids. Retrieved from red3d.com: https://www.red3d.com/cwr/boids/

Rouse, M. (2016, August). fuzzy logic. Retrieved from whatis.com: http://whatis.techtarget.com/definition/fuzzy-logic

Rouse, M. (2016, July). Neural Network. Retrieved from Whatis: http://searchnetworking.techtarget.com/definition/neural-network

Simpson, C. (2014, July 7). Behavior trees for AI: How they work. Retrieved from Gamasutra: https://www.gamasutra.com/blogs/ChrisSimpson/20140717/221339/Behavior_trees_for_AI_How_they_work.php

Simpson, C. (2014, July 7). Behavior trees for AI: How they work.. [Online Image]. Accessed: [1 November, 2017]. Available from: <https://www.gamasutra.com/blogs/ChrisSimpson/20140717/221339/Behavior_trees_for_AI_How_they_work.php>

Soren D. (2017, August 13). Teaching an AI to play a simple game using Q-learning. Retrieved from Practical AI Learning: https://www.practicalai.io/teaching-ai-play-simple-game-using-q-learning/

Tangness, D. (2014, November 6). Game Analysis - Guard AI in Splinter Cell: Blacklist. Retrieved from Davidtangness.con: http://www.davidtangness.com/journal/2014/11/6/game-analysis-guard-ai-in-splinter-cell-blacklist

Thompson, T. (2016, February 26). AI 101 - Part 10 - A* Search. [Online Image]. Accessed: [1 November, 2017]. Available from: <http://aiandgames.com/ai-101-part-10-search/>

Thompson, T. (2016, February 26). AI 101 - Part 10 - A* Search. Retrieved from Davidtangness.con: http://www.davidtangness.com/journal/2014/11/6/game-analysis-guard-ai-in-splinter-cell-blacklist

tvtropes. (n.d.). The All-Seeing A.I. Retrieved from tv tropes: http://tvtropes.org/pmwiki/pmwiki.php/Main/TheAllSeeingAI

YongYea. (2017) Metal Gear Solid V - Ocelot on Codec, Wall Knocking Addressed, and More!. [Online Video]. 30 March 2014. <https://www.yout.ube.com/watch?v=GazC3A4OQTE> [Accessed: 1 Novermber 2017]

Learning Techniques

Machine Learning

Learning techniques, also known as Machine Learning (ML), is a way of teaching machines to learn by itself. Machine learning strategies involve implementing an algorithm by giving it a set of guidelines that it must follow. Machine learning techniques have only become so popular in games within the last few years, but they are a better focus for games that have more complex AI as “most game AI that currently exists is hand coded, consisting of decision-trees with sometimes up to thousands of rules. All of which must be maintained by hand, and thoroughly tested.” (Juliani, 2017). This is highly inefficient from a programming point of view -having to code the behaviour manually of each individual AI. ML allows AI to be maintained by adding in generic guidelines of behaviour, and only telling the AI that good result should be done again, whereas poor result require further solutions.

Q Learning and Artificial Neural Networks

The Q-learning algorithm is “The Q-learning algorithm is a reinforcement learning algorithm. Reinforcement learning algorithms are a set of machine learning algorithms inspired by behavioral psychology. The basic premise is that you teach the algorithm to take certain actions based on prior experience by rewarding or punishing actions. Similar to teaching a dog to sit by giving it treats for good behaviour.” (Soren D, 2017). It’s essentially a trial and error algorithm which is able to learn from past situations.

In an interview with (Greenawalt, 2013), (Orland, 2013) asks him about the driving game Forza and how the AI is adapting its behaviour based upon player data. Greenawalt explains that the AI is gathering data not just from the player’s point of view, but from all cars on the track. When explaining an overtaking manoeuvre, he explains: "Why would you dodge all the way to the outside of the tracks? Is it because you're a bad driver or because you're going around somebody? It's just data, but we over-record the data because it's possible that any of those other pieces of information, like where the other cars are, will be important to the actual data itself." All of these data points are crucial in defining the behaviour of the AI. They can analyse how players use the track in order to performing overtaking manoeuvres, and that the AI can learn from these trends and use them to block the manoeuvre.

The way in which the Forza AI is learning is called an Artificial Neural Network (ANN). According to (Rouse, Neural Network, 2016) the definition of a neural network is “a system of hardware and/or software patterned after the operation of neurons in the human brain. Neural networks -- also called artificial neural networks -- are a variety of deep learning technologies”. These ANNs are used in many applications and have recently made an appearance in the gaming industry.

Hardware Constraints

Hardware constraints must be considered with all implementations of the above techniques. Every AI technique is just more code for the CPU to process. The more calculations that are being made in a game, the higher the toll on the CPU and GPU. Therefore, franchises such as Age of Empires implemented population caps. Some of the reason these caps exist is from a gameplay side, but hardware consideration comes into it. Another way to think about it is that you wouldn’t consider trying to play an intensive PC game such as Civilization VI on a mobile device. (Makuch, 2016) outlined the technical specifications which were released by the developers, 2K [fig. 8]. These are the minimum and recommended requirements for the game. Civilization VI is a game that requires good processing power due to all the calculations made by the AI.

Games should be as optimised as they possibly can to run on a variety of systems, from low range to high range. 60 frames per second (Frames Per Second) is the common framerate to run at for most games nowadays, with some games on consoles previously have been capped at 30FPS. It’s not fun playing a game where it feels more like you are scrolling through a presentation on PowerPoint, constantly dropping frames. This incurs feeling of frustration but more importantly, it breaks immersion. It’s hard to lose yourself in a game for a long period of time when you can feel every frame, or rather every frame-drop. (Linneman, 2014) explains the concept of locking a frame-rate: “a locked frame-rate produces steady input and consistent visual feedback, allowing a stable experience across the board”. Locking the frame rate is a good optimisation strategy, it conserves memory at the cost of visual enhancement. Typical refresh rates of monitors are only designed to run at 60FPS (60Hz). This means that for some games, locking the framerate is an ideal strategy. It helps to reduce the number of frame-drops, especially if the AI in said game is making lots of calculations in small spaces of time. Frame drops don’t just affect the game visually either, it can lead the game to be unresponsive in terms of user input. (Linneman, 2014).