This project focussed on creating the infamous game “Flappy Bird” and then using Artificial Intelligence to learn and improve to beat the game!
The code below runs through all the classes, and functions defined to create the successful program.
This program imports multiple modules and packages such as pygame, neat, os and more, but essentially Pygame builds on the Flappy Bird game and NEAT is the evolutionary algorithm that creates the artificial neural networks in order to beat the game.
Overview of Neural networks
Neural networks, as a brief overview, come in layers. The input layer consists of the information that is fed into our network. In this case, for every single bird, there are 3 neurons: we input the position of the bird, and the distance from the bird to the top and bottom of the upcoming pipe. The output layer consists of 1 neuron which is simply whether the bird jumps or does not jump. There are also hidden layers within the network which are the secret ingredient to the success of the network (they find the complex patterns which give us the intended output).
Overview (in context)
In order to do this, we have weights on the connections between neurons which represents how weak or strong the connection is. We are using a feed-forward neural network (connections between neurons do not form a cycle – information only travels forwards). Therefore, with each of these weights on the connections, we calculate a weighted sum which is where you take each weight and multiply it by its corresponding input value. Next, we add a bias to this weighted sum, which is required to tweak the sum into the correct position.
Finally, we use an activation function which is what evaluates our model and decides on the next course of action. In our case, we use a F = tanH activation function (which is a hyperbolic tangent function). As shown in the graph below, this function gives us an output between -1 and 1, which means we can evaluate our output neuron. If the activation function returns a value of greater than 0.5 then we make the bird jump, else it does not jump. As simple as that!
NEAT Overview
You may be wondering what NEAT is and how it works?
NEAT stands for NeuroEvolution of Augmenting Technologies and ultimately, it is an algorithm that gradually learns and picks up patterns to progress through the game. After a few iterations, the algorithm exponentially develops, until it is unbeatable (for a simple game like Flappy Bird, this can be very fast…)
NEAT is inspired on natural selection (Charles Darwin’s theory of natural evolution), which reflects how the fittest individuals are selected for reproduction to produce offspring of the next generation. Similarly, in our version of flappy bird, NEAT initialises the algorithm with a large population of birds, and assigns a neural network to each bird (known as a genome). NEAT allows this population, from the first generation, to attempt the game, and next it assesses and evaluates which genome performed best in the population.
These select-few genomes (birds) are allowed to breed and mutate, and thus they create a brand-new population, which will perform better than the previous generation since they are the offspring from the top-performing birds. (see flowchart below)
A vital part of this entire program is the fitness function which is a way of evaluating how good the birds within a population. If this “scoring” of the birds does not make sense, then the entire Artificial Intelligence process will fail, since it cannot learn from its mistakes. Even more so, the process of “survival of the fittest” will fail, since we will not be able to know which are the “fittest” birds out of the population.
Our fitness function is simple: whichever bird is able to travel the greatest distance within their population gets the greatest fitness score. This is because the NEAT algorithm will assign a random neural network to every bird, and will try different combinations of the weights and add new neurons. The best neural network for the Flappy Bird game will result in the greatest fittest score.
Results & Testing Stages
The NEAT algorithm found this game very easy. Even with a relatively small initial population size of 100, the algorithm created every single possible neural network which meant that out of the 100 initial birds, 2 birds ended up beating the game in the first generation! This is remarkable. (See video below)
Therefore, I tested the algorithm’s effectiveness with a lower population size. Having now reduced the population size to only 20, the effect of neural evolution truly came into play. The algorithm was unable to create an unbeaten bird until the 8th generation, which is a significant increase from the previous population size. (see video below)
I have also added a statistics section while running the program to see how the model is performing. For example, the information given is age, fitness score, generation number, run time etc.
The Code
Please see below for the code. There is a Generation counter in the top left, and a Score counter in the top right (whenever it passes a pipe successfully).
There are comments at each of the key pieces of code to help you understand the purpose of that line. (please note: this project was run locally on the PC, and all modules and packages as well as other images and files were downloaded on the laptop.)
Thank you for reading, and please leave any comments or questions below!
References
- Many thanks to Tech with Tim’s YouTube channel and his tutorial for this program
- NEAT documentation
