Creating AI for GameBoy Part 1: Coding a Controller

Creating AI for GameBoy Part 1: Coding a ControllerAaron FrederickBlockedUnblockFollowFollowingJan 22Released in 2003, Fire Emblem, The Blazing Sword is a strategy game so successful that its characters are featured in Super Smash Bros and the 15th installment of the series will be released in early 2019.

The game is played by selecting characters (aka units), making decisions on where to move them, and then deciding from a set of options to attack, use items, and more.

Because of how much I enjoyed this game as a pre-teen and teenager, I wanted to make an AI for this Gameboy Advance classic.

This post will be the first of many as I break this endeavor into more bite-sized chunks in building a game-playing AI from scratch.

The code presented here can be found at github.

com/aaronfrederick in its most recent form for those looking to create similar projects.

At the outset of this project, I divided the tasks into 4 chunks:Creating a controller in pythonObtaining data from the gameAutomating the playing process to continuously generate dataImplementing an algorithm to optimize gameplayTo play Fire Emblem and create an AI, I downloaded the VisualBoy Advance emulator and a Fire Emblem ROM to my computer.

The emulator, in conjunction with the ROM, allows one to play games from different consoles on a computer.

Another example of this is the Dolphin emulator, which allows for Wii and Gamecube games to be played on the computer.

Pyautogui will not work unless the box is checked!Now that we have an emulator setup in place, we need to be able to programmatically access it.

In preparation, I have turned on the emulator and placed it in the upper-left corner of my computer screen unobstructed by other apps.

It is crucial to have a consistent place where we will click to select the emulator, but this consistent location is even more important to gain information from the screen when the time comes to screencap the emulator for data.

The pyautogui library is great for controlling your computer with python, but if you are on a Mac like myself, you need to let the terminal access your computer as shown on the left.

The first thing we need to do is select the emulator so that it runs, which we can do by moving our mouse over the app and clicking.

import pyautoguipyautogui.



click()The code above with a 4 second time to reach destination for clarityThe arguments passed to the moveTo function are x pixel location, y pixel location, and time taken to get to the location.

Because the upper left corner is (0,0), we are moving the mouse close to the top left, but not all the way so that we can have the cursor over the app.

In order to press buttons once we have selected our emulator, we will need a function that uses pyautogui again.

The press_key function below simplifies this for the future so that we can just type press_key(key, # of times) to control the game with ease.

def press_key(key, n_times = 1): for _ in range(n_times): pyautogui.

keyDown(key) pyautogui.

keyUp(key)Now that we can press buttons, we need to apply this to the game.

The three functions I’ll be presenting for this example are select_next_unit, move_unit, and wait.

These functions will be doing the bulk of the work for this AI, so it is important to get them right.

As shown below, select_next_unit simply presses the “q” key and then the “ ‘ “ key.

This is equivalent to pressing L and then A on the Gameboy Advance, which are the controls to moving the cursor to the next available unit and pressing A.

This will bring up an interface allowing us to select where to move our unit.

def select_next_unit(): press_key('q') press_key("'")The move_unit function as implemented on my github is slightly more complicated than what’s shown here — it allows for random moves — but the functionality is largely the same.

The basic idea behind this function is that we want to feed in a set of coordinates to move, then press the keys to achieve those coordinates.

The inequalities are for random move generations so that we only press left if the value of left is greater than right, for example.

def move_unit(left=0,right=0,up=0,down=0): ret_list = [left,right,up,down] if left>right: press_key('a', left-right) elif right>left: press_key('d', right-left) if up>down: press_key('w', up-down) elif down>up: press_key('s', down-up) press_key("'") time.


2) return ret_listUsing W, A, S, D as up, left, down, and up, respectively, we can press the keys the amount of times that correspond to the coordinates we pass in.

Once we move our unit, we see a menu that allows us to either use an item or wait.

Using the wait function, we will keep our unit (Lyn) where we moved her and end our turn.

def wait(): time.


2) press_key('w') time.


2) press_key("'")The code to the right playing Fire Emblem!Putting these three functions together, we can take our first turn in the first level of Fire Emblem!.The GIF to the left shows the code below at work.

#Select the Emulatorpyautogui.





3)#Play the Gameselect_next_unit()move_unit(left=2, up=2)wait()In part 2, I’ll be discussing how to get information out of this game so that we can create a dataset and loss function for machine learning.

This will involve some image processing techniques and creating an image classifier to intelligently extract data from the screen.


. More details

Leave a Reply