AI generating football video game commentaryChintan TrivediBlockedUnblockFollowFollowingJun 25My approach to generating dynamic commentary in real time for Google’s football environment using GPT-2 language model.
Sample kickoff commentary generated in real-time by the model trained on commentary lines from the FIFA video game series.
If you have been playing the FIFA video game series for many years now, you have probably heard these lines in the game’s commentary many many times:-“Hi there everybody, Martin Tyler here along with Alan Smith and our match today comes from the Premier League.
”“He’s read the intentions of the opposition there to make the interception.
”“Well, we could get some news on the injury.
I think Geoff Shreeves is on the case.
”The reason these same phrases and lines come up so often in the game is because they are prerecorded during development of the video game and are played back from memory during specific events in the match.
Majority of new lines are recorded and released only every few years in the game series, which makes the commentary quite repetitive and boring, especially if you play the game as much as I do (unfortunately, lot more than I should).
I took up this project with the aim of exploring whether an AI could help generate dynamic, off-the-cuff lines every single time the same event occurs in the game.
Hopefully, this would make the game more realistic and closer to how real football matches have commentary.
The Game EnvironmentGoogle Research Football Environment [source]For this project, I’m using a football simulation environment that the Google AI team released recently.
While it was introduced for the purpose of training Reinforcement Learning agents, we can use it for other purposes as well, like in our case for generating commentary.
Getting the environment up and running is as easy as writing the following code.
Running random actions on an 11-vs-11 instance of the environment.
[source]The beautiful thing about this environment is that it allows us to play the game step by step, and at each step it gives us the complete information about the state of the game using OpenAI's gym API.
State Observations available through the API.
See full list here.
Information like goals scored, yellow cards given, current game mode (free-kick, goal-kick, etc.
) and much more is readily available to us in python code.
This makes it very easy to infer match events and feed them to a language model to speak about these specific events during the game.
The Text Generator[image source]This brings me to the commentary generation part of this project.
I’m using OpenAI’s GPT-2 language model, which is something I have covered before in my previous article in case you are interested to learn more about it.
Basically, given a small text prompt about a certain topic, this neural network can generate long and random sentences regarding that topic, making it ideal for our use case in this project.
By feeding the game’s information as a prompt to this model, we can let it generate rest of the commentary.
Collecting Training DataI downloaded the English annotations from YouTube videos of FIFA and PES matches giving me a corpus of standard football commentary lines.
Unfortunately, these annotations come with lot of mistakes and lack any punctuation.
The only way I could think of correcting this was to manually edit the corpus, which took up majority of my time in this project!YouTube auto-generated captions available via YouTube API.
Training and ResultsNext, I fine-tuned the pretrained 345M version of GPT-2 model on this dataset.
Using this, we can give the match information from our football environment as input to this model and let it complete rest of the commentary text, as shown in the example format below.
INPUT: And that’s a goal for this side!OUTPUT: Well, if you keep going in to that sort of area the ball will drop for you and it dropped perfectly for him.
That’s a lesson to anybody in during the goal to help keep getting into the positions, in the 18 yard box, in the six yard box and you will score!To recreate the real-time commentary experience while playing the game, I’m using a python text-to-speech engine called pyttsx3 to read out the output of the GPT-2 model.
The real-time audio commentary results are shown in the video embedded below.
Final ThoughtsBased on my experience with this approach, I noticed/learnt the following:-GPT-2 produces dynamic and varied sentences every time but many phrases within these sentences are repeated due to the large model memorizing the small training dataset quite easily.
I tried limiting this issue by ending the training process early enough to prevent it from over-fitting the input.
However, the ideal solution would be to massively increase the training set size hopefully leading to even more dynamism in the model’s output.
We need to provide a lot more context as input to the model, like a longer term game summary (match time, scoreline, etc.
) along with a more concise short term detail of what event occurred (who scored the goal and in what manner — header or long range, etc.
).
With more context, the model will hopefully be able to describe the specifics of the event rather than just making random guesses like in my case.
While addressing these limitations were out of scope for this work, I am still amused by what this AI is able to do currently.
It demonstrates the tremendous potential that language models in AI hold to completely revamp video game narrations!Feel free to check out this project’s source code at the GitHub repository linked below and please let me know if you manage to improve upon these results! :)ChintanTrivedi/football_ai_commentaryAI generating real-time football commentary.
Contribute to ChintanTrivedi/football_ai_commentary development by…github.
comI would also like to thank Jonathan for useful discussions surrounding training the GPT-2 model on YouTube annotations and how to improve upon the results.
Thank you for reading.
If you liked this article, you may follow more of my work on Medium, GitHub, or subscribe to my YouTube channel.
.. More details