This week’s Thursday Theory post features an interview with Julian Togelius. Julian got his Ph.D. by doing research into computational intelligence (CI) — a branch of artificial intelligence (AI) inspired by biological systems. He has published multiple papers about applying these techniques to video games, notably racing games.
Figure 1: Using a neural network to control a helicopter.
Alex Champandard: Hi Julian. Thanks for taking the time to answer these questions. Could you introduce yourself briefly and give us a little background on your research?
Julian Togelius: Hi Alex, and thanks for inviting me! Well, I recently finished my PhD at the University of Essex, with a thesis called “Optimization, Imitation and Innovation: Computational Intelligence and Games” (PDF, 3.4Mb). What I’ve been doing is trying to find ways to use evolutionary algorithms, which are a type of algorithms inspired by biological evolution, in computer games. (Basically, an evolutionary algorithm is a systematic trial-and-error process.) I’ve also been looking at how we can use computer games as test beds for developing artificial intelligence using evolutionary algorithms.
Basically, I think that there is a lot to gain for both game developers and AI researchers: game developers have an untapped resource of tools that can help make games more fun, and computer games could replace robots (expensive and slow) in many experiments AI researchers do. To demonstrate this, I present a series of experiments using a car racing game.
Related blog entries by Julian Togelius:
AC: Congratulations for finishing your Ph.D. by the way! Tell us more about the best ideas and your favorite algorithm in there…
“The idea of personal content creation is to model the style of a human player, and use it to evolve entertaining content.”
JT: Thanks! From a game developers perspective, the most interesting stuff in my thesis is probably the personalized content creation (which I worked on together with Renzo De Nardi). The idea is to model the playing style of a human player, and use this model to evolve entertaining content for that player. In my proof-of-concept implementation I first record the way players drive on test track using a couple of different quantitative measures. I then evolve a neural network-based driver to drive in the same way as the modelled human player - well, in the same way according to those quantitative measures, at least. Finally, I evolve tracks that are supposed to be fun for the modelled player. When the fitness function tests a new track, it runs the driver on the track and measures how “fun” the ride is, according to another set of quantitative measures.
Figure 2: Automatically generated tracks based on player models.
AC: How do you feel about CI/ML/AI technology today? Did you run into many problems in the process of applying it to make a game prototype?
JT: There is an enormous amount of expertise available in the computational intelligence community (evolutionary algorithms, neural networks, fuzzy logic, swarm intelligence etc.), and there are many algorithms that work very well for hard problems. And even more important, even when there is no problem to solve, there are algorithms out there that could help make entirely new types of games possible, if they are incorporated at the design stage!
The game development community doesn’t seem to realize this. Maybe this is because these algorithms are often not so powerful when taken off-the-shelf (or from the textbook), but you actually need to hire a CI expert that masters this somewhat dark art.
Video 3: Using evolved controllers on a remote-controlled car.
AC: How can computational intelligence help make car racing more fun?
JT: Some ways I explored in my theses is to create fun tracks for particular player, to create sets of behaviorally diverse opponents, and to create opponents that have a particular skill level on a track that has not be seen before (e.g. it has just been created by the user). Such techniques could help game developers dispense with some of the expensive level design and opponent AI programming process. But above all, they could make racing games more interactive and open-ended. I imagine a game where you just start driving, and continue driving for as long as you like on a never-ending track; the game senses what sort of driving you are good at and you think is fun, and comes up with new track section and competing drivers that are exactly the ones that you would have wished for. I couldn’t sit down and program such a game tomorrow, but with a couple of years’ more directed research effort I think it could be done.
Towards Automatic Personalised Content Creation for Racing Games Togelius J., De Nardi R. and Lucas S. IEEE CIG 2007 (PDF, 3.1Mb) Multi-population Competitive Co-evolution of Car Racing Controllers Togelius J., Burrow P. and Lucas S. CEC 2007 (PDF, 169Kb) Evolving Controllers for Simulated Car Racing with Object-oriented Genetic Programming. Agapitos A., Togelius J. and Lucas S. Proceedings of Gecco 2007 (PDF, 314Kb)
AC: My best memories of racing games are from NFS III: Hot Pursuit. I always loved playing with class C cars and beating the AI with class A cars! Is that the kind of situation where tracks personalized based on your car choice would help the most?
JT: The situation you describe is clearly a case of you having exhausted the challenges in the game, and making up a new one for yourself in order to be able to continue playing, in my ears. I think this happens all the times in games - you choose to do something other than the “main quest” because the main quest is too simple, too hard or just not fun enough. What CI algorithms could do here is identify such situations, and come up with new challenges for you before you get bored.
Screenshot 4: The TORCS framework for prototyping racing algorithms.
AC: In all your writings about procedural track generation, you discuss the fitness function at length. The maximum speed difference between trials and improvement over time seem like good metrics already… What’s the next step in estimating fun? Maybe something that works on a personalized basis?
JT: Yes. Definitely. There are some theorists that discuss what makes games fun in general, such as Thomas Malone and Raph Koster, but I haven’t seen much research on how to measure what particular individuals think is fun. It could be so easy as just look at what the players choose to do. In a car racing game, if a player tries to get into an oversteer all the time, maybe he would like some curves coming up where a nice oversteer really would come in handy?
Related blog entries by Julian Togelius:
AC: The University of Hull has been doing research into emotions of players in Unreal Tournament, and they’ve found that the emotional drain of such FPS is equivalent to a rugby match! No doubt that new style controllers like the Wii (and future sensors) can infer such emotions from the movement, or even meta-factors such as the amount of time played, number of restarts, etc.
What kind of research would you like to see done in this area?
JT: Wow, that’s cool! Isn’t it amazing what a large role games play in many people’s lives?
As you say, we could measure how involved people are in a game from how they wave their hands, how fast they click through menus etc., but the real challenge is distinguishing emotions from each other. Ever so often we hear about the need for games, if they are to be taken seriously and appeal to new demographics, to evoke other emotions than the raw joy of driving fast or firing an automatic weapon. But can we measure such emotions? For this, we probably need to collaborate with psychologists as well.
AC: Do you have any advice for game developers looking to investigate CI/AI, and personalized content generation?
“The best thing is for studios to contact a university with some good CI researchers.”
JT: I think the absolutely best thing they can do is to get into contact with a university with some good CI researchers - they would most likely be overjoyed to point to provide concrete ideas, and discuss collaborations. Virtually every other tech industry features academic/industrial collaborative projects, so it is a bit surprising that this is relatively absent in the digital entertainment sector. For those who are in the UK, my former supervisor Simon Lucas at the University of Essex just started a network aimed at facilitating such collaborations together with a number of game companies and academics at other universities.
One thing that should be remembered, though, is that even though very many CI researchers are interested in games (I recently attended a meeting on future directions for CI research with some of the leading figures in the field, and basically everyone agreed on this), academics might be more interested in using your games to test their own algorithms, and might simply not understand your needs. So be careful to explain what you are interested in.
AC: Where would you like to take the ideas of your Ph.D. — both on a theoretical and practical level?
JT: I am currently not planning on getting involved in trying to take any of my ideas to the market, as what I really is coming up with new ideas and exploring virgin territory (though this could change!).
One of the paths I’m thinking about is using raw visual input to learn to play first-person shooter games. The primary aim of this would be to explore how evolutionary neural networks can be made to tackle more complex problems, though it could have applications for creating more interesting opponents in FPS games. Generally, FPS games fascinate me because they are so immensely popular despite their apparent simplicity. And that so many players choose to go online to fight it out is testament to that most such games feature rather boring opponents.
Figure 5: Monitoring the improvement of CI models as they evolves.
AC: What’s in store for you next? I hear you’re moving to IDSIA in Lugano, Switzerland.
JT: Yes, IDSIA is a very good institution, that has produced some breakthroughs in CI research over the years, so I’m very happy to be going there. (Also, life in Italian-speaking Switzerland seems to be sweet…) I will be working on new algorithms for reinforcement learning and neuroevolution, probably using games as testbeds, and hopefully I will also find time to work on CI in game design.
AC: Many thanks for taking the time to answer these questions Julian!
JT: And many thanks for asking them, Alex!