Article
files/neuron

Do You Think Neural Networks Are Useful for Game AI?

Alex J. Champandard on November 13, 2007

The applicability of computational intelligence and neural networks (NN) in games is a touchy subject here at AiGameDev.com, and in game developer forums generally. This is particularly interesting because everyone seems to be approaching the issue from a different direction:

  1. Most developers starting out in game AI seem to gravitate towards neural networks by default. (I’m guilty of that myself!)

  2. Developers with a certain level of experience in NNs tend to find classical game AI techniques more useful for typical problems.

  3. With a solid background in machine learning, NNs become a very powerful tool when the application is right.

Rather than tiptoeing around the issue, this week’s developer discussion dives right in! (Remember you can win an AiGameDev.com T-Shirt this month by writing interesting comments…)

Neural networks have found a useful role in industry via data-mining, but games applications seem to be lagging.

  • Is the lack of NN applications in games a sign that there are no suitable problems?

  • Do game developers and designers lack the necessary skillset to apply NN?

  • Are NN missing some key technology that would allow more widespread adoption?

You’re welcome to discuss neural network technology in general, but keep in mind that single- and multi-layer perceptrons trained using back-propagation have had the most practical success in games to date.

Join the discussion by posting a comment below!

Discussion 15 Comments

Andrew on November 14th, 2007

I don't have nearly enough experience with neural nets practically to talk about them - however, I'd pose that there is, as far as I've researched,a huge lack of tools to debug the behaviour of neural nets - something that is a major obstacle. Having terrible behaviour from some random occurrence is a bigger problem then the bonuses gained from having it learn via. neural nets, if you cannot debug any of the potential problems they have. That black box doesn't look too good if you don't know what is going on inside it easily :) if anyone knows different I'd eagerly check it out.

Ian Morrison on November 14th, 2007

Most developers starting out turn to Neural networks? Damn it, I'm in the minority again. I haven't actually touched them myself yet, since they seem to be more of a black box AI that you don't really understand the workings of (which sends me into fits just thinking about it). However, I've seen them do really neat stuff, so it's something I'd like to look at sometime in the future, when I have the time to goof around with the concept.

FuriCuri on November 14th, 2007

In my opinion what holds NN to be implemented in games is the complexity of algorithms - it's not so simple to put NN in realtime with all other game tasks required to calculate. Maybe with dominance of multicore CPU we will start to use NN more frequently. Speaking in general with implementing NN we give subject tools to change it's behavior algorithms in realtime based on accumulated knowledge. It's good and bad for games. For example, we can develop amazingly smart enemies: imagine that enemy "marine" will change it's tactic based on how much he saw marines like him were killed by player, when those marines used tactic "A" - when he saw that tactic A fails maybe he can somehow change his own tactic "A" to some other tactic ("B") and use it. This is great, but players expect enemies to have many knowledge already and react almost like human do. And the current state of NN science does not allows that - there is simply no algorithms yet that can implement minimum of what current game player expects from AI. So we just can fully rely to NN at this moment. So we must implement some kind of hybrid. But then a "NN hard to debug" problem comes on it way. So, I think it's not about games, it's about NN - current level of NN science is too low for game's AI to be fully relied on it. And more general and more personal thoughts. I think that whole computer AI science is a dead-end road. We can sharp our simulation algorithm but lay claim to be able think at least like some animal - it's just impossible due the fact that animal's mind operate with on different basis, not on discrete basis like our computers do. Basis state everything: computers can make simple calculation billions times faster that brain, but they never get a chance against, for example, brain image recognition (precise and fast).

togelius on November 14th, 2007

LM: Allow me to disagree sharply. Two decades of modern NN (and other CI) research have come up with some very sophisticated algorithms that can solve very complex tasks. And there are always some spare clock cycles if you think what you can do with them is more important than e.g. 5% longer drawing distance. (Some of the early NN research was carried out on hardware that was computationally on par with a Gameboy Advance...) On the other hand, most game developers don't have the required expertise in NN and other CI techniques. That's why they should hire/collaborate with CI experts. Don Bloomfield: Agree wholeheartedly! Andrew: I'm starting to understand just how much of a deal-breaker the black-box problem is for developers. I'd be happy to work on evolving/learning some easier to understand structure for game AI, so that developers could go on tweaking the structure after (or in-between) learning. But what would such a structure look like? Decision trees? FSMs? Rule sets?

FuriCuri on November 14th, 2007

Julian Togelius, there are some very interesting facts for me in your answer, thank you. My knowledge of NN and other AI-science knowledge were abandoned when I graduate from my institute so it would be awesome if you could share you knowledge with us. Since we have tons of materials about NN and general AI science it would be nice to see how those complicated theory can be used in games.

alexjc on November 14th, 2007

LM, See [URL=http://aigamedev.com/interviews/racing-games-computational-intelligence]Making Racing Games Fun With Computational Intelligence[/URL] for references and insights into Julian's research. Alex

FuriCuri on November 14th, 2007

And yes, Gwaredd Mountain - thats what I wanted to say too. Again - back when I study NN in my institute (that was in short course form) we did some laboratory works and create some very simple NN applications. The big problem was that it require a big database and, in general, we can't force system to do what we want - the more NN app become completed the more unmanageable it becomes. And if you look into an article about Halo's AI development you see that designers want to control where the game can flow and users want to see some transparent behavior of the mobs in game.

gware on November 14th, 2007

Alex : yep, Touchy subject isn't it ? ;) My thoughts on this : First of all, I think it can be useful to be a bit more precise. What's the meaning of "NN used in game" ? Do you mean NN being "evolved" in-game? or Using more static NN (which would be trained off line) ? Do you really mean at run time ? What kind of NN ? For what task ? Most debates about NN and their use in games are all about these questions ... One will say "but hey it's expensive to use (ie train) NN in games" other reply "hey hell no ! I had a NN (ie Xor) on my Atari back in the days" ... Do you really think we could train and use a spiking NN at run time in a GBA game? I'd say that we would prefer not to : in term of memory (remembering inputs, world wide facts and results), cycles (training at wich frame?, using it at which frame ?), code size. Everything counts here. Still I'm not saying we should never use any NN. of course We can evolve a NN off line, and use it to classify some patterns in game. Said that : If we already know what patterns (and believe me , you want that if you don't have a very good patching system:), do we still need the NN ? Julian, Saying that "most game developers don’t have the required expertise in NN" is just your thought and not a fact. I think that game AI programmers, the one that usually decide to use NN or not, usually have enough background on the subject, or are making tests. What are the principal uses of NN in the industry : - Classification / data mining - decision making - prediction - compression - probably tons of other ... Ok, now, what do game programmers need ? - Classification / pattern & sequence recognition Not sure about this. Sure we could "classify" gameplay schemes used by player at runtime. But we can also use a few "tricks" that will classify in the same way, fine tuned, and with no training at all. - Decision making Decision making using NN is interesting if the domain you're working on is partially known or not at all. Is it the case in game logic ? I'd say no. Can it help ? sure, in some (very specific) places. I believe most wii developpers are using some NN to classify input from the wiimote. - prediction Same as above I believe. - Compression Well, I pass. "using of NN" can have lot of meanings, you can use them off line in your pipe, or online in your game. But when talking about online NN, you should always keep in mind that most game developpers have very specific issues to deal with : - memory - cycles - requierements for licensing. NN can eat memory AND cycles depending on what you expect them to do. Maybe one can say that game programmers are bad programmers since they are always fighting to get their title att60 FPS and fitting in memory. Maybe we could put less content, display less objects, animate less characters and put some incredibly new hiped techniques in the build and still meet the FPS & memory constraints. Maybe we could take a bit more time and go back study the subject. Maybe this will let us make better games. Maybe not. Game developpers are making games, and when possible using new techniques (by new I mean "new in the video game industry") if it help them create better game in a better way. Researchers are creating new techniques, and demonstrating their use in different context. Let's everyone do their job and we'll see more and more NNs used at runtime in games.

Andrew on November 14th, 2007

Julian, since you addressed me; I've no idea what a NN would look best like in a debug state, but some form of graph is a certainty from how they work. I'm not developing anything but my own AI stuff and am still learning, so I can't help say what would work best - It's likely easier to put forward some examples to potential people who'd use it then discuss it anyway :) it'd be hard to say which would be best without trying them against each other.

togelius on November 15th, 2007

Gabriel Ware: Interesting post, and sorry, I didn't mean to step on anyone's toes. Of course there are game developers that know a lot about NN and CI. However, there are others that don't, and as you point out, our respective areas of competence are typically complementary. As for your taxonomy of uses of NN: You forgot function approximation. One use of function approximation is to learn to imitate how a player behaves; you can then use decision making (learnt through evolution/reinforcement learning) to come up with agents that behave in an interesting way, when trained together with the model of the player's behaviour. Similar to what Don Bloomfield suggested above. Andrew: I might just get down to inventing a couple such structures, do some experiments, and write up a paper...

qwerty01 on November 15th, 2007

Just a link. An interesting application of a game with NN can be found here: http://nn.cs.utexas.edu/keyword?stanley:cig05 and the related game can be downloaded from here: http://www.nerogame.org

anpd on November 15th, 2007

I think NN can be great during the development cycle. I belive it was Colin Macrae 2 that used NN to train their ai drivers. The final training results was then used by the ai in the game.

FuriCuri on November 16th, 2007

Don Bloomfield, yeah - I've been talking about this application of NN too. But think about it from other side: what do the newbie of that NN MMORPG do when he encounter a mob that was trained for a month on 10k players? I think that we forgetting one main thing about game - it must bring player joy. It's bad when enemies become too hard to fight, and NN can do that with them - moreover, it's hard to adjust enemy level after NN DB has been trained.

birken on November 16th, 2007

Hi, from my experience (no I haven't used machine learning in any commercial game, but I did write my master thesis at a game developer on the subject), I'd say that the keep it simple stupid rule applies most of the time. Using a NN to choose a strategy from a set of good strategies, will make sure that the ai doesn't do anything you REALLY don't want it to do, while making some space for learning/flexibility/chaos/tuning. I've mostly tried reinforcement learning, having a 2 layer nn as my function approximator, and while usually learning the task at hand, the results often don't look soo good when playing a human. Why? For one, you often need a lot of examples to train the nets properly, thus the ai needs to play some other ai (or itself, although risky). MMOs could probably fix this though, 1 million people can create a lot of data... Secondly, if the nets have too much control, they'll find loopholes and bugs for you (which can be a good thing ofcourse). They usually become very good, but boring to play, or useless if they've trained with a flawed ai that they'd learn how to trick. Don & LM: I think the difficulty setting could be part of the training. If using reinforcement learning, the ai could be punished for being too good. Also, the difficulty could be set from outside the net altogether, only allowing the ai to perform actions from a limited set when encountering new players.

Veksi on December 15th, 2007

Whilst reading these comments today, I did wonder, if anyone thought of doing neural network related AI in server side? It seems to the industry practice with large server clusters churning data with help of Microsoft BI Solutions (http://www.microsoft.com/sql/solutions/bi/default.mspx), for instance. As, Björk then suggested in his comment, MMOs seem to be a good candidate, but perhaps other kinds of data collection arrangements could created also. Collecting game data and running analysis on it to train game response could solve the problems with tight development schedule and soft real-time constraints. Privacy concerns and on-going salary, hardware and networking costs would be some new issues to consider.

If you'd like to add a comment or question on this page, simply log-in to the site. You can create an account from the sign-up page if necessary... It takes less than a minute!