Article
i/2008/08/marrionette

Puppet Strings, Remote Control, or Hands Off?

Dave Mark on August 20, 2008

In this week’s developer discussion on AiGameDev.com, Dave Mark sets the scene for a polarized discussion of methodologies. Let everyone know which camp you fall into and post a comment below!

Tell me again what happens when the strings get tangled?

I have recently been privy to a discussion amongst a very learned and experienced group of AI developers. They were all expressing their opinions on a subject that seems to be ever lurking in the virtual halls of the AI world. That subject is, in a broad sense, how much control should we have over our AI agents? Alternately, how much is even necessary? Interestingly, the answers to the “should” and “necessary” questions differ significantly depending on whom you ask. There are a number of schools of thought in this regard. I will try to touch on them here (although this will not be an exhaustive list, I’m sure.

First, there is the “script everything” crowd. The belief here is that conveyance of the designers intent and vision as well as a properly presented level of entertainment is only possible if the agents in the game are treated much the same as actors in a film. We hand them not just the plot and screenplay but also micromanage what they say, how they say it, how they move around the set, how they even more their hands and face. The director of the film is analogous to the designers in the game world.

Second, there is the “autonomous agent” lobby. These are people that believe in the complete hands off approach. Design independent agents with the ability to perceive the world model, analyze their goals, and act on those goals without any intervention whatsoever. I suppose this maps over rather well to the “reality TV” model. You can populate your world with agents about which you have a general notion — but you can only vaguely guess at what they will do based on your perception of their “personality”. Once you toss a bunch of independent agents in a room, you never really know what you are going to get.

Show me again where the Kill Player button is?

A third group, however, is what came up in this discussion. This group was based on more of a “repeat after me” model. The idea was that using various technologies, game entities could be shown how they were supposed to act by example. In some cases, this involved neural nets learning situational based responses. In other cases, this was based on a rough equivalent of what MoCap is to animation… i.e. “this is the way it is supposed to look.”

Obviously, this triangle is not composed of three completely disjoint points but rather outline a continuum of possibilities. The most common word that illustrates this would be “except”. For example, I want my agent to be completely autonomous except under this particular circumstance when I need it to be scripted. There is a wide range of possible combinations at that point… and yet it is deciding where that word “except” needs to be that seems to be the problem. Each of the above methods has its advantages and shortcomings… far more numerous than what I can delve into in this column. (After all, one could make the claim that I have been doing too much “research” this week.) Suffice to say that none of them are a “silver bullet”. At least not yet.

Much of the separation seems to appear along genre lines. Not the genre as a whole, but rather the genre role that any given agent is playing at one time. For example, you could have RPG elements mixed into an FPS game. During those times, strictly using a solution that is good for an FPS agent would fall significantly short of providing a solution for the RPG needs. One example that was given in the discussion I mentioned before was particularly poignant… a solution that involves the “let me show you how to do this” approach — which is suited well to fighting games — would be tedious if it were applied to an NPC shopkeeper whose daily life involves baking bread, putzing around his store and serving customers that came in to his humble establishment. (Of course, I have seen plenty of people in MMOs who were perfectly happy to live out their online lives doing exactly this. I admit that in the first week after the release of Ultima Online, I made a killing by dying people’s clothes for them. Was that really 10 11 years ago?)

So, my question for the week is along the lines of ferreting out where those dividing lines are. What benefits are there to each of those three approaches? What caveats exist? As there merit in the industry slowly drifting towards any one of those three apexes?

Discussion 12 Comments

PaulTozour on August 21st, 2008

Good article, Dave. Some amount of scripting is always important in any game, and there's always a need to provide tools for designers to specify their intentions for the AI, whether through scripting or some other interface. However, this approach isn't going to get us any closer to the Holy Grail of game AI -- it's not really "AI," since it's just a fixed set of commands from the designer, sometimes with a small amount of conditional branching. From my point of view, the behavior capture / game observation capture ("GoCap") approach really just boils down to a different interface for scripting. It can only "learn" to do what you show it, and ultimately amounts to a fancy way of splicing pre-recorded gameplay clips together. GoCap by itself has no knowledge base, no world representation, no sense of its goals, no way of making educated guesses about the player's strategy, no way of making predictions or doing any form of planning ... it's a nice scripting tool, but it's far from being a useful tool in any but certain limited situations, and it will never get us to the Holy Grail of game AI. While we'll always need to do some scripting, I think the "autonomous agent" approach is the really rewarding path in the long run. It's the only approach that's going to get us better results for less designer effort. (Of course, your characterization of it as a "hands off" technique which produces very unpredictable results is not really accurate, as I'm sure you realize. It definitely needs to give designers ways to specify behaviors, but the autonomous agent approach encourages them to do this at a high level -- in terms of goals, knowledge bases, personalities, etc, rather than asking designers to script a lot of discrete action sequences.) What we need is a system that gives every artificial agent the ability to understand its goals, its capabilities, its environment, its knowledge base, and the player's goals and behavior, to do some amount of prediction of the future game state and the player's behaviors, and to put all of those together in all sorts of novel ways at runtime that result in interesting and believable behaviors that live up to the design team's expectations.

MieszkoZ on August 21st, 2008

Good points there, Paul. What bothers me is the question what is The Holy Grail of Game AI (or did I miss some article/discussion on that?)? And I do mean [U]Game AI[/U], because for The Holy Grail of (academic) AI I'm pretty sure it's "AI that always acts reasonably, consistently and with purpose, plans and reacts, all of it in a human fashion, and passes Turing test". I'm a kind of AI programmer that thinks about himself as a provider of tools to enable designers to create believable and fun experience regarding enemy opponents or NPCs. From this standpoint Holly Grail of Game AI would be a set of programming tools (such as algorithms and techniques) allowing me to implement my AI so it's: * easily controllable, manageable, extensible, debuggable, * can do anything designers might want it to Of course those values can still be obtained by a skilled, experienced programmer, but it takes, well, skill and experience ;) So (in my opinion and understanding) The Holy Grail of Game AI would be a technique or a way of doing things that would allow even a programmer with medium experience to implement all requited things with desired quality. Of course it's quite utopian, so it looks like (at least for me) there will never be no Holy Grail... Oh well, back to gathering skill and experience then ;) PS: and on topic: I'm a fan of autonomous agents that are at any point willing to follow scripted sequences or take hints.

zoombapup on August 21st, 2008

Well, my take on this, is that I'm not after "intelligence" from my agents. I'm after behavior. I dont really much worry about how that behavior is derived as long as it is contextually appealing. Now the bigger question becomes "if I'm after some behavior from an AI agent, what sort of interface would be most useful to describe that". I see this problem as one of a sort of author/director role. Currently the authorship of AI is far too complex, because essentially every director has to also be the cameraman, the key grip, the gaffer etc. Basically I want me AI agents to be digital actors that I can use some easy interface to that say "go here and fight anyone you see", "act crazy", "when youre health is low, act scared", "dont let the player reach this position". Those kind of things. If you look at it, you arent micro-managing the actor when you're telling them these things as much as you are giving them context within which to act. Good directors let actors pursue thier own take on the character they are playing, maybe we need to have designers who do the same kind of thing?

PaulTozour on August 21st, 2008

[QUOTE=MieszkoZ;4398] What bothers me is the question what is The Holy Grail of Game AI (or did I miss some article/discussion on that?)? And I do mean [U]Game AI[/U], because for The Holy Grail of (academic) AI I'm pretty sure it's "AI that always acts reasonably, consistently and with purpose, plans and reacts, all of it in a human fashion, and passes Turing test". [/QUOTE] A really good actor can jump into a scene, and with only minimal direction and a brief description of what character he's playing, fill a role naturally, and improvise where needed. IMO the Holy Grail of game AI is when game characters can do the same thing -- play their roles convincingly and entertainingly, with a minimum of direction from the design team. I think the test for when we've reached that is a 2-part test very different from the Turing Test: [LIST] [*]When your customers can play the game and say "It's so believable, it feels like I'm interacting with that character real life ... that's exactly, in every way, how I expect that human (or dragon, or elf, or battlemech, or cyborg, or whatever) to act in that situation." [*]When a designer and a professional animator can both look at it and say, "that's at least as good as what we would have done if we'd scripted every single behavior and hand-animated every moment, but we barely had to do anything at all, and it's a lot more robust and can handle unanticipated situations far more smoothly than we could have ever done by hand." [/LIST] I'm not sure what we'd call that ... maybe the Tozour Test? ;)

Dave Mark on August 21st, 2008

[quote=PaulTozour;4405]A really good actor can jump into a scene, and with only minimal direction and a brief description of what character he's playing, fill a role naturally, and improvise where needed. IMO the Holy Grail of game AI is when game characters can do the same thing -- play their roles convincingly and entertainingly, with a minimum of direction from the design team.[/quote] One vote for GOAP. [quote=PaulTozour;4405]When your customers can play the game and say "It's so believable, it feels like I'm interacting with that character real life ... that's exactly, in every way, how I expect that human (or dragon, or elf, or battlemech, or cyborg, or whatever) to act in that situation."[/quote] One vote for my book. [quote=PaulTozour;4405]When a designer and a professional animator can both look at it and say, "that's at least as good as what we would have done if we'd scripted every single behavior and hand-animated every moment, but we barely had to do anything at all, and it's a lot more robust and can handle unanticipated situations far more smoothly than we could have ever done by hand."[/quote] Another vote for GOAP. [quote]I'm not sure what we'd call that ... maybe the Tozour Test? ;)[/quote] Oh brother...

PaulTozour on August 21st, 2008

[QUOTE=Dave Mark;4407] Oh brother...[/QUOTE] Kidding, of course ... :P

alexjc on August 21st, 2008

Dave, The stuff Paul is talking about will primarily be achieved by the hard work of designers and animators specifying a huge database of special case behaviors that can work together if necessary. Sure, GOAP or behavioral mathematics may help, like many other technologies we talk about on the site, but it really is all about the content! Your comment seems to point in a different direction, so I thought I should set the record straight :-) Alex

Dave Mark on August 21st, 2008

Agreed... an algorithm alone can't do anything without someone specifying when, how, how often, why, etc.

Sergio on August 22nd, 2008

We often put down scripted behaviour, but we rarely stop to think why we do it. For me, there are two main drawbacks: extra work and fragility. Extra work means that designers can't just drop a few enemies and have their missions play beautifully on first try. The Holy Grail may help here, and we are constantly working on improving scripting mechanisms to make it easier for designers to control the AI. But the main problem I see is that of fragility. Designers are not usually good programmers, they forget possible situations, don't code defensively, and make all sorts of mistakes. That means that the behaviour of the AI will not be robust, it will show bugs or inconsistencies. Every time a designer asks me to do something that is too complex or too specific I panic. I see bugs raining down. If they tell me, "I want this character to focus more on the player", that's perfectly fine, we adjust a number in the priority of the targeting system and the basic structure of behaviour remains unchanged. However, other times they say "I want this enemy to stay put and use only distance attacks, then when it's hit he'll react in this way". Then I freak out. There are a thousand questions in my mind, that the designer may or may not have thought of. What will happen if the player fights that enemy in melee? What if the player can attack from outside of its range? Is the enemy supposed to react in that way when hit by other AIs or just when hit by the player? I can see inconsistencies, because that behaviour is too specific, not well tested and not robust. There is no way to guarantee robustness without removing control, so I don't think we can dodge this issue. We can only supervise the scripting ourselves, and pray.

zoombapup on August 22nd, 2008

Sergio: I agree with your assertions. But the issue you brought up about the designer asking for something very specific brings in a different side issue, that being that designers often are unable to specify what it is they want in enough detail for implementation. What we should read between the lines from any designer request is "and in all other cases, do something sensible". I think that is implicit when a designer tells you it should do one thing particularly. He is taking for granted that all other cases are handled appropriately. This makes the developer role a lot more complex, but when you think about it, its very hard to consider so many potential situations up front isnt it. We are essentially talking about designing AI to be more robust and I think its for exactly this situation. The one caveat there, is that the designer must understand that outside of those exact parameters, the behavior may not be exactly as he designed it.

gwaredd on August 23rd, 2008

I think "script everything" can be a slightly confusing term - since to me I think you are referring to what I call a rule based system (not the be confused with rule based systems). Whether the 'rules' are provided by designer or programmer is really just a minor production detail, the important point is the behaviour is a human authored solution - and this should not be readily dismissed. We have had thousands of years evolving very sophisticated neural networks that can spot effective strategies for complex situations ;) This accounts for (OK - ignoring path finding) pretty much all of the AI code from games I've been involved with. So a FSM would count, so would a BT, scripts, etc. The problem with these scripted systems is they grow in size and complexity exponentially, but we can manage the complexity - and tbh that is normally enough ... but not quite as fun or interesting as playing around with more emergent systems ;) In terms of "ferreting out those dividing lines" between rules and autonomous systems, the line is how the systems are implemented. Autonomous systems are always afaics implemented as some kind of search of a 'space'. This the computer looking for a set of actions (behaviour) which (may) achieve some kind of desired result. Planning systems (and hence path finding) fall into this category, also case based reasoning (searching a set of examples looking for something that fits). This is very much like the difference between procedural and declarative programming languages. In one system you specify the sequences of steps to take, in the other you declare how your world works and let the computer figure it out. The problems with autonomous systems searching large complex spaces can be expensive and take a bit of memory - and potentially can involve non-trivial tests (such as "can I see the target from this position" ray-test). Also, the AI may act in unexpected ways - which can appear dumb to the player, even if it is not. You can see this in path finding systems where agents sometimes "go the wrong way", where in fact they have worked out a quicker route - although it is not how a human player would naturally behave. It is not enough to make the AI intelligent, you also have to show the player why they behave as they do. For example, if a door ahead the path finder routes around it where as a human player would probably try the door first and then look for an alternative - as the AI typically operates in a perfect information environment they do not behave in this way. Although there is a really nice and elegant solution (using the notion of 'belief') that falls out naturally that gives this behaviour and can be the impetus for ancillary AI behaviour - such as acting surprised or speech. On the plus side they are nice and modular, "linearly" complex in terms of authoring them and can cope better with dynamic environments, but not all problems naturally fit declarative systems. It can be awkward and unnatural to force particular problems into this model - if anyone has a good rule of thumb or practical experience of what works and what doesn't here I would love to hear ;) Systems that are 'self modifying' fall into the 'repeat after me' or learning systems (NN's, GA's, CBR where the cases can self modify - e.g. in the minstrel system - interactive narrative is a personal penchant). Typically in games we have not had much success with these kinds of systems. We want more control, we have tighter deadlines and our worlds are quite well defined - so systems that can adapt in this kind of way do not really fit, despite appearing to be academics wet dreams :o I don't see this changing any time soon, although I suspect there is a bit of mileage in "case based" systems - particularly in the narrative space. These groups are probably not a comprehensive taxonomy of AI techniques, and it is not obvious where some things should go - or even if they fit at all (markov chains? celluar autonoma?), but they seem to fit all the things I care about quite nicely ;)

Kevin Dill on August 26th, 2008

I've been thinking more and more lately that there are lots of ways to skin an AI... errr, implement a cat... you know what I mean. People seem to get religious about architectures. GOAP. Behavior Trees. Learning. State Machines. Scripting. Goal-based AI. [There - I put mine last. Happy? ;)]. What we forget when we do this is that these are tools, and different tools are suited to different jobs. You can pound a screw in with a hammer and it will sort of hold. You can implement complex strategic AI in a state machine and that will sort of hold as well. But in both cases, you're not using the best tool for the job. On the other hand, if you've got a nail (or a simple, cyclic behavior, as the case may be)... For my money, the next step is going to be figuring out how to combine our tools together. Use Behavior Trees at the high level to pick what to do, and then script the low level actions so the designers control exactly how they look. Or, use scripts at the high level to guide the story, but use goal-based AI at the low level to make your animation selection dynamic and reactive. Or, use a behavior tree at the top level to pick high level actions, with support for scripts where appropriate, feed that into a planner which determines how to accomplish the selected actions, and then feed that into a goal-based AI at the low level to make it happen in a nice dynamic fashion. *Now* we're playing.

If you'd like to add a comment or question on this page, simply log-in to the site. You can create an account from the sign-up page if necessary... It takes less than a minute!