Review
B000FRU0NU

Teaming Up with Halo’s AI: 42 Tricks to Assist Your Game

Alex J. Champandard on October 29, 2007

Halo caused a bit of a controversy here on AiGameDev.com when it ranked so highly in the Top 10 Most Influential AI Games. But after reading through the record 42 tips in this technical review, you’ll quickly realize how much influence Bungie has over the industry by sharing insights into its development process.

Taking a cue from the presentations at the Game Developer’s Conference from Jaime Griesemer and Chris Butcher in 2002 (Halo’s designers), then Damian Isla talk in 2005 (Halo’s lead AI programmer), this article looks into the artificial intelligence technology behind the best-selling franchise — what to reuse for your game as well as what to avoid.

Design Principles

Squad Escort

Screenshot 1: Squad escorting a vehicle in Halo 3.

1) Focus on the Illusion of Intelligence

Let’s face it, the games industry uses artificial intelligence technology only to help entertain the player. It’s not about pure logical reasoning, as Chris Butcher and Jaime Griesemer (designers at Bungie on the Halo series) commented at the start of their GDC 2002 presentation [1]:

“If you’re looking for tips about how to make the enemies in your game intelligent, we don’t have any for you we don’t know how to do that, but it sounds really hard.”

It still takes lots of design and technology to maintain the illusion of intelligence, but it’s important you remember the true nature of the problem.

2) Remember, Tough Means Smart!

It’s obvious in retrospect, but Bungie found a correlation between how smart the AI is, and how tough it is.

  • When you improve the AI relative to the player, obviously the game gets tougher.

  • However, when you make the AI tougher (e.g. with more health points), the players believe they are smarter.

In extensive playtests, players confirmed simply making the AI stronger helps it appear more intelligent. The fact that stronger AI characters live longer to show off their behaviors is certainly a factor, but it’s not the only one.

3) Balance Your Difficulty Levels

Players have a very specific opinion about games when they fail to entertain them. As Jaime and Chris phrase it [1]:

“If the game goes from easy to hard and back again, players attribute the easy parts to poor AI and the hard parts to evil level designers.”

So by taking care to provide a consistent challenge, you can avoid these pitfalls in your game too.

4) Make Behaviors Even More Obvious

You’d be surprised how many players don’t notice things in games, despite them being visible. In Halo, from the focus testing feedback, the developers kept having to take it another level. As Chris and Jaime mentioned:

“Initially nobody noticed so we had to keep adding clues to make it more obvious. By the time we shipped we had made it so not only does every single Grunt run away every single time an Elite is killed.”

Even with the exaggerated animations and sounds, they estimate that less than a third of the players understood the connection.

5) Remember, Fuzzy Logic is Hard to Understand

During the design of Halo, the initial model for the actor’s state was based on fuzzy logic. This is how the prototype worked [1]:

  1. Actors would have four fuzzy variables: fear, anger, defensiveness, surprise.

  2. Each variable would have thresholds and decay rates on a hidden variable.

  3. Inputs and events around the would affect these variables.

  4. Actions would be driven by these multiple variables.

In the end, this proved to be too hard for players to understand, so only an analog measure of threat was kept.

6) Focus On Building More Behaviors!

In terms of behaviors, the more responses you have to the different situations, the better it gets. Damian Isla, Lead AI Programmer on Halo 2 & 3, explains it best [2]:

“When it comes to game AI, more is often better, in the sense that the more well-rounded an AI’s behavioral repertoire is, the more unique triggers an AI can recognize, and the more unique ways an AI can respond, the more likely we are to see the AI as a common-sensical kind of creature.”

As such, if you want more common sense from your actors, you need to be designing your system to handle as much logic as possible!

7) Borrow Intelligence from The Designers

Looking at things pragmatically, the role of the team is to build a system that shows signs of intelligence. The best strategy for that is to find the right balance between human supervision and technology. As Chris & Jaime said [1]:

“Ideally you want to borrow the intelligence of the designers to make the AI seem smart, but you want to use the generalization of code to make the intelligence procedural (you can’t ship a designer on a CD).”

The question is though, where do you draw the line? The next set of tips looks into team organization…

Methodology and Workflow

At Bungie they have well defined roles for the team members.

Squad Movement

Screenshot 2: Squad movement in Halo 3.

8) Decide on Responsibilities of Design and Code

In the Halo series, the technical side is responsible for controlling the AI autonomously from one second to another (for about 30 seconds [1]). This involves:

  1. Intelligent decisions, for example to find valid cover.

  2. Instant reactions, e.g. when grenade is about to explode.

On the other hand, the designers are responsible for giving more meaning to the AI, over the course of minutes (typically around 3 minutes of playing scope [1]).

  1. Strategic Purpose, for deciding what to do in the situation.

  2. Racial personalities, specifying the high-level behaviors.

As such, the design provides motivations for the AI that are not represented in the code itself, like protecting certain areas, or securing perimeters. The overlap between design and code happens in the combat behaviors.

9) Reuse these Sensible Design Goals

To provide a fun experience for your players, ideally you want the AI to be:

  • Intelligible — The AI should be easy to understand.

  • Interactive — All in-game actors should focus on the player.

  • Unpredictable — Gameplay should not become repetitive.

The solutions to each of these goals are discussed in the rest of this article…

10) Playtest as Much as You Can

The Halo franchise has used focus testing right from the start. The idea is to get 25-35 non-hardcore players who’ve never seen the game, let them play and fill in a survey about their experience. So the strategy is quite simple:

“Playtest early, playtest often!”

The results from the tests are both objective and accurate, which helps with design and level structure greatly. Bungie even has designers watch the playtesting, to give them the motivation to build a game that people want to play without getting frustrated!

11) Cover Up Weaknesses with Design

A good design always takes into account problems with AI and gameplay, and addresses these issues. In Halo for example, two specific problems were fixed my negatively reinforcing certain playing tactics:

  • Rushing certain levels with the assault rifle isn’t fun because the levels end too quickly. To fix this, Bungie made the AI more accurate the longer the player is out of cover.

  • Also, the AI isn’t very good at dealing with close targets moving quickly, so the design was extended to give them powerful melee attacks.

Using play testing, you can refine the design to fix flaws in your AI behaviors and emphasize what works, for example, making sure the AI is engaged from the ideal range, and doesn’t look to robotic.

12) Have Important Assets Reviewed

There are typically a couple important AI scripts or configuration files that have a significant impact on the functioning of the logic. When that’s the case, it’s best to have these approved by technical people and the AI designer, to make sure that nobody breaks the build accidentally. As Damian Isla puts it [2]:

“The designers have a small style library to choose from when setting up an order (each style in the library having gotten the seal of approval from both the lead designer and the AI programmer).”

13) Identify Details that May Appear as Bugs

Players are always on the lookout for little details that will break their suspension of disbelief. According to Jaime and Chris:

“Early versions of Halo’s AI had a short pause between seeing you and recognizing that you were an enemy. This was to allow people to get the jump on the AI or retreat if they were getting in over their heads. Playtest feedback showed that players assumed this Feature was a Bug and they all said it made the AI look dumb.”

In the end, they removed the reaction time feature, even though it was intended to help gameplay. Be on the lookout for these kinds of problems in your AI.

14) Use Heavy Scripting to Create Novel Situations

When you have a specific story in mind, and want to entertain the player using “set-pieces” then scripting is the right solution for you. This has the advantage of being fully controllable, and creates the feeling of a movie. However, there are downsides:

  • Players are limited to a mostly linear experience.

  • It’s time consuming for the level designers.

Regardless, Halo uses scripts extensively to guide the AI, most notably during battles…

Battles & Squad Behaviors

Vehicle Battles

Screenshot 3: Vehicle battles in Halo 3

15) Create Strategic Spaces to Emphasize the AI

The role of designers includes creating the levels where the gameplay happens. Obviously, they should be designed to play to the strengths of the AI. At Bungie, they have simple guidelines for establishing such battle lines [1]:

  1. Battle Area — Linear corridors aren’t much fun to have battles in. Instead there needs to be lots of interconnected space for flanking and other tactical maneuvers.

  2. Killing Zones — Since large spaces can be chaotic, the designers must establish an open area between two positions with good cover. This makes the battle feel more directed.

Such layouts are applicable to many action games, although you might need to make some adjustments depending on your AI and gameplay.

16) Use Tactical Scripting to Improve Battles

In Halo, the designers use Attacking and Defending states to control the flow of battles [1]. Essentially, each group of actors is provided with:

  • An aggressive territory that it should occupy by attacking.

  • A defensible location for the group to fallback to.

Specific retreat conditions can be scripted also, for example taking into account the number of casualties. This allowed the designers to capture the dynamic flow of a raging battle.

17) Annotate Firing Positions to Control Battles

As well as scripting and setting up territories for groups, in Halo the designers can add annotations to the level to specify firing positions. This provides better time-dependent control over the combat, as Damian Isla explains in [2]:

“In Halo 1, AIs were grouped into encounters, which also contained a set of firing positions. In Halo 2, the basic ideas remain the same, although the representations are different. Instead of having a single encounter structure, we now have squads [as] groupings of AI and areas [as] groupings of firing positions.”

By assigning squads to areas, you effectively issue an order, which is the name given to it in the implementation.

18) Emphasize the Tactical Goals of Squads

Throughout the scripting and level design, it’s still important to focus on the goal of making the AI intelligible. In Halo, the developers tried to make sure that group behaviors are easy to understand. Luckily, this can be easier than you think, as Chris and Jaime point out:

“Players are naively confident in your AI programmer. They will assume your AI knows they are supposed to be protecting the tractor beam power switch until you prove that they are just standing around.”

So by simply having strategic objectives in mind when creating squads, most of the work is done already.

19) Make it Easy to Control Friendly AI

In action games, having multiple squad mates often makes the game more exciting. However, this can cause many problems [1]:

“Unfortunately the Player is often forced to be a manager instead of a hero the end result is that the Player often ignores this option.”

Now, in this case, Bungie didn’t do as well as the other points. The friendly AI actors are certainly autonomous, but they can still get into trouble with grenades, etc. A few Halo 3 reviews mention these bugs in some way… Co-op AI is certainly not an easy problem to solve.

20) Rely on Emergent Combat Dialog

Many action games don’t need sophisticated dialog outside of the scripted cut-scenes. So a good solution is to use simple logic to get an illusion of squad communication. For example:

Grunt says “Kill you, human.”
Sarge says “Keep it steady, marines. We’ll beat them.” (custom for group of marines)
A nearby marine screams “No! you covenant b*!” (custom by killer’s race)
another marine shouts “Someone take that thing out!” (custom by damage type)

Since Halo’s AI engine is event-driven, each event can trigger a response. These are picked based on the different parameters for the group [1].

Biologically Plausible Sensory Model

Grunts and Elites

Screenshot 4: Dealing with grunts and elites.

21) Maintain Individual Actor Knowledge

The players assumes that the AI has similar capabilities than they do. So actors should only see things the player can, and conversely, they should not see things the player cannot. There’s no “cheating” in the sensory department because it helps with the illusion.

The AI system can’t access the world state directly, only the local knowledge of the actor. This is used to simulate four senses: vision, hearing, touch and a sixth sense for scripting.

22) Remember Only Important Information

The problem with having individual sensory models is that memory is limited. So only data that’s relevant to combat is stored by Halo’s AI. As Chris and Jaime explain:

“Actors only track characters that are important to them, around 3 for friends, 5 for enemies. Every actor’s decisions depend on their own local situation which is easier for the player to understand.”

Other information is accessed directly by scripting which can trigger the “sixth sense.”

23) Keep Perception Biologically Plausible

The benefit of using a biologically plausible model is that the behaviors are much more entertaining for the players: they can fool the AI. But it also results in more life-like behaviors, which you get for almost nothing. For example [1]:

  • Turning to look at new characters

  • Watching friends as they walk by

  • Investigating dead bodies

  • Spinning around to track gun fire

24) Allocate Actor Memory Wisely

There are four different kinds of information that an actor would want to store [2]:

  1. Per-behavior (persistent) — This information is stored in variables of the actor class (e.g. grenade throws).

  2. Per-behavior (short-term) — This is allocated and freed dynamically on behavior’s execution scope.

  3. Per-object — Stores perception information, last seen position and orientation.

  4. Per-object per-behavior — last-melee attack time, search failures, pathfinding failures.

The first two rely on freeform C++ variables, while the last two are stored as props, as described next.

25) Perceptual Information Objects

All the individual information per-actor is stored in a separate data structure, as Damian Isla explains [2]:

“In Halo2, these representations are called props, and their primary function is as a repository for perceptual information on objects in the world. Having this state information (position, orientation, pathfinding location, etc.) distinct from the actual world-state and gated by the actor’s perception filter allows the two representations to occasionally diverge – thus the actor can believe things that are not true, and we now enter the realm of AI that can be tricked, confused, surprised, disappointed, etc.”

These objects are allocated in a pool to help reduce memory fragmentation and recycling problems.

Character Behaviors & Decision Making

A lot of effort in Halo went towards making the behaviors transparent, which means that hidden influences (e.g. internal state) is an obstacle for making fun AI.

Halo Explosions

Screenshot 5: Dealing with explosions in Halo 3

26) Make It Obvious Why…

Basically, as Chris and Jaime put it:

“It should be obvious why the AI is doing what it is doing.”

Based on Halo’s ideas, you have two ways achieving this in your game:

  1. Limit the AI to basic cause and effect decisions; if you shoot something the enemy investigates, if you throw a grenade the enemy will flee, etc.

  2. Develop predictable strategies, for example the searching pattern should move at a linear speed and never turn around, etc.

This helps the players make a mental model of the behaviors, and allows them to anticipate what’s going to happen.

27) Alert the Player of Decisions

Making it easy to understand why is only part of the equation. It helps to make it obvious when the decision is made. This is done in three ways in Halo [1]:

  • Language — Various recorded audio dialogs: “wort wort wort.”

  • Posture — Animation styles like sneaking or running.

  • Gestures — Specific movements such as dive/roll.

This helps convey the actors’ decisions, but it also makes the experience much more involving for the player.

28) Use Distinguishable Character Personalities

Halo has many different character types, each with their own personalities expressed through the animations and the behaviors. Establishing these personalities helps by:

  1. It makes the design process smoother, as it’s often clearer what the behaviors should be.

  2. It helps the player understand the behavior and intentions of the actors.

As Jaime and Chris point out [1], pay attention to the first impressions. The initial few seconds are often cut short, so it’s important to get the personality across as soon as possible.

29) Inherit Parameters via a Character Hierarchy

All these parameters add up quickly, and can become a maintenance nightmare if the system is not designed to handle them. Halo uses the traditional character hierarchy to solve this problem, as Damian Isla points out:

“A white elite fights more aggressively and is tougher than an ordinary red elite, and so is a different character type. In all other respects however, the white elite and the red elite are identical. It is therefore a waste to have to create an entirely new full set of behavior parameters when we’re really only interested in the fight and vitality parameters. What we therefore have, is a system which allows us to define only those parameters that are truly distinctive to a character, and then to rely on a ‘parent’ character for the rest.”

30) Think of Custom Behaviors as Additions to the Default Behaviors

Just like maintaining many parameters becomes a nightmare as the number of character types grows, keeping logic simple is also a challenge. As Damian discusses [2]:

“Custom behaviors are inserted into the [behavior] tree, in this case in a preprocess step, so that the final prioritized list of children does not need to be recomputed every time. In this way, we can add any number of character-specific impulses, behaviors or behavior subtrees – and it is through these additions that a good amount of the personality of the characters comes through.”

31) Uses Styles Along with Orders

Another way for the designers to control the behaviors in a context-sensitive way is for them to use styles when giving orders to the actors. This allows them to specify exactly what the AI is allowed to do or not. Here’s how Damian Isla explains it [1]:

“The style represents the final and perhaps most direct mechanism through which we can control the structure of an AI’s behavior tree. The style is really just a list of allowed and disallowed behaviors. Just as a behavior cannot be considered if its tag does not match the actor’s current state, a behavior cannot be considered unless it is explicitly allowed by the order’s style.”

Interactivity and AI

Assaulting a Base

Screenshot 6: Assaulting an aircraft in Halo 3.

32) Respond to the Player as the Top Priority

The designers at Bungie know from experience [1] that a game where the AI doesn’t react to the player isn’t very entertaining… so presumably, with more reactions to the player, the experience would be more fun!

Now of course they use the typical combat reactions to improve responsiveness to the player, but on top of that they added many little lifelike reactions to make the game more engaging. For example, the soldiers always say something when the master chief arrives, and all enemies have their unique response too.

33) Make the Player Feel like an Action Hero

According to the designers at Bungie, the players dominating their enemies and like them to know when they’ve been beaten. They achieve this by using the breaking point concept [1]:

“A breaking point is a distinct change in behavior that indicates the AI knows it is going to lose. For example, when a Grunt sees enough friends die or sees an Elite killed he flees in terror. When an Elite takes a large amount of damage he loses his temper and charges the player.”

Groups also have a breaking point, so they retreat when they’ve taken too many casualties.

34) Rely on the Player to Help Make Behaviors Unpredictable

Since the AI must be easy to understand, it can’t be random. As such, Halo tries to break the repetitiveness of the game by harnessing unpredictability from the player. This is done with a feedback loop [1]:

“The player is unpredictable, which puts the AI in unpredictable situations, causing unpredictable reactions, and this leads to a unique experience.”

The simple chaotic element of the simulation makes the game replayable, including after saved games are reloaded.

Working with Behavior Trees

Behavior Tree

Figure 7: The behavior tree used in Halo 2, see [2]

35) Use Modularity to Allow Custom Logic

The AI logic is essentially made up of a tree of tasks which combine together to create behaviors. When these tasks are implemented modularly according to a standard interface, then they are free to manage their child tasks as they see fit. As Damian explains [1]:

  1. The parent task can make the decision using custom code, or

  2. The children can compete, with the parent making the final choice based on the child task’s desire-to-run, or relevancy.

Given a suitable API, both these options can be unified into one, so every task would be capable of expressing whether it wishes to run or not.

36) Make Binary Decisions on Prioritized Behaviors

Making fuzzy decisions involving all behaviors is not computationally feasible on current hardware if you have many actors. It’s also much harder for the player to understand. As a consequence, Halo uses a binary activation for each behavior, and then sort them by priority.

In this model, the designers essentially create selectors in the behavior tree, which contain ordered child behaviors. These behaviors are run in order to see if they can be activated, and if not, a fallback lower down the list is used. This algorithm forms the basis of the best-first search algorithm at the heart of the behavior tree.

37) Handle Dynamic Priorities by Splitting up Behaviors

Since child behaviors are ordered statically within parent selectors, Halo uses special tricks to deal with dynamic priorities:

  1. Identify the situations where the priority varies.

  2. Split the behavior into two for each priority.

  3. Insert a new behavior in the tree in the correct order.

In the end, each sub-behavior only activates for a subset of the original behavior, which does the equivalent of dynamic prioritization. The high-priority sub-behaviors are called impulses [2].

38) Use Direct Transitions if Necessary

The behavior tree is a great way to structure the AI logic, and it works in a very regular fashion that’s easy to understand. However, in certain cases, it’s necessary to jump to another part of the behavior tree.

“When the impulse wins the child competition either the current stack of running behaviors is redirected to the position of the referenced behavior, or the referenced behavior is simply run in the position of the impulse.”

Implementing this feature isn’t straightforward, however, as it can require winding-up the stack of tasks and setting up a new one in the new context. (This is similar to basic support for continuations in the behavior tree.)

39) Side-Step Conventions for Debugging

In the logic, no behavior is allowed to do any work without having been officially chosen. This is established by convention to prevent any problems. However, during development, this can be used to execute arbitrary code at any location in the tree, for example:

  • Debug breakpoints.

  • Console printing.

  • Data logging.

Of course, these items in the tree can (and should) be removed in final builds!

40) Track your Hacks!

A little hacking here and there is necessary in the software industry, but it seems particularly popular in the games industry! One thing you can do to prevent problems is to keep a list of hacks so you can keep on top of technical debt. Damian describes how his system deals with this [2]:

“Hacks are going to happen. When they do, we must make sure we have a way of containing them. This system also imposes a healthy discipline on our hacks, since one is required to label and, in the case of the Halo2 codebase, list them in a global list of impulses, thus making it very unlikely that we will lose track of them and forget that they are there.”

Implementation Details

Group Attack

Screenshot 8: Attacking a group in Halo 3

41) Cache You Sensory Queries

This advice is rather commonplace, but it’s good to reiterate: don’t do sensory calculations on-the-fly! If you want to run your AI logic every frame, then you’ll bring down the whole framerate. Raycasts account for 60% of the AI computation budget — almost 10% of the total budget.

In Halo, they use a bitvector to store information about the actor so it can be easily compared [2]:

“The idea of behavior tagging is to move these common conditions out of the relevancy function (thereby avoiding having to write the same code over and over again) and encode them in a tag for the behavior, which is checked directly at decision-time. In Halo2, these conditions are encoded as a bitvector, which is then simply compared with another bitvector representing the AI’s current actual state.”

42) Optimize Logic when Checking for Rare Events

With an event driven AI engine like the one they use at Bungie, it can be very wasteful to poll every frame to check for a condition. This is especially wasteful if the condition itself is rare, for example a death event. Here’s how the behavior tree handles it:

“One way we might consider doing this is through a stimulus behavior. This is a behavior or impulse that does not appear in the static tree structure, but is instead dynamically added by an event-handler to a specific point in the tree.”

In the GDC 2007 presentation about behavior trees, Max Dyckhoff (AI developer from Bungie) mentioned they found a more elegant solution for this in Halo 3 which didn’t involve dynamic allocations but simply activating a flag.

References

[1] The Integration of AI and Level Design in Halo
Jaime Griesemer, Chris Butcher
Download Slides (PPT, 14.3Mb)
[2] Managing Complexity in the Halo 2 AI System
Damian Isla
Download Slides (PPT, 4.6Mb)
Download Paper (DOC, 3.7Mb)
[3] In the Mind of the Enemy: The Artificial Intelligence of Halo 2
Robert Valdes, Chris Butcher
HowStuffWorks.com, 2004

http://entertainment.howstuffworks.com/halo2-ai.htm

Discussion 4 Comments

Ian Morrison on October 30th, 2007

I remember reading some of these in the past. Thanks for highlighting all the good stuff!

emmanuel_d on October 30th, 2007

This series of article begin to be more and more interesting. Hope you'll find the time to put everything into a book :)

diegix on November 1st, 2007

Halo has without doubt one of the most commented AI in games that are not AI oriented like Black and White. There are a lot great additions to the AI and the fact that grunts run away when their boss is dead and how that is understandable by the player is one of the most commented things in interviews for AI positions :D I always liked the behavior architecture of halo2 as it seems very clean and readable and usable in a game without loosing flexibility to add variety and complexity in decision As you say in the article, a lot of things in the Halo AI depends on the design both directing the behaviors and building interesting levels. In the same way as the AI in Fear, the planner is great and all, but as Jeff Orkin said the coordination and flanking was just the result of a clever level design. The conclusion I get from the AI from Halo or Fear is that an AI system could be great, but without a level design oriented to help the AI, it's just academic paper material

Ian Morrison on November 1st, 2007

What I got from them is that communication of intelligence is essential... an AI seems a thousand times smarter when he's screaming out his tactics and situation, instead of the player needing to guess from his behaviours. Transparent behaviours are a must.

If you'd like to add a comment or question on this page, simply log-in to the site. You can create an account from the sign-up page if necessary... It takes less than a minute!