Open Discussion
OFDR_Statistics.medium

Non-Determinism and Statistical Outcomes in OPERATION FLASHPOINT 2 (aka. You Shot My Pilot!)

Alex J. Champandard on April 13, 2010

I've been enjoying the OPERATION FLASHPOINT: DRAGON RISING campaign recently. In brief, the game is a large-scale open-world sandbox tactical combat simulation whose feature list would make programmers around the world whimper. The AI does a pretty impressive job considering the challenges!

My playthrough, however, has occasionally thrown unintended challenges at me — in particular reloading automatic checkpoints that end in disasters within 15 seconds. But rather than start the mission from scratch, this obviously seemed to me like an invitation to play around with the simulation and figure out its underlying mechanics...

In this short article, I'll look into a particular scene that ends in one of my rescue targets being shot at close range while I'm rushing away towards the pick-up zone like a coward. This raises interesting questions about determinism, level of detail and statistics — and makes a perfect topic for a developer discussion!

NOTE: If combat AI and sandbox games interests you, be sure not to miss our very own Paris Game AI Conference on June 23rd and 24th, which bring together a variety of developers from FPS and action games.

It Doesn't End Well (Mostly)...


Screenshot 1: In the full 3D game, the pilot (left) ends up being shot at close range by the enemy soldier (middle) and often shoots me trying to grab a good screenshot too!

OF:DR is not a deterministic game, like a majority of first-person shooters. Everytime I reload the checkpoint something slightly different happens, even when I don't move a muscle. When I move, obviously, things turn out even more differently. Due to the Sandbox nature of the game, this turns out to be much more interesting to play as you're forced to keep rethinking situations even after you reload...

However, in my situation, the pilot I was supposed to rescue almost always ends up being shot at close range by an enemy soldier. He occasionally escapes, but as you'd expect from an un-escorted target, he indeed bites the bullet a majority of the time — despite my best efforts to find creative solutions to this predicament!

Trying to make the most out of the ~15 seconds I had at my disposal to save the pilot, I tried various orders to my squad from the game, but switched to the map view when that failed. That's where things got interesting...

Safety in Map Mode


Screenshot 2: Switching to the map view, my squad (white) manages to shoot the enemy soldier (red) while moving around the center building, or the pilot (in blue) manages to sneak past without being harmed.

By switching to the top-down map view, the statistics of the outcome changed significantly. Most of the time, the pilot managed to escape from the building he was being ambushed in! The ratio of success to failure was reversed, now 80% to 20%.

Again, this wasn't guaranteed, but two new situations were the most common in map mode (that never happened otherwise while in 3D):

  • If I switched to the map instantly, my squad would manage to take out the enemy threat almost immediately while moving towards my position. Without switching the map view, they'd never deal with the enemy soldier and simply end up just following me.

  • When waiting a moment before switching to the map, the pilot managed to sneak past the enemy target in many situations. He did occasionally get hit but that was an exception rather than a rule, unlike it was previously...

The question arises, what could cause such a radical change in the outcome of this situation, at the mere press of the 'M' key to engage the map?

Combat Simulation Statistics...

There are two ways to interpret such a large deviation in the simulation:

  1. There's a different simulation that kicks in when you enter map mode. This different simulation operates in such a simplified fashion (e.g. no animation) compared to the full detailed simulation that the two different encounters end up differently in practice.

  2. The game's non-determinism is significantly affected by entering map mode. It's possible that disabling the 3D rendering allows more resources to be spent on AI. My squad responded better and the rescue target managed to deal with his threat more capably!

It's certain that OF:DR has a level of detail implementation. You couldn't build such a game without it, and in this developer commentary on Intel's site, they mention that the animation in particular is handled differently. However, it's unlikely that the simulation would be affected by the LOD just by the press of a key to switch into map mode.


Screenshot 3: My squad follows me around when I try to run to the target rendez-vous point like a coward rather than defend the pilot!

The Impact of Non-Determinism

This kind of in-game situation (and I guess this article too) raises more questions than it does answers.

  • Does it matter that the same situations play out significantly depending on the hardware configuration and rendering settings?

  • Assuming you want to preserve the performance benefits of non-determinism, how can you make sure your simulation behaves similarly everwhere?

  • How many developers treat different hardware configurations (PC) as different SKUs and balance each of them accordingly?

  • If the game's simulation statistics can be affected significantly at runtime (in different situations), will it even be noticed or is it a loophole?

  • Is determinism a solution to all of these problems, or does it just mean we implement a lowest common denominator for gameplay and AI?

If you have any thoughts on the subject, feel free to post a comment below!

Discussion 19 Comments

jadnohra on April 14th, 2010

was the game fun? .

bknafla on April 14th, 2010

Perhaps the AI behaves differently: - in 1st person mode it follows you and tries to be visible so you know it works (aka unseen intelligent behavior is no intelligent behavior). - in map mode the AI is visible wherever it is - to look intelligent it doesn't need to be near you so other behaviors and goals are possible. I haven't played the game so this is possibly completely wrong.

alexjc on April 14th, 2010

Jad, Should I deduce from that cryptic comment that you don't think that determinism matters at all for gameplay? :-) As I wrote in the introduction, I've generally been enjoying the game but the automatic checkpoint logic is broken and if you run into the wrong place a bit early then it'll save the game while you're in serious trouble. That's what caused the experiments I described, and while it was fun seeing this situation play out differently, when it happened before near a helicopter I could not salvage the situation and it was a frustrating game-breaking experience. (Radu apparently gave up on the game because of this.) I think players in single player campaigns will naturally end up doing the things that are fun (or quit) so even if the game is 10x easier from map mode it may not matter at all. What concerns me is that I could change my screen resolution and potentially get a very different game experience. That kind-of ruins all the balancing the designers did, so if you consider that as unimportant then non-determinism also doesn't matter either. Random games FTW! Alex

alexjc on April 14th, 2010

As an addition to my previous post, I don't think non-determinism on consoles or the same PC hardware (without LOD) is much of a problem. For example, if your floating point calculations are slightly different, then the simulation will slowly drift out of sync., but it'll mostly be the same. What's interesting to think about is how significant non-determinism (like in this case) can affect gameplay. Bjoern, indeed. Both those options are possible! It's a bit hard to say, I'll see if I can find more information. Alex

Doug on April 14th, 2010

For most games, I think deterministic Gameplay/AI is important for many reasons, this example points out one of them. When non-logical actions (e.g. taking your eyes away from the action by looking at a map or just looking down) improve your chances for success, then immersion is lost. I'm guessing most people who encounter this issue will say "This game is broken" and I'd say they're right.

bknafla on April 14th, 2010

Unsure if determinism is important for the player but it is unquestionable important to enable debugging and make Q&A life easier. Interestingly the situation described above - automatic saving at a very bad moment - could be solved by concepts like Valve's AI Director - where a save-game is analyzed and if it questionable to survive the next seconds the game could change some parameters (health, aggression of NPCs, etc.) so the player doesn't need to restart the level from the beginning which could mean a substantial time-loss. Cheers, Bjoern

alexjc on April 14th, 2010

[QUOTE=bknafla;49409]Unsure if determinism is important for the player but it is unquestionable important to enable debugging and make Q&A life easier.[/QUOTE] At least that's the common wisdom, but not many games do it! I think this example definitely raises more questions about the gameplay perspective... Doug, I wonder if you'd have noticed or just quit in frustration without an explanation. Alex

SilentBob on April 14th, 2010

It'd be interesting to see if changing graphics settings does change the simulation, rather than just speculating. It is possible that the friendly AI do things differently when the player is in map mode to when the player is running around shooting. If the player is actively moving and shooting, the friendly AI may hold off in order to let the player feel like he's more important and "being the hero". But if the player is looking at the map, you may want the friendly AI to be a little more autonomous. It would really suck if the player took a moment to look at the map and got shot in the head when a friendly could have seen & shot the sniper (ok, it's more 'realistic' for the player to get shot in the head while looking at the map, but it's not necessarily fun). My point is that it might be a game-play design decision to make enemies less accurate and friendlies more accurate when the player is looking at the map.

Doug on April 14th, 2010

[QUOTE=alexjc;49416]Doug, I wonder if you'd have noticed or just quit in frustration without an explanation.[/QUOTE] I would have noticed because I write & break gameplay code for a living :), but I'm not sure if a 'normal' player would have. This sort of non-determinism drives me crazy as a programmer. I had to bite my tongue at an Intel lecture teaching us how to make all our games behave randomly depending on the hardware load (and how to destroy your game balance when computers ship with more cores).

William on April 14th, 2010

Didn't Halo randomize NPC positions when loading a check point? Might have helped reduce the size of the save game, and added some degree of non-determinism.

t_sato on April 15th, 2010

[QUOTE=SilentBob;49421]My point is that it might be a game-play design decision to make enemies less accurate and friendlies more accurate when the player is looking at the map.[/QUOTE] That sounds really a good compromise. Right after having read this article, I bought this on Steam, played, and often get killed without recognizing the situation ... I always feel unfair in the beginning when playing FPS. NPCs know everything, but I don't. They can shoot me through bush from where I can't see. In this game, even a single shot puts an end to me. Yes, opening a map is as good as telling "it's not fair!".

alexjc on April 15th, 2010

t_sato, The game certainly takes getting used to compared to most shooters. You have to be very aware of any presence (either on the map or on your HUD) and not expose yourself in any way... It does feel a bit random in terms of death at first, it's not ideal but it does better than Far Cry 2. SilentBob, You're absolutely right, here are some interesting statistics. I reloaded the game 10 times for each mode. It's not statistically significant, but it's the best I can do with the time I have without having the codebase :-) I basically reloaded the checkpoint and did not move at all. For map mode I tried to press 'M' as soon as possible. 'K' indicates the pilot was killed, and 'L' indicates he lived. [CODE] 640x480 LKKLLLKLKL 1920x1200 LKKKKLLLKK MAP MODE KLLLLLLKLK [/CODE] That's basically 70% survival rate in the map mode, 60% survival rate in 640x480, and 40% survival rate in 1920x1200 with full graphics turned on and MSAA 8x. In map mode, I felt it was more like a skill. The times the pilot died were when I didn't press as fast as the other times. It think I could hit at least 80% survival rate for the map mode. While then numbers are certainly not scientific, it does point towards a frame-rate dependent AI problem -- maybe on top of the design decision to improve the AI in map mode. Interesting! Alex

jadnohra on April 15th, 2010

Maybe we are all taking this too far and it simply is a big bad bug/sloppy coding?

alexjc on April 15th, 2010

[QUOTE=jadnohra;49456]Maybe we are all taking this too far and it simply is a big bad bug/sloppy coding?[/QUOTE] :-) This particular train of thought may have been triggered by an autosave bug, true. But there's lots to be learned I think... Plus you normally love digging into topics even if they are philosophical -- why not this one? Alex

Doug on April 15th, 2010

[QUOTE=jadnohra;49456]Maybe we are all taking this too far and it simply is a big bad bug/sloppy coding?[/QUOTE] I'm not sure if it is sloppy coding or a sloppy idea. There is a big push to make everything highly scalable (which is important, since we are no longer going to get a free-ride with higher clock speeds), but the techniques being pushed by Intel and other companies tend to sacrifice determinism. This is fine for things like graphics and sounds but (IMHO), not so great for gameplay and AI.

jadnohra on April 15th, 2010

[QUOTE=Doug;49459]This is fine for things like graphics and sounds but (IMHO), not so great for gameplay and AI.[/QUOTE] Do you think their AI is such that it would not cope if it was not parallelized to such a degree that it would loose determinism? I don't think parallelization is an acceptable excuse if it was really the cause. AJC: what gave you the impression I don't like digging into this?

Doug on April 16th, 2010

[QUOTE=jadnohra;49461]Do you think their AI is such that it would not cope if it was not parallelized to such a degree that it would loose determinism? I don't think parallelization is an acceptable excuse if it was really the cause.[/QUOTE] My guess is that they could have written their AI to be both highly parallelized and deterministic, but they didn't think determinism was important for their game. The reason I think this is because of the recent Intel lectures I went to at GDC2010. They presented a lot of techniques for threading AI, but didn't address determinism.

alexjc on April 16th, 2010

[QUOTE=Doug;49467]The reason I think this is because of the recent Intel lectures I went to at GDC2010. They presented a lot of techniques for threading AI, but didn't address determinism.[/QUOTE] From our experience with TBB, there's not much about multi-threading that makes your code non-deterministic. If you have a fixed workload and homogeneous computation resources (i.e. the same processor types) then you'll end up with the same results. This particular case in OFDR could be just as easily caused by a single threaded implementation that has a timer to scale the AI calculations based on the amount of time that's available. Whether you're multi-threaded or not, that kind of time-dependent behavior just doesn't work for determinisim. Alex

t_sato on April 22nd, 2010

[QUOTE=alexjc;49455]t_sato, The game certainly takes getting used to compared to most shooters. You have to be very aware of any presence (either on the map or on your HUD) and not expose yourself in any way... It does feel a bit random in terms of death at first, it's not ideal but it does better than Far Cry 2. [/QUOTE] I finished a series of campaigns two times(NORMAL and EXPERIENCED), but didn't notice the difference in map mode. But anyway, I opened a map frequently to take a good position(higher than enemies). I'm not sure if it is intended, but as in real, having higher position always gave me an advantage. Also, I noticed one thing. Friendly fires happened because NPCs cross line of fire. They look indifferent. That could happen more than once in any missions. You're right. This gets more interesting as I get used to. I like it!

If you'd like to add a comment or question on this page, simply log-in to the site. You can create an account from the sign-up page if necessary... It takes less than a minute!