I've been enjoying the OPERATION FLASHPOINT: DRAGON RISING campaign recently. In brief, the game is a large-scale open-world sandbox tactical combat simulation whose feature list would make programmers around the world whimper. The AI does a pretty impressive job considering the challenges!
My playthrough, however, has occasionally thrown unintended challenges at me — in particular reloading automatic checkpoints that end in disasters within 15 seconds. But rather than start the mission from scratch, this obviously seemed to me like an invitation to play around with the simulation and figure out its underlying mechanics...
In this short article, I'll look into a particular scene that ends in one of my rescue targets being shot at close range while I'm rushing away towards the pick-up zone like a coward. This raises interesting questions about determinism, level of detail and statistics — and makes a perfect topic for a developer discussion!
It Doesn't End Well (Mostly)...
Screenshot 1: In the full 3D game, the pilot (left) ends up being shot at close range by the enemy soldier (middle) and often shoots me trying to grab a good screenshot too!
OF:DR is not a deterministic game, like a majority of first-person shooters. Everytime I reload the checkpoint something slightly different happens, even when I don't move a muscle. When I move, obviously, things turn out even more differently. Due to the Sandbox nature of the game, this turns out to be much more interesting to play as you're forced to keep rethinking situations even after you reload...
However, in my situation, the pilot I was supposed to rescue almost always ends up being shot at close range by an enemy soldier. He occasionally escapes, but as you'd expect from an un-escorted target, he indeed bites the bullet a majority of the time — despite my best efforts to find creative solutions to this predicament!
Trying to make the most out of the ~15 seconds I had at my disposal to save the pilot, I tried various orders to my squad from the game, but switched to the map view when that failed. That's where things got interesting...
Safety in Map Mode
Screenshot 2: Switching to the map view, my squad (white) manages to shoot the enemy soldier (red) while moving around the center building, or the pilot (in blue) manages to sneak past without being harmed.
By switching to the top-down map view, the statistics of the outcome changed significantly. Most of the time, the pilot managed to escape from the building he was being ambushed in! The ratio of success to failure was reversed, now 80% to 20%.
Again, this wasn't guaranteed, but two new situations were the most common in map mode (that never happened otherwise while in 3D):
If I switched to the map instantly, my squad would manage to take out the enemy threat almost immediately while moving towards my position. Without switching the map view, they'd never deal with the enemy soldier and simply end up just following me.
When waiting a moment before switching to the map, the pilot managed to sneak past the enemy target in many situations. He did occasionally get hit but that was an exception rather than a rule, unlike it was previously...
The question arises, what could cause such a radical change in the outcome of this situation, at the mere press of the 'M' key to engage the map?
Combat Simulation Statistics...
There are two ways to interpret such a large deviation in the simulation:
There's a different simulation that kicks in when you enter map mode. This different simulation operates in such a simplified fashion (e.g. no animation) compared to the full detailed simulation that the two different encounters end up differently in practice.
The game's non-determinism is significantly affected by entering map mode. It's possible that disabling the 3D rendering allows more resources to be spent on AI. My squad responded better and the rescue target managed to deal with his threat more capably!
It's certain that OF:DR has a level of detail implementation. You couldn't build such a game without it, and in this developer commentary on Intel's site, they mention that the animation in particular is handled differently. However, it's unlikely that the simulation would be affected by the LOD just by the press of a key to switch into map mode.
Screenshot 3: My squad follows me around when I try to run to the target rendez-vous point like a coward rather than defend the pilot!
The Impact of Non-Determinism
This kind of in-game situation (and I guess this article too) raises more questions than it does answers.
Does it matter that the same situations play out significantly depending on the hardware configuration and rendering settings?
Assuming you want to preserve the performance benefits of non-determinism, how can you make sure your simulation behaves similarly everwhere?
How many developers treat different hardware configurations (PC) as different SKUs and balance each of them accordingly?
If the game's simulation statistics can be affected significantly at runtime (in different situations), will it even be noticed or is it a loophole?
Is determinism a solution to all of these problems, or does it just mean we implement a lowest common denominator for gameplay and AI?
If you have any thoughts on the subject, feel free to post a comment below!
Discussion 19 Comments
was the game fun? .
Perhaps the AI behaves differently: - in 1st person mode it follows you and tries to be visible so you know it works (aka unseen intelligent behavior is no intelligent behavior). - in map mode the AI is visible wherever it is - to look intelligent it doesn't need to be near you so other behaviors and goals are possible. I haven't played the game so this is possibly completely wrong.
Jad, Should I deduce from that cryptic comment that you don't think that determinism matters at all for gameplay? :-) As I wrote in the introduction, I've generally been enjoying the game but the automatic checkpoint logic is broken and if you run into the wrong place a bit early then it'll save the game while you're in serious trouble. That's what caused the experiments I described, and while it was fun seeing this situation play out differently, when it happened before near a helicopter I could not salvage the situation and it was a frustrating game-breaking experience. (Radu apparently gave up on the game because of this.) I think players in single player campaigns will naturally end up doing the things that are fun (or quit) so even if the game is 10x easier from map mode it may not matter at all. What concerns me is that I could change my screen resolution and potentially get a very different game experience. That kind-of ruins all the balancing the designers did, so if you consider that as unimportant then non-determinism also doesn't matter either. Random games FTW! Alex
As an addition to my previous post, I don't think non-determinism on consoles or the same PC hardware (without LOD) is much of a problem. For example, if your floating point calculations are slightly different, then the simulation will slowly drift out of sync., but it'll mostly be the same. What's interesting to think about is how significant non-determinism (like in this case) can affect gameplay. Bjoern, indeed. Both those options are possible! It's a bit hard to say, I'll see if I can find more information. Alex
For most games, I think deterministic Gameplay/AI is important for many reasons, this example points out one of them. When non-logical actions (e.g. taking your eyes away from the action by looking at a map or just looking down) improve your chances for success, then immersion is lost. I'm guessing most people who encounter this issue will say "This game is broken" and I'd say they're right.
Unsure if determinism is important for the player but it is unquestionable important to enable debugging and make Q&A life easier. Interestingly the situation described above - automatic saving at a very bad moment - could be solved by concepts like Valve's AI Director - where a save-game is analyzed and if it questionable to survive the next seconds the game could change some parameters (health, aggression of NPCs, etc.) so the player doesn't need to restart the level from the beginning which could mean a substantial time-loss. Cheers, Bjoern
[QUOTE=bknafla;49409]Unsure if determinism is important for the player but it is unquestionable important to enable debugging and make Q&A life easier.[/QUOTE] At least that's the common wisdom, but not many games do it! I think this example definitely raises more questions about the gameplay perspective... Doug, I wonder if you'd have noticed or just quit in frustration without an explanation. Alex
It'd be interesting to see if changing graphics settings does change the simulation, rather than just speculating. It is possible that the friendly AI do things differently when the player is in map mode to when the player is running around shooting. If the player is actively moving and shooting, the friendly AI may hold off in order to let the player feel like he's more important and "being the hero". But if the player is looking at the map, you may want the friendly AI to be a little more autonomous. It would really suck if the player took a moment to look at the map and got shot in the head when a friendly could have seen & shot the sniper (ok, it's more 'realistic' for the player to get shot in the head while looking at the map, but it's not necessarily fun). My point is that it might be a game-play design decision to make enemies less accurate and friendlies more accurate when the player is looking at the map.
[QUOTE=alexjc;49416]Doug, I wonder if you'd have noticed or just quit in frustration without an explanation.[/QUOTE] I would have noticed because I write & break gameplay code for a living :), but I'm not sure if a 'normal' player would have. This sort of non-determinism drives me crazy as a programmer. I had to bite my tongue at an Intel lecture teaching us how to make all our games behave randomly depending on the hardware load (and how to destroy your game balance when computers ship with more cores).
Didn't Halo randomize NPC positions when loading a check point? Might have helped reduce the size of the save game, and added some degree of non-determinism.
[QUOTE=SilentBob;49421]My point is that it might be a game-play design decision to make enemies less accurate and friendlies more accurate when the player is looking at the map.[/QUOTE] That sounds really a good compromise. Right after having read this article, I bought this on Steam, played, and often get killed without recognizing the situation ... I always feel unfair in the beginning when playing FPS. NPCs know everything, but I don't. They can shoot me through bush from where I can't see. In this game, even a single shot puts an end to me. Yes, opening a map is as good as telling "it's not fair!".
t_sato, The game certainly takes getting used to compared to most shooters. You have to be very aware of any presence (either on the map or on your HUD) and not expose yourself in any way... It does feel a bit random in terms of death at first, it's not ideal but it does better than Far Cry 2. SilentBob, You're absolutely right, here are some interesting statistics. I reloaded the game 10 times for each mode. It's not statistically significant, but it's the best I can do with the time I have without having the codebase :-) I basically reloaded the checkpoint and did not move at all. For map mode I tried to press 'M' as soon as possible. 'K' indicates the pilot was killed, and 'L' indicates he lived. [CODE] 640x480 LKKLLLKLKL 1920x1200 LKKKKLLLKK MAP MODE KLLLLLLKLK [/CODE] That's basically 70% survival rate in the map mode, 60% survival rate in 640x480, and 40% survival rate in 1920x1200 with full graphics turned on and MSAA 8x. In map mode, I felt it was more like a skill. The times the pilot died were when I didn't press as fast as the other times. It think I could hit at least 80% survival rate for the map mode. While then numbers are certainly not scientific, it does point towards a frame-rate dependent AI problem -- maybe on top of the design decision to improve the AI in map mode. Interesting! Alex
Maybe we are all taking this too far and it simply is a big bad bug/sloppy coding?
[QUOTE=jadnohra;49456]Maybe we are all taking this too far and it simply is a big bad bug/sloppy coding?[/QUOTE] :-) This particular train of thought may have been triggered by an autosave bug, true. But there's lots to be learned I think... Plus you normally love digging into topics even if they are philosophical -- why not this one? Alex
[QUOTE=jadnohra;49456]Maybe we are all taking this too far and it simply is a big bad bug/sloppy coding?[/QUOTE] I'm not sure if it is sloppy coding or a sloppy idea. There is a big push to make everything highly scalable (which is important, since we are no longer going to get a free-ride with higher clock speeds), but the techniques being pushed by Intel and other companies tend to sacrifice determinism. This is fine for things like graphics and sounds but (IMHO), not so great for gameplay and AI.
[QUOTE=Doug;49459]This is fine for things like graphics and sounds but (IMHO), not so great for gameplay and AI.[/QUOTE] Do you think their AI is such that it would not cope if it was not parallelized to such a degree that it would loose determinism? I don't think parallelization is an acceptable excuse if it was really the cause. AJC: what gave you the impression I don't like digging into this?
[QUOTE=jadnohra;49461]Do you think their AI is such that it would not cope if it was not parallelized to such a degree that it would loose determinism? I don't think parallelization is an acceptable excuse if it was really the cause.[/QUOTE] My guess is that they could have written their AI to be both highly parallelized and deterministic, but they didn't think determinism was important for their game. The reason I think this is because of the recent Intel lectures I went to at GDC2010. They presented a lot of techniques for threading AI, but didn't address determinism.
[QUOTE=Doug;49467]The reason I think this is because of the recent Intel lectures I went to at GDC2010. They presented a lot of techniques for threading AI, but didn't address determinism.[/QUOTE] From our experience with TBB, there's not much about multi-threading that makes your code non-deterministic. If you have a fixed workload and homogeneous computation resources (i.e. the same processor types) then you'll end up with the same results. This particular case in OFDR could be just as easily caused by a single threaded implementation that has a timer to scale the AI calculations based on the amount of time that's available. Whether you're multi-threaded or not, that kind of time-dependent behavior just doesn't work for determinisim. Alex
[QUOTE=alexjc;49455]t_sato, The game certainly takes getting used to compared to most shooters. You have to be very aware of any presence (either on the map or on your HUD) and not expose yourself in any way... It does feel a bit random in terms of death at first, it's not ideal but it does better than Far Cry 2. [/QUOTE] I finished a series of campaigns two times(NORMAL and EXPERIENCED), but didn't notice the difference in map mode. But anyway, I opened a map frequently to take a good position(higher than enemies). I'm not sure if it is intended, but as in real, having higher position always gave me an advantage. Also, I noticed one thing. Friendly fires happened because NPCs cross line of fire. They look indifferent. That could happen more than once in any missions. You're right. This gets more interesting as I get used to. I like it!