Since I began writing this column for AiGameDev.com a month ago, Alex and I have occasionally spent an amusingly large amount of time deciding on the titles for the articles. People who have seen my writing before know that I am often inclined to tongue-in-cheek or intentional double-meaning in order to illustrate a point. This column’s title is one such example… and one that I won’t let Alex edit because the question that it asks is very illustrative of a disturbing quandary that AI programmers face.
You may recognize the loaded question that is being parodied by this title: “Does this [clothing] make my [body part] look [good/bad]?” It is a comedy staple due almost entirely to the reaction that it elicits from people (mostly men) as they decide how to split the difference on the equally-damning answers. Is honesty the best policy, however? If he lies to her, isn’t he helping her lie to himself? And if she decides that she does look ridiculous, is she going to hold it against him for not saying so in the first place? We truly feel for the poor sop as he opens his mouth knowing that, no matter how he decides to answer, he’s going to wish that he had chosen the other route (or none at all). And does she really want to hear the answer anyway?
The title question can be thought of in much the similar vein. Instinctively, we want to answer “no”. It’s intuitively wrong. A blatant contradiction contained in one eight word package. After all, since the inception of game AI, we have been trying to have our agents make better decisions, not worse ones. We have been trying to eliminate stupid behaviors, not encourage them. We have been striving for more realism, not less… but wait a minute… Isn’t this where we might be lying to ourselves? And do we really want to hear the answer if it means we have been wrong all along?
A Rational Definition of Rational
John von Neumann was the father of game theory and, in a broader sense, decision theory. Most AI programmers who are familiar with his minimax model also know that one of the requisite premises of applying it is perfect knowledge of the game environment. Given that, the minimax model is spectacular in calculating the best possible move at that moment. Even when faced with uncertainty (i.e. imperfect knowledge) surrounding the game space, there exists an entire arsenal from which we can draw. Bayesian math, for example, helps us to fill in the logical gaps that may occur when we don’t know the whole story. However, this is still one shortcoming that John von Neumann, his writing partner Oskar Morgenstern, and even the legendary game theorist John Nash all acknowledged. All this fancy math to predict decisions based on maximized return is based on the premise that the decision maker is rational — which, much to the dismay of many, is not always the case.
“All this fancy math to predict decisions based on maximized return is based on the premise that the decision maker is rational — which is not always the case.”
The issue that the above-named luminaries were concerned with came to the fore in a roundabout way. Their algorithm had to generate a decision based on what the same algorithm said the other side would do. That is, “if the other player makes his best decision, it will be X, to which I will respond with Y.” However, all of it fell on its face if the other party was not rational — and therefore could not be counted on to make the “best” decision. The point is, humans were not rational — or at least not perceptive and calculating enough to determine what that maximized solution to any given problem may be. (Rational, as used here, is almost a catch-all for omniscient, perfectly calculating, unemotional, and other such non-human traits.)
One common situation would be where the competing options are not significantly different enough from each other so that it is apparent that one is the “best”. A purely mathematical approach may be able to quickly calculate the intricacies and assign a score to the options from which a rank can be determined. For humans, however, this is not always an intuitive process. Let’s face it, there are entire self-help books dedicated to teaching people how to construct a simple “pro/con list” — and that doesn’t even get into the subjectivity that may be involved with assigning values to items on the list. Purely rational — i.e. algorithmic — decision making is just not something we excel at. Often, we have to “go with our gut” — which, if not perfect, may at least lead to a reasonable decision.
But that’s not how we program our AI, is it? As I mentioned above, in modeling our decisions, we have constantly been focused on finding the best that we can accomplish. The A* algorithm finds the best path. We take great comfort in the fact that A* even guarantees that it will find that best path or our money back. We can roll up the n-depth tree search with thousands of leaf nodes into a handful of individually scored decisions, pick the highest ranked one from the pile, and feel quite comfortable that we have selected the “best” possible path. In fact, if our AI is criticized for being not human, we instinctively look for ways to make our best decisions even better! All of this is based on that the same fallacious ideal that haunted John von Neumann — “Humans are always rational; Humans always make the best available decision.”
What is Better Than the Best?
If their algorithms were to accurately model human behavior, the fathers of decision theory had to eventually accept, account for, and try to model the very fact that humans do not always act rationally. This was a daunting task that has only recently been addressed by delving into the realm of psychology. Thankfully, this field is well beyond the scope of this article. (Although I do recommend “A Beautiful Math” for light reading on the subject.) One advantage that we have over von Neumann and Nash, however, is that they were trying to accurately predict human behavior whereas we are only trying to model a sample of it.
Even with that disclaimer in force, the spectre of the imperfection of perfection still looms over us. If the optimal decision isn’t the best one, then what is? Certainly the collection of “good” decisions will tend more towards that optimal one than they will towards a purely random one. Constructing the stochastic equivalent of a blindfolded dart-throw isn’t going to be the answer. In the situation where there are multiple, similarly scored alternatives, picking amongst them could generate a unique collection of behaviors that isn’t precisely deterministic. No matter how we select things, we would want to limit the choices to ones that that are, if not the optimal ones, at least somewhat reasonable. Anything blatantly stupid only makes us just look bad. No amount of explanation of how “we were modeling a form of stress-induced psychosis” will rescue us at that point. Therefore, our “mistakes” (I’m beginning to prefer “inconsistencies” just to avoid the stigma) must be on an order of degrees rather than magnitudes.
“Your illogical approach to chess does have its advantages on occasion, Captain.”
— Mr. Spock to Captain Kirk in “Charlie X”
So, the Question is…
Again, a proper treatment of this subject would go well beyond the caffeine supply I have in the house tonight — and strays somewhat from the purpose of the column. The quandary that I want to pose here is not a technical one - that is how we could make our agents less than optimal. What I want to ask is more subjective (and therefore bound to be delightfully contriversial). Should we allow (or force) our agents to occasionally not do their best… to sometimes not select the best weapon… to once in a while not dash for the perfect cover… to typically not make impossibly swift analysis and decisions (see last week’s column)… should we allow our agents to make mistakes - and thereby look human? In fact, if our agents ever are going to look more human, don’t they almost have to make mistakes?
Can we seriously walk into a room and ask our designers, producers, publishers, reviewers, and ultimately our players, the loaded question: “Does this mistake make my AI look smarter?” Or maybe it is just better not to tell them that our agents aren’t “perfect”. That we actually designed them to be that way. It may be safer than asking the question. I know that I cringe at the thought of the answer.
One thing is for sure… if we get it wrong, they are not likely to be delicate about their answer. (And do we really want to hear the answer anyway?)
