A Game of Lies

How game theory can help make better decisions in Diplomacy and maybe in life

This piece has been inspired by two of my former colleagues from my Maastricht years, Christian Seel and Mehmet Ismail, who, in addition to their academic careers, have started creating content about their competitive boardgame matches, chess and shogi by the former, and backgammon by the latter. Like them, I have an online boardgame hobby, my poison of choice being the game Diplomacy, but unlike them, I don’t play tournaments. Nevertheless, I’m currently ranked 22nd on the largest site, webdiplomacy.net, so if there ever is a time when I can speak of a competitive online game with some authority, it is now. Rather than talking about specific matches, I present why Diplomacy is such a rich field experiment for many aspects of game theory and what insights a game theorist might have for many aspects of Diplomacy.

I’m a game theorist by profession and, among many things, a strategy board gamer in my spare time. Those who play with me regularly know that I’m quite average in board games. In fact my butt has been kicked so thoroughly and so many times by my group of friends (most of whom are not game theorists) in such a wide variety of games that I’ve recently developed a bit of an aversion to them. And while my professional identity has not yet been thrown into crisis because of this, I’ve often asked myself the question:

“if my knowledge of game theory can’t help me make better decisions in the sterile environment of strategy board games, how can I expect it to inform policy in more complex environments?”

And while I (think that I) understand (some of) the limits of game theory, and therefore know that not being able to play Agricola or Gaia Project super well is not really a failure of the field, I was really on the lookout for a board game where I can find some advantage of the game theory mindset.

I think Diplomacy is one such game. There are patterns of play that even a skilled board gamer will only learn after they encounter them enough times, but a game theorist might recognize immediately. It might be that I’m more invested in it or more suited for it than other types of board games, or it might be that my opposition in Diplomacy is not as strong as in my other board games. It may be that whatever success I have is independent of my game theory knowledge. In any case, at present, I believe they are not independent and I found Diplomacy to be a veritable treasure throve for game theory concepts which I collected in a handful of examples.

Quick overview of the rules

Diplomacy is a game of strategy and negotiation with a historical flavor. There are seven players who play as the Great Powers of Europe on the eve of the Great War: Austria, England, France, Germany, Italy, Russia, and Turkey. The game is played on a map of Europe that has 34 Supply Centers (shortened to Centers) on it. Each country has as many units, armies or fleets, as the number of Centers they currently own. Countries move their units around to fight each other and capture more Centers. The countries start with 3 Centers and 3 units, 2 armies and 1 fleet, except for England who starts with 2 fleets and 1 army, and Russia who starts with 4 Centers, 2 armies, and two fleets. The objective is to reach 18 Centers.

Every region on the map can hold at most a single unit with some sensible restrictions: sea regions can be occupied only by fleets, inland regions by armies, coastal land regions by either. Units can receive 5 orders:

Hold: The unit stays where it is.

Move: The unit moves to an adjacent region (obeying the sensible restrictions for fleets and armies), or armies can move to regions reachable by convoy.

Support hold: The unit supports the hold order of another unit in an adjacent region. Support hold orders on moving units are canceled, and the supporting unit holds instead. Support hold orders also cancel if a different unit moves on the supporting unit's region.

Support move: The unit supports another unit's move order into a region adjacent to both the supporter and the mover. If the supported unit does anything else other than that move order, the support move order is canceled, and the unit holds instead. Again, support move orders cancel if a different unit tries to move on the supporting unit's region (except if it's from the supported move's target region).

Convoy: Fleets ferry armies through bodies of water.

If a single unit moves to an empty region, it succeeds. If a unit moves to an occupied region, or two or more units move to the same empty region, combat occurs which is resolved like this: If a unit moving to a region is supported by a larger number of units than any other units moving into the same region, and the number of support holds in that region, then it succeeds, dislodging the unit occupying the region. In all other cases the move fails.

The play of the game is as follows:

1. Spring Phase: All countries issue orders to their units simultaneously.

2. Retreat: Success and failure of Spring moves evaluated, dislodged units retreat to empty provinces or are destroyed.

3. Autumn: All countries issue orders to their units simultaneously.

4. Retreats: Success and failure of Autumn moves evaluated, dislodged units retreat to empty provinces or are destroyed.

5. Winter Phase: Number of Centers held at the end of the Autumn Phase evaluated. Countries with more Centers than units can build in their empty Home Centers (if there are any), those with more units than Centers must remove units from the board.

The complexity of the game comes from communication. Players can talk to each other, make deals, pacts of non-aggression, or alliances. However, there’s no coordination mechanism that would enforce these deals, so players can just as well lie, break alliances, and betray one another. It's just seven people and their words.

The game is thus truly that of diplomacy, to have any hope of winning the game you must find allies. But, since the entire point of the game is to reach 18 Centers alone, you must prepare to betray those allies, and be prepared that they may betray you at any point.

The Big Game

A game of Diplomacy can end in one of two ways.

1. One player wins (called a "solo") by owning 18 or more Centers at the end of the Autumn phase.

2. All surviving players unanimously decide to end the game in a draw.

The most popular scoring system online works like this: If a solo occurs, the winning player receives all points in the pot, all others lose and go home with 0 no matter who else survived to the end. If there is a draw, the drawing players distribute the pot equally no matter the number of Centers currently held by the players. So let’s assign a payoff of 1 for winning the game by solo, a payoff of 1/n for an n-way draw, and a payoff of 0 if someone other than you wins the game or the game ends in a draw but you were eliminated.

Diplomacy is a dynamic game, which are usually not too easy to solve, even when small. And this game is not small. For comparison, chess has 20 legal first moves and 400 first-move-second-move combinations. Diplomacy has over 5.000.000.000.000.000, that is 5 million billion first moves (just counting the number of possible move and hold orders). If we restrict the number of openings any country can play to just 3 to eliminate some bad-looking openings (a very conservative estimate) there are still over 2000 first moves.

Even if we can get around the size of the game using some miracle computer, game theory will still not help you solve the game as it would for games like chess. This is because chess is a sequential game with perfect and complete information: we can define and even calculate a play that is "optimal". We can thus find a solution to chess and we can predict that under this optimal play, either white has a winning strategy, black has a winning strategy, or the game is always a draw if both players play perfectly. There is no such hope with Diplomacy. Since the players move simultaneously, there is no single optimal play, just as there is no single optimal play in the rock-paper-scissors game.

But, unlike rock-paper-scissors, we can still say something about playing the game "well" and "not well" in many situations, and game theory is instrumental in finding and resolving these situations!

Example #1: The Prisoner's Dilemma

"It's good to trust others, but not to do so is much better."

Benito Mussolini

To understand the type of insight that game theory provides, let's look at the relationship between France and England at the very start of the game. England and France are neighbors which puts them in an excellent position to either work together closely or get into an early tussle. The key region in their relationship is the English Channel. Bordering the French home Center of Brest and the English home Center of London, as well as the neutral Center of Belgium, whoever controls the Channel in Autumn 1901 takes a huge advantage in case of a conflict. If left empty, however, the two countries can avoid coming to blows and focus on growth and conquest elsewhere.

Suppose that France has made up his mind on two of his three starting units: he wants to move Paris to Burgundy to challenge either Belgium or Munich, and Marseilles to Spain, to pick up one of Spain or Portugal. The question is where to move with Brest? Two options present themselves: Either move to the English Channel towards England or to the Mid-Atlantic-Ocean (MAO) to pick up the other Iberian Center by the end of the first year. This is what the dilemma looks like on the map:

On the other shore, England is presented with a similar choice. Suppose that the decision to move one of the starting fleets to North Sea is made, eyeing Belgium, Holland, Denmark, and Norway, as well the move by the starting army to Yorkshire. The question is, should London move to the Channel towards France, leaving Edinburgh to move to the North Sea, or should it be London moving to North Sea, and Edinburgh to Norwegian Sea, picking up Norway from the Norwegian Sea fleet and use the North sea fleet to convoy to another neutral Center like Belgium, Holland, or Denmark?

This is England's dilemma on the map:

If both countries decide to move to the Channel, then these moves cancel out and both countries' fleets return to their starting points. In this case both wasted their moves as there's no Center in reach for them to capture in the Autumn. It would have been better for both to choose the other opening and be able to use their fleets productively in the Autumn period.

Capturing the Channel, however, puts either country in a secure position with many opportunities: they have the option to invade the other country or go for neutral Belgium instead. The opponent, on the other hand now must either return to the home Center to prevent an attack and waste both moves in the first year, or continue towards a neutral Center at the risk of losing the home Center.

We can't map this dilemma with exact payoffs, but we can still make a qualitative payoff-table, that would look something like this:

France

Channel MAO

Channel bad, bad very good, very bad

England

Norwegian very bad, very good good, good

The first phrase describes England's position and the second phrase describes France's.

We have a prisoner's dilemma on our hands: no matter what the other power does, both France and England is better off opening towards the Channel than opening towards a neutral Center, but the Channel-Channel outcome is worse for both than the Norwegian-MAO outcome.

What insight does game theory provide? Leaving the channel open as France or England is either a risky gamble, a wasted opportunity, or a big leap of faith for your alliance.

Example #2: Matching Pennies with a dominated option.

"He who defends everything, defends nothing" Frederick the Great

The second example is less flashy, but it highlights the bread-and-butter of tactics. Take this scenario: Austria and Russia agreed to ally against Turkey at the start of the game. To solidify this, they made a pact that neither will occupy Galicia, which is basically their equivalent of the English Channel; whoever controls it will dominate the conflict between the two, while leaving it empty is a good way to avoid hostilities.

In this case, Austria followed the agreement and made the following moves: Army Vienna to Budapest, Army Budapest to Serbia, Fleet Trieste to Albania. Russia, however, dishonored the pact, moving Army Warsaw to Galicia, Army Moscow to Ukraine and Fleet Sevastopol to Black Sea, directly threatening Budapest and Vienna.

Now Austria can defend both by moving Budapest to Vienna and Serbia to Budapest, so no matter which one Russia attacks with the Galician army, he will be able to bounce it back. Notice that no matter which one Russia attacks, Austria's Serbia army will stay in Serbia -- either because Austria and Russia bounce in Vienna, in which case Serbia can't move because Austria's own Budapest army is in the way, or because Austria and Russia bounce in Budapest. This is good because this way Austria secures 1 new Center for the Winter adjustment phase.

On the other hand, however, Russia may decide to move to neither, instead moving Galicia to Rumania supported by Ukraine and Black Sea. In this case the Austrian armies just march back home. Turkey is pretty sure to move his Bulgaria army to Greece in the Autumn, so Austria will not be able to take Greece with the fleet in Albania unless the Serbia army supports it. (An important note here: the fact that Austria visits Serbia in the Spring doesn’t count for the winter adjustment phase if he does not have an army stationed there at the end of Autumn. On the map this is represented by Centers not switching color in the Spring moves). So, in the end Austria is left with 0 new captured Centers.

Just looking at Austria's options, it seems clear that Both is an inferior choice as no matter what Russia does, Austria is better off defending either Budapest or Vienna. Similarly, Russia is better off by not choosing the Neither strategy, and instead attacking Austria's home. Once you eliminate the dominated options, you get the Matching Pennies game.

What insight does game theory provide? You should first make a list of your options and then do some thinking to see if you can toss out any that are dominated. Many beginner Austrias defend both their home Centers to avoid the risk of guessing wrong, but here the risky options are the best.

Finding friends

Even with a solid tactical effort one country has a hard time beating another, let alone a coalition of many countries. The only way to succeed is to find allies. I find that the tools of game theory are extremely helpful in identifying which alliances help you accomplish which strategic goals and how to negotiate in them.

The network approach

Theoretically any set of countries can decide to be allies, but practically the options are limited. The main limitation is that most communications are bilateral: it's only pairs of countries chatting with one another. Because of this, I focus on the bilateral relationships, which can mean only one thing: networks! So, let’s simplify the map to a network with 7 nodes with links signifying geographic neighborhood.

As Italy, you have the option of being at war with Austria, France, and Turkey. It is up to you and your neighbors if you fight these wars sooner or later in the game and the network helps you formulate a game plan.

For instance, you can start the game by securing your Western border by having a pact of non-aggression with France and move East by offering Turkey to ally against Austria, a common rival (top left panel). At the same time, you can approach Russia for a future alliance against Turkey who will become a common rival once Austria is defeated (top right panel). If it works, you're almost guaranteed to end up having to fight Russia, for which you may try enlisting Germany's help if he's still around (bottom panel).

The other advantage of the network thinking is defensive. If your planned alliances don’t work out but others’ do, you’ll have to be reactive in your diplomacy. As my hypothetical Italy with the above plan, if the France pact doesn’t work out, or you see evidence of a Russo-Turkish alliance, you're well advised to reach out to Austria even if your opening moves were directed against him. Continued warfare between the two of you may lead to Austria’s quick death, but you’re going to be next, and you’ll be out of options for useful allies against Turkey.

In short, the network mindset helps in identifying potential friends and foes, lets you prepare a long-term strategy by detecting future friends and foes, and helps you be reactive in your diplomacy based on what other alliances appear.

(Non-)zero-sum games

Securing alliances is much more than just cheap talking each other. It’s about finding mutually beneficial goals and signaling with your moves that you’re willing to act on them. The best time to do this is the early game, when there are 12 neutral Centers on the board waiting to be shared. The opening stage is the best chance you get to show your willingness to bargain, your ability to find mutually beneficial deals, and be altruistic. Overlooking the benefits of altruism, especially in this cut-throat setting is a common mistake as not all people understand the non-zero-sum property of the early game.

Example #3: The Swedish question

Take the relationship between Germany and Russia which is usually established by how they handle Sweden. You'll often see an opening move from Germany into Denmark with his starting fleet in Kiel, while Russia almost always opens to the Gulf of Bothnia with St. Petersburg. If England chooses to open towards Norway, then the scramble for Scandinavia will look like this at the start of Autumn 1901:

Russia is pretty much guaranteed to move to Sweden in Autumn 1901. Germany can either move Denmark to Sweden in order to bounce Russia out or hold in Denmark and gift Sweden to Russia. In either case Germany picks up Denmark in the Autumn. Now if you've seen my payoff tables and the network map you might be thinking there's no reason whatsoever for Germany to just hand Russia a free Center.

And this would be true if it wasn't for England. Russia, uniquely for a corner nation has 4 neighbors with at least one of Austria and Turkey being hostile to them in almost every game. England’s move to the North is a clear signal that he intends to go to war with Russia for Northern dominance. With Russia stretched across the map from Turkey to Norway he doesn't stand much of a chance if Germany also messes with them. And an England that controls St Petersburg usually spells disaster for Germany.

So, Germany is very much advised to take advantage of the non-zero-sum nature of the Sweden situation and consider letting Russia into Sweden to weaken England in the future!

The second common early game mistake is to think too much in terms of the non-zero-sum nature of the first year and forgetting about the incipient mid-game which is very much zero-sum in Center terms. If Germany frames the Sweden situation as a gift to Russia, then he has failed to get the most out of the early game. Instead of deciding in the Autumn, Germany is advised to negotiate the fate of Sweden BEFORE making the Spring moves of 1901 with Russia AND England! The possibility of a Sweden gift can be used as a reward for Russia in exchange for an anti-English opening or as a threat to goad England into an anti-French one! Or both!

In short, take every opportunity to find deals that are mutually beneficial even if the benefits are indirect - such as Germany's Sweden decision. But don't forget that today's friends are tomorrow's rivals in this game; make sure you get enough out of the deal to justify gifting free Centers.

The end of an alliance

We've talked about how to form alliances but by far the more interesting part is when they end. We've seen in the networks example with Italy that alliances can end because the common goal has been achieved, but there are two other reasons for a split, both having very much to do with game theory! If you recognize the games, you can get the most out of these splits.

Example #4: The ultimatum game

“Never accept ultimatums, conventional wisdom, or absolutes”

Superman

Take the following situation: Germany had a strong early game while France has been struggling against England. Soon enough though, the two countries unite against England and push him back. At this point, Germany is on 6 Centers and France is on 5. They are now on the verge of taking the fight to English home territory when this happens:

The off-map situation is the following: France and Italy are at peace with no units on each other's borders, Italy and Austria are allies against Turkey and Russia, and are fighting a war without much progress made. Germany and Russia are at peace, but Germany is much stronger on their common border due to Russia's entanglement down in the South.

With the convoy to England, Germany makes a claim on the English home Centers. This is basically an ultimatum to France: he can either respect this and try to expand against Italy with whom he is currently at peace or challenge it and go to war with Germany. The problem is that France is already a junior partner in the alliance with Germany. If he accepts German conquest of Britain, Germany will become a superpower right next door, quite likely taking up to 11 centers (3 in Britain, 1 in Norway, 1 possibly in Warsaw) before France gets his 6th from Italy (Tunis, which Italy can defend easily as France will take long to move units into the Mediterranean Sea). Continuing the alliance leaves France either subservient to Germany's goals or his next target. On the other hand, Germany is currently stronger than France with 6 German Centers to 5 French ones.

Before this phase, Germany was facing a choice: either push into English home territory or attack Russia in Scandinavia and Poland. He clearly chose the former but let let's discuss the alternative. If Germany had claimed Scandinavia instead, both would have gone on to take 8 centers in very short order: Germany could have taken Norway and St Petersburg or Warsaw, while France could've taken the English home Centers. It would have been a win-win for them with close to equal number of Centers. There's plenty of ways their alliance could have broken down in the future, but by that time both would have been in a stronger position compared to the other nations on the board.

Let me try to graph this into a sequential game.

Game theorists will recognize this as a version of the ultimatum game. The only equilibrium solution of this game is for Germany to Claim England and France to attack Italy but there are many reasons to expect the other outcomes as well.

If France can sell that upon seeing the greedy option (Claim England), he will attack Germany as punishment, Germany might reconsider and go for the mutually beneficial option instead. On the other hand, if Germany ignores Frances's request, France may choose the suboptimal option (War with Germany) due to the broken trust.

In short, France must recognize the ultimatum game as it's happening and must do everything diplomatically to prevent Germany from choosing the greedy option. Germany on the other hand needs to prepare for France’s adverse reaction towards him.

Which brings me to the next example.

Example #4: The spite game

“If an injury has to be done to a man it should be so severe that his vengeance need not be feared.”

Niccolò Machiavelli

By far the most interesting, satisfying, and frustrating part of the game is backstabbing, where a country turns on a former ally. If unanticipated, a backstab can cause immense damage to the victim and give an insurmountable advantage to the aggressor in his new war.

Backstabbing works best when the target is already fighting a war he cannot easily disengage from. What's often underestimated, however, is the target's willingness to harm the backstabber even if this increases the risk of his own elimination. This can take several forms: The backstabbed victim may ally his former foes to try to eliminate the backstabber even if this helps the former foes win the game. They can even allow those foes to take over their territory while doing their best to deny the backstabber from any further conquests, intentionally throwing the game. This may seem like irrational play but that’s a word too often thrown around.

Take this example: Italy, Austria, and Russia are allies against Turkey. Secretly, Italy and Russia had agreed to take out Austria once Turkey is gone (recall the network approach). Austria has a chance to betray Italy and take Venice (after which the rest of Italy would fall quickly) but decides to keep his alliance with the two and attacks Germany instead. Meanwhile Russia is having trouble dealing with a stray Turkish unit that made it to his home territory as Turkey’s own country is getting dismembered. This is what the South-Eastern corner of the map looks like at a critical point of the game.

As Russia is quite powerful with no strong opposition, the optimal play for Austria and Italy is to band together against him. Instead, Italy becomes greedy, and as per his agreement with Russia, he attacks Austria before Russia could attack Austria from the other side (I was the one playing Russia here and I was delaying intentionally).

For Austria this is a betrayal by Italy! The betrayal notwithstanding, the optimal play for both is still to ally against Russia. Instead, Austria moves all units from his German front, as well as the Balkans to fight Italy.

This allows Russia to sweep through Austrian home territory and the Balkans, eventually winning the game by solo. Here’s the game as a game theorist might see it:

The payoffs are my guess for the likely Centers that each country might have taken had they followed these strategies (except for the second Attack Italy outcome where both countries get 0 since Russia wins in that case).

The equilibrium of the game, which we can get by backward induction, is for Austria to Attack Italy in the first decision node. This is because, if Austria’s second decision node is reached, his optimal choice is to Attack Russia, so Italy, anticipating this, will Attack Austria in his own decision node rather than go for the jointly optimal outcome of Attack Germany followed by Attack Russia. Knowing this, Austria’s best choice is to break the alliance immediately. Instead, Austria went for the cooperative outcome hoping Italy would reciprocate. Italy didn’t, so Austria got angry and made it his business to ruin Italy’s game.

There are several aspects to disentangle here. One is of course the emotion of spite Austria felt against a friend-turned-foe which can easily overshadow the desire or the ability to play optimally and stop Russia from winning. Many backstabbers are taken aback by this and then deride their opponents for not even trying to play to win.

It takes a high level of emotional awareness and a deep understanding of the evolutionary role that spite has played in the development of human societies to understand, respect, and anticipate the punishment strategy played by Austria. Game theory gives a shortcut through the Folk Theorem. Simply put, if someone plays selfishly, it is in the best interest of society to punish them and thus ensure cooperation in the future. Most games we play don’t end at the final whistle of the match. There’s always the next match, the next board game, the next rest of your life to play and our emotions are there to provide a shorthand on how to play them.

In short, in an alliance, it can be useful to appear vengeful to give potential backstabbers a second thought. As a backstabber, never underestimate your victim’s willingness to hurt you even if it’s costly for them.

The emotional highs and lows of Diplomacy

Even though I enjoy the game greatly, I cannot recommend it to everyone. Emotions, useful as they are, can be hard to handle in the game the same way they are in life. Many people don’t need feelings of suspicion or anger when they’re trying to have fun. Even when the game is going well for you, it is natural to feel guilty for the lies you told. Again, these are hard-wired evolutionary shortcuts to help you play the game of life by being a more socially compatible person. If you’re not prepared for them, you’re not prepared for Diplomacy either.

Then again, I think most of us could benefit from some emotional training and Diplomacy is a sterile environment to do so. Add to that the complexity of the strategy and tactics and the infinite opportunities that the use of language provides, and you have a truly unique and exhilarating cocktail that challenges you in many ways. If you’re up for it, hit me up on webdiplomacy.net, my username is boylee.

October 3, 2020.

Footnotes and links:

1. The matching pennies game is a two-strategy equivalent of the rock-paper-scissors game with no ties. Player 1 wins if they choose the same strategy as player 2, otherwise player 2 wins.

2. Here’s a funny illustrative drawing about the Prisoners’ Dilemma game and how it relates to morality: https://www.smbc-comics.com/index.php?db=comics&id=1899.

3. For more advanced diplomacy players here’s further reading on the diplomacy and game theory topic https://www.lesswrong.com/posts/jkf2YjuH8Z2E7hKBA/diplomacy-as-a-game-theory-laboratory.

4. If you start playing Diplomacy it’s good to keep in mind that you’re much more likely a lose the game than to win it. Below is a table of outcome chances from data collected from tens of thousands of games. Overall, 45% of games end in solo wins, 55% end in draws.

5. The fact that I’m ranked so highly is probably an example of the law of small numbers: a few lucky wins and draws can propel you high on the list and I've been playing low-risk style ever since. If I’m proud of any Diplomacy-related accomplishment, it is my 24% win rate and my 77% success rate (solo + draw).

Solo Draw Survived Eliminated Resigned

England 6,12% 32,46% 18,79% 35,52% 7,11%

France 7,63% 36,23% 19,90% 29,44% 6,79%

Italy 4,73% 28,53% 18,95% 42,27% 5,52%

Germany 6,04% 29,00% 15,97% 42,63% 6,36%

Austria 5,13% 25,78% 12,67% 52,24% 4,17%

Turkey 6,75% 31,94% 19,35% 35,96% 6,00%

Russia 8,34% 25,82% 14,22% 46,05% 5,56%

Comments and questions should be addressed to peter.bayer7@gmail.com