Péter Bayer - Blog 8: Brexit and the Condorcet paradox

Blog 8: Brexit and the Condorcet Paradox

Brexit and the Condorcet Paradox

An evaluation of the British withdrawal from the EU through social choice theory

Unlike most discussions on Brexit which focus on isolationism, migration, terror, trade, or campaign lies, this article investigates the social choice aspects. Although I don’t claim that the other concerns are not valid or not interesting, I assert that we've reached a point when the discussion can only be advanced by abstraction. Therefore, I present no overt opinion on the matter of Brexit. I claim that the division of the British electorate on Brexit features a Condorcet cycle, try to prove this point with available data, and review the possible solutions that have been put forward from a theoretical perspective.

A disclaimer is in order: While I am indeed a fledgling economic theorist with a pretty good idea of what I’m talking about in this post, now is the first time I touch political economy. So, if anyone’s interested in the topic on a deeper level, I can point to you at least ten other people in my vicinity who understand this far more than I do.

Condor what?

First, I introduce the concepts. Most readers with an economics background will be familiar with this and are advised to go through the notation I use and then jump to the next section. Social choice theory is the mathematical discipline of aggregating individual preferences into a collective decision. Depending on the definition and the analogy, it can be viewed as a prodigy, a little brother, or as a cousin of the science of strategic decision making, game theory. The two main components of a social choice problem are the alternatives, in my story these will be “Soft Brexit” (denoted by S), “Hard Brexit” (H), and “Remain” (R), and preferences, which are orderings of the alternatives. For example, someone might have the preference HRS, indicating that his favorite outcome of the Brexit process is to leave the EU with no deal, his second favorite is to remain in the EU, third favorite is to leave with a deal. Arguably, there are more outcomes to Brexit, but for simplicity I use only these three. Obviously, the electorate has more than one voter, and once again, for simplicity, I will pretend we know everyone’s preferences. We do not allow ties, voters cannot be indifferent between any two alternatives, and they can’t reveal anything other than their ordinal preferences. For example, the above voter might think H is just a little better than R which is itself just a little better than S, or he might think H is the best thing ever, infinitely better than R and S which are sort of the same with a slight preference to R. No matter how strongly the voter feels about his preferences, we know, and therefore care about, only the order, nothing more.

Social choice theory studies the possible mechanisms through which we turn votes into collective decisions. For instance, I tell you there are 9 million HSR voters, 8 million SHR voters, 6 million RHS voters, and 10 million RSH voters, and you must tell me which of the three outcomes should win.

The most common way democracies approach this problem is via pairwise comparison. You can imagine this as three referendums with two options on each ballot, H vs R, R vs S, and S vs H. If we do the comparison, you can see that 17 million voters prefer H to R which would get 16 million votes, approximately reflecting the 2016 vote (in this example it doesn’t matter whether a “Leave” vote represents H alone, or S alone, or H and S together). If we measure R against S the result is 16 against 17 again (I told you), and if we measure S against H we get 18 against 15. So, S beats both other alternatives and is therefore called the Condorcet winner (named for French mathematician and philosopher, Nicolas de Condorcet), while H is better than R, making R the Condorcet loser. So, the decision should be S, leave the EU with a deal. The overall preference of the population is SHR, although only 8 million voters actually have this exact preference.

The problem is that pairwise comparisons doesn’t always produce a Condorcet winner. It may just happen that the electorate is divided in a way that the collective pairwise preferences are cyclic. For instance, if we have 9 million HSR voters, 8 million SRH voters (this is the only one I changed compared to previous), 6 million RHS voters, and 10 million RSH, the results are: H beats R by 17 to 16 (again producing the 2016 referendum result if a “Leave” vote means just H alone, or H and S together), R beats S by 24 to 9, and S beats H by 19 to 14. This is called a Condorcet cycle. The fact that such intransitive cycles can exist with voters possessing transitive preferences is called Condorcet paradox (yes, everything is named after the same guy here).

The terrible implication of the paradox with respect to Brexit is the following:

If the electorate’s preferences produce a Condorcet cycle, then no alternative can be chosen for which a majority of voters couldn’t find a better alternative.

This is a pretty harsh situation because it seemingly contradicts the idea of democracy which is that the majority rules. Now it turns out that whatever we pick, the majority can’t rule.

I’m not the only or the first one to propose something like this with respect to Brexit. Simon Kaye writes the following in his article “however things play out, there will always be a majority in the UK that will find the ultimate outcome of the Brexit process to be very far from their first choice”. This is true, but it was true even for my first example which didn’t feature a Condorcet cycle (although I must stress there is no such thing as “very far” in this model). Jonatan Portes’s excellent blog post (written two weeks prior to the referendum) ends with “whichever [option] we choose, there's an alternative preferred by a majority of the electorate”, which is eerily prophetic. I go beyond this conclusion and will present what social choice theory says about resolving these situations.

But first I hear you saying: “Peter, you just made up those numbers!”, which is true, of course I did. “The British people aren’t dumb, they wouldn’t have such nonsense preferences!”. So let’s examine the validity my argument’s premise, i.e. whether Portes’s prediction came true.

Do British preferences produce a Condorcet cycle?

There are two useful ways to interpret “British preferences”: the electorate’s preferences and the MPs’ preferences. In this section I’ll talk about the electorate, Parliament will be very briefly touched upon later on.

On the electorate’s preferences, we can rely on the 2016 vote as well as the subsequent polls. Deltapoll elicits preferences on three alternatives: May’s deal (which, since there’s no other way to avoid H, I will use as proxy for S), No deal (clearly this would be H), and R. The results are as follows:

People with a clear preference (those who didn’t answer “would not vote” or “don’t know”, approx. 60% of the whole sample)

Whole sample (after those who didn’t report clear preferences were pressed to make a choice):

For the whole sample, the full preference list was compiled (sadly I couldn’t find it for people with a clear preference):

As one can see, all six orders appear, and all three outcomes have something going for them: In both samples, R is the most popular best choice, but it is also the most popular worst choice. S is the second most popular best choice, the most popular second choice, and the least popular worst choice. Arguably, H is not doing well, but in the whole sample it beats the most popular best choice, R by 52% to 48%. In the whole sample, S beats H by a lot, 59% to 41%, and S beats R by 56% to 44%, leaving S as the Condorcet winner and R as the Condorcet loser.

However, among people with a clear preference, R is doing much better. Unless, SRH loses to HSR by more than 5% from those with a clear preference (in the whole sample, which is much more H-friendly, SRH beat HSR by 4%), R beats H. Moreover, if HSR is within 5% of SRH from those with a clear preference (which was true in the whole sample), R beats S as well. Finally, S beats H, as long as SHR does not lose to RHS by more than 16% (in the whole sample SHR beat RHS by 11%). So, from people with a clear preference (and hence, more likely to turn up for a referendum), R is the likely Condorcet winner and H is the almost definite Condorcet loser.

In both cases, there’s one result which avoids the Condorcet cycle. In the whole sample, overturning the R vs S comparison would result in a cycle (which almost certainly happens in the clear preference case), while of those with a clear preference S beating R would produce the cycle (which happens by a large margin in the whole sample case). None of this is definite proof that a Condorcet cycle exists, after all, these are polls, but I think it’s a clear indication that the possibility exists. The fact that the cycle appears somewhere in between the whole sample and those with a clear preference indicates further that it isn’t just possible that an election (where most people with a clear preference but only some people with a weak preference participate) would produce a cycle, but highly probable. In any case, the possibility can’t be ignored.

With the premise having been explored, I present and review the resolutions that were put forward by other sources.

How can we resolve this?

As stated above, if the Condorcet cycle exists, there’s no way to select an alternative for which at least 50% of the electorate could not find a better one. This means that, in addition to the changes brought about by any potential Brexit decision, hard or soft, unrest and continued political instability are inevitable. Nevertheless, an alternative must be and will be chosen. Here are the possible ways to do so.

The easiest way to choose is to avoid the choice. Since Article 50 was triggered on March 29, 2018, by not doing anything, before March 29, 2019, Parliament would implement H by procedure. The advantage here is that the popular support vs R and S is not revealed, hence the outcome will appear legitimate and democratic, as it does not contradict the 2016 vote. The main disadvantage is that the procedure decides the final outcome and not the electorate or Parliament. Given that H isn’t measured to be the Condorcet winner under any circumstance, this creates a problem even without the presence of a Condorcet cycle.

Despite both May’s and Corbyn’s firm position that there wouldn’t be a second referendum, most bookmakers think the odds of holding one are close to 1:1. Most people seem to agree that a first-past-the-post system is not the right one for three alternatives. There were several other options put forward, all good and all bad.

The first such option is to use cardinal methods. The idea here is to convert ordinal preferences into cardinal ones, giving each first choice, each second choice, and each third choice a weight, then sum up the weights given by each voter. For instance, we can evaluate the first choice weighing twice as much as the second, and three times as much as the third. This is called the Dowdall system. The results we get:

Clear preference: R gets 45+14/2+41/3=65.67, S gets 29+53/2+18/3=61.5, H gets 26+33/2+41/3=56.17, so R wins.

Whole sample: R gets 39+14/2+47/3=61.67, S gets 33+49/2+18/3=63.5, H gets 28+37/2+35/3=58.17, so S wins.

In both cases we were able to reproduce the Condorcet winner, however, this property is not generally true. The major advantage here is that even when a Condorcet cycle exists we can pick a winner with some legitimacy. The major drawback is that the result is very sensitive to which weights we put on the choices. If we assign three points to every first choice, two points to every second choice, and one point to every third choice we get following results:

Clear preference: R gets 45*3+14*2+41=204, S gets 29*3+53*2+18=211, H gets 26*3+33*2+41=185 points, so S wins.

Whole sample: R gets 39*3+14*2+47=192, S gets 33*3+49*2+18=215, H gets 28*3+37*2+35=193, so S wins again.

This rule is called the Borda system and is the classic weighing method. This is the method recommended by an article in The Conversation (co-authored by an actual political economist I met once when he presented in Maastricht, Dimitrios Xefteris). Not only did the Condorcet winner R lose in the clear preference case, the result flipped by switching the weights from Dowdall to Borda. Hence, by setting the correct weight, whoever sets the rules will set the winner as well – the more we reward the first choice the more likely it is for R to win, a steep linear rule will favor S, and, interestingly, if we set close to equal weights (which looks strange but is theoretically possible), even H can win the whole sample case.

The next method is the simplest, sequential pairwise voting. In this method we ignore any potential collective intransitivity and pretend that once an alternative is defeated by another, we can eliminate it. For instance, we vote between H or R. Then we vote between the winner of the first round and S, and then implement the winner of the second round. Doing so would leave R as the winner in the clear preferences case and S in the whole sample. Taking the 2016 vote as a first round of such a vote (which was not intended to be so by any side) would leave S as the winner again (again, it doesn’t matter here whether “Leave” meant S or H). As you can imagine, this is a very popular alternative, recommended by the Independent among many others, including pro-H sites (which I will not link because I closed that tab and I can’t be bothered to browse through their articles again). The disagreement, of course is which two options should be on the first ballot, and whether the 2016 referendum was indeed the first round. Theresa May’s idea was very similar to this method: first S/H defeated R in 2016, then she thought S would win against H in Parliament. Of course, she could not credibly sell to Parliament that R is completely off the table, losing support from the R MPs in the Tory party as well as Labour’s S MPs who would not vote across party lines. (Yes, this is me summarizing all that prattle about the backstop, Irish border, Custom’s Union, and Single Market in one sentence. Polecon is awesome!)

The advantage of this method is that if there is a Condorcet winner, we will find it, no matter who participates in the first round. Another advantage is that if there is a Condorcet cycle, it is never revealed, hence the decision appears legitimate. The major obvious disadvantage is that in case a cycle exists, whoever sets the rules decides the result by choosing who participates in the first round.

Next is the famous, but often misunderstood instant runoff voting (commonly known as the “alternative vote”). Voters rank their choices on a single ballot. If there is a first choice that exceeds 50% support, it is implemented. Otherwise, the least popular first choice is eliminated, and voters who ranked that first are added to one of the remaining options based on their second choices. This method was recommended for Brexit by the Institute for Government. Interestingly, Britain had a (two-way) referendum in 2011 on whether this method should replace the first-past-the-post method in their Parliamentary elections, the motion was rejected quite handily.

By this method, H is eliminated in both cases as it was the least popular first option, and it comes down to R vs S. In the whole sample this leaves S getting 33% (24% SHR + 9% SRH) from first preferences and 23% (HSR) from second preferences for 56% in total. R gets 39% (26% RSH + 13% RHS) from first preferences and 5% (HRS) from second preferences for 44% in total, meaning that S wins. In the clear preference case it comes down to whether HSR is within 5% points of SRH, probably leaving R as the winner.

This method always breaks Condorcet cycles and is much less manipulable by the arbiter, but there are many drawbacks. First, it may be that the Condorcet winner is eliminated. Second, it reveals Condorcet cycles when they exist, hence its winner won’t have that strong appearance of legitimacy as the pairwise comparison. Finally, the result still depends on the method of elimination. Taking out the least favorite first choice (the least loved alternative) is justified, but so would be to take out the most popular third choice (the most hated alternative). In that case R would be eliminated and S would win decisively in both cases.

A combination of the previous two methods is the runoff method most commonly known as a two-round election. This is used most famously in French Presidential elections. Voters pick their favorite on a ballot featuring all alternatives, then the top two choices advance to the second round. If the preferences don’t change between the two rounds, this method produces the same results as the previous one, R winning the clear preference case and S winning the whole sample. This method is recommended for Brexit by the Guardian although they call it the “alternative vote” and may have believed this is the method that was rejected in the aforementioned 2011 referendum (an illustration how these colloquial names can cause misunderstandings). The method has the same properties as the instant runoff method, but does not reveal Condorcet cycles, so the result is more legitimate.

The final method is that of delegation. Instead of the electorate, Parliament decides. Since Parliament votes on a Yes-No basis, no Condorcet cycles are revealed this way. The problem that this creates is that, S, unlike R and H, requires a negotiated settlement with the EU. Whichever party negotiates and passes that settlement scores a huge political win. Or, to put it differently, failing to do so is a win for the other party. Hence, as I’ve alluded to earlier, Labour won’t support a Tory deal delivered by May, nor will the Tories support a Labour deal delivered by Corbyn. Parliament’s decision is not three-way anymore, but at least four-way: R, H, May S, Corbyn S. This splinters the S camp in Parliament, which, not only would be a compromise solution, but would likely prevail under most of the methods I described above (including, being a likely candidate for a Condorcet winner).

I compile the methods and their evaluations into a table.

As the table clearly shows, the British are screwed. The only method that finds the Condorcet winner when one exists, sequential pairwise, is also the most problematic if a cycle exists. If we’re reasonably sure that a cycle exists we must still choose one of three imperfect methods, with various levels of legitimacy and manipulability. To me it seems like the runoff method might be the least problematic, which would entail not only a second Brexit referendum, but also a third, in a climate that may not even tolerate a second. So, if a referendum is held at all, it will probably come down to a choice between the cardinal and the instant runoff methods, a choice between manipulability and legitimacy.

Is democracy over?

Brexit is one of the greatest challenges to Western democracy. It is a test not only of collective common sense and decency, but of the limits of our voting systems, and the electorate’s faith in the voting systems as those limits are reached. As I’ve said repeatedly, no voting system is perfect, but in most cases, we were able to avoid situations when their failings could be made explicit. Whatever the outcome, the choice will create a precedent on how to deal with the Condorcet paradox in a situation where the stakes are high.

I’ve written above that if the cycle exists, then the majority will be unhappy, no matter what we pick. This is still true. However, I argue that doesn’t contradict the principles of democracy. The principle, to me, is that we discuss and vote on these things. By public conversation, debate, or passionately shouting at each other if necessary, we settle on which of our imperfect decision rules we use and live with the consequences. The Condorcet cycle is not the real wrench in the system, that would be the dishonesty of the arbiters and the misinformation of the electorate. Instead, I think of the Condorcet paradox of Brexit as an opportunity to review and renew our faith in democracy. So that even when confronted with its failings we can still choose to live by its principles.

January 26, 2019.

Footnotes:

Maybe* here means that if the first, second, and third choices are all on the same ballot, then the cycle is revealed because we learn the full distribution of all six preferences. If the first, second, and third choices are all on different ballots, we only learn the marginal distributions and the cycle is not revealed.
None** comes with the caveat that there are of course other ways to specify a runoff vote but they require more than three alternatives.
As political economists will no doubt notice, in this comparison I completely ignore the possibility of strategic voting, i.e. supporting your true preferences by reporting a different one. I believe all methods permit this possibility, and hence I didn't consider this to be an important distinction.
The LSE blog post contains a serious error, possibly due to an earlier version of Deltapoll’s illustration. One of their figures claims that in the whole sample R beats S but this is not true and has already been corrected on Deltapoll's page.

Links:

Comments should be addressed to peter.bayer7@gmail.com.

Google Sites

Report abuse