Local elections and national polls: modelling change
PART 1 The first two sections
Abstract
1 Introduction
2 A simple account
2.1 This week’s by-elections: the headlines
2.2 Six recent by-elections
2.3 A simple model
2.4 Beyond the simple model
2.5 Conditional statements about the next general election
Abstract
The UK Conservatives are doing badly. They lost two by-elections this week and did badly in local government elections last May. There are swings of over 20%. The national step ratio is 0.61 …
Is the swing the same across all localities? Is the ‘step ratio’ constant? …
How do local elections relate to national polls? A useful notion is that the national results are a weighted mean of the local results. How does the change in local elections relate to the change in national polls? In other words how do the results ‘now’ relate to the results ‘then’? (… across all localities.) The linear model is a simple model: the results ‘now’ have a straight line relationship with the results ‘then’. The linear model has two special cases. One case is that the change in each percentage is the same across all localities – cf the assumption of a constant ‘swing’. [In all possible circumstances this cannot be true!] Another case is that ‘now’ is a constant proportion of ‘then’ – in other words the ‘step ratio’ is constant across all localities.
These general ideas are explored for: the six recent by-elections; the local government elections in May; opinion poll results for the regions; West Bradford in 2011; and the Scottish parliament results in 2011.
The main part of this report – the most substantial analysis - is section 7.2 which looks at the three ‘historic’ by-elections of 20th July 2023.
Section 2 provides a short simple account based on the six recent by-elections.
1 Introduction
If the dominant party is doing badly in the national polls then it will do badly in local elections. One can speculate: will the dominant party do badly in the next general election?
In the UK the Conservatives have been doing badly in the national polls all year. They have also been doing badly in local elections. They did badly in the local government elections in May. They did badly in three local by-elections in the summer and this week they did badly in two local by-elections in the autumn. All these results were in England. In Scotland, the SNP has been doing badly and they lost the local by-election in in.
Here the phrase ‘doing badly’, means badly relative to an earlier performance in a previous election, namely the 2019 general election. In other words, we are concerned with change. ‘Doing badly’ means ‘doing worse’.
How might we measure change? The headlines talk about swing and they talk about the drop in the percentage. Less well known is the concept of volatility. And here I introduce the notion of step ratio. There are equations relating the different measures of change.
Is the local change the same as the national change? Is change the same everywhere? The headlines compare different swings and wonder whether the swings are the same. A further analysis looks at the comparison for other measures of change. My particular interest is whether the step ratio is much the same everywhere.
Is there a model for the changes? The simplest model is that some measure of change is constant, nationally and across different locations.
An important model of change is the flow model with flows of voters between parties, and associated flow rates. The matrix of flow rates is an example of a transition matrix. The step ratio is the aggregate flow away from a party.
How might we explain the flow rate matrix? One possibility is a gravitation al model with parties of different sizes (masses) and at varying distances in political space.
There is an important distinction between the single flow away from the dominant party and the several flows to the several other parties. The first is more uniform whereas the latter is more variable – not least because of tactical voting.
2 A simple account
2.1 Last week’s by-elections: the headlines
On Thursday 19th October 2023, there were two by-elections. The UK Conservatives lost both seats by historic amounts.
Yesterday’s Times (Saturday 21 October 2023):
“Tories plot stamp duty cut to win over voters.” 1, 2
“No silver lining this time.” John Curtice. 2
“Bruised Tories suffer dissent in ranks as Sunak’s reset flounders. Seismic by-election results have hit party’s morale hard.” 12-13
“Is this 1997 all over again? Conservatives facing uphill battle to win.” 12
“Starmer must raise hopes to win power, Mandelson warns.” 13
“Celebrity support gives Tamworth candidate the X-factor.” 13
“Tories in trouble.” Letters. 30
“Swing Time. As the Tories reel from by-election defeats, they must avoid being panicked into headline-grabbing policies and focus on good government.” 31
https://www.thetimes.co.uk/past-six-days
Today’s Observer (Sunday 22 October 2023):
“Sunak warned of electoral threat from right posed by Reform UK. Successor to Brexit party could wipe out Tories in red wall seats.” 12
“Tories despair at scale of defeats and admit: ‘even our safe seats are in play’. After a crushing week, MPs fear they’ll need a miracle to hold on to power.” 12-13
“Labour to add rural areas and central Scotland to election hit list. Last week’s victories and a belief that Tories have abandoned whole areas of the electorate prompt Keir Starmer’s party to widen its campaign focus.” 14
“They haven’t a clue’: why Tories will worry if next by-election is here. In the seat of Peter Bone, who had the whip withdrawn for bullying, supporters are thin on the ground.” 14-15
“Jeremy Hunt ‘set to quit as MP’ in fear of Portillo moment. Chancellor will stand down before election as likelihood of defeat in new Surrey seat looms, senior Tories say.” 15
“Labour’s double whammy of byelection victories should make the Tories more scared.” 49
“Spare us devout MPs like Peter Bone. If weever had faith in them, it is long lost. Despite a direct line to grace, the moralising politician fell short on morals about bulling.” 51
“Growth comes at a price.” Letter. 54
“A property price rise could be the Tories’ last trump card.” 61
https://www.theguardian.com/theobserver/2023/oct/22
2.2 Six recent by-elections
There have been six by-elections in the UK in the past four months. A summary of the results is given in the table below. The localities/nations are listed in order of decreasing Conservative vote in 2019.
The first two numerical columns give the Conservative percentage vote in the general election of 2019; and the Conservative percentage vote in 2023, either in a by-election or in a national opinion poll. The third column gives the difference between these two percentages. The fourth column gives the ratio of the two percentages, what I refer to as the ‘step ratio’.
The final column is different in that it does not derive from the percentages of the first two columns. Instead it derives from a separate study of survival rates in different regions of the UK. It is the percentage of Conservative voters in 2019 who ‘survive’, namely who intend to vote Conservative in the next general election. In the table each entry is the survival rate for that region in which the given locality is located.
The final row gives the symbols which I shall use in the next section to refer to these variables.
Consider the difference in the two percentages – between then in 2019 and now in 2023. It is not the case that the differences are all the same. Indeed it would be impossible for the last three rows to have a difference of -28.7 – because these three rows have less than that in 2019! What is the case is that the larger the vote in 2019, the more negative is the difference – although this is only true when comparing the top four rows with the bottom four rows. The top four rows have much the same difference.
Consider the step ratio of the two percentages – the vote in 2023 divided by the vote in 2019. The step ratios are (mostly) moderately close together, ranging between 0.47 and 0.61. This range includes four local results and two national results. However there are two exceptions: London has a high ratio and Rutherglen has a low ratio. In London the green policy of the Labour London leader was unpopular. In Rutherglen, some Conservatives may have voted tactically elsewhere in order to defeat SNP.
Consider the regional survival rates. Compare the rates (percentages) with the corresponding step ratios. Tamworth’s step ratio of 0.61 equals its regional survival rate of 61%. In general the step ratios are moderately the same size as the corresponding survival rates. A later section will show that under certain circumstances they should be equal (namely if the locality reflects the region and if there is no flow back to the Conservatives). Here too Rutherglen is an exception -probably for the same reason.
Table Measuring change from 2019 to 2023
Locality vote vote differ. step regional
. 2019 2023 pcent ratio survival rate*
Tamworth 66.3 40.7 -25.6 0.61 61
Selby 60.3 34.3 -26.0 0.57 72
Mid-Beds 59.8 31.1 -28.7 0.52 61
Frome 55.8 26.2 -29.6 0.47 60
Uxbridge 52.6 45.2 - 7.4 0.86 81
[UK] 43.6 26 -17.6 0.60 61
[Scotland] 26.1 16 - 9.9 0.61 80
Rutherglen 15.0 3.9 -11.1 0.26 80
. P1 P2 P2-P1 P2/P1
*See section 3
2.3 A simple model
“Abstract. How do local elections relate to national polls? A useful notion is that the national results are a weighted mean of the local results. How does the change in local elections relate to the change in national polls? In other words how do the results ‘now’ relate to the results ‘then’? (… across all localities.) The linear model is a simple model: the results ‘now’ have a straight line relationship with the results ‘then’. The linear model has two special cases. One case is that the change in each percentage is the same across all localities – cf the assumption of a constant ‘swing’. [In all possible circumstances this cannot be true!] Another case is that ‘now’ is a constant proportion of ‘then’ – in other words the ‘step ratio’ is constant across all localities.”
Let us denote the percentage ‘then’ as P1 and the percentage ‘now’ as P2. The change in percentage is (P2-P1). The step ratio is P2/P1.
A linear model is given by the straight line equation:
P2 = c + m P1 [1]
A constant change-in-percentage model is given by the equation:
P2 = c + P1 [2]
A constant step ratio model is given by the equation, m being the step ratio:
P2 = m P1 [3]
Note that equation [2] corresponds to equation [1] in the case that m=0.
Note that equation [3] corresponds to equation [1] in the case that c=0.
The linear model can come about in an imaginary example involving just two locations.
Suppose that P1=0 in one location and that P1=100 in another location.
Suppose that P2=c in the first location and that P2=b in the other location.
Then the two locations satisfy the equation:
P2 = c + (b-c)(P1/100) = c + ((b-c)/100) P1 [4]
This corresponds to equation [1] if we take m=((b-c)/100).
[Note: a fairly standard notation for the straight line equation is y=mx+c.]
2.4 Beyond the simple model
The simple model is based solely on the percentages ‘then’ and ‘now’. It is based on the following notion:
The percentages now depend on the percentages then.
The next step is to consider what might be called ‘conditional percentages’, specifically percentages based on answers to the question, ‘If you voted for the Conservatives in 2019, will you vote for them in the next general election?’. Those who answer ‘yes’ might be referred to as Conservative ‘survivors’, giving rise to the notion of ‘survival rate’. These survival rates are discussed in Section 3, and appear in the final column in the table in Section 2.2.
Alongside the survivors are other voters flowing from one party to some other party. Alongside the survival rate are flow rates (sometimes referred to as ‘transition rates’). The rates can be put in a flow rate matrix (or transition rate matrix). The flow model is based on the following notion:
The percentages now depend on the percentages then - and also on the flow rates.
How might we explain the flow rates? One notion is that there are larger flows to bigger parties; and another notion is that there are larger flows to nearer parties. Now this is like gravity in physics: the force of gravity is stronger towards bodies with bigger masses (like the sun); and the force of gravity is stronger towards bodies that are nearer in distance (like the moon to the earth). By analogy is a political gravity model:
The flow rates between parties depend on the size of the parties (percentage vote) and on the distance between the parties in political space.
There are other models which are motivated by rather different ideas. One notion is that a model of aggregate percentages can be based on a model of individual voters – the aggregate percentage is the mean of some individual voter attribute (such as ‘party fitness’ discussed in Section 3). Another notion is that percentages now depend on just on the previous percentages but also on the trajectory of previous percentages, leading to recency-and-frequency models.
2.5 Conditional statements about the next general election
What do these current results tell us about what will happen at the next general election? This is the question that is occupying people minds – see the headlines in Section 2.1.
In general a prediction makes certain assumptions and so should be thought of as a conditional statement about the future: “if A then B”. In logic a conditional statement is true except in the case that statement A is true and statement B is false. Two key questions are: is A true?; is the implication true? In the following statement A includes “On the basis of a +17 LAB lead over CON” and the assumptions of the MRP model.
“On the basis of a +17 LAB lead over CON, the MRP results show in GB (ex NI) LAB on 426 seats CON: 141 LD: 25 SNP: 36 GRE: 1 Others: 5”
https://politpro.eu/en/united-kingdom/institute/survation
Gordon Burt, 22nd October 2023
THE END