Local elections and national polls: modelling change
PART 2 Sections 3 to 8
3 Regions and constituencies: density-dependent survival rates?
4 How the vote changes: percentage change, step ratio or change in z-score?
5 Scotland parliamentary elections: SNP and LibDem step ratios, 2007 - 2011
6 Local government elections, 4th May 2023 … step ratios
7 Local by-elections
7.1 George Galloway and Bradford West, March 2011
7.2 The main part … England: the ‘historic’ by-elections, 20th July 2023
7.3 Scotland: SNP loses Rutherglen, 5th October 2023
7.4 England: another two losses, 19th October 2023
3 Regions and constituencies: density-dependent survival rates?
This is a section in:
W3 First party: the trajectory for the UK Conservative Party, 1945-2023
See also Section 7.2: The ‘historic’ by-elections, 20th July 2023: a modelling approach.
“… a third of those who voted Conservative in 2019 now intend to switch to other parties …” [In other words the survival rate is two thirds.]
When, as now, there is a slump in the vote, the step ratio, the survival rate and the transition matrix give much the same ratio or percentage.
Note that the survival rate is an entry in the transition matrix. The transition matrix approach is more sophisticated than the survival rate. The point is that only a few voters are switching to the Conservatives. Note that most opinion polls give the transition matrix in their full data tables and the national single transition rate has been slightly above 60% for the past six months or more.
[See Section 7.2: The ‘historic’ by-elections, 20th July 2023: a modelling approach.]
Sunday’s newspaper reported the Conservative survival rate and the loss rate:
“… a third of those who voted Conservative in 2019 now intend to switch to other parties …” [In other words the survival rate is two thirds.]
The Observer, October 1 1, 8-9; Opinium.
Nationally the Conservative survival rate is 66%. The loss is substantial in all regions but there is some variation across regions: the survival rate varies between 60% and 81%. In order: South 60%, Midlands 61%, Wales 70%, North 72%, Scotland 80% and London 81%.
It is noticeable that regions with a high Conservative percentage in 2019 have a low survival rate in 2023 – and vice versa. In other words there is an inverse relationship between the percentage vote in 2019 and the survival rate now in 2023 … namely, ‘density-dependent’ survival rates. The figures in the Table below are consistent with an equation:
% survival to 2023 = 115 - % vote in 2019
Table Voting for Group A by region in 2019; % survival in 2023
Group A: Conservative, Brexit Party, UKIP
. Group A in 2019 % survival to 2023
V S S=100-V/2 (simple model)
East 58 60/61? 71
East Midland 56 61 72
South East 55 60 72
West Midlands 55 61 72
South West 53 60 73
York & Humber 49 72 75
North East 46 72 77
North West 41 72 78
London 33 81 83
Scotland 26 80 87
https://en.wikipedia.org/wiki/2019_United_Kingdom_general_election_in_Scotland
Gordon Burt. The Brexit election of 2019. The results. Draft 30 December 2019.
Density-dependent survival rates … the survival of the fittest
A simple model illustrates how the inverse relationship between voting percentage in 2019 and survival rate in 2023 might arise. Suppose each voter has a certain level of ‘Conservative fitness’ f, where f varies between 0 and 100. The population of voters have a uniform distribution over the interval [0,100]. In the 2019 election V% vote Conservative. All these voters are the ones with the greatest Conservative fitness – they are in the range [100-V, 100]. The mean fitness of the group is 100-V/2. Suppose the probability of an individual voting Conservative in 2023 is equal to the individual’s fitness. Then the survival rate S for the group will be equal to the mean fitness, 100-V/2:
S = 100 - V/2
Constituencies
The results of the three by-elections in July show a similar regional ordering with step ratios of 0.47 in Frome in the South; 0.57 in Selby in the North; and 0.86 in Uxbridge in London. Also a one-off local poll in Mid-Bedfordshire in the South shows a step ratio of 0.48. See details about Mid-Bedfordshire and Tamworth below, both due to hold by-elections on October 19.
In 2019 in Mid-Bedfordshire the Conservatives had 59.8% of the vote. Using the above national figure of 66% gives a ‘prediction’ of 39.5% for the byelection; and using the above south figure of 60% gives a ‘prediction’ of 35.9%. In my report last week on the Liberal Democrats, I ‘predicted’ 35.7%, using the national transition matrix. Also a single local poll ‘predicted’ 29%.
In 2019 in Tamworth the Conservatives had 66.3% of the vote. Using the above national figure of 66% gives a ‘prediction’ of 44.4% for the byelection; and using the above south figure of 60% gives a ‘prediction’ of 40.0%.
https://electionresults.parliament.uk/election/2019-12-12/Results/Location/Constituency/Tamworth/ ;
https://www.theguardian.com/politics/2023/sep/30/tamworth-voters-byelection-tories-chris-pincher;
4 How the vote changes: percentage change, step ratio or change in z-score?
Note. I wrote this back in 2012 and I refer to “proportional change” whereas now I prefer the term “step ratio”. The two concepts are opposite to one another: proportional change equals one minus step ratio.
The forthcoming local elections (2012) promise to exhibit large changes in the percentage vote for Conservatives, Labour, Liberal Democrats and (in Scotland) SNP. Rallings and Thrasher (2012) note that the three main parties gained 43%, 24% and 23% respectively in the local elections of 2008 whereas the parties’ current standing in national opinion polls is 29%, 40% and 11% (with UKIP and others having 10%
apiece).
Rallings and Thrasher (2012) Local voters ready to punish Tories. Sunday Times, April 29. p. 17.
Sunday Times (2012) Tories at lowest ebb for 8 years. April 29. p. 1.
How are these changes to be measured? The ‘swing’ is a commonly used method
based on the change in the percentages between the two leading parties. As well as
looking at the change in percentages, there is merit in looking at the proportional
change and also at the change in z-scores. Two examples are presented the victory of
George Galloway in the recent Bradford West bye-election; and the SNP gains and
the Liberal Democrat losses in last year’s Scottish Parliament elections.
See later sections.
Curtice, J. (2012) Was this the biggest shock in by-election history? Not quite ... The Independent. Saturday 31 March.
Percentages or z-scores?
What I have called z-score here is the inverse of the standard normal cumulative distribution with mean 0 and standard deviation 1.
Percentages or z-scores – which are better? Percentages are better in that they are more easily understood and are understood by more people. Percentages are more direct representations of the data. From another viewpoint percentages and z-scores are equally valid in that the one is a monotonically increasing function of the other and indeed in the middle of the range is approximately a linear function of the other. However outside the middle of the range – and indeed across the full range there is a non-linear relationship between the two. In these circumstances the two measures provide different results. The choice between them depends on which provides a better account of the data and which is in better accord with the theory. The conclusion from the previous section is that z-scores can sometimes provide better accounts of the data. The following section introduces a theory and shows how z-scores are in better accord with that theory.
5 Scotland parliamentary elections: SNP and LibDem step ratios, 2007 - 2011
Note. I wrote this back in 2012 and I refer to “proportional change” whereas now I prefer the term “step ratio”. The two concepts are opposite to one another: proportional change equals one minus step ratio.
The starting point for this investigation occurred earlier with the following observation. In the Scottish Parliament elections, between 2007 and 2011 the SNP added another 13% to its percentage vote and this figure was fairly uniform across all eight regions. In contrast the Liberal Democrat vote declined not by a constant amount but proportionately, by around a half. See below. This contrast between constant change and proportionate change can be reconciled by using z-scores as discussed in the previous section.
The Times (2011) Briton of the Year. Thursday 27 December. Pp. 1, 2, 12, 13.
Figure 1 Constant gain in the percentage vote for the SNP 2007 to 2011, as a (constant) function of their percentage vote in 2007, across all eight regions, Scottish Parliament
Figure 2 Proportional loss in the percentage vote for the Lib Dems 2007 to 2011, as a function of their percentage vote in 2007, across all eight regions, Scottish Parliament
6 Local government elections, 4th May 2023 … step ratio
Note. I wrote this back in May and I refer to “proportional loss” whereas now I prefer the term “step ratio”. The two concepts are opposite to one another: proportional loss equals one minus step ratio.
“Local elections were held in parts of England on 4th May 2023. Conservative poll ratings had plummeted in the summer of 2022 and had only partially recovered under Rishi Sunak’s premiership. So it was anticipated that the Conservatives would do badly, even worse than they had done four years earlier. And that was what happened: they lost 1000 seats. The gains were shared between the Labour, Liberal Democrat and Green parties.
An article by Robert Ford reported how the party votes had changed. He looked at the “percentage point change in vote share across key wards from 2022 to 2023” for Conservative and Labour parties (and in one graph, the Green party); and the swing from Conservative to Labour. These voting changes were looked at for different Leave vote areas in the first graph; different graduate level areas in the second graph; and different Conservative 2022 vote areas in the third graph.
The variable with the greatest effect was the previous vote – the third graph. In particular the largest decline was in areas where the Conservative vote had been largest. Straight-line equations gave approximations to the effect of previous voting on the Conservative vote, on the Labour vote and on the swing from Conservative to Labour. Straight-line equations also gave approximations to the effect of the Leave vote on the Conservative vote, on the Labour vote and on the swing from Con to Lab. The effect of the leave vote was seen as arising from the relationship between the Conservative vote and the Leave vote, and an equation was identified to represent this relationship. A high graduate level was shown to give a high vote for the Greens.
These specific effects exemplified certain features of elections in general. The past vote was the best predictor of the present vote. The dominant feature of some elections is the sharp decline in the vote for one specific party. In such cases losses exhibit constant proportional loss. It is a secondary matter as to how the other parties gain from these losses. Proportional loss can be conceptualised in terms of the mean of a distribution of individual propensities to vote for a particular party.
[Previous studies of change and trajectory are noted. The changes in voting between elections results are routinely analysed descriptively. Between 2007 and 2011 the votes in Scotland showed constant gains for the SNP across regions and constant proportional losses for the Liberal Democrats across regions.] See previous section.
[Between 2017 and 2019 the votes in the UK were analysed in terms of proportional flows between Remain and Leave parties. Between 2015 and 2017 the votes in the UK were analysed in terms of proportional flows between parties according to a mass-and-distance gravitational model. The trajectories in voting across elections are also routinely analysed descriptively – one study looked at elections in Ireland, 1901-2016. Another study presents a time series model for UK general elections, 1945-2015. A further study presents a damped pendulum model for US presidential elections, 1789-2020. Finally, Conservative voting in three consecutive elections is analysed in terms of a population binary process model.]”
Abstract for Section 9.2 to 9.5, particularly 9.2 pages 7-11, in:
9 A postscript: UK local elections and a coronation
Table Contents, Sections 9.2 to 9.5
9.2 The local elections in England, 2023
How the party votes have changed, 2022 to 2023
The size of the effect of the other two variables
The effect of the Leave vote
9.2.1 The aftermath of the local elections
The Lib Dem voters in May: their future flow to other parties
Conservative MPs: the distribution of opinion
9.3 Elections in general
Swing the same across the country – necessarily so?
Biggest losses in party’s heartlands – necessarily so?
Hypothesis: the loss proportion is constant
The best predictor of the current vote is the previous vote
Difference, proportion and swing
9.4
The proportional loss model (mathematical)
9.5 Change and trajectory
Change: local elections 2012
Change: descriptive
Change: the loss proportion model
Change: the mass and distance gravity model
Trajectory: time series
Trajectory: the mathematics of the damped pendulum
7 Local by-elections
7.1 George Galloway and Bradford West, March 2011
7.2 The main part … England: the ‘historic’ by-elections, 20th July 2023
… see Part 3
7.3 Scotland: SNP loses Rutherglen, 5th October 2023
7.4 England: another two losses, 19th October 2023
This section analyses the six most recent by-elections in the UK, the first three in England all on the same day in July; next one in Scotland at the beginning of October; and finally two this past Thursday.
7.1 George Galloway and Bradford West, March 2011
Note. I wrote this back in 2011 and I refer to “proportionate difference” whereas now I prefer the term “step ratio”. The two concepts are opposite to one another: proportionate difference equals one minus step ratio.
‘Galloway hails a ‘Bradford spring’ while Labour licks its wounds’
The Guardian (2011) Saturday 31 March, pp, 1, 4, 5.
Table 1 below presents the results of the Bradford West bye-election on Thursday 29 April 2011. The columns show the percentage votes in the previous general election (p1), the percentage votes in the bye-election (p2); the difference in percentage votes (dp); the proportionate difference in percentage votes (dp/p); and the corresponding figures for z-scores, (z1, z2 and dz). The main story is of course the dramatic victory of George Galloway and his Respect Party adding 53% to their previous minimal 3%. But among the losers who came off worst?
In terms of the difference in percentage vote, dp, Labour and Conservative were the worst losers, much the same as one another, the former losing 20% and the latter losing 23%. The Liberal Democrats and the Green did not lose nearly as much, 7% and 1%, respectively. Note too that UKIP changed hardly at all, adding just 1% to their vote – and the Democratic Nationalist lost only 0.1%
In terms of the proportionate difference in percentage vote, dp/p, a very different picture is presented. Here, the Conservatives are the worst losers, dp/p=-0.73. (Their proportion of the vote fell by 73%). Lib Dems are slightly less bad with dp/p=-0.61 and, less bad still, Labour with dp/p=-0.45 and Green with -0.37. For the Democratic Nationalist, dp/p=-0.09. Turning to gains, UKIP’s gain was dp/p=+0.66 and Respect a dramatic dp/p=17.26.
The picture for z-scores is somewhat similar to the picture for dp/p-scores. Here too, the Conservatives are the worst losers, dz=-0.89. Labour and Lib Dems are not nearly so bad, and much the same as one another, dz=-0.56 and dz=-0.49, respectively. The Greens’ loss was modest in comparison, dz=-0.19. UKIP’s gain was similar in magnitude, dz=+0.22. For the Democratic Nationalist, dz=-0.03.
The nature of the contrast is that the first measure is the simple difference in the percentage whereas the second measure, the difference in the z-scores, is very roughly speaking some approximation to the proportionate change in the percentages.
Which of the three measures provides a more authentic account of what has happened? See Table 1.
Table 1 p and z-scores; dp, dp/p1 and dz changes in scores
party
p1
p2
dp
dp/p1
z1
z2
dz
Respect
0.0306
0.5589
0.5283
17.26471
--1.8720
0.148181
2.020229
Labour
0.4535
0.2499
-0.2036
-0.44895
-0.1168
-0.6748
-0.55798
Conservativ
0.3115
0.0837
-0.2278
-0.7313
-0.4916
-1.38061
-0.889
Lib Dem
0.1167
0.0459
-0.0708
-0.60668
-1.1916
-1.68598
-0.49433
UKIP
0.02
0.0331
0.0131
0.655
-2.0537
-1.83707
0.216682
Green
0.0232
0.0147
-0.0085
-0.36638
-1.9917
-2.17808
-0.18634
Dem Nat
0.0115
0.0105
-0.001
-0.08696
-2.2734
-2.30798
-0.03455
Monster
0
0.0034
0.0034
0
#NUM!
-2.70648
#NUM!
7.2 The main part … England: the ‘historic’ by-elections, 20th July 2023
See:
PART 3 The main part, Section 7.2
England: the ‘historic’ by-elections, 20th July 2023
Link:
7.3 Scotland: SNP loses Rutherglen, October 2023
“Labour celebrates seismic win in Scottish by-election”
BBC https://www.bbc.co.uk/news/live/uk-scotland-67009605
A by-election was held on October 5th 2023 in the SNP-held constituency of Rutherglen & Hamilton West. The analysis here of the Rutherglen results follows the procedure which is outlined in more details in a later section.
Tables 1 to 4 all relate to Scotland and Rutherglen.
Table 1 shows the national change between the 2019 general election and opinion polls in August-September 2023. The volatility is high at 20.9%, composed mainly of gains for Labour and a little for Scottish Greens and Reform (compared to Brexit in 2019); and substantial loses for SNP and the Conservatives. There are substantial swings from SNP to Labour and from Conservatives to Labour.
Table 2 shows step ratios of 0.82 for the SNP; 0.61 for the Conservatives (much the same as for UK as a whole – see section below); and 0.74 for the LibDems.
Table 3 shows the Rutherglen results, the local change between the 2019 general election and the by-election October 2023. The Rutherglen volatility is 30.0%, higher than the national volatility of 20.9% in Table 1. The Rutherglen swings are 20.4% from SNP to Labour and 17.6% from Conservatives to Labour – again higher than the corresponding national swings of 11.7% and 12.7%.
Table 4 shows Rutherglen step ratios of 0.62 for the SNP; 0.26 for the Conservatives; and 0.56 for the LibDems. In all three cases these are substantially lower than the national step ratios. This is particularly so for the Rutherglen Conservative step ratio – possibly reflecting ‘UK unionist’ tactical voting for Labour in order to defeat Scottish nationalist SNP.
Table 1 Scotland: national change between the 2019 general election and opinion polls in August-September 2023
. SNP Cons Lab LibD sGreen Brexit UKIP Other
/Reform
2019 GE 45.0 26.1 18.6 9.5 1.0 0.5 0.1 0.1
A-S 2023 37 16 34 7 5 2 0 0 median
A-S 2023 35-38 15-17 27-35 6-8 2-6 2-4 0 0 range
difference -8.0 -9.9 +15.4 -2.5 +4.0 +1.5 -0.1 -0.1
swing 12.7 from Conservative to Labour
swing 11.7 from SNP to Labour
[volatility 20.9]
General election, December 2019
https://en.wikipedia.org/wiki/2019_United_Kingdom_general_election_in_Scotland
Current voting intention, five polls within 13-20 Sept 2023
https://www.electoralcalculus.co.uk/polls_scot.html;
Table 2 Step ratio from 2019 GE to A-S 2023: A-S 2023 divided by 2019 GE
. SNP Cons Lab LibD sGreen Brexit UKIP Other
/Reform
step ratio 0.82 0.61 1.83 0.74 5.0 4.0 0 0
Table 3 Rutherglen: local change between 2019 general election and October 2023
. SNP Cons Lab LibD sGreen Brexit UKIP Other
/Reform
2019 GE 44.2 15.0 34.5 5.2 - - 1.2 -
Oct 2023 27.6 3.9 58.6 2.9 2.0 1.3
difference -16.6 -11.1 +24.1 -2.3 +5.9
swing 17.6 from Conservative to Labour
swing 20.4 from SNP to Labour
[volatility 30.0]
Rutherglen https://en.wikipedia.org/wiki/Rutherglen_and_Hamilton_West_(UK_Parliament_constituency) ;
https://members.parliament.uk/constituency/2501/election/397 ;
Table 4 Rutherglen: step ratios from 2019 GE to July 2023: July 2023 divided by 2019 GE
. SNP Cons Lab LibD sGreen Brexit UKIP Other
/Reform
step ratio 0.62 0.26 1.70 0.56 5.8
7.4 England: another two losses, 19th October 2023
NOTE: The commentary for this section has not yet been written. Sorry.
National, UK
Table 1 Change between 2019 general election and … July 2023 / October 2023
. Cons Lab LibD SNP Green Brexit Other
/Reform
2019 GE 43.6 32.1 11.6 3.9 2.6 2.0 4.2
July 2023 26 45 11 3 5 6 median
July 2023 24-29 44-48 10-13 3-4 4-5 6-8 range
difference -17.6 +12.9 -0.6 -0.9 +2.4 +4 [-0.2]
swing 15.3 from Conservative to Labour
swing 8.5 from Conservative to Liberal Democrat
[volatility 19.3]
October 2023 27 44 11 2 4 6 median
October 2023 25-30 42-47 9-14 1-3 2-7 5-7 range
difference -16.6 +11.9 -0.6 -1.9 +1.4 +4 []
swing 14.3 from Conservative to Labour
swing 8.0 from Conservative to Liberal Democrat
[volatility 17.3]
General election, December 2019
https://en.wikipedia.org/wiki/2019_United_Kingdom_general_election ;
https://commonslibrary.parliament.uk/research-briefings/cbp-8749/;
Current voting intention, six polls within 13-19 July 2023;
https://en.wikipedia.org/wiki/Opinion_polling_for_the_next_United_Kingdom_general_election
Current voting intention, five polls within 12-19 October 2023;
https://en.wikipedia.org/wiki/Opinion_polling_for_the_next_United_Kingdom_general_election
Mid-Bedfordshire
Table 3 Mid-Bedfordshire: local change between 2019 general election and October 2023
. Cons Lab LibD Green Brexit UKIP Other
/Reform
2019 GE 59.8 21.7 12.6 3.8 2.1
Oct 2023 31.1 34.1 23.1 1.8 3.7
difference -28.7 +12.4 +10.5 -2.0 +3.7
swing 20.6 from Conservative to Labour
[volatility -30.7]
Mid-Bedfordshire
https://en.wikipedia.org/wiki/2023_Mid_Bedfordshire_by-election
Table 4 Mid-Bedfordshire: step ratios from 2019 GE to July 2023: July 2023 divided by 2019 GE
. Cons Lab LibD sGreen Brexit UKIP Other
/Reform
step ratio 0.52 1.57 1.83 0.47
Tamworth
Table 5 Tamworth: local change between 2019 general election and October 2023
. Cons Lab LibD sGreen Brexit UKIP Other
/Reform
2019 GE 66.3 23.7 5.3 2.0 - 1.8 0.9
Oct 2023 40.7 45.8 1.6 1.6 []2.3 1.7m 5.4
difference -25.6 +22.1 -3.7 -0.4 +7.6
swing 23.8 from Conservative to Labour
[volatility 29.7]
Tamworth
https://en.wikipedia.org/wiki/2023_Tamworth_by-election
https://electionresults.parliament.uk/election/2019-12-12/results/Location/Constituency/Tamworth
Table 6 Tamworth: step ratios from 2019 GE to July 2023: July 2023 divided by 2019 GE
. Cons Lab LibD sGreen Brexit UKIP Other
/Reform
step ratio 0.61 1.93 0.30 0.8 3.0
Gordon Burt, 22nd October 2023
THE END