Local elections and national polls: modelling change
PART 3 The main part, Section 7.2
England: the ‘historic’ by-elections, 20th July 2023
1 Introduction
Part 3.1 Statics - the situation at one point in time: 2019
Part 3.2 Change
Part 3.3 Flow
Part 3.4 Political space (not yet available)
Part 3.5 The distribution of size (not yet available)
Part 3.6 Gravitational flow (not yet available)
Contents
1 Introduction
The ‘historic’ by-elections, 20th July 2023
Sources
Part 3.1 Statics - the situation at one point in time, the 2019 general election
The seats
“The people” … the sets of individuals
The candidates in 2019
The results in 2019
Individual preferences and the social choice mechanism output
The winner and the ordering of the parties
The size distribution
The size distribution as a consequence of a spatial distribution
The parties in political space, July 2023
The distribution of size in political space
Part 3.2 Change
2 Measures of change
3 The results, national and local
The winner and the ordering of the parties, 2019 and 2023
National change: the 2019 general election and the July 2023 polls
Local change: Uxbridge, Frome and Selby; 2019 and 2023
General and specific volatility: the results
4 Changes in the Conservative vote
Measures of change in the Conservative vote: difference, swing and step ratio
Predicting the local Conservative vote using the national measures
Part 3.3 Flow
5 The flow of individual voters
6 Flows between Conservative and Other
Staying Conservative: the survival rate, 2019 to 2023
Becoming Conservative: flow/transition rates, 2019 to 2023
Conservative and Other: the 2x2 transition matrix
The national 2x2 transition equations, 2019 and 2023; local ‘predictions’
Using the transition matrix to predict the Nadine Dorries Mid-Bedfordshire by-election
7 Flows between the individual parties
Staying with the party: the national survival rates for the other two main parties (2019GE – July 2023)
The transition rates for the three main parties
The three main parties: the transition matrix
The national 3xn transition equations, 2019 and 2023, local ‘predictions’
Part 3.4 Political space (not yet available)
Part 3.5 The distribution of size (not yet available)
Part 3.6 Gravitational flow (not yet available)
1 Introduction
The ‘historic’ by-elections, 20th July 2023
The year 2022 had been a disastrous year for the Conservative Party. Public opinion and opinion within the party turned against prime minister Boris Johnson. He was replaced by Liz Truss. But the newly elected leader’s policies led to an economic crisis and she lasted only a few weeks in office. Would the new prime minister Rishi Sunak succeed in rescuing the party’s fortunes?
Into 2023 with the polls remaining low, the May local elections were bad for the Conservatives. The situation was no better when by-elections for three Conservative seats were called for July 20th 2023. Back in 1968, Harold Wilson’s Labour government had also faced three by-elections all on the same day - and had lost all three of them. It was widely anticipated that Rishi Sunak’s Conservative government would suffer the same fate.
[Acton, Dudley, Meriden, by-elections on 28th March 1968]
https://en.wikipedia.org/wiki/1968_Acton_by-election
Before the by-elections:
“The prime minister … faces the prospect today of losing three by-elections …
losing all three would be the worst result for a governing party for more than
half a century.”
“The party chairman said they would lose 3-0 ...”
After the by-elections:
The Conservatives lost their seat in Frome to the Liberal Democrats; and they lost their seat in Selby to Labour:
“The second highest swing from the Tories since 1945.”
“We’ve turned over a 20,000 majority, the biggest majority we’ve ever turned
over.”
The Times: Thursday July 20 2023: 6. Saturday July 22 2023: 8, 1, 6.
However, the Conservatives kept hold of their Uxbridge seat.
Sources
General election, December 2019
https://commonslibrary.parliament.uk/research-briefings/cbp-8749/;
https://en.wikipedia.org/wiki/2019_United_Kingdom_general_election ;
https://en.wikipedia.org/wiki/Opinion_polling_for_the_next_United_Kingdom_general_election ;
…
Current voting intention, July 2023
https://en.wikipedia.org/wiki/Opinion_polling_for_the_next_United_Kingdom_general_election
…
National figures, 10-11 July 2023. YouGov/Times. page 1, “Vote in 2019 GE” columns in:
https://docs.cdn.yougov.com/98dsl0fsbn/TheTimes_VI_230711_W.pdf
Part 3.1 Statics - the situation at one point in time, the 2019 general election
This report is about the change which took place from 2019 to 2023. The first step is to consider the situation at the start, at just the one point in time, namely the results of the general election in 2019.
The seats
The UK general election in 2019 involved contests in 650 seats across the country. Three of the seats are the ones which we are particularly interested in. They are the ones which held by-elections four years later, on July 20th 2023. All three seats had been won by the Conservatives in 2019 – with an absolute majority. The seats were all in England – none in Scotland, Wales or Northen Ireland. However, the seats were in quite different parts of England: Uxbridge and South Ruislip in London; Somerton and Frome in the South West; and Selby and Ainsty in Yorkshire in the north.
[From now on I shall refer to the seats simply as ‘Uxbridge’, ‘Frome’ and ‘Selby’.]
https://en.wikipedia.org/wiki/Uxbridge_and_South_Ruislip_(UK_Parliament_constituency) https://en.wikipedia.org/wiki/Somerton_and_Frome_(UK_Parliament_constituency)
https://en.wikipedia.org/wiki/Selby_and_Ainsty_(UK_Parliament_constituency)
“The people” … the sets of individuals … in 2019
Elections are sometimes said to give “the people” an opportunity to decide and the results are said to express “the will of the people”. Who are “the people”? A number of different sets of individuals are involved:
the whole population
the population 18 or over on election day
the electorate (those eligible to be on the electoral register)
the electorate (those on the electoral register)
the electorate (those eligible to vote in a general election)
the electorate (those on the electoral register and eligible to vote in a general
election)
those who voted
those who voted who cast neither blank nor invalid votes
The table below gives the numbers of people in some of these sets and the percentages for some sets in relation to other sets. Registered electors exclude younger people and so form about 70% of the total population; and about 70% of the electorate vote. So about half the population votes (about 49%).
Table The sets of individuals, numbers and rates … in 2019
UK Uxbridge Frome Selby
Population 66,796,807 ? ? ?
% 18+/Pop [79.3 in 2023]
Registered electors:
. 47,587,254 70,365 85,866 78,398
% Reg/Pop 71.2 ? ? ?
Voters 32,014,110 48,187 64,896 56,418
% Turnout 67.5 68.5 75.6 72.0
Blank/invalid 117,919 ? ? ?
UK population 2019
The candidates in 2019
Nationally there were 88 parties putting forward candidates and an additional 224 non-party individuals putting themselves forward, giving a total of 312 entities.
In Uxbridge there were 6 parties putting forward candidates and an additional 6 non-party individuals putting themselves forward, giving a total of 12 entities.
In Frome there were 4 parties putting forward candidates.
In Selby there were 5 parties putting forward candidates.
The results in 2019
Individual preferences and the social choice mechanism output
“The people” “choose”. The process is as follows: individuals express their preferences; this gives a distribution of preferences; and this distribution is used by a social choice mechanism to select a winner. The mechanism used is first past the post.
The winner and the ordering of the parties, 2019
The three main parties
The Conservative party was the winner in the 2019 general election, Labour was next and the Liberal Democrats third. Let us denote this ordering: C>L>LD.
Turning now to the three constituencies, in 2019 the Conservative party won all three seats. In Uxbridge and Selby the ordering was the same as the national one of C>L>LD. In Frome the ordering was C>LD>L.
Restricting our attention to these three main parties, in all there are six possible orderings. The orderings can be arranged in a hexagon – see below. Adjacent orderings in the hexagon differing by an adjacent pair transition in the ordering.
. L>C>LD C>L>LD (A)
L>LD>C C>LD>L (B)
. LD>L>C LD>C>L
Nationally the ordering was (A) as it was in Uxbridge and Selby. However in Frome the ordering was (B). Note that (A) and (B) are adjacent orderings in the hexagon.
All the parties
Consider now all the parties and the full ordering.
Nationally, the full ordering was:
Cons > Lab > LibD > SNP > Green > Brexit > DUP(NI) > Sinn Féin (NI) > Plaid Cymru > Alliance(NI) > SDLP(NI) > UU(NI) > Yorkshire > Scot Green > Speaker > UKIP > … > TIG > …
https://en.wikipedia.org/wiki/2019_United_Kingdom_general_election
In Uxbridge, the full ordering was:
Cons > Lab > LibD > Green > UKIP > …
https://members.parliament.uk/constituency/3817/election/397
In Frome, the full ordering was:
Cons > LibD > Lab > Green
https://members.parliament.uk/constituency/3735/election/397
In Selby, the full ordering was:
Cons > Lab > LibD > Yorkshire > Green
The size distribution
Quite apart from the specific parties we can order the party percentages from largest to smallest.
Nationally, the sizes in descending order were:
43.6, 32.1, 11.6, 3.9, 2.6, 2.0, 0.8, 0.7, 0.6, 0.5, 0.4, 0.4, 0.3, 0.1 …
In Uxbridge, the sizes in descending order were:
52.6, 37.6, 6.3, 2.3, 0.6, 0.3, 0.1, 0.1, 0, 0, 0, 0
In Frome, the sizes in descending order were:
55.8, 26.2, 12.9, 5.1
In Selby, the sizes in descending order were:
60.3, 24.6, 8.6, 3.4, 3.2
Elsewhere I have referred to such distributions as truncated stepwise power distributions. In some cases the distribution is close to a power distribution. Here Frome is close to the theoretical distribution 50, 25, 12.5, 6.25, each term half the previous one. Also Selby is (not quite so) close to the theoretical distribution 67, 22, 7, 2, 1, each term a third of the previous one. On the other hand Uxbridge displays a series of steps.
Mathematical note
Consider an infinite geometric series of percentages which add to 100%. If the first highest percentage is X%, then the constant ratio is (100-X) as a ratio.
If X=50% then 100%-X%=50% (half) … cf Frome above.
If X=67% then 100%-X%=33% (third) … cf Selby above.
The size distribution as a consequence of a spatial distribution
A size distribution can arise as a consequence of the distribution of the parties in some space.
2 The parties in political space, July 2023
Although the focus of Part 1 is the situation in 2019, in this section the location of parties in political space is based on the flows between parties over time, an aspect which is discussed later in the report.
It is common to locate voters and parties along a one-dimensional Left-Right continuum. As Brexit came to dominate UK politics, a second dimension acquired significance: the Leave-Remain dimension. The ordering of the parties along the first dimension is similar to the ordering of the parties along the second dimension.
The ordering of parties on a Left-Right dimension, July 2023
The work of Part 5 on gravitational flow suggests the following ordering of the parties on a Left-Right scale.
SNP Lab Green Lib Dem PC Cons Other Refrm UKIP
Source: Part 5
Remain scores for parties, July 2023
The ordering on the Leave-Remain dimension is similar except that on the Left-Right scale Labour is further to the left and Lib Dem and Green are more towards the centre.
The Remain score for a party is a measure of the support amongst current party members for the Remain option which they had expressed at the time of the referendum in 2016 – in other words the percentage of the current party’s supporters who voted Remain in 2016, the base being those who voted either Remain or Leave.
Unsurprisingly UKIP has a zero Remain score and Nigel Farage’s Reform Party has an almost zero Remain score. Conservatives and Other have a low Remain score while Plaid Cymru are at the midpoint of the scale. Labour, Lib Dem and Green have high Remain scores and the SNP has a very high Remain score.
Table The Remain scores for the parties, July 2023
UKIP Refrm Other Cons PC [mid] Green Lab LD SNP
0 5 19 22 49 [50] 68 70 72 85
Source: YouGov, July 2023
3 The distribution of size in political space
The three main parties
We now return to our earlier discussion of orderings. The hexagon of orderings is repeated below. Here however we relate the orderings to the location of the parties in political space.
. L>C>LD C>L>LD (A)
. v-shaped v-shaped
. [M-shaped] [M-shaped]
L>LD>C C>LD>L (B)
end-peaked end-peaked
[middle-peaked] [middle-peaked]
. LD>L>C LD>C>L
. ….. middle-peaked …...
Part 3.2 Change
2 Measures of change
The amount of change can be measured in various ways: difference, step ratio, swing (aggregate and two-party) and volatility (general and specific). Most media coverage reports either the difference or the swing. However the other measures can also provide useful insights.
Difference. The simplest measure is the subtracted difference in the percentage vote for a party in 2019 and in 2023.
Step ratio. This is the July 2023 percentage divided by the 2019 general election percentage and I refer to this as the step ratio. (The phrase ‘step ratio’ is prompted by the notion of the common ratio in a geometric series.) When the percentage stays the same. the step ratio is 1.
Swing. If there are only two parties, then swing equals difference. If there are more than two parties and the parties are divided into just two groups, then aggregate swing equals aggregate difference. If there are more than two parties but the focus is on just two of the parties, then a two-party swing can be defined between each pair of parties. Usually the focus is on the two largest parties.
Volatility (general). First consider the overall amount of change in the percentage votes for the parties. One measure of this is volatility: it can be thought of as the aggregate swing from the group of losing parties to the group of gaining parties.
[Consider a set of parties competing in an election or opinion poll at two points in time. Consider the difference in the percentage share of the vote for each party between the two points of time. Either there is no change or some parties will be winners and some parties will be losers.]
Volatility (specific). In the above, the set of parties is partitioned into two groups, winners and losers. However this is just one way of partitioning the set into two groups, and much of the above reasoning applies to any way of partitioning the set into two groups. Each specific partition gives rise to a specific volatility. The specific volatility is necessarily less than or equal to the general volatility.
In this situation, the two-group volatility v equals the difference d which is equal to the swing s.
v=s=d
[Mathematical Aside: the step ratio in the two-group situation]
In the two-group situation how does the step ratio r relate to the other measures?
In what follows we think in terms of the proportion of the vote rather than the percentage of the vote. (For example the proportion 0.57 rather than the percentage 57%.)
Suppose group A in aggregate have x at the first point in time and then y at the second point in time. Suppose y>x, y/x>1.
So group B in aggregate will have u=1-x and then w=1-y. w<u, w/u<1.
So the differences are +d=y-x, and -d=w-u=-y+x.
The step ratio for the winners is y/x.
The step ratio for the losers is w/u. w/u=(1-y)/(1-x)
3 The results, national and local
The winner and the ordering of the parties, 2019 and 2023
The Conservative party was the winner in the 2019 general election, Labour was next and the Liberal Democrats third. Let us denote this ordering: C>L>LD. Four years later, in the national opinion polls in July 2023, Labour was the winner and the ordering was L>C>LD.
Turning now to the three constituencies, in 2019 the Conservative party won all three seats. In Uxbridge and Selby the ordering was the national one of C>L>LD. In Frome the ordering was C>LD>L.
In the 2023 by-elections, the Conservatives again won Uxbridge, but Labour won Selby and the Liberal Democrats won Frome. The orderings were C>L>…>LD; L>C>…>LD; and LD>C>…>L, respectively.
Thus where change occurred, in the two seats which the Conservatives lost, the change involved a switch between the first and second parties in that constituency. Another feature was that in all three by-elections the party which had been placed third in 2019 was overtaken by at least two other parties, Green and Reform (or Reclaim).
Restricting our attention to the three main parties, in all there are six possible orderings. The orderings can be arranged in a hexagon, adjacent orderings differing by an adjacent pair transition. See below. Nationally (1) has become (2). In Uxbridge (1) stayed as (1); in Selby (1) became (2); and in Frome (3) became (4).
. L>C>LD (2) C>L>LD (1)
L>LD>C C>LD>L. (3)
. LD>L>C LD>C>L (4)
National change: the general election in 2019 and the polls in July 2023
How did party preferences change at the national level between the general election in December 2019 and the by-elections in July 2023?
Summary of the results. There was a large change in the percentage support for the parties. The largest change was the fall in the Conservative vote. The largest gain was a rise in the Labour vote. There was a large change in the percentage votes across a binary divide between left and right, corresponding also to the Brexit divide between Remain and Leave – there was a move from right to left. Within the Left, Labour and the Greens gained; and within the right, Brexit/Reform gained slightly. Thus there was a gain at the two extremes as well as for the Labour left. See Tables 1 and 2.
First consider the overall amount of change in the percentage votes for the parties. The volatility between 2019 and 2023 was 19%: in aggregate, losing parties lost 19% and gaining parties gained 19%.
Almost all the loss was lost by the Conservatives and almost all the gain was gained by Labour (with small gains by the Greens and by Nigel Farage’s Brexit/Reform parties). There was little change in the votes for the Liberal Democrats and for the SNP. Consider a binary divide between left and right, corresponding also to the Brexit divide between Remain and Leave. The right/leave parties lost 14% and the left/remain parties gained 14%.
There are various measures of the Conservative loss: the difference in the percentage vote was -17.6%; there was a swing from Conservative to Labour of 15.3%; and the step ratio was 0.60.
There are various measures of the Labour gain: the difference in the percentage vote was +12.9%; there was a swing from Conservative to Labour of 15.3%; and the step ratio was 1.43.
There are various measures of the Liberal Democrat change. There was hardly any change: the difference in the percentage vote was -0.6%; [there was a swing from Conservative to Liberal Democrat of 8.5%]; and the step ratio was 0.99.
There are various measures of the Green gain: the difference in the percentage vote was +2.4%; and the step ratio was 1.92.
There are various measures of the Brexit/Reform gain: the difference in the percentage vote was +4%; and the step ratio was 3.00.
Table 1 Change between 2019 general election and July 2023
. Cons Lab LibD SNP Green Brexit Other shape
/Reform
2019 GE 43.6 32.1 11.6 3.9 2.6 2.0 4.2 M
July 2023 26 45 11 3 5 6 median M
July 2023 24-29 44-48 10-13 3-4 4-5 6-8 range
difference -17.6 +12.9 -0.6 -0.9 +2.4 +4 [-0.2]
swing 15.3 from Conservative to Labour
swing 8.5 from Conservative to Liberal Democrat
[volatility 19.3]
General election, December 2019
https://en.wikipedia.org/wiki/2019_United_Kingdom_general_election ;
https://commonslibrary.parliament.uk/research-briefings/cbp-8749/;
Current voting intention, five polls within 13-20 July 2023;
https://en.wikipedia.org/wiki/Opinion_polling_for_the_next_United_Kingdom_general_election
Step ratio. The Conservative step ratio is well below 1; and the Labour step ratio is well above 1. Both Green and Brexit/Reform parties also have step ratios above 1.
Table 2 Step ratio from 2019 GE to July 2023: July 2023 divided by 2019 GE
. Cons Lab LibD SNP Green Brexit/Reform
step ratio 0.60 1.43 0.99 0.89 1.92 3.00
Local change: Uxbridge, Frome and Selby; 2019 and 2023
How did local party preferences change between the general election in December 2019 and the by-elections in July 2023?
Uxbridge
Quoted below is the same as what was said above about the national change – but underlined below is how Uxbridge is different from the national situation:
“There was a large change in the percentage votes for the parties - not so large in Uxbridge. The largest change was the fall in the Conservative vote. The largest gain was a rise in the Labour vote. There was a large change in the percentage votes across a binary divide between left and right, corresponding also to the Brexit divide between Remain and Leave – from right to left … [not so large in Uxbridge]. Within the Left, Labour and Green gained [slightly]; and within the right, Brexit/Reform gained. Thus there was a gain at the two extremes as well as the left. Notable in Uxbridge was the fall in the Lib Dem vote.”
Table Uxbridge and South Ruislip by-election, 2019 and 2023
. Cons Lab LibD Green Reclaim Other shape
2019 GE 52.6 37.6 6.3 2.2 - 2.3 M
20 Jy 2023 45.2 43.6 1.7 2.9 2.3 4.3 M
difference -7.4 +6 -4.6 +0.7 +2.3 +2.0
swing 6.8 from Conservative to Labour
step ratio 0.86 1.16 0.21 1.32 +++ 1.87
[volatility 12.0]
https://members.parliament.uk/constituency/3817/election/397;
https://en.wikipedia.org/wiki/2023_Uxbridge_and_South_Ruislip_by-election
Frome
Quoted below is the same as what was said above about the national change – but underlined below is how Frome is different from the national situation:
“There was a large change in the percentage votes for the parties. The largest change was the fall in the Conservative vote. The largest gain was a rise in the Labour vote -not so in Frome – the largest gain was by the Lib Dems, and there was a large fall in the Labour vote.
There was a large change in the percentage votes across a binary divide between left and right, corresponding also to the Brexit divide between Remain and Leave – from right to left. Within the Left Labour and Green gained - not so in Frome – Lib Dem gained and Labour fell; and within the right, Brexit/Reform gained. Thus there was a gain at the two extremes as well as the left -not so in Frome, where there was a gain in the centre.”
Table Somerton and Frome by-election, 2019 and 2023
. Cons Lab LibD Green [Brexit] Other shape
Reform
2019 GE 55.8 12.9 26.2 5.1 - - unimodal
20 Jy 2023 26.2 2.6 54.6 10.2 3.4 3.0 M-shaped
difference -29.6 -10.3 +28.4 +5.1 +3.4 +3.0
swing 20.0 from Conservative to Liberal Democrat
[swing 9.7 from Conservative to Labour]
step ratio 0.47 0.20 2.08 2.00 +++ +++
[volatility 39.9]
https://en.wikipedia.org/wiki/2023_Somerton_and_Frome_by-election
Selby
Quoted below is the same as what was said above about the national change – but underlined below is how Selby is different from the national situation:
“There was a large change in the percentage support for the parties. The largest change was the fall in the Conservative vote. The largest gain was a rise in the Labour vote.
There was a large change in the percentage votes across a binary divide between left and right, corresponding also to the Brexit divide between Remain and Leave – from right to left. Within the Left, Labour and Green gained; and within the right, Brexit/Reform gained. Thus there was a gain at the two extremes as well as the left. Notable in Selby was the fall in the Lib Dem vote.”
Table Selby and Ainsty by-election, 2023
. Cons Lab LibD Green [Brexit] Yorks Other shape
Reform
2019 GE 60.3 24.6 8.6 3.2 - 3.4 - M
20 Jy 2023 34.3 46.0 3.3 5.1 3.7 4.2 3.4 M
difference -26 +21.4 -5.3 +1.9 +3.7 +0.8 +3.4
swing 23.7 from Conservative to Labour
step ratio 0.57 1.89 0.38 1.59 +++ 1.23 +++
[volatility 31.3]
https://en.wikipedia.org/wiki/2023_Selby_and_Ainsty_by-election
https://en.wikipedia.org/wiki/2023_Uxbridge_and_South_Ruislip_by-election
https://en.wikipedia.org/wiki/2023_Somerton_and_Frome_by-election
https://en.wikipedia.org/wiki/2023_Selby_and_Ainsty_by-election
General and specific volatility
The “volatility” that is reported in the above tables is the general volatility, namely the swing between losers and winners.
Average local volatility over the three by-election seats is much greater than national volatility. The volatility nationally is 19.3% and the average local volatility is 27.8%. This greater volatility may be partly due to tactical voting.
Note that ‘volatility’ refers to aggregate amount of change. Later we shall discuss the amount of individual change and find it to be much higher, perhaps around 29 at the national level.
Consider the various measures of change in the Conservative vote.
How does the local volatility relate to national volatility? The three constituencies vary substantially amongst themselves in terms of volatility. The local average lies between the two local extremes – as does the national figure.
Table A measure of change in the vote: general volatility; the 2019 general election, polls in July 2023
. national Uxbr Frome Selby ave local
volatility 19.3 12.0 39.9 31.3 27.8
In the above, the set of parties is partitioned into two groups, winners and losers. However this is just one way of partitioning the set of parties into two groups. Each specific partition gives rise to a specific volatility. The specific volatility is necessarily less than or equal to the general volatility.
There is a specific volatility between Conservative and Other. It is simply the fall in the Conservative vote.
Table Specific volatility from Conservative to Other; the 2019 general election, polls in July 2023
. national Uxbr Frome Selby ave local
volatility 17.6 7.4 29.6 26.0 21.0
Another way of grouping the parties into two groups is between pro-Brexit Leave and anti-Brexit Remain. The Leave group contains mainly the Conservatives and the Brexit/Reform parties. This grouping of the parties corresponds also to a Right v Left & Centre grouping.
Table Specific volatility from Right to Left & Centre; the 2019 general election, polls in July 2023
. national Uxbr Frome Selby ave local
volatility 13.6 5.1* 26.4 22.3 17.9
*Conservative and Reclaim
4 Changes in the Conservative vote
It is clear from the preceding tables that the dominant feature of the change between 2019 and 2023 is the fall in the Conservative vote.
Measures of change in the Conservative vote: difference, swing and step ratio
Consider the various measures of change in the Conservative vote. How do the local measures relate to the national measures? Although the three constituencies vary substantially amongst themselves in terms of each measure, the average over the three constituencies for each measure is quite close to the figure for the national measure. The average over the three constituencies for the step ratio is particularly close to the national step ratio.
The difference nationally is -17.6% and the average local difference is -21.0%. The swing from Conservative to Labour nationally is 15.3% and the average local swing from Conservative to Labour is 13.4%. The step ratio nationally is 0.60 and the average step ratio is 0.63.
Table Measures of change in the Conservative vote: difference, swing and step ratio
. national Uxbr Frome Selby ave local
difference, Cons -17.6 -7.4 -29.6 -26.0 -21.0
swing, Con to Labour 15.3 6.8 9.7 23.7 13.4
step ratio, Cons 0.60 0.86 0.47 0.57 0.63
Predicting the local Conservative vote using the national measures
We can use the national change to predict the local changes and the average local change. Each measure gives a different prediction. All the predictions are in error. However it is the prediction using the step ratio that gives the lowest error.
The last line in the table gives the predictions based on the approach in the following section where we discuss the 2x2 transition matrix. This matrix is a development from the step ratio.
Table Predictions of Conservative vote, using difference, swing and step ratio
. national Uxbr. Frome Selby ave local
e: error
2019 general election 43.6 52.6 55.8 60.3 56.2
2023 actual …
… [poll]/ election [26.0] 45.2 26.2 34.3 35.2
predicted Conservative vote in July 2023, using national:
swing from Conservative to Labour 37.3 40.5 45.0 40.9 e+5.7
difference 35.0 38.2 42.7 38.6 e+3.4
step ratio 31.6 33.5 36.2 33.8 e-1.4
2x2 transition matrix (see below) 31.7 33.4 35.9 33.7 e-1.5
Part 3.3 Flow
5 The flow of individual voters
We imagine individual voters flowing from their party preference in 2019 to their party preference in July 2023. In other words there is a transition from one party to another party. A portrayal of this flow is provided in a diagram in the section, Where have 2019 voters gone?, in a report by More in Common.
https://www.moreincommon.org.uk/our-work/research/voting-intention-july-2023/
The flows in the diagram are of different thickness representing the flow rates or transition rates. The transition rates can be presented in a transition matrix. See below.
6 Flows between Conservative and Other
Staying Conservative: the survival rate, 2019 to 2023
One of the flows is from being Conservative in 2019 to being Conservative in 2023. For those who voted Conservative in December 2019, what percentage of them now in July 2023 say they intend to vote Conservative at the next general election? Let us refer to this as the staying rate or survival rate. In the week before the by-elections, opinion polls reported survival rates with a median survival rate of 57.5%; (56, 56, 59 and 70). An earlier YouGov poll had reported a survival rate of 61%.
Note that there is also a flow from being Conservative in 2019 to not being Conservative in 2023. If the survival rate is 57.5% then it follows that the non-survival rate is 42.5%.
The five surveys and the reported Conservative survival rate:
Omnisis
https://twitter.com/Omnisis/status/1679853645540499457
56% Redfield and Wilton, N=2000
https://redfieldandwiltonstrategies.com/latest-gb-voting-intention-16-july-2023/
56% Deltapoll, N=1000
https://deltapoll.co.uk/wp-content/uploads/2023/07/Deltapoll-230717_trackers.pdf
70% More in Common, N=2000 … access via:
https://www.moreincommon.org.uk/our-work/research/voting-intention-july-2023/
59% Techne, N=1628
https://www.techneuk.com/wp-content/uploads/2023/07/R77-UK-2023-7-21-DATA.pdf
National figures, 10-11 July 2023. YouGov/Times; N=. page 1, “Vote in 2019 GE” columns in:
https://docs.cdn.yougov.com/98dsl0fsbn/TheTimes_VI_230711_W.pdf
Becoming Conservative: flow/transition rates, 2019 to 2023
Some of the flows are from voting for other parties in 2019 to voting Conservative in 2023. For those who voted for other parties (or who did not vote) in December 2019, what percentage of them now in July 2023 say they intend to vote Conservative at the next general election? In the week before the by-elections, the same opinion polls reported flow/transition rates with a median transition rate of 3%; (4, 3, 2 and 4). An earlier YouGov poll had reported a transition rate of 2%.
[Correction: these numbers are in fact the transition rates from Labour in 2019 to Conservative in 2023. Here, for the sake of exposition, we assume that these numbers also represent the overall transition rates from other parties and sources in 2019 to Conservative in 2023. A more accurate account of the situation will be discussed later.]
Note that there is also a flow from being Other in 2019 to staying Other in 2023. If the becoming Conservative rate is 3% then the staying Other rate is 97%.
Conservative and Other: the 2x2 transition matrix
The previous two sections have considered staying Conservative and becoming Conservative, respectively, with flow rates of 57.5% and 3%, respectively.
The sections have also considered not staying Conservative and not becoming Conservative, respectively, with flow rates of 42.5% and 97%, respectively. It is customary to present these four numbers in a transition matrix:
57.5% 3%
42.5% 97%
The percentages can be expressed as decimals:
0.575 0.03
0.425 0.97
The national 2x2 transition equations, 2019 and 2023; local ‘predictions’
There were 44% voters for the Conservatives in 2019 and 56% voters for Others in 2019. What are the votes now in July 2023?
We can use the numbers in the transition rate matrix to obtain the transition equations:
Conservative voters now: 0.575 (44%) + 0.03 (56%) = 25.3 + 1.68 = 26.98%
Other voters now: 0.425 (44%) + 0.97 (56%) = 18.7 +54.32 = 73.02%
The numbers in these two equations can be represented in terms of the transition matrix and the ‘vectors’ for December 2019 and July 2023. The matrix equation is:
[ 0.575 0.03 ] [ 44 ] = [ 26.98 ]
[ 0.425 0.97 ] [ 56 ] [ 73.02 ]
Using the same method we can derive predictions for the local by-elections - using the national transition matrix.
Uxbridge
[ 0.575 0.03 ] [ 52.6 ] = [31.7]
[ 0.425 0.97 ] [ 47.4 ] [68.3]
Frome
[ 0.575 0.03 ] [ 55.8 ] = [33.4]
[ 0.425 0.97 ] [ 44.2 ] [66.6]
Selby
[ 0.575 0.03 ] [ 60.3 ] = [35.9]
[ 0.425 0.97 ] [ 39.7 ] [64.1]
It is these predictions which are given in the table above.
The general case
We can also consider the transition matrix applied to the general case.
(Here we shall use proportions that add to 1 rather than percentages which add to 100%.)
Suppose there are C voters for the Conservatives in 2019 and (1-C) voters for Others in 2019.
Applying the proportions in the national transition rate matrix gives:
Conservative voters now: 0.575 C + 0.03 (1-C) = 0.03+ 0.545 C … [1]
Other voters now: 0.425 C + 0.97 (1-C) = 0.97 - 0.545 C
Using the transition matrix to predict the Nadine Dorries Mid-Bedfordshire by-election
Consider the anticipated by-election to replace Nadine Dorries in Mid-Bedfordshire. In 2019 she obtained 59.8% of the vote. Assuming the national surveys stay the same,
applying the equations in the previous section, we can predict the new Conservative candidate gaining 35.7% (0.03+0.327) of the vote.
“Dorries told: quit as MP, you’ve failed your voters.” The Times, July 28, 2023, p. 6.
https://members.parliament.uk/constituency/3603/election/397
Since then Dorries resigned and there has been further acrimony between Dorries and her party – including extracts trailed from her book.
Later in mid-September an opinion poll showed 29% support for the Conservatives in the mid-Bedfordshire seat (the national figure at that time was 26%).
The by-election is now due on 19 October 2023.
“Tory vote is collapsing in Dorries’ old seat – but divided rivals could let victory slip away.” The Observer, September 17, 2023, 10-11.
https://www.survation.com/mid-bedfordshire-by-election-update/
7 Flows between the individual parties
The previous section has looked at flows between the Conservative party and other parties in aggregate. We now look at the flows involving each of the parties individually.
Staying with the party: the national survival rates for the other two main parties (2019GE – July 2023)
In an earlier section we noted that a YouGov poll had reported a Conservative survival rate of 61%, and a non-survival rate of 39%. The same poll reported that …
… for Labour voters, the survival rate was 81% and the non-survival rate was 19%.
… for Lib Dem voters, the survival rate was 52% and the non-survival rate was 48%.
Thus the Labour survival rate is the highest of the three, much higher than the rate for the Liberal Democrats, and the rate for the Conservatives is lower still.
Source: see below.
The transition rates for the three main parties
As well as the survival rates there are the other transition rates to other parties. Consider the three groups who voted for each of the three main parties in the general election of December 2019 and how these groups say they would vote now in July 2023…
… for those who voted Conservative in 2019, they would now vote:
Conservative 61%; Labour 12%; Lib Dem 5%; and Other 22%.
… for those who voted Labour in 2019, they would now vote:
Conservative 2%; Labour 81%; Lib Dem 5%; and Other 13%.
… for those who voted Lib Dem in 2019, they would now vote:
Conservative 5%; Labour 36%; Lib Dem 52%; and Other 7%.
Source: see below.
The three main parties: the transition matrix
These numbers can be put into a transition matrix:
…
The YouGov transition matrix: three parties and Others
The YouGov table includes a transition matrix for the three main parties and other parties:
. Vote in 2019 GE
. ….. YouGov ….. Delta
. Con Lab LD Others
Intention 2023:
Con 61 2 5 1
Lab 12 81 36 9
Lib Dem 5 5 52 4
Others 22 12 7 86
Total 100 100 100 100
…
Source: National figures, 10-11 July 2023. YouGov/Times. page 1, “Vote in 2019 GE” columns in:
https://docs.cdn.yougov.com/98dsl0fsbn/TheTimes_VI_230711_W.pdf
Delta, page 2, https://deltapoll.co.uk/wp-content/uploads/2023/07/Deltapoll-230717_trackers.pdf
The national 3xn transition equations, 2019 and 2023, local ‘predictions’
There are 44% voters for the Conservatives in 2019 and 32% for Labour, 12% for Liberal Democrats, 3.9% for SNP, 2.6% for Green, 2.0% for Brexit and 4.25 for other parties. What are the votes now in July 2023?
We can use the numbers in the transition rate matrix to obtain the transition equations. We ‘multiply’ the 2019 percentages by the transition matrix to obtain:
Conservative voters now: 0.61 (44%) + 0.02 (32%) + 0.05 (12%) + 0.01 (12%)
= 28.08%+0.12%=28.2%
Labour voters now: 0.12 (44%) + 0.81 (32%) + 0.36 (12%) + 0.09 (12%)
= 35.52%+1.08%=36.60%
LibDem voters now: 0.05 (44%) + 0.05 (32%) + 0.52 (12%) + 0.04 (12%)
= 10.04%+0.48% = 10.52%
Other voters now: 0.22 (44%) + 0.12 (32%) + 0.07 (12%) + 0.86 (12%)
= 14.36%+10.32% = 24.68%
Total % =88%+12%=100%
These derived figures of 28%, 37%, 11% and 25% compare with the headline figures in the same report of 25%, 43%, 11% and 21%.
The numbers in these four equations can be represented in terms of the transition matrix and the ‘vectors’ for December 2019 and July 2023. The matrix equation is:
[ 0.61 0.02 0.05 0.01 ] [ 44 ] = [ 28.2 ]
[ 0.12 0.81 0.36 0.09 ] [ 32 ] = [ 36.6 ]
[ 0.05 0.05 0.52 0.04 ] [ 12 ] = [ 10.5 ]
[ 0.22 0.12 0.07 0.86 ] [ 12 ] = [ 24.7 ]
Using the same method we can derive predictions for the local by-elections.
Uxbridge
[ 0.61 0.02 0.05 0.01 ] [ 52.6 ] = [ 33.2 ]
[ 0.12 0.81 0.36 0.09 ] [ 37.6 ] = [ 39.4 ]
[ 0.05 0.05 0.52 0.04 ] [ 6.3 ] = [ 7.9 ]
[ 0.22 0.12 0.07 0.86 ] [ 3.5 ] = [ 19.5 ]
45.2 43.6 1.7 9.5
+12 +4.2 -6.2 -10
Frome
[ 0.61 0.02 0.05 0.01 ] [ 55.8 ] = [ 35.7 ]
[ 0.12 0.81 0.36 0.09 ] [ 12.9 ] = [ 27.0 ]
[ 0.05 0.05 0.52 0.04 ] [ 26.2 ] = [ 17.3 ]
[ 0.22 0.12 0.07 0.86 ] [ 5.1 ] = [ 20.0 ]
26.2 2.6 54.6 16.6
-9.5 -24.4 +37.3 -3.4
Selby
[ 0.61 0.02 0.05 0.01 ] [ 60.3 ] = [ 37.8 ]
[ 0.12 0.81 0.36 0.09 ] [ 24.6 ] = [ 30.7 ]
[ 0.05 0.05 0.52 0.04 ] [ 8.6 ] = [ 13.1 ]
[ 0.22 0.12 0.07 0.86 ] [ 4.5 ] = [ 20.1 ]
34.3 46.0 3.3 16.4
37.8 30.7 13.1 20.1
-3.5 +16.7 -9.8 -3.7
https://en.wikipedia.org/wiki/2023_Uxbridge_and_South_Ruislip_by-election
https://en.wikipedia.org/wiki/2023_Somerton_and_Frome_by-election
https://en.wikipedia.org/wiki/2023_Selby_and_Ainsty_by-election
Gordon Burt, 22nd October 2023
THE END