Validation
Summary: Expectation Shock methodology has been applied to a large number of project challenges including business and social issues, local and global community, anti-social behaviour and anti-bullying - engaging both adults and children. It has been externally validated by University of Essex (ethics, questionnaire, statistics), University College London (overall process and validity of results) and by doctoral thesis. In essence:
Expectation shock provides a high impact, memorable, enduring experience which drives enhanced perception in the form of individual self esteem, aspirations and achievement, and of participating organisations eg business, educational, police and community.
To be enduring, this enhanced perception of both self and others depends on three factors following the Expectation shock event with:
Active engagement – actually DOING something, undertaking a project and coming up with THEIR solutions
Working with others – simply working round the table with, for example, a policeman or business expert or local role model – somebody who actually listens to them – can provide a huge expectation shock!
Over time re-enforcement – the initial high impact event has an enduring impact in itself, it can be further enhanced and re-enforced by ‘light, top-up’ activities such as visits or videoconferencing.
This Perception shift can be measured: We benchmark and MEASURE students Perceptions pre- and post activity with a 60-second Questionnaire. We ask questions on a five-point scale, for example, 'To what extent do you think you can do something about bullying?' or 'To what extent do you think business is interesting?' In the example below, the graph shows a large Perception Shift: Pre-event nearly all 14 children had a low or average view (2-3), whereas post-event their Perceptions had shifted and were much higher (4-5):
NB: The following statistical analysis was undertaken to validate the well-foundedness of this simple, 60-second tool. (This means we do NOT have to undertake the statistical analysis or indeed read the stats below ☹)
------------------------------
Expectation Shock! is based on the doctoral thesis: Expectation Shock! 2005 The copyright of this thesis rests with the author and no quotation from it or information derived from it may be published without the prior written consent of the author. The thesis can be viewed at University College London.
In this activity, students were asked to explore social networking technology such as Facebook. After a ‘hands-on’ experience using the technology they were asked for their views on Reliability and Usability – and also Social Interaction and Fun. The measured Perception shift was obtained as follows. There were four questions of the following format (taking the example of question 1 on Reliability):
Q1 Reliability: including ability to use the service dependably, securely and accurately.
How important is it? (on a scale of 1: low 5: high)
What did you expect before using the technology? (on a scale of 1: low 5: high)
What did you find after using the technology? (on a scale of 1: low 5: high)
The mean values for the responses to each question and its three component parts were calculated, together with the Perception shift (P-E). The data for the 18 students are presented in columns 2,3,4 below in the following table. This is followed in column 5 by the Perception shift, i.e. the difference between the student Expectation before using the technology and the student Perception after using the technology (P-E):
Importance
Expectation
Perception
Perception Shift (P-E) 0.7
Reliability
4.9
3.7
4.6
0.9
Usability
4.2
3.7
4.4
0.7
Social
4.2
3.8
4.3
0.5
Fun
4.3
3.8
4.6
0.8
Importance, Expectation and Perception: 18 students
The activity gave an opportunity to undertake further statistical analysis which supported the above findings.
Significant Differences between Perception and Expectation Means for All Variables
One-sample t tests were undertaken which suggest that the differences between the Perception and Expectation means for all variables are significant (< 0.01).
Standard Deviations of Means & Dispersion
The standard deviations of the mean values gave a perspective on dispersion, for example: The individual values for Reliability Importance are tightly grouped around the mean (Mean 4.9 SD 0.3). The individual values for Reliability Expectation are more dispersed (Mean 3.7 SD 0.9). The individual values for Reliability Perception are less dispersed (Mean 4.56 SD 0.598). However, this pattern is not repeated for the other variables. Examination of the table below shows that Expectation means are all lower and are more dispersed than Importance and Perception means. This might suggest the possibility that subjects are more diverse in their Expectation than their Importance or Perception.
Minimum
Maximum
Mean
Std. Deviation
Reliability I
4
5
4.89
0.314
Rel E
2
5
3.72
0.931
Rel P
3
5
4.56
0.598
Usability I
2
5
4.22
0.711
Us E
2
5
3.72
0.870
Us P
2
5
4.44
0.831
Social I
4
5
4.22
0.416
Soc E
2
5
3.83
0.764
Soc P
2
5
4.33
0.745
Fun I
3
5
4.33
0.667
Fun E
2
5
3.83
0.764
Fun P
3
5
4.61
0.591
Standard Deviations of Mean
Correlation Coefficients
The following table shows the Pearson correlation coefficients for the dimensions of Importance, Expectation and Perception values of the 4 variables. The largest coefficients with high significance (less than 0.01) are highlighted. Subsets of this data are then used to examine the interesting features in a hopefully clear way
Rel I
Rel E
Rel P
Us I
Us E
Us P
Soc I
Soc E
Soc P
Fun I
Fun E
Fun P
Rel I
P Corr
1
-0.105
0.033
-0.138
0.294
-0.024
0.189
-0.309
-0.079
-0.088
0.386
0.067
Sig. 2-tail
.
0.667
0.894
0.573
0.222
0.924
0.438
0.199
0.748
0.719
0.103
0.787
Rel E
P Corr
-0.105
1
0.377
0.261
0.042
0.662
0.016
0.716
0.454
0.507
0.482
0.511
Sig. 2-tail
0.667
.
0.112
0.281
0.865
0.002
0.948
0.001
0.051
0.027
0.037
0.025
Rel P
P Corr
0.033
0.377
1
0.102
0.190
0.732
0.174
0.324
0.208
0.371
0.081
0.454
Sig. 2-tail
0.894
0.112
.
0.679
0.436
0.000
0.477
0.176
0.394
0.117
0.742
0.051
Us I
P Corr
-0.138
0.261
0.102
1
-0.170
0.115
-0.167
0.477
0.594
0.195
0.375
0.470
Sig. 2-tail
0.573
0.281
0.679
.
0.488
0.640
0.495
0.039
0.007
0.423
0.114
0.042
Us E
P Corr
0.294
0.042
0.190
-0.170
1
0.094
0.017
-0.153
-0.200
0.256
0.349
0.114
Sig. 2-tail
0.222
0.865
0.436
0.488
.
0.702
0.945
0.531
0.412
0.291
0.144
0.642
Us P
P Corr
-0.024
0.662
0.732
0.115
0.094
1
0.036
0.554
0.478
0.234
0.204
0.578
Sig. 2-tail
0.924
0.002
0.000
0.640
0.702
.
0.885
0.014
0.038
0.335
0.402
0.010
Soc I
P Corr
0.189
0.016
0.174
-0.167
0.017
0.036
1
-0.233
-0.239
0.134
-0.408
-0.101
Sig. 2-tail
0.438
0.948
0.477
0.495
0.945
0.885
.
0.336
0.324
0.585
0.083
0.682
Soc E
P Corr
-0.309
0.716
0.324
0.477
-0.153
0.554
-0.233
1
0.488
0.327
0.238
0.349
Sig. 2-tail
0.199
0.001
0.176
0.039
0.531
0.014
0.336
.
0.034
0.171
0.326
0.143
Soc P
P Corr
-0.079
0.454
0.208
0.594
-0.200
0.478
-0.239
0.488
1
-0.224
0.293
0.547
Sig. 2-tail
0.748
0.051
0.394
0.007
0.412
0.038
0.324
0.034
.
0.357
0.224
0.015
Fun I
P Corr
-0.088
0.507
0.371
0.195
0.256
0.234
0.134
0.327
-0.224
1
0.436
0.329
Sig. 2-tail
0.719
0.027
0.117
0.423
0.291
0.335
0.585
0.171
0.357
.
0.062
0.169
Fun E
P Corr
0.386
0.482
0.081
0.375
0.349
0.204
-0.408
0.238
0.293
0.436
1
0.595
Sig. 2-tail
0.103
0.037
0.742
0.114
0.144
0.402
0.083
0.326
0.224
0.062
.
0.007
Fun P
P Corr
0.067
0.511
0.454
0.470
0.114
0.578
-0.101
0.349
0.547
0.329
0.595
1
Sig. 2-tail
0.787
0.025
0.051
0.042
0.642
0.010
0.682
0.143
0.015
0.169
0.007
.
N
19
19
19
19
19
19
19
19
19
19
19
19
Pearson Correlation Coefficients for Importance, Expectation and Perception
Reducing the table to those rows and columns containing high correlation values with high significance (< 0.01) produces the following:
Us P
Soc E
Soc P
Fun E
Reliability E
Pearson Correlation
0.662
0.716
0.454
0.482
Sig. (2-tailed)
0.002
0.001
0.051
0.037
Reliability P
Pearson Correlation
0.732
0.324
0.208
0.081
Sig. (2-tailed)
0.000
0.176
0.394
0.742
Usability I
Pearson Correlation
0.115
0.477
0.594
0.375
Sig. (2-tailed)
0.640
0.039
0.007
0.114
Fun P
Pearson Correlation
0.578
0.349
0.547
0.595
Sig. (2-tailed)
0.010
0.143
0.015
0.007
High Correlations with High Significance
From the above table, it can be seen that the highest correlation is between the Reliability Perception and the Usability Perception (0.732). Perhaps this might be due to an underlying variable relating the two or it might be a general Perception shift. This possibility will be returned to.
The next highest correlation is between the Reliability Expectation and the Social Interaction Expectation (0.716). Perhaps this might be due to an underlying variable relating to the Security and Privacy aspects of both Reliability and Social Interaction. However a similarly high correlation is not found in the Perceptions of Reliability and Social Interaction: so this is inconclusive.
There were no high significant correlations in Importance between the variables: suggesting perhaps that students see these four areas as different with no large underlying variable.
Similarly, from the Expectation Correlation Coefficients table below, there were no high, significant correlations between Expectations, with the exception of the Reliability Expectation and the Social Interaction Expectation (0.716).
Rel E
Us E
Soc E
Fun E
Reliability E
Pearson Correlation
1
0.042
0.716
0.482
Usability E
Pearson Correlation
0.042
1
-0.153
0.349
Social E
Pearson Correlation
0.716
-0.153
1
0.238
Fun E
Pearson Correlation
0.482
0.349
0.238
1
Expectation Correlation Coefficients
In contrast, in the Perception Correlation Coefficients table below, there were more, higher significant correlations between Perceptions: thus lending support to the conclusion in the body of the report suggesting a general Perception shift.
Rel P
Us P
Soc P
Fun P
Reliability P
Pearson Correlation
1
0.732
0.208
0.454
Usability P
Pearson Correlation
0.732
1
0.478
0.578
Social P
Pearson Correlation
0.208
0.478
1
0.547
Fun P
Pearson Correlation
0.454
0.578
0.547
1
Perception Correlation Coefficients