Research Findings

Background--> Research Findings

Descriptives, Group Differences, & Correlations

Descriptives 1: Are there any group differences (Gamification Group vs. Control Group) on the Time 1 and Time 2 variables on Flow, Learning Approach, and Course Experience, and on the performance measures?

Independent-Samples T-tests: IV=Group, DV=Time 1; DV=Time 2 (See Table 1); DV=Performance (Table 3)

Ans: No.

At Time 1 (before the intervention), when rating on their previous sense of flow, use of learning approach, and course experience in previous courses taken at the university, both the Gamification education students and the Control psychology students did NOT differ from each other.

At Time 2 (after the intervention), both groups did NOT differ from each other on their Time 2 ratings on the sense of flow, the use of learning approach, and course experience felt during their experience of the gamified (Gamification Group) or group-based (Control) teaching and learning sessions. Their team performance and individual quiz performances also did NOT differ from each other.

Research Question 1: Are there any differences (i.e., changes) between the Time 1 and Time 2 ratings on Flow, Deep Learning Approach, Surface Learning Approach, Good Teaching as Course Experience, and Generic Skills as Course Experience for the Gamification Group and for the Control Group?

Paired-Sample T-Tests: IV=T1; DV=T2 (Table 1)

Ans: Yes!

For the Gamification Group, there was a significant within-group difference between Time 1 and Time 2 on Flow (M = 3.15, SD = .41; M = 3.50, SD = .53; t(35) = 3.447; p < .01); that is, for the Gamification Group, there was a moderately statistically significant increase in the sense of flow in the education students during the gamified teaching and learning.

On the other hand, for the Control Group, there were also significant within-group differences between Time 1 and Time 2 on Flow (M = 3.18, SD = .38; M = 3.38, SD = .53; t(48) = 2.531; p < .05) and Good Teaching as Course Experience (M = 3.44, SD = .66; M = 3.65, SD = .61; t(48) = 2.496; p < .05). That means, for the Control Group, the sense of flow and the evaluation of good teaching as course experience in the psychology students have slightly statistically significantly increased from Time 1 and Time 2 during the group-based teaching and learning.

Research Question 2: Can the experience of Gamification of teaching and learning lead to an increase in Flow, the use of Deep Learning Approach, and Good Teaching and Generic Skills as course experience, and a decrease in the use of Surface Learning Approach in the students between Time 1 (based on previous experience) and Time 2 (based on experience during this course)?

Regression Analysis: Predictor=Group; Criterion=Flow_Increase, DeepAppr_Increase, SurfAppr_Decrease, GdTeach_Increase, GenSkill_Increase

Ans: No.

Gamification (i.e., the gamified elements) did NOT predict an increase in Flow, Deep Approach, Good Teaching, and Generic Skills, or a decrease in Surface Approach.

One-Way MANOVA: IV=Group; 5 DVs=Flow_Increase, DeepAppr_Increase, SurfAppr_Decrease, GdTeach_Increase, GenSkill_Increase

Statistic result of Wilks’ lambda = .955; F(5, 79) = .752, p = n.s.

Ans: No group differences were observed on these dependent variables measuring change.

Descriptives 2: Are there any group differences between the Gamification Group and the Control Group on the students’ experience during the gamified or the group-based (control) teaching and learning processes assessed by the variables Curiosity as a Feeling of Interest and Curiosity as a Feeling of Deprivation (Intolerance, Competence, Problem Solving) and by the Gamification Assessment measures (namely, Motivation Processing, Social Processing, and Cognitive Processing)? (Table 2)

Correlations, means, and standard deviations (Table 2)

Observation 1: For the Gamification Group students driven by the rewards and recognition, the higher their level of curiosity as feeling of deprivation in competence, the lower their confidence in performing the tasks well but the higher their sense of social processing in terms of teamwork and competition (CurAsDep_Competence X -GA_MotPro_CF; CurAsDep_Competence X +GA_SocPro). (But, these links are NOT found in the Control Group students.)

Observation 2: For the Gamification Group students driven by the rewards and recognition, the higher their level of curiosity as feeling of deprivation in problem solving, the lower their confidence in performing the tasks well but the higher their sense of enjoyment in completing the tasks (CurAsDep_ProblemSolv X -GA_MotPro_CF; CurAsDep_ProblemSolv X +GA_MotPro_EN). (But, these links are NOT found in the Control Group students.)

Observation 3: For the Gamification Group students driven by the rewards and recognition, the higher their level of curiosity as a feeling of interest, the higher their level of cognitive processing (mental effort) put into completing the tasks (CurAsInterest X +GA_CogPro). (But, this link is NOT found in the Control Group students.)

Independent-Samples T-Tests: IV=Group; DVs=Curiosity variables; DVs=Gamification Assessment variables (Table 2)

ANSWER: No. Both groups did NOT differ on the curiosity variables and the gamification assessment variables.

CONCLUSION (1):

Despite the lack of statistically significant group differences on all of the variables measured, the correlation analysis revealed that the Gamification Group and the Control Group were SOMEHOW DIFFERENT in the way the variables were correlated. Mere correlation analysis, however, could NOT reveal how the two groups were different.

Regression Analysis

Follow Up: To order to observe whether the Gamification Group and the Control Group were different on how the predicting variables (curiosity and gamification assessment) predicted flow, learning approach, and course experience in the students, a series of regression analyses were performed on each sample with each of these outcome variables as criterion variable (CV).

Regression Analysis 1 (Stepwise): CV=Flow_Increase

o Gamification Group (driven by the rewards and recognition): Motivation Processing (Attention, Relevance, Satisfaction, Feedback, Enjoyment) (p = .000) + Final Quiz Performance (p = .022) + Understanding (p = .003) + Team Performance (p = .014) → Flow Increase

o Control Group: Enjoyment (p = .001) à Flow Increase

Regression Analysis 2 (Stepwise): CV=DeepAppr_Increase

o Gamification Group (driven by the rewards and recognition): Enjoyment (p = .002) + Final Quiz Performance (p = .047) → Deep Approach Increase

o Control Group: Curiosity as Deprivation in Problem Solving (p = .002) → Deep Approach Increase

Regression Analysis 3 (Stepwise): CV=SurfAppr_Decrease

o Gamification Group (driven by the rewards and recognition): Curiosity as Deprivation in Competence (p = .004) + Overall Quiz Performance (p = .047) → Surface Approach Decrease

o Control Group: No significant predictors being entered → Surface Approach Decrease

Regression Analysis 4 (Stepwise): CV=GdTeach_Increase

o Gamification Group (driven by the rewards and recognition): No significant predictors being entered à Good Teaching Increase

o Control Group: Curiosity as Deprivation in Problem Solving (p = .017) → Good Teaching Increase

Regression Analysis 5 (Stepwise): CV=GenSkill_Increase

o Gamification Group (driven by the rewards and recognition): Enjoyment (p = .007) → Generic Skills Increase

o Control Group: Enjoyment (p = .008) → Generic Skills Increase

CONCLUSION (2):

(a) The series of regression analysis on each of Change variables showed that Motivation Processing (including Attention, Relevance, Satisfaction, Feedback, and Enjoyment), Understanding, Team Performance, and Individual Final Quiz Performance—all have led to a significant increase in their sense of flow experienced during the gamified teaching and learning process.

(b) Moreover, the Gamification Group students seemed to be driven by a sense of Enjoyment and Achievement at the Final Quiz, both of which have predicted the increase in their use of the Deep Learning Approach in studying for this course.

(c) In addition, with the pressure of team competition, the Gamification Group students were aroused by their curiosity as a feeling of deprivation in competence, leading to better quiz performances, and then resulting in a decrease in their use of surface learning approach in studying for this course.

In contrast, without the rewards and recognition in their group-based teaching and learning sessions, the Control Group students’ increase in their sense of flow was resulted ONLY from the individuals’ sense of Enjoyment felt during their completion of the tasks.

Descriptives 3: How are changes in flow, learning approach, and course experience linked to team performance and individual performance for the Gamification Group and for the Control Group? (Table 3)

Correlations, means, and standard deviations (Table 3)

Observation 1: For the Gamification Group students driven by the rewards and recognition, the larger their increase in their sense of flow from Time 1 (dispositional flow measured based on their previous experience) and Time 2 (flow state measured with respect to the gamified teaching and learning), the larger their increase in their use of deep learning approach in studying for this course, the better their team performance in the tasks, and the better their individual performance on the final quiz. (But, this link is NOT found in the Control Group students.)

Observation 2: For the Gamification Group students driven by the rewards and recognition, the larger their increase in their use of deep learning approach, the larger their increase in their evaluation of good teaching as course experience for this course.

Hierarchical Regression Analyses

Follow Up: A series of hierarchical regression analyses were performed on each sample with the following model predicting team performance and individual performance:

Predictors (Stepwise; Level 1): Curiosity variables (CurAsDep_Intolernace, CurAsDep_Competence, CurAsDeprivation_ProblemSolv, CurAsInterest) and Gamification Assessment Variables (*MotPro_AT, *MotPro_RE, *MotPro_CF, *MotPro_SA, *MotPro_FE, *MotPro_UN, *MotPro_EN, *MotPro, *SocPro, *CogPro)

Mediating variables (Stepwise; Level 2): Flow_Increase, DeepAppr_Increase, SurfAppr_Decrease, CourExp_GdTeach_Increase, CourExp_GenSkill_Increase

Hierarchical Regression Analysis 1 (Stepwise; 2 Levels): CV=TeamPerformance

o Gamification Group (driven by the rewards and recognition): Flow Increase (p = .018) → Team Performance

o Control Group: Curiosity as Deprivation in Problem Solving (p = .000) + Curiosity as Deprivation in Intolerance (p = .005) → Team Performance

Hierarchical Regression Analysis 2 (Stepwise; 2 Levels): CV=Final

o Gamification Group (driven by the rewards and recognition): Flow Increase (p = .002) + Surface Approach Decrease (p = .025) → Individual Final Quiz Performance

o Control Group: Curiosity as Deprivation in Problem Solving (p = .002) + Gamification Assessment on Cognitive Processing (p = .006) → Individual Final Quiz Performance

Hierarchical Regression Analysis 3 (Stepwise; 2 Levels): CV=Overall

o Gamification Group (driven by the rewards and recognition): Surface Approach Decrease (p = .012) + Flow Increase (p = .025) → Individual Overall Quiz Performance

o Control Group: Gamification Assessment on Understanding (p = .047) → Individual Overall Quiz Performance

CONCLUSION (3):

(a) Definitely, for the Gamification Group students, their increase in their sense of flow experienced during the gamified teaching and learning sessions has led them to developing higher level of team performance in the course of the gamified teaching and learning activities.

(b) In terms of their individual quiz performances, their increase in their sense of flow during the series of gamified teaching and learning tasks, coupled with their decrease in their use of surface learning approach resulting from the process, has led to their enhancement in their individual final quiz performance!

(c) In the Control Group, with the curiosity variables entered and the gamification assessment variables entered, NONE of the outcome variables were significant predictors of their team performance during their group-based teaching and learning tasks.

(d) In fact, the Control Group students’ team performance were found to be an artifact of individual differences on curiosity as a feeling of deprivation in problem solving and intolerance. In other words, the Control Group psychology students tended to have higher levels of curiosity as a feeling of deprivation in problem solving and intolerance, that which have driven them to attain better team performance.

(e) And, in terms of their individual final quiz performance, the Control Group psychology students were driven more by their level of curiosity as a feeling of deprivation in problem solving, plus their level of cognitive processing (i.e., mental effort) exerted into their solving the MC questions in the final quiz.