Design planning

Design planning

The New World Kirkpatrick Model (Kirkpatrick Partners, 2021) can be used to evaluate success in digital storytelling (DST), or other teaching methods. The Kirkpatrick Model focuses on four levels of evaluation: Level 1 is the “Reaction” of the students to the learning and applicability to their jobs; Level 2 is the “learning”, how much knowledge and skills did the students gain; Level 3 is the “behaviour”, how much do students apply their new training in the real world; Level 4 is the “Results” how much can the outcomes of the learning be measured and what was the achievement level of these outcomes. A review by Moreau et al. (2018) using the Kirkpatrick model to evaluate DST just 16 articles with suitable data based on the Kirkpatrick Model. In terms of “learning” the learning of health professionals was enhanced if they co-created the stories with patients. When patients' digital storytelling on its own was used, there was minimal impact on health professional learning and therefore their “behaviour”. Only three of these studies looked at our context of continuing professional development (CPD). It seems that digital storytelling was superior to hypothetical case studies in at least one study (Price et al., 2015), a similar benefit (although no comparator used) was found by Cueva et al. (2016) when looking at healthcare professionals combining traditional learning with personal digital storytelling about cancer. These results were measured in terms of the first three levels of Kirkpatrick’s model but lacked an objective measurable outcome to meet level four requirements.

Stacey and Hardy (2011) showed an improved transition to practice (“behaviour”) for newly qualified nurses with the use of digital storytelling, they suggest that the creation process itself enhanced the nurses’ reflective practice and emotional awareness, it also allowed them to see the stories of others and realise that their experiences/struggles were shared by their peers, making them easier to deal with (“reaction”). The use of co-creation of stories helped with intergenerational interactions between new medical students and caring for older adults, also reducing the signs of ageism (Hewson, Danbrook and Sieppert, 2015). Once again the “results” level four section of the Kirkpatrick model seems under evaluated.

The Games-to-Teach Research Team (2003) at Massachusetts Institute of Technology (MIT) suggested that previous decades of gaming research in education had found no effectiveness in supporting learning, but an ability to motivate learners to do repetitive tasks. Games have the potential for developing collaborative problem solving skills, practice caring for digital life forms with a possible carry over to real life care, engaging with wider communities online and the development of tactical planning in chess, sports management or military style games. Typical pedagogical approaches to achieve some of these aims would include case-based learning, goal-based scenarios, anchored instruction and problem-based learning. Medical education is no stranger to role-playing situations for doctor/nurse-patient interaction. The MIT team suggest that context is a critical component of successful learning environments for educational games. Narratives set the scene for the situation to be problem solved, they define the goals and constrain unwanted or unnecessary actions. If designed correctly they should provoke reflection and emotional responses. When designing our CPD course we need to set up detailed background stories of the characters involved in the digital story to promote these emotional responses and to give the students characteristics and realistic belief scenarios to identify and evaluate.

For a game format to be successful in educating students it also needs to meet students’ expectations of what a game will normally do. In the modern gaming world this will include a progression through levels, easier or harder modes, or characters progressing in ability levels over the course of the game with “power-up” bonuses as rewards for good or regular play (Games-to-Teach Research Team, 2003). Lloyd Rieber (1996) defined “exogenous play” as play that is part of the learning experience not separate from it. The MIT team came up with 7 conceptual design frameworks for games based on this principle. The simplest model was to take a simulation of a real world situation, such as a patient-doctor interaction used in medicine, and sprinkle game elements into it. This requires the addition of goals for the student to attain in the simulation and also appropriate constraints on the situation that make it a challenge for them to overcome. Ultimately “winning” a game is based on either luck or working out the underlying logic on which the game is based. If the game is designed around the logic a student should follow in medical practice then the game should, in theory, teach them to adapt to the simulation in a way that will enhance their real world practice. The use of power-ups can add to the student learning experience if correctly designed. Gamers expect bonus power –ups to make certain portions of the game easier, in a medical simulation scenario the power-up could be the use of a piece of technology like an MRI scan or it could be the giving of a clue if a player/student was struggling to make progress.

The third framework of the MIT team revolves around the idea that games are a series of contested and non-contested spaces. If these spaces are well designed they can promote greater motivation and skill in problem solving when in the contested spaces and greater exploration and reflection when in the non-contested spaces of the game environment. This is based on work by Jenkins and Squire (2002). This can also be combined with the fourth idea in terms of how much transgressive play to allow. For example, if we are designing a scenario based around improving person centred care and empathy, do we allow a scenario where the accepted social/cultural practices in this area are temporally pushed or removed to allow students to explore the consequences of not doing the accepted behaviour. Au (2001) talks about these as liminal spaces and they can be fun for students but need to be carefully designed to avoid promoting unwanted behaviour.

The fifth framework from MIT is about using information to solve complex problems in simulated environments. This is combined with the sixth framework of providing choices and consequences in simulated worlds. The trick is to combine the educational information the educators want the students to learn, with a narrative that engages them to play and explore, but also learn from and have to problem solve. This is where things like Computer Supported Collaborative Learning (CSCL) (Klopfer et al., 2005) can be effective with the use of clearly defined roles for players that restrict what one individual player can achieve, thereby forcing them to collaborate with others. This idea of differentiating roles is part seven of the MIT team’s framework.

References

Au JW, (2001) A Spector haunts gaming: Inside the mind of Deus Ex designer Warren Spector. Game Slice, Available at: http://www.gameslice.com/features/spector/index.shtml.

Cueva M, Kuhnley R, Revels L, Schoenberg N, Lanier E, Dignan A. (2016) Engaging elements of cancer-related digital stories in Alaska. J Cancer Educ. 2016;31:500–505. doi: 10.1007/s13187-015-0826-z.

Games-to-teach Research Team, (2003) Design Principles of next-generation digital gaming for education. Educational Technology September-October 2003 Vol 43, No.5, pp.17-23.

Hewson J, Danbrook C, Sieppert J. Engaging post-secondary students and older adults in an intergenerational digital storytelling course. Comtemp Issues Educ Res. 2015;8:135–42.

Jenkins H, Squire K, (2002) The art of contested spaces. In L.King (Ed), Game On! London: Barbican.

Kirkpatrick Partners, (2021) The New World Kirkpatrick Model. Accessed online at: The New World Kirkpatrick Model (kirkpatrickpartners.com) (Accessed 26 May 2021).

Klopfer E, Perry J, Squire K, Jan MF, (2005) Collaborative learning through augmented reality role playing. Accessed online at: (PDF) Collaborative learning through augmented reality role playing (researchgate.net) (Accessed 17/5/2021).

Moreau KA, Eady K, Sikora L, Horsley T, (2018) Digital storytelling in health professions education: A systematic review, BMC Medical Education 18; 208.

Price DM, Strodtman L, Brough E, Lonn S, Luo A, (2015) Digital storytelling: an innovative technological approach to nursing education. Nurse Educ. 2015;40:66–70.

Rieber L, (1996) Seriously considering play: Designing interactive learning environments based on the blending of microworlds, simulations and games. Educational Technology Research and Development, 44(2), 42-58.

Stacey G, Hardy P. Challenging the shock of reality through digital storytelling. Nurse Educ Pract. 2011;11:159–64.