SAGSET 2008 – 38th Annual Conference
Nottingham, UK 17-18 July 2008
GROUNDING SIMULATIONS IN REALITY: A CASE STUDY FROM AN UNDERGRADUATE POLITICS DEGREE
Dr Simon Usherwood
University of Surrey, Dpt. of Political, International and Policy Studies and Fellow of the Surrey Centre of Excellence in Professional Training and Education (SCEPTrE)
ABSTRACT
One key issue in the use of simulation games to engage, support and enrich active student learning is the possibility that such games become disconnected from those real-life scenarios that they aim to recreate. In this paper, we consider how the possibility of disconnection can be managed and minimised, using the example of a simulation in an undergraduate politics degree. The paper looks at how academic research in advance of the main game can form the basis of a more rich and realistic simulation, when coupled with extensive feedback from the teaching staff before, during and after the negotiation sessions. This process of grounding the simulation generates a number of beneficial side-effects and, it is argued, generates a much more positive environment for student learning than would be expected from the substantive and procedural knowledge acquired by using simulations per se.
INTRODUCTION
Implicit in the notion of simulations is that of a connection with the real-world: we seek to recreate some aspect of that world within a controlled environment, so that we might better understand what is happening. Thus, the question of how we validate that simulation is one that concerns all those involve in simulating. In this paper, we consider one particular aspect of this validation, using a case study drawn from a university degree programme in Politics, which aims to study the process of negotiation itself. In essence, the paper explores the tension between being simple and being realistic which Loveluck (1994) notes lies so ambiguously at the heart of simulations. This paper starts by making some general observations on validity and verisimilitude, before moving on to describe and discuss the case study and drawing out some key points for consideration. A note on phrasing is appropriate here, since political science offers some particular pitfalls. Realism (with a capital letter) is a major school of political thought, understanding people as selfish and being concerned primarily with their own security against others. To avoid confusion, the word ‘realism’ is avoided wherever possible, and uses of reality and realistic are understood in their more generic senses.
THE NOTIONS OF VALIDITY AND VERISIMILITUDE
While it might appear to be axiomatic that simulations are what their name implies – simulations of the real world – it is uncommon to find much explicit discussion of this relationship or of how it can managed. To take just one example, Gredler’s very useful handbook (1992) on simulation design makes no overt attempt to address this issue. Instead, she notes a series of core issues in design that must be resolved if a simulation is to be successful: the purpose of the game, the designation of roles, the method of scoring, the relationship of surface and deep structures, and the relationship of all these elements together (30-41). It is left to the reader to extrapolate that ‘success’ is understood as an accurate recreation of some real-world interaction. Typically, the matter is discussed at the level of validity. Feinstein and Cannon (2003) explore the notion of external validity, in the sense of the extent to which simulations correspond to relevant events outside the simulation. They note that there is little exploration in the literature on how simulations simplify and reproduce the real-world, instead focusing on the validity of experiments and of instruments, neither of which address the specific concerns raised by the objective of the case-study in hand. As part of an undergraduate degree in Politics, it is concerned primarily with skills development and improving substantive student knowledge about the subject matter. It is through the gaming out of diverse starting positions that students learn about the practices of presentation, negotiation and compromise, and come to appreciate better the complexities of the matter in hand. Consequently, the importance of verisimilitude here is of a specific nature, namely that students need to have a coherent, well-rounded and consistent set of understandings about the subject matter, so that they can enter into the simulation more comfortably, negotiate more effectively and profoundly and then reflect upon how their knowledge and understanding has been shaped by the experience. Even in the case of simulations relating to specific political events (as used in this paper), there is something of an ambivalence about the extent to which real-world knowledge and understanding is necessary. Meerts (1994) notes that in simulating topical diplomatic situations it is important to make sure that games are ‘up-to-date’ (134) by means of modifying structures and pre-game information provided to players. However, he also notes that: “The ideal international political simulation game should be rather simple as far as content in concerned (the participants must be able to play the game without much knowledge beforehand), but should become complicated because of the many opposed and cross-cutting issues involved.” (137) Likewise, Horne (1999) notes the tendency in military war games towards ‘essence’, rather than verisimilitude’ through the use of agent-based models, as a means of side-stepping these issues. The closest we might come is Jones’ discussion of ‘reality of function’, where participants ‘become’ their roles, e.g. chairman (see Gredler, 1992). This includes not just their actions, but also their thoughts; and is grounded in giving participants bona fide roles, together with “sufficient documentation...so that the participants can behave in a professional manner”, all within a framework that encourages them to play out their roles faithfully. However, the question does still remain of how reality of function can best be integrated with a desire for participants to explore and make sense of a set of real-world events or facts, as it desired in this case study.
THE CASE STUDY
For the past two academic years, the author has run a module for second year undergraduates entitled ‘Negotiating Politics’ at the University of Surrey. The module is compulsory for all BA Politics students (approximately 50 in the year 2007-8) and is weighted as one-sixth of that year’s study, as well as counting towards the students’ final degree classification. The module is structured around a series of sessions running through the autumn semester, leading up to the main simulation just before Christmas, and followed by a feedback session in the New Year (see figure 1). The logic of the structure was informed to a significant degree by two basic observations from the first iteration of the module. In that iteration, the negotiation sessions were short (3 hours) and repetitive, being spread through the spring semester, with a reflective portfolio being submitted at the very end of the module. Both students and the author had noted that this structure, while allowing for repeated interactions, was not very suitable for creating an immersive environment within which students could fully explore their roles; a key element in the purpose of using simulations. More pertinently to the current discussion, it was also noted by all involved, that the structure of sessions and of assessment was not very conducive to producing a realistic negotiation and outcome. With these points in mind, the current iteration of the module was significantly modified. Firstly, the main negotiation was concentrated into a three day slot at the end of the autumn semester: this allowed students both to immerse themselves in their roles and to start to understand the role of mental and physical fatigue and engagement in negotiations. Secondly, much more emphasis was placed on creating structures to point students towards realistic processes and positions. This was done using a holistic approach, treating the matter at all three key stages of the simulation; namely, before, during and after (see figure 1). The pre-game phase is essential in establishing and managing expectations on the part of students. In terms of negotiation skills, this is where they receive instruction on theory, practise in a limited environment and receive rapid feedback. It is also where the groundwork for building substantive knowledge and realistic negotiations is laid. This is done in three ways. Firstly, the module leader provides a lecture on the substantive topic. In the last iteration of this module, this lecture covered the negotiation framework used (essentially based on the Conference on Cooperation and Security in Europe that started in the 1970s); the dynamics within that original set of negotiations; some basic information on the Middle East; identification of key issues to consider, and; how the framework might (and might not) work in the case of the Middle East. This is the sole direct input of substantive knowledge by the module leader into the game, and so is accompanied by as much question-and-answer as students feel is useful to them. The second pre-game element is an individual essay, on the student’s group’s position on the subject of the game (so, in this case, their country’s position on improving Middle East cooperation). This serves both as a standard piece of academic work, but also clearly gives scope for preparing realistic and comprehensive positions for the game. Indeed, it is worth noting that the quality and quantity of research by students for this piece of work is significantly more developed than for other modules: it is not unusual for students to arrange interviews with the group they are representing and to gather other primary sources. Feedback for this essay is given before the game starts, so that any issues pertinent to the game can be addressed in a useful fashion. Essay feedback is given both in writing and orally, in conjunction with the final element of pre-game preparation, the group position paper. This is a joint piece of work by the members of each group, briefly outlining group objectives, strategy and tactics, as well as identifying potential allies and obstacles. This aims to get groups working together (a key skill the module aims to develop) and to start applying their research-based knowledge towards practical ends. Typically, there are issues in this process of application, as students will often construct narrow and inflexible positions, rather than considering the underlying interests of their group; and the feedback stresses the need to reflect on, and address, such matters. Taken together, the pre-game elements are designed to give students access to substantive facts, to start a process of application and to give the module leader a sense of how far students have engaged with the matter in hand. On this last point, the module leader uses the feedback at this stage to flag up any issues that students have missed, so that they are not completely unprepared for when they arise. It is also here that the issue of positions of non-engagement are resolved: some groups might feel that their real-world group would not participate in the negotiations, which clearly runs against the other objectives of the module. In such cases, students are advised to modify their position only as much as is necessary to allow them to participate: in most cases, this consists solely of the group playing a rather negative part in proceedings, since they have little interest or desire for negotiations to proceed very far. While this does undermine the verisimilitude of the game for these students, it does still allow for them to follow the large majority of their interests. During the game itself, students are essentially left to their own devices, creating a space and agenda for negotiation by themselves. In the current iteration, the main game lasts for three days, and students were asked to send daily summaries back to their national capitals (i.e. the module leader) at the end of the first two days and again before any final decision. The intention here is for the module leader to give students the freedom already mentioned, while still being able to question any matters that do not fit with the real-world positions of the groups. To take just one example, in the current case, students began to pledge specific amounts of funding to cooperation projects, in a fashion that was not realistic, so they were told simply to indicate their support for funding such projects, with amounts to be agreed later. Thus students are drawn into a two-level game that would be difficult to run in the current institutional setting. This raises two points. Firstly, the figures that students were mentioning were drawn from official sources and media reports which they accessed during the game. Since negotiations inevitably covered subjects that have not been anticipated by all groups (despite the position paper), students used their internet access to find information and incorporate it into the game: browser logs from the game show that this was true for all groups, throughout the game. Thus students were attempting to integrate their pre-game knowledge with new specifics on the hoof. However, this leads to the second point, which is the module leader’s knowledge is also limited, so in some cases it was very difficult to ascertain whether a particular agreement was realistic or not, except in the most general of terms. Thus, while useful in guiding the general thrust of the game, the mechanism outlined did have its problems. Throughout the pre-game and game phases, students had been asked to keep a record of their activities, which they then submitted a few days after the game, together with a reflective essay. This essay asked students to discuss various aspects of their work in the module, including one section on the verisimilitude of their agreement. Here students were being asked to take a step back from their immersion in negotiations and to consider more dispassionately their work. In so doing, they not only were able to identify any issues of improbable agreement, but also developed their ability to use academic tools in a real-world environment. In the final feedback session run by the module leader, the question of verisimilitude is addressed. Students are typically clear about the limits of what they have done and, perhaps more importantly, are clear about the game has forced into less-realistic positions (as in the case of non-engagement, mentioned above). This understanding of their work is essential in allowing students to identify, evaluate and internalise the nature of political negotiations, both in the specific case and more generally.
DISCUSSION AND CONCLUSIONS
Loveluck (1994) talks about a tension between the striving for simplicity in a simulation, so long as it does not compromise the verisimilitude, and the desire to increase complexity, so as to make the simulation more realistic. It is this tension which is the most apparent within the case study. On the one hand, the simulation aims to achieve a realistic capture by participants of the salient points of fact, the real-world actors’ interpretations of those facts and an understanding of the theory and practice of political negotiation. On the other, there is an understanding that it is impractical for participants to discover all the relevant information, or for them to accurately balance out interpretations, or for them to experience every aspect of negotiating. Indeed, within the confines of a university module, it is as much as can be hoped for that students are introduced to all of these elements, without real scope to fully investigate them. On this note, it is worth mentioning that many of the students who have passed through this model have become heavily involved in other negotiation fora, such as the Model United Nations, their interest having being whetted by their study. Clearly, the exigencies of a programme of university study impose limits on what can be attempted. Firstly, a position of non-engagement, as discussed above, is not variable in an academic module where a student is to be assessed on their participation and reflection on their negotiation. Secondly, the structure of academic (and social) timetables limits the ability of students to devote themselves completely to this one activity and the ability of the module leader to provide completely flexible and exhaustive resources. Finally, the very nature of the simulation is intended to be introductory, to show students through their own work how they can start to understand both a set of practices and a particular case-study. Despite these limitations, it is important to stress that the effort to maintain realistic simulations also develops significant additional benefits. Firstly, verisimilitude helps to provide objective benchmarks for encouraging and supporting reflective practice, which is an essential aspect of using simulations, on the sides of both the participants and the module leader. Secondly, verisimilitude as practised here allows for the playing of more complex and involving games than would otherwise be possible, due to resource constraints. Thirdly, it helps students to engage more fully with the real-world situation, by translating their academic research for their essay into practical activity, which necessarily requires interpretation and application of that research, within a supported framework. In the current context of encouraging transferable skills in graduates, this is an opportunity to break down the divide between study and work. One of the key strategies for managing this simulation has been to be explicit about these dimensions with students. In both the pre- and post-game phases, this is a strong attempt to lay out the limitations of the exercise, exactly so that students can begin to see how it shapes their behaviour and their interactions. As is explained to students, this is a nested game, since they are shaped by their existing relationships with other participants, by their programme of study, by their relationship with the module leader, indeed by their understanding of human nature. This is then the ultimate means by which the question of verisimilitude is addressed: by making students aware of the limitations of the simulation, and by asking them specifically to consider such limitations, they can see how those limitations affect them and their actions, so that they can start to see how things might be in the real world. This is not an ideal process, nor a wholly successful one, but it offer a methodology that has scope for incremental improvement over future iterations and is one that has broadly stood up in practice.
REFERENCES
More information about the module can be obtained from the author, by email: s.usherwood@surrey.ac.uk
Figure 1: Indicative Structure of Module ‘Negotiating Politics’ in Academic Year 2007-8