Report into the development, delivery and evaluation of an online learning activity delivered to second year, undergraduate, student nurses.

Introduction:

This report presents the development, delivery and evaluation of an online learning activity, it begins by presenting the aims, objectives and rationale for the project, and in particular the rationale for the chosen area of investigation, it then goes on to present the design for the activity demonstrating how it is learner centred, pedagogically sound and accessible; after which a discussion is held regarding the professional and practical issues which arose from the implementation and how these were managed; the next section provides a critically reflective evaluation of the impact of the initiative, drawing on student and staff feedback.

Reflection on personal and professional development in relation to e-learning is included throughout the report.

Aims:

The project aims to assess student engagement and participation in online discussions.

Objectives:

Develop an evaluation framework;

Develop an assessment rubric to operationalise the framework;

Pilot the framework to evaluate the student experience.

Learning outcomes:

For me as a learner, through completion of this project I would develop my skills in project planning and prioritisation; writing learning aims and outcomes; delivering digital media within a VLE; using CourseGenie to develop course content; and developing and designing an evaluation framework.

For the student as a learner, on completion of the online learning activities students will demonstrate their development of knowledge and understanding of dementia and its impact on the person with a learning disability and their carers. They will also develop their ability to use IT as a communication and learning tool; to access and manage information and use IT to present ideas.

For the School of Health & Social Care, upon completion of the project and the successful development, implementation and evaluation of the assessment rubric, the rubric can then be considered for development and use in other modules delivered in the School of Health & Social Care.

Methodology for the project:

Initially I constructed an activity development framework to guide the development of the online activity within this I defined the learning outcomes for the students.

Figure i - Map of activity development within existing module

Figure ii - Design for learning activities

Student Learning Outcomes:

On completion of these online learning activities students will have developed knowledge and understanding in order to:

• Understand what dementia is and how it impacts on the person;

• Recognize how dementia is assessed in people who have a learning disability;

• Identify the needs of the person with dementia and their families and carers;

• Identify approaches to dementia care;

• Identify medication used in the treatment of dementia;

• Develop an understanding and appreciation of risk assessment in relation to supporting a person with dementia;

• Explore and identify the role of the nurse in meeting the needs of the person with dementia.

This was followed by the development of the evaluation framework and the creation of the evaluation tools.

The online learning activity was delivered during which time I facilitated the student learning through participation in asynchronous discussion within the Virtual Learning Environment (VLE).

Figure iii – Survey 1

Figure iv – Survey 2

Upon completion of the online activity I commenced the evaluation stage. Evaluation was achieved through quantitative and qualitative analysis of student engagement with online activity within the VLE and administration of two evaluation surveys. This data informed the students’ final assessment, the assessment was made against an assessment rubric (figure v), adapted from the work of Edelstein & Edwards (2002).

Assessment criteria

Assessing Effectiveness of Student Participation in Online Discussions

(Adapted from: Edelstein & Edwards (2002)

Total possible points: 24

Figure v – Assessment criteria

Rationale:

The rationale for my project arises from the need for a technology equipped workforce, to provide a safer service to patients, and develop a highly skilled and efficient workforce which is able to more flexible in meeting the needs of patients (Connecting for Health, undated), because of these needs it is important to evaluate student learning to determine whether we have been successful in achieving these policy objectives.Furthermore Howatson-Jones (2004) identifies the personal challenges to health care professionals brought about from the need to maintain lifelong learning, and the pressures on resources in meeting these demands, and she emphasized the need for creative solutions to meet the needs of both the learner and the organization in which they work. Gibbon (2006) identifies an increase in seeking information from websites and other technology, it is important that students develop the necessary skills to do this. Within the module and the online activity students would be working to achieve the transferable skills identified by the university, specifically those relating to Information Technology (Oxford Centre for Staff and Learning Development, undated).

A further concern is raised by Blair (2002) who identifies the often isolated working environments of learning disability nurses, and the limited access they may have due to geographical and financial boundaries, many being employed by small organisations without the capital to fund higher education, in the traditional sense. Because of these restrictions on resources considerable importance is placed on investment in the technology to support professional development (Connecting for Health, undated). Blair (2002) also emphasised the importance of e-learning opportunities for this small professional group as a means of communicating via networking and access to resources for continual professional development. I concluded in this light that students need the opportunity to further develop their skills in communication & information technology.

Through participation in a structured online learning activity students would further develop of the necessary skills to make the most of online resources and would have the opportunity to further develop their communication skills, together this would contribute towards equipping them with the necessary skills for continued professional development in an increasing technologically enhanced workplace. Students have been introduced to using the university VLE in previous modules, and have engaged in asynchronous discussion based activities; however this is the first occasion that their contribution to discussion based activities will be assessed.

Evaluation:

Sharpe (undated) identifies the benefits of using multiple methods or data sources to confirm or refute observations. To gain stronger evidence on which to base my conclusions, I selected three methods of data collection this is known as triangulation.

My evaluation framework consisted of three methods of data collection:

1. an assessment rubric;

2. an evaluation survey;

3. a face-to-face evaluation activity.

To assess the effectiveness of student participation in online discussion activity I adopted an existing assessment rubric developed by Edelstein & Edwards (2002), this assessment rubric was developed in a University in the USA to provide a dynamic objective assessment of student participation in an online discussion forum. Participation assumes that the students presented are reflective on different levels in their contributions/participation, that the promptness of each post varies, and that the postings are different to some degree in most of the assessment categories.

I previously identified that the School of Health & Social Care, had not until that time embarked on summative assessment of students online learning, it is hoped that this can be considered for development in other modules delivered in the school.

Due to the range of topics covered in the module and the number of guest speakers and specialist lecturers who support the module. To help to avoid evaluation or questionnaire fatigue I have applied a variety of approaches, this is endorsed by Szwelnik (2005) who promotes a diverse approach to evaluating modules to avoid student fatigue.

I have always found it beneficial to evaluate the weekly content and delivery at the end of the module rather than complete weekly evaluations. Evaluation of the weekly content and delivery is facilitated face-to-face, and timetabled in the final week of the module. The evaluation activity is semi structured, based on the structure of a focus group. McAteer (1998) identifies that focus groups are seen as useful for summative evaluation and can serve as a single, self-contained method of evaluation. Students are focused towards each weeks learning activity and asked to comment on the content and deliver with an aim to identify aspects which should be retained and aspects which need to be developed.

To evaluate the student experience in participating in the online activity I developed an eleven question survey, using Respondus ™, the questions asked students to rate how well the learning outcomes had been achieved, and it also invited suggestions for improvement to the activity (figure iii). The evaluation survey was administered within the VLE. The questions devised were appropriate in gauging student perception of how well they felt the learning outcomes had been achieved.

My previous experiences in delivering online learning activities which were ‘supplementary’ to core module content had mixed results. In previous module evaluations students had stated that placement, study (core material), family and paid work commitments and lack of access to internet off campus had influenced whether students would prioritise time in completing supplementary learning activities. These motivational factors are comparable to those identified by (Timmis et al, 2006). With this in mind I had made the activity a core component of the module; however I continued to anticipate that some students would not engage with the online activity. To determine why this might happen I devised an additional Survey, consisting of a single question (figure iv). The evaluation surveys were administered to the 10 students in the final week of the module.

Development of the learning activity:

Rationale for the learning activity:

The theme of the on-line activity is Dementia; the rationale for selecting this theme for inclusion in the module on understanding complex health needs is due to the demands from practice.

It is well recognised that there is increased longevity of people with learning disabilities surviving into old age, Patel et al (1993) investigated this phenomenon and identified with survival into old age dementia is becoming an increasingly significant issue for services, with prevalence rates for dementia at least twice as high in people with learning disabilities compared to those in the general population.

Students were previously introduced to Dementia in their science module, where they were taught about the underlying physiology and factors which increase the likelihood of developing this disease, this on-line activity seeks to develop their knowledge of dementia in relation to practice and the impact the disease has on the person and their circle of support (Falvey et al, 1997).

Design of the learning activity:

The on-line learning activity was developed for delivery to an undergraduate, second year; practise based module. The focus of the module is Meeting Complex Health Needs in Learning Disability Nursing.

The students have been together one year and in that year they have engaged in on-line learning, and have been afforded the opportunity to develop the necessary skills to log in, navigate around, participate in on-line discussions, post comments and demonstrate all the necessary skills required for the proposed on-line activity at an introductory level. It was anticipated that this on-line activity would further develop their skills in using the VLE and support knowledge construction (Salmon, 2002) through reading, reflecting, and discussion.

I began with a student learning needs analysis (Bostock, undated); I considered several factors including what relevant knowledge and skills they held in relation to professional knowledge and skills; based on my observations of the students in the previous semester I identified some variation in their knowledge and learning styles, variation in levels of engagement in on-line learning activities. I considered their motivation, interest and attitude to teaching/learning methods encountered in the previous semester. I also considered obstacles to their learning, such as anxiety, dyslexia, and colour blindness, lack of concentration, computer access, and placement hours.

I then reviewed the core material for the activity, which is derived from material previously delivered to students in their second year of the learning disability nursing programme (academic year 2005-06). This material was initially developed for face to face delivery; I anticipated that it would require substantial development to become an on-line learning activity. I then reviewed the learning outcomes from the previous delivery of the activity and updated these to apply them to the on-line activity.

Before taking the development any further I felt it was necessary to consult the relevant literature on educational theories and frameworks. I considered how my teaching could be enhanced through Constructive Alignment (Biggs, 1996), (Biggs, 1999) following the 3P model of learning and teaching, namely:

Presage – Where I consider students prior knowledge, experience and motivation and the learning objectives, assessment methods, teaching methods, climate of e-learning within the institution.

Process – Focusing on development of the learning activities seeking a deep approach (Prosser & Trigwell, 1999) to learning;

Product – Focusing on learning outcomes, what knowledge, skills, and involvement would be expected in and from the completion of the activities.

It was also necessary to consider how to develop the reflective practitioner, skills which are essential for a Learning Disability Nurse and how to engage students by considering the range of learning styles and preferences. To achieve this I consulted the work of Honey & Mumford (1986) on learning styles and that of Fleming (Undated) and the VARK guide to learning preferences. I also considered how students may learn through their experiences and focused on the Experiential Learning Cycle (Kolb, 1984).

To assist the development of eLearning skills within the module I have applied the 5 stage model (Salmon, 2004); in supporting my own personal development as an e-moderator I have been guided by the work of Salmon (2003), the design for learning activity is shown in Figure ii.

It is important to recognise that students are likely to utilise several learning preferences and may favour more than one, as in the multimodal approach (Fleming, undated). Therefore the methods of teaching delivery have been developed to meet a range of learning preferences as identified in table 1. You can see from the table that learning achieved through engaging with a VLE has the facility to meet each of the components identified by Fleming (undated).

Table 1 - Learning preferences & resources (adapted from Fleming N (undated)).

The activities have been developed to ensure that students learn from experience, experiential learning (Kolb, 1984) is a key aspect of developing nursing practice. To facilitate this each activity has a component of reflection. The reflection activities require the student to reflect on their reading, their practise and their role as a nurse. The amended model of Kolb (1984) adapted by Chapman (2006) will be utilised to shape the reflection and identify the relationship between reflection and learning style (Honey & Mumford, 1992) (see figure vi).

Figure vi - Kolb’s Learning Styles (Chapman, 2006)

In previous runs of the module online activities had been included as alternatives to face-to-face teaching to provide students with flexibility in engaging in learning online (Salmon, 2002). Regrettably participation from students had been rather poor, citing lack of time due to placement commitments,lack of access to PC’s when off campus and perceived limited value as the activity did not contribute to the modular assessment.

To increase the likelihood of student participation in the activity, I identified time within the module timetable for completion of the activity, Salmon (2002) emphasises the importance of designing etivities, online activities, but cautions the designer to remember they need time and energy to get them to work.

In order to reduce the impact of the statedlack of time the activity was considered equivalent to 3 hours of classroom based learning activities; this was one weeks contact time. Students also had a further private study week allocated later in the semester. Students could either complete the online activity on or off campus. I also hope to influence participation through the assessment of the activity and marks being awarded as part of the modular assessment. Brown (2001) identifies that the assessment defines what students consider as important, and this influences how they spend their time; however this needs to be approached with caution, one needs to be mindful of the quality of the learning experience and design the activity to encourage deep learning rather than surface learning (Prosser & Trigwell, 1999).

Constructing the learning activity:

To create the course content for the activity I choose to use CourseGenie (superceded by Wimba Create ). Wimba Create is a software application which converts Microsoft word documents into HTML. I chose to use this software application as it was one I was not yet fully acquainted and I consider it a valuable application to become more experienced in using, on previous occasions for the creation of web content I had used Macromedia Dreamweaver MX software. To help my skill development I had attended an introductory workshop and a follow up workshop, I now consider that I have an adequate working knowledge of the application.

I initially reviewed and updated the teaching materials I had used previously, and transferred the text content into a Microsoft Word document; I then applied the necessary formatting within Wimba Create, I embedded a video file and images adding descriptions via <alt> tags to ensure the content was accessible. I created a multiple choice self assessment within Wimba Create and once completed I created the html file by running CourseGenie.

Other technical resources were utilised during the project, consideration was made regarding these resources to ensure they are available to students on and off campus, and where students chose to complete the learning activities off campus I made sure the appropriate drivers, plug-ins or software readers were available at no extra cost to students (see Table 2).

*application available online, free of charge.

Table 2 – Technical resources utilised within the project

Implementing the learning activity:

To ensure students have access to the online activity, I checked that all the students were registered and enrolled on the module during week 0.

At the beginning of each module I present an outline of the module to the students I present the learning outcomes, timetable, assessment and assessment criteria, and held a group tutorial to help students begin to focus on the work ahead. During this introductory seminar students were presented with an outline of the online activity, they were informed that the development of the activity was to meet my personal learning outcomes. I explained the sequence of the tasks and my proposed method of evaluation. I clarified the time scale involved; I emphasised that the discussion based task focusing on Dementia was required for the module and needed to be completed within the university Virtual Learning Environment (VLE) WebCT (Web Course Tools). I also identified that this was an assessed activity, the first of its kind within the School of Health& Social Care, and presented the assessment rubric so they were aware of what was expected of them (figure iv). The students were apprehensive, indicating that this was the first occasion that their contribution to online activities was being formally assessed, I attempted to provide the necessary reassurance. I was cautious to ensure that the weighting of the marks for the activity ensured that a student could pass the module even if they choose not to participate in the online activity. Finally I invited their evaluation of their experience in engaging with the activity at the end of the module.

The activity commenced during the week 3 seminar and ended prior to the start of the seminar week 12. Students were given the following information and instructions in readiness for engaging in the activity.

Figure vii – instructions for Dementia online activity

Students were presented with a series of structured activities on the theme of dementia in relation to people with learning disabilities. Each activity could be completed wholly online, or where readings are indicated a student could choose to download and print off the paper and then go online to complete the discussion component of the activity. Each activity within the theme comprised of reading, reflecting and discussion; with students presenting reflections in discussion area and peers reading and then reflecting and commenting on the discussion posting.

During the period between week 3 and 12, I facilitated the students’ engagement in the online activity through the use of an asynchronous discussion board and through additional encouragement and responding to questions raised during face to face sessions; upon completion of the module the online activity was evaluated through a survey delivered in the Quizzes and Surveys facility within the VLE.

Supporting learners:

Students were supported through the components of the activity using the 5 stage model (Salmon, 2004) (see figure ii)

1. Assessment & Motivation - Students have previous experience of accessing and using the VLE (WebCT) and engaging in etivities in year 1 modules and in second year semester 1 module.

2. Online socialisation - Students have prior experience of sending and receiving messages and getting to know each other as a group, this cohort of students have been together for three semesters and have engaged in discussions as individual learners in the previous semester.

3. Information exchange - Students will read identified material, complete the learning activities, reflect on what they have read and then present their reflections to their colleagues for discussion in the online discussion area. The emoderator will facilitate the tasks and discussions, supporting students with the learning materials acting as ‘trouble-shooter’ if required.

4. Knowledge construction – Students will begin to develop their own knowledge from their reading this will be evident in the reflections they present, the discussion dialogue presented in the discussion topics and the personal learning goals which they set and achieve during the activity.The emoderator will facilitate the tasks and discussions.

5. Development – Students will engage in discussions further developing their knowledge on Alzheimer’s and Dementia and further develop their skill in communicating with others in a virtual learning environment.

Results:

There were 10 students registered on the module.

All students participated in the activity, meaning that they read the topic posted and responded to the questions posed by the module leader; six students shared resources with their peers or raised questions for their peers to respond to. The questions posed were a combination of open and closed questions; however these did not develop into active discussions. Two students responded to their peers; and two students answered the questions but did not raise any new questions or comments for peers. All students responded to response questions which I posed during the activity, but once again these did not flourish into an active discussion.

Seven students commenced the activity at the start of the semester, while one student commenced the activity near the end of the semester and completed in one sitting. Those students who commenced the activity at the start of the semester posted their responses for the activity and then revisited the discussion board on a later occasion, reading and responded to the posting of other students. The two students who did not respond to questions posed by their peers completed the majority of the aspects of the activity except for commenting on the responses of others. The students who commenced the activity later in the semester appeared to adopt a similar approach with the viewing and responding to comments made by peers during the completion of the activity.

Discussion Postings:

On the whole student participation was positive, students completed the activity on average in two sittings, and they made a single post in response to each component of the activity. Three students posted between 2 and 4 responses to a few of the questions, these responses were affirmation statements; agreeing with their peers, or were information sharing statements; posting up URLs or references to further reading materials.

Of those students who completed the activity near the end of the semester their postings indicated that they had read comments from their peers and again provided affirmation statements.

None of the postings made by students were critical of the postings of others.

Engagement of the students was likely to be influenced by my role as facilitator and eModerator in a position as participant observer. Tarling & Crofts (2002) identify that the role of participant observer is potentially problematic when carried out in your own work area. The role presents a challenge to the students as the person may be perceived to influence the engagement of the students in completing the activity. The second influencing factor for engagement in the activity arose form the activity being assessed (Brown, 2001). On this occasion the students may have engaged in the activity because I was influencing their contribution in my role or they may have been influenced by the assessment. If student participation were to be further researched it would be important to determine which of these factors the influence was.

Dementia activity evaluation survey:

Student feedback from evaluation survey:

Four people completed the evaluation survey. Results from their responses shown below:

Question 1

Understand what dementia is and how it impacts on the person; student responses were equally split between fully achieved & partially achieved.

Question 2

Recognize how dementia is assessed in people who have a learning disability; students identified that this was partially achieved.

Question 3

Identify the needs of the person with dementia and their families and carers; student responses were split between fully achieved (1) & partially achieved (3).

Question 4

Identify approaches to dementia care; student responses were equally split between fully achieved & partially achieved.

Question 5

Identify medication used in the treatment of dementia; student responses were split between fully achieved (1) & partially achieved (3).

Question 6

Develop an understanding and appreciation of risk assessment in relation to supporting a person with dementia; student responses were equally split between fully achieved & partially achieved.

Question 7

Explore and identify the role of the nurse in meeting the needs of the person with dementia; student responses were equally split between fully achieved & partially achieved.

Question 8

At the beginning of the activity you were asked to identify personal learning goals, please rate how well these were achieved; student responses were split between fully achieved (1) & partially achieved (3).

Question 9

What were the best aspects of the activity for you? (maximum answer 10 lines)

-The activity that asked us to think about the lived experience of a person with dementia was really interesting and made me consider the condition in a way I had never thought of before. I was also very keen to understand relevant medications and looking at other people's answers and links to this question was useful.

-The opportunity to build up knowledge over the whole semester and gradually construct my posts.

-It was mostly interesting and enhanced my knowledge

Question 10

What were the worst aspects of the activity for you? (maximum answer 10 lines)

- It was hard to complete an activity based on the internet, as i dont have the internet at home nor is there assess at my placement, this limits me to using the university computers. There are a very limited number of computers avaliable at marston road! I feel that this is not a sutible activtity to assess students on. It would be better to have a longer essay or another alternative.

- When I finished the activities I realised that although I had a much better understanding of dementia, I was not really considering it specifically in the context of people with learning disabilities, and maybe if I were to do the activities again I would apply this focus more consciously. I have found it difficult to approach these activities as a discussion as generally people seem to be completing the questions at very different times during the semester. This was particularly difficult for the final few questions as for a while I was the only person to post a response, and so it was difficult to gauge my answers with other people's responses.

-The lack of information about dementia in the module. I was lucky because it was an issue in my placement. Other students found it very hard.

-It was too time consuming including this.

Question 11

What improvements to this activity would you suggest?

- It was very time consuming, and was hard to fit in as well as the essay for this module and the other 3 we had from other modules. I would suggest not having this as part of the assessment.

- I know it's unrealistic but a shorter or more focused time frame would encourage discussions. I think I would have also found it beneficial to discuss the activities regularly as part of the Friday morning class with your guidance and feedback, because it often seemed a very 'faceless' activity, and dementia is such a big and important topic to cover. Generally though, I think the activity content was interesting and the links were very informative.

-More relation between the activity and topics covered in lectures.

- Not to give it such a weighty percentage of your coursework, if any at all

Figure viii – Survey 1 - results

None of the students completed survey 2 (figure iv)

Face-to-face evaluation:

Student feedback from face-to-face evaluation:

Eight students completed the evaluation survey. Results from their responses shown below:

Table 3 - Face-to-face evaluation

Discussion:

As an e-moderator I found it particularly challenging to motivate this group, and felt that some members oriented towards the information exchange stage, stage 3 of Salmon’s model (2004), receiving information and posting a response rather than engaging in active discussion. As previously identified, some students participated in the discussion from the outset while other joined in nearer the end of the semester, this staggered engagement proved problematic. It created two sub groups within the students, with vocal comments raised in face-to-face discussions.

One facet which was important to recognise, success is reliant on cooperation and trust, Goodyear emphasises “an important goal of (distance) learning is the creation of an environment of cooperation and trust among students and the instructor so that meaning can be built and shared… Students need a supportive environment, models of appropriate behaviour and a general sense of security if they are to launch themselves on the risky process of knowledge co-construction” (2001:106).

I too felt that this was important, students need to feel safe and secure in the classroom, this is equally important in a virtual classroom. I strived to ensure that when I constructed my responses to student postings I was selective in the words I used to provide a supportive, encouraging demeanour. I also posed questions in such a way to challenge the understanding of the students, I was equally mindful of how these were posed to provide a positive challenge.

Several students expressed that it was ‘not right’ that their colleagues left it so late to join in as it effected the discussion. My own observation as I mentioned before was that the contributions read more as postings than as a discussion. It is important to consider timing when setting an activity, usually a reasonable period of time is necessary for the success of asynchronous discussion (Salmon, 2002) however on this occasion the design of the activity lent itself to a specific focused task and could have been achieved in a fixed time period.

The construction of the focused tasks heavily influenced the actions of the students, the activities invited a formal response to a series of questions, they also invited reflection on experience and critical comments from guided reading, the result of which assisted students to orient towards the fourth stage, Knowledge Construction, students read and constructed responses a few made additional comments, they began to demonstrate development of their own viewpoints and the postings by a few students aimed to assist with widening and appreciating the perspectives others.

One particular challenge arose when it came to assessing the students’ participation in the activity against the assessment rubric, regrettably the rubric did not take into account the participation of a student lat in the semester and their completion of the activity in one sitting. This certainly needs to be rectified in the revision of the assessment rubric.

A further complication arose in the weighting of the marks, with the activity being marked out of 24point = 24% of the module total the combined mark for the module assessment moved downwards, this is wholly unsatisfactory for the student and is being rectified through negotiation with the Dean of the modular programme. On reflection it would have been beneficial to have had more discussion with the course team on this assessment weighting, further development will be discussed with the school Learning & Teaching Committee who are currently revising the marking criterion for student coursework.

It is evident from reading the responses to the open questions presented in the activity evaluation survey that some students had not fully appreciated that the activity was a self contained learning activity. The relation to the module was evident in the module title and theme, Understanding complex needs. The module sought to help students to develop the knowledge to underpin care delivery and support care planning and to build and maintain professional partnerships with service users, families and service providers. Aspects of the activity focused on each of these module components. It was positive to read that having completed the on-line activities, students had on the whole considered that they had, at least in part, met their own and the module learning outcomes.

One student indicated that a shorter more focused timescale for the activity would have influenced their participation with discussion during face-to-face seminars. This is a valuable point to raise and certainly one to be considered in the next delivery of the activity, however it challenges my perception of the value of an on-line activity within my module as the thought behind the activity was to reduce the face-to face contact time and introduce a more flexible method of learning, if I were to timetable discussions on the activity during the semester I would be building in an expectation that students would have engaged in the activity at each face-to-face discussion, I would also need to reduce the delivery time of other content to free up time for more discussion. It may be more beneficial to invest more time in engaging in discussion on-line although that time would be an additional commitment in workload planning calculations; I believe it would be a valuable investment to make.

Eight students contributed to the evaluation of the dementia activity during the face-to-face evaluation. They were split 50/50 in their enjoyment of the module, and comments for changes to the activity include the length of the activity and the weighting for marks awarded compared to the weighting of marks for the case study paper. These comments are definitely being taken into consideration in the next revision of the module.

Despite the challenges presented by the students in engaging with the technology, prioritising time to engage and complete the on-line activities within the given time scale all the students completed the activity and the majority identify a degree of increased confidence and knowledge development in the subject area.

Reflection on own learning:

Personal learning outcomes:

Through completion of the project I aimed to develop my skills in project planning and prioritization, I feel that I approached the project with positive ideas and with support of my personal tutor shaped this into a plan, unfortunately I deviated from plan and struggled with prioritizing my academic work as a learner against the other demands in my posts.

My second learning outcome focused on writing learning aims and outcomes, I feel more self-assured in this aspect and am more confident in differentiating between knowledge and skill components.

My next learning outcome focused on delivering digital media in a VLE, I integrated a single video file into the online learning activity; sourcing and selecting the file was problematic. Currently, in the school where I work, we have very few video files available for use in teaching which are small enough to embed in an on-line learning activity, so I was overly cautious in selecting the file not only considering its suitability and appropriateness for the learning activity but also in ensuring it was attributed correctly. I would like to develop the skill in creating and manipulating media files for use in the school; however I recognize that there are others who already have this skill who I can use as a resource. With the development of a Learning Object Repository within the institution, there may be suitable artefacts which I can use in the near future.

Creating course content was also a learning outcome for this activity, I already felt confident using Microsoft word and was familiar with using styles and attributes, which was useful skill to have when, learning to use CourseGenie/ Wimba Create. The two workshops I attended provided me with supervised experience in using the application and through these I gained in confidence, I included text, images, video clip and a self assessment activity using the multiple choice questionnaire facility to good success. The finished artefact was user friendly and met WW3 accessibility guidelines.

My final learning outcome focused on developing my skills in designing an evaluation framework, I chose to use Respondus™ to develop an evaluation survey. I selected this as I was already familiar with functionality of the application. I chose to design a short survey of 11 questions (figure iii) to evaluate the students experience and determine whether they had met their learning outcomes. The questions devised were appropriate in gauging student perception of how well they felt the learning outcomes had been achieved. To evaluate the module overall I had previously developed an evaluation survey, questions in the module evaluation asked about peer engagement, peer support and tutor support, access to computers etc. To avoid duplication these questions were intentionally excluded from the evaluation for the on-line activity.

I also anticipated that some students may choose not to engage with the learning activity, to establish why that may have happened I posed a single question (figure iv), this may be perceived as cynicism, however I would see it as realism informed by previous experience.

In hindsight to get a fuller and richer amount of feedback for the on-line activity it would have been beneficial to have removed these questions from the standard module evaluation and integrate them in the evaluation for the activity. In the next run of the module I will definitely revise the evaluation questions for the on-line activity and the module evaluation.

Student learning outcomes:

For those students who completed the dementia activity evaluation survey it is clear that they have developed their knowledge and understanding of dementia and its impact on the person with a learning disability and their carers.

Through the contribution presented in the discussion tool within the VLE they have demonstrated an ability to use IT as a communication and learning tool; to access and manage information and use IT to present ideas.

During the face-to-face evaluation students were divided in opinion, however on balance the positive comments indicate that there has been an increase in knowledge and an increase in confidence in the subject area. The less favourable comments were targeted towards the length of the activity in comparison to the weighting of the assessment. On reflection the activity developed into something larger than the original three our activity and certainly needs to be reduced. The assessment method and weighting for the assessments proved a challenge and certainly needs revising before the next delivery of the module.

School learning outcomes:

For the School of Health & Social Care, completion of this project and the development, implementation and evaluation of the assessment rubric, has shown that it is a useful tool for assessing effectiveness of student participation in on-line discussions. However the tool is far from ready for wide spread application to online learning.

It is important that the tool be developed to consider situations including late participant engagement, engagement and completion of the activity in one sitting and single responses to questions without generating discussion. In hindsight the activity and the tool were poorly matched. The tool needed to apply to a less structured learning activity, which had an active discussion basis rather that the formal structured learning activity administered. The tool also needs to be formally aligned with the revised marking criterion in the school.

I do believe with more development and greater consideration on the type of learning activity being delivered that an assessment of assessing effectiveness of student participation in on-line discussions would be valuable for the school.

References:

Biggs J (1996) Enhancing Teaching through Constructive Alignment. Higher Education, 32(3): 347-364

Biggs J (1999) Teaching for quality learning at university. Buckingham: The Society for Research into Higher Education & Open University Press.

Blair J (2002) E-learning: a virtual challenge for educators. Nursing Times. 98(31): 34 – 35

Bostock S (undated) Student Needs Analysis Process. Available at: http://www.elt.ac.uk/ELT%20documents/materials/needs-anal.ppt(accessed 10/01/07)

Brown G (2001) LTSN Assessment Series 3:A guide for lecturers. Available at: http://www.swap.ac.uk/docs/ltsn/assess/03Lecturers.pdf (accessed 23/01/07)

Chapman A (2006) Kolb’s Learning Styles. Available at:http://www.businessballs.com/freematerialsinword/kolb_learning_styles_diagram_colour.doc(Accessed 30/0107)

Connecting for Health (undated) The case for IT in healthcare – supporting the patient safety agenda. Available at: http://www.connectingforhealth.nhs.uk/about/case/the-case-for-it-in-healthcare(accessed 26/06/07)

Edelstein S & Edwards J (2002) If You Build It, They Will Come: Building Learning Communities Through Threaded Discussions. Online Journal of Distance Learning Administration, V (I), Available at:

http://www.westga.edu/~distance/ojdla/spring51/edelstein51.html(accessed 10/01/07)

Falvey M, Forest M, Pearpoint J & Rosenberg R (1997) All My Life's a Circle. Using the Tools: Circles, MAPS and PATH. Toronto: Inclusion Press.

Fleming N (undated) VARK – A guide to your learning preferences. Available at: http://www.vark-learn.com/english/index.asp(accessed 10/01/07)

Goodyear P (2001) Effective networked learning in higher education: notes and guidelines. Available at:http://csalt.lancs.ac.uk/jisc/Guidelines_final.doc(accessed 10/01/07).

Honey P & Mumford A (1986) A Manual of Learning Styles. Maidenhead: Peter Honey.

Howatson-Jones L (2004) Designing web-based education courses for nurses. Nursing Standard; 19(11): 41-44

Kolb DA (1984) Experiential Learning: Experience as the Source of Learning and Development. London: Prentice-Hall.

McAteer (1998) Focus groups. Cited in Harvey J (Ed.) (1998) Evaluation Cookbook. Available at:http://www.icbl.hw.ac.uk/ltdi/cookbook/cookbook.pdf(accessed 23/01/07) (38-39)

Moon J (2002) The module & programme development handbook: a practical guide to linking levels, learning outcomes & assessment. London: Kogan Page

Oxford Centre for Staff and Learning Development (undated) Oxford Brookes List of Transferable Skills. Available at:

http://www.brookes.ac.uk/services/ocsd/2_learntch/trans_skills.html (accessed 23/01/07)

Patel P, Goldberg D & Moss S (1993) Psychiatric morbidity in older people with moderate and severe learning disability. II: The prevalence study. The British Journal of Psychiatry163: 481-491

Prosser M & Trigwell K (1999) Understanding Learning and Teaching, on Deep and Surface Learning. Buckingham: Society for Research into Higher Education & Open University Press.

Salmon G (2002) E-tivities: The key to active online learning. London: Routledge.

Salmon G (2003) E-Moderating - The Key to Online Teaching and Learning. (2^ndedition). London: Routledge.

Salmon G (2004) Running etivity plenaries. Available at: http://www.atimod.com/e-tivities/5stage.shtml (accessed 10/01/07)

Sharpe R (undated) Monitoring and evaluating blended e-learning. Available at:http://www.heacademy.ac.uk/assets/York/documents/ourwork/research/literature_reviews/blended_elearning_monitoring_and_evaluating.pdf(accessed 26/06/07)

Tarling & Crofts (2002) Venturing into the field. In Tarling & Crofts (Eds) The essential researchers handbook (2nd Edition). Edinburgh: Baillière Tindall, 136-157

Timmis S et al (2006) SOLE Project (Students Online Learning Experience). Available at: http://sole.ilrt.bris.ac.uk/index.html(accessed 26/06/07)

Szwelnik A (2005) Improving module evaluation and feedback. Available at:http://www.qube.ac.uk/QuBE/toolbox/resereps/summaries/mod_eval(accessed 17/07/07)

Bibliography:

Benyon D, Stone D, & Woodroffe M (1997)Experience with developing multimedia courseware for the World Wide Web: the need for better tools and clear pedagogy. International Journal of Human Computer Studies 47. 197-218

Berge Z L & Collins M P (1996) (eds) Computer Mediated Communication and the Online Classroom. Volume ll: Higher Education.Creskill: Hampton Press Inc.

Bull J & McKenna C (2001) Blueprint for Computer-assisted Assessment. (Draft Version 4). Luton: The CAA Centre.

Continuing Professional Development Centre & Begbroke Science Park (2007) The Learning Needs Analysis Toolkit. Available at:http://tall.conted.ox.ac.uk/lnat/ (accessed 10/01/07)

Cook J (2001) Evaluating Learning Technology Resources: Workbook. Available at:http://www.ltss.bris.ac.uk/publications/guides/evalwkbk/evalwkbk.htm(accessed 23/01/07)

Entwistle NJ & Marton F (1984) Changing conceptions of learning and research. In: Marton F, Hounsell D & Entwistle N (eds.) The Experience of Learning. Edinburgh: Scottish Academy Press, pp. 211-236

Heard S, Chapman K & Heath S (1997) Protocol for the implementation of summative computer-assisted assessment examinations. Aberdeen: UniversityofAberdeen, MERTaL Publications.

Heard S, Nicol J & Heath S (1997) Setting effective objective tests. Aberdeen: UniversityofAberdeen, MERTaL Publications.

Joint Information Systems Committee (undated) JISC e-Learning focus. Available at: http://www.elearning.ac.uk/ (accessed 10/01/07)

JISC (2004). Effective practice with e-Learning: A good practice guide in designing for elearning. Available at:http://www.jisc.ac.uk/uploaded_documents/ACF5D0.pdf (accessed 10/01/07)

Laurillard D (2002) Rethinking university teaching: a conversational framework for the effective use of learning technologies (2nd Edition). London: Routledge Falmer

McConnell D (2000) Implementing computer supported cooperative learning (2nd Edition). London: Kogan Page

Maier P & Warren A (2000) Integrating technology in learning and teaching, a practical guide for educators, London: Kogan Page

Milne J & Heath S (1997) Evaluation Handbook for successful CALcourseware development. University of Aberdeen. MERTaL Publications.

Moule P (2006) E-learning for healthcare students: developing the communities of practice framework. Journal of Advanced Nursing. 54 (3), 370-380.

Phillips R (Ed) (2000) A handbook for learning centred evaluation of computer facilitated learning projects in higher education.Available at: http://www.tlc.murdoch.edu.au/archive/cutsd99/handbook/handbook.html (accessed 10/01/07)

Respondus ™ (2003) Testing, assessment and survey applications for the e-learning market. Available at: http://www.respondus.com/(accessed 10/01/07)

Stephenson J (Ed) (2001) Teaching and learning online: pedagogies for new technologies. London: Kogan Page.

Ward C (1981) Preparing and using objective questions. Cheltenham: StanleyThornes (Publishers) Ltd.

Weller M (2002) Delivering learning on the net. London: Routledge Falmer.

© J Pawlyn July 2007