Guidebook Chapter 8 – Research and Evaluation of Effectiveness
The best way to find things out is not to ask questions at all. If you fire off a question, it is like firing off a gun – bang it goes, and everything takes flight and runs for shelter. But if you sit quite still and pretend not to be looking, all the little facts will come and peck around your feet, situations will venture forth from thickets, and intentions will creep out and sum themselves on a stone; and if you are very patient, you will see and understand a great deal more than a person with a gun does (Huxley, 1982, p 20).
Taking the time to plan how to do something and what you questions you need to ask are just as important as the questions and the answers that you will be given once you ask the questions. Taking the time to evaluate a course or a program is extremely important if you want to have success within the course not only for the program but also for the instructors and the students.
There are several key issues that an institution must look at when evaluating a program, course or instructor and there are several different ideologies behind the evaluation models and philosophies. MBC Group does believe that when evaluating anything you must look at all the parts involved and not just one single part. We look at the macro-level, meso-level, and micro-level as discussed by Hew, Liu, Martinez, Bonk and Lee in their study (Hew et al 2004). When looking at these three levels the entire program, specific courses, and learners to be more specific we believe that you get a better picture of how each piece works in relation to the whole. We do however also like to use within this basic structure the AEIOU approach (Simonson, et al (2009) p 352-354).
With the AEIOU Approach we are using quantitative and qualitative methods of evaluation. We will be looking at information related to the implementation of the project and information about the value of it and its activities. With this approach we are looking at five different areas: Accountability, Effectiveness, Impact, Organizational Context, and Unanticipated Consequences.
When looking at just what the AEIOU stands for it may be hard to understand what each actually means when looking at evaluating a program or a course, as a result, we want to give a brief explanation of each. With accountability we will be asking the question of did the project planners and designers do what they said they were going to do? With regards to effectiveness, how well done was the project? Did the project, course or program make a difference or have the desired impact? Organizational context refers to what structures, policies or events in the organizational or environment helped or hindered the project in accomplishing its goals. Finally, when we look at the unanticipated consequences, we are looking at what changes or consequences of importance happened as a result of the project that were not expected (Simonson, et al (2009) p 352-354).
Even with all of these ideas and concepts in mind, we want to make sure that our courses and programs are in line with the National Education Association (NEA) and Western Cooperative for Education Telecommunications (WCET) best practices documents within the United States. These two organizations believe each program’s educational effectiveness and teaching/learning process should be assessed through an evaluation process that uses different methods and several different standards. Secondly, they also believe that when exam are used they should be written to include firm measures for student identification. Third, there should be data on enrollment, costs, and successful or innovative uses of technology which are used to evaluate a program’s effectiveness. Finally, we need to look at the intended learning outcomes and other overall program effectiveness measured. (Anderson, et al (2004) p 421).
MBC Group’s main focus on evaluation is looking at how we can improve our products, services, and make our products fit the needs of our customers. We want to look at what is available and how we can make our products and services even better. We are not able to do this without ongoing evaluations and performance reviews. We want to make sure that all of the programs and courses that we design for you meet all of your needs and wants. We always welcome feedback from the companies, instructors, and students that use our courses and are in our courses. If we did not receive this feedback, we would not know how well things are working or what needs to be changed in order to make the courses work they way they were designed to work or the way our customers want them to work. We believe that if you stop evaluating and making needed modifications to your courses that you will be left behind and be selling outdated products. We want to stay on the forefront of online learning and make sure our customers get exactly what they have asked for and more!
References
Anderson, T. & Elloumi, F. (2004).Theory and Practice of Online Learning, Canada: Athabasca University. Please note, this book can be retrieved athttp://cde.athabascau.ca/online_book/.
Hew, K. F., Liu, S., Martinez, R., Bonk, C., & Lee, J-Y. (2004) Online Education Evaluation: What Should We Evaluate?
Huxley, E. (1982). The flame trees of Thika: Memories of an African childhood. London: Chatto and Windus
Simonson, M, Smaldino, S, Albright, M, & Zvacek, S (2009). Teaching and learning at a distance: Foundations of distance education: Fourth edition. Boston: Pearson/Allyn and Bacon.