by Chloe Pozderac
Demir, Bruce-Kotey, and Alenezi investigated three learning management systems (LMS), Canvas, Blackboard, and Moodle, to determine what components of an LMS are most valuable from the standpoint of the instructor. The three research questions they were pondering were how do these three LMS differ in their usability, satisfaction with the system, and most used functions. This study was conducted by in-service teachers at the graduate level. They were required to use their chosen LMS as a platform and post as if they were conducting a class through it using all the available features. The results showed that between Canvas, Blackboard, and Moodle there was a difference in usability and satisfaction from both the student perspective and teacher perspective. Canvas was found to have the highest satisfaction with a SUS score of 70.6 which correlated to a C (Demir, pp. 58). Blackboard came in a close second as it also obtained a C but with a lower SUS score of 65.3. Moodle scored a D from a SUS score of 52.5. A similar pattern was observed for usability and learnability. Lastly, Demir (2022) addressed which features were most important to users. Individuals who investigated Blackboard determined that the grade book, assignment tab, announcement function, and syllabus were the top components used. Moodle users reinforced Blackboard users’ views as their findings had the highest percentage of strongly agree and agree for the all of same features. Canvas users highlighted some varying features in their top selections. Their highest frequency use feature was to conduct polls/tests and quizzes. Conducting polls/tests and quizzes was then followed by the same features that were highlighted by both users of Blackboard and Moodle.
Based on the findings of Demir, Bruce-Kotey, and Alenezi’s (2022) paper, I came to some conclusions on why many K-12 schools chose to use Google Classroom as their preferred LMS. Google Classroom contains all the top-used functions such as conducting polls, tests, and quizzes, a built-in grade book, and the easy ability to post assignments, announcements, and syllabi. When it comes to LMS, teachers and students alike desire something that quickly presents all the information in a user-friendly format that is accessible when needed. Some additional aspects that Google Classroom has to offer are the “Reuse post from another class” and “Schedule”. This allows the teacher to copy an assignment from another class and easily post to a different class or to seamlessly post to multiple classes at once for a later date. This further increases the usability of the platform that all teachers need.
Demir, F., Bruce-Kotey, C., & Alenezi, F. (2022). User experience matters: does one size fit all? Evaluation of learning management systems. Technology, Knowledge and Learning, 27(1), 49-67.
Comment by Tristan Silver
Although it was interesting to read your synthesis of Demir et al.'s 2022 paper regarding how instructors felt that three different Learning Management Systems (LMS) compared, one study is not enough to convince me that any one of these LMS is better than another. The instructors being allowed to choose what LMS to conduct a class through for the purpose of the study, instead of the LMS being randomly assigned to them, introduces a strong bias. For example, if a teacher had already been using Canvas for some years and chose to use Canvas for the purpose of the study, it is likely that they would rate this LMS higher than those that they are not as familiar with. I enjoyed how you connected what you read in the study to what you have seen in K-12 instructor use of Google Classroom but also feel that this point would have been made stronger if preceded by more than one LMS study.
Comment by Kelli Mohn
I have been analyzing quantitative and qualitative research on different LMS for a different class, so it was interesting to read this research and compare it to the research I've found. Specifically, one study that I read analyzed how student interaction with Blackboard affects student performance, and if there is a trend between how much time students spend interacting with the LMS and the final course grade. It found a weak positive linear correlation between login times and final grade proving that the more time students spend on Blackboard the higher their overall course grade was. Another study researched students' thoughts about learning management systems and found that communication, discussion, learning activity support were some of the most liked features of the LMS. These studies were both from the viewpoint of the students, but if used with your study can provide districts with a more illustrative overall view of what their students and staff desire from a LMS. One aspect that neither study addresses is cost. Google Classroom appears to be priced at $3, $4, and $5 depending on the type of license while Blackboard is priced at $300 per classroom and $9000 per department. While student and teacher needs are hopefully considered, cost is definitely a factor that districts consider when purchasing an LMS.
The Private Academic Library Network of Indiana (PALNI) is a 501(c)(3) owned by twenty-four supported private academic institutions including Butler University and University of Indianapolis. Their goal is to collaborate to enhance teaching through optimizing library resources and services while partnering with others groups to reduce costs. They created a LMS Evaluation Task Force dedicated to developing, documenting, and supporting a shared process with PALNI supported organizations to assist with LMS evaluation and investigate possible partnerships. The task force created a tool kit to prepare districts to review an LMS, walk them through the piloting process, provide them with rubrics to compare the LMS options, suggest tips for implementing a new LMS. Below are three rubrics that can be used for evaluating and comparing LMS. The first is a general evaluation rubric that could be used during a vendor presentation or during a pilot session. The second and third rubrics are designed to compare several LMS against each other. They are downloadable and editable so the author can change the vendors as needed.