Embedded Files
Last Updated: 08/21/2023
Evaluations of Online Learning
Goals
Goals
Develop & design global digital learning.
Discuss & examine global digital learning.
Discuss & examine evaluations of online learning.
Instructor's Note
Instructor's Note
Assessing online learning has two different parts: assessing online instructional design and assessing online teaching.
In this lesson, we will focus on assessing online learning from an instructional design aspect. Within the four components of technology integration, evaluation is a process to assist us to know whether our design is effective, whether students learn the content, and whether you achieve your goals and objectives etc. Effective online learning assessment should be able to assess content, integrated online technology, and integrated online instructional strategies. Frequently, educators only assess the content and conclude whether their online instructions are effective. This is a mistake that we should avoid. We need to assess online technology and online instructional strategies that we integrate so we know whether our designs (online technology and online instructional strategies) are effective. For example, if we integrate online discussion board (online technology), we need to assess whether online discussion board is an effective tool for your design. Could other online technologies, such as chat room or video conferencing, be more effective than online discussion board? If we integrate learner-learner interaction to support online collaboration, do we know that learner-learner activity is effective or should we consider a different strategy to support online collaboration.
In our assessment, our evaluation methods, and evaluation instruments must be capable to evaluate all three areas so we are informed whether our online instructions are effective.
Activity
Activity
Online Discussions (4 Points)
Assessing Online Learning Communities: Strategies and Insights
Introduction:
This integrated discussion focuses on the comprehensive assessment of online learning experiences through diverse lenses. How might we effectively evaluate an online learning community structured according to the frameworks of Online Communities of Practice (OCoP) and/or Communities of Inquiry (CoI)? Is a sole assessment of students' content acquisition sufficient, or should we also consider their engagement in social interaction, a focal point of constructivist strategies? Furthermore, how can we appraise the efficacy of the instructional design you've incorporated, analyze the quality of the instructor's teaching methods, and gauge the extent to which the community exhibits congruence and collaborative dynamics?
Additionally, it is imperative to consider the inherent data resources available in LMS such as Canvas, or Web 2.0 platforms like Nearpod. These backend data can significantly assist in the assessment of diverse presences, as well as network and community development. For instance, as demonstrated in Assignment 2, we delved into the assessment and analytical tools provided by LMS, particularly focusing on the often overlooked analytic features which can greatly benefit instructional designers and educators alike. In alignment with Harasim's findings, her chapter highlights contextual indicators of achievement, primarily centered on "fundamental and easily accessible usage data that can offer insights into the community's dynamics, complemented by qualitative indicators that shed light on its social and intellectual activities." It is also important to remember the potential backup quantitative data accessible through Learning Management Analytics (LMA) or similar educational tools.
By combining assessment strategies with the insights derived from Social Network Analysis, this discussion aims to equip educators and instructional designers with holistic perspectives on evaluating online learning experiences. We delve into the importance of assessing various dimensions of online education and how leveraging data-driven insights can lead to more impactful and engaging digital learning communities.
Read:
Assessing online learning communities
Read:
Read the section: Assessment in online learning.
Assessment strategies for online learning: Engagement & Authenticity by Dianne Conrad & Jason Openo (2018)
Select one chapter from Chapter 1-7 to read based on your own interests.
Chapter 5 & 6 are advantageous to your final PD unit.
Office of Educational Technology, Department of Education (DOE), USA. Assessment: Section 4: Measuring for Learning.
Important article to understand the trend of technology-based assessment.
Optional:
Social Network Analysis (SNA): Using SNA, as technology-driven assessment, to assess a learning community
What is Social Network Analysis? YouTube Video, Length: 3:45. By Mod•U: Powerful Concepts in Social Science
Social Network Analysis by ETC.
If necessary, conduct further online research.
Be sure you have good understanding on different SNA, such as In-Degree; Out-Degree; Betweenness Centrality; Closeness Centrality; Eigenvector Centrality; PageRank; Clustering Coefficient; Reciprocated Vertex Pair Ratio etc.
In your lesson discussion feedback, we will integrate data-driven feedback/instructions with different SNA scores to help you to understand the role you plan in the class discussion network/community.
Optional
Applying Social Network Analysis to Web-Based Student Interaction by Virginia Commonwealth University.
The link is unavailable. If you know the new link, inform the instructor.
Tu, C. H., Yen, C.-J., Özkeskin, E. E., & Sujo-Montes, L. (2022). From online interaction to social learning analytics and community building. In K. Seo & S. Gibbons (Eds.) Technology in Interaction, Interaction in Technology (pp.134-157). Routledge.
Discussion Questions:
Part I: Expanding Assessment Horizons
Briefly reflect on the limitations of solely assessing content or cognitive presence in online learning. Share your perspective on whether it's adequate to assess content alone or if evaluating teaching presence, social presence, and instructional design is equally crucial.
Delve into the challenges and solutions surrounding the assessment of teaching presence and social presence in online education. Discuss how we can measure and evaluate these elements to ensure effective online teaching and community building.
Part II: Unleashing Insights with Social Network Analysis (SNA)
Throughout our lesson discussions, you've actively participated both as a participant and moderator, and your social network performance has been reflected in our weekly discussion feedback. Discuss the potential of Social Network Analysis (SNA) as an alternative method for evaluating online learning communities, focusing on interaction and collaboration within Online Communities of Practice (OCoP) and Communities of Inquiry (CoI). Offer instances of how various stakeholders, including teachers, instructional designers, moderators, and administrators, can employ SNA to enrich the sense of community within digital learning environments.
Assignment: (Assignment 4 & 5)
Assignment: (Assignment 4 & 5)
Assignment 4: Final Project: Online Professional Development Unit for Global Digital Learning Community (Signature Assignment)
Assignment 4: Final Project: Online Professional Development Unit for Global Digital Learning Community (Signature Assignment)
Grade: 22 points
Group Project
Easy (1) - Challenging (5): 5 (Previous students' ratings)
Submission:
Submit: One person from the group should submit the project URL to Canvas.
Instructions
Goal
Apply the UX (User Experience) Design model to guide your team design process.
Each group will create a complete online professional development website/community/unit by integrating a LMS to model and to teach teachers, parents, or all stakeholders about Global Digital Learning and global digital citizenship.
Your group project should demonstrate that you and your group members, as educational technology leaders/directors candidates, meet Standard 2 & 5 in terms of Standard 2 & 5 (see below) in addition to the criteria listed under the rubrics (see below).
Standard 2
2.5. Learning Communities-Locate, develop, and disseminate models of local and global learning communities and support implementation through local infrastructure and policies (In other words, ensure your online PD integrates online technology to support the local and global learning communities.)
Standard 5:
5.1. Digital Equity- Facilitate equitable access to digital tools and resources, including adaptive and assistive technologies, to support the diverse needs of all learners (In other words, ensure your unit integrates online technologies, and resources to support the diversified learners.)
5.2 Policies for Safe, Healthy, Legal, and Ethical Use- Lead district in the collaborative development, dissemination, and evaluation of district policies and procedures designed to guide, in the least restrictive manner possible, the safe, legal, and ethical use of digital information and technology resources among all stakeholders (In other words, ensure to research and to evaluate your school's or districts' online learning policies and procedure.)
5.3. Programs for Safe, Healthy, Legal, and Ethical Use- Develop and implement programs to educate all stakeholders about the safe, legal, and ethical use of information and technology resources, including district policies and procedures regarding filtering, netiquette, cyberbullying, acceptable use, and fair-use guidelines for online resources (In other words, design and develop an online PD unit to teach colleagues, parents, or the community about these listed topics.)
5.4. Diversity, Cultural Understanding, and Global Awareness- Model and promote diversity, cultural understanding, and global awareness by providing and supporting digital-age communication and collaboration tools for students and faculty to engage in cross-cultural, collaborative projects addressing global issues (In other words, ensure your units model and promote these listed topics)
Tasks
Scenario:
While approaching to another semester, online learning or remote learning will remain in the pandemic and the post-pandemic era. While educators, parents, and the stakeholders continue strengthening their online learning or remote learning, seminal contributions should be made to global digital citizenship. Your school or college has asked your instructional design team to advance the competency of digital citizenship for school’s teachers, instructional designers, staff, administrators, parents, or the stakeholders.
Research and evaluate your school's school district's, or college's Online Learning Policies for Safe, Healthy, Legal, and Ethical Uses, or Computing Policy.
Design, develop, and disseminate a complete online professional development website/community/unit to teach and to promote global digital learning for teachers, parents, or all stakeholders based on your research and evaluations on the school online learning policies and procedures.
Integrate a LMS as your Community of Practice central location to promote and disseminate your instructions to your community members.
CoP Website/Pages
Home Page:
Title of your CoP
An introduction to explain the purposes of this online community
Community news/announcements
Unit/Module Pages: A complete online PD unit (You can integrate multiple pages to deliver different components of your unit.)
Unit Requirements/Criteria:
Goals and objectives
Clear achievable and measurable objectives for each lesson.
Include them in your unit.
Unit size
At least three lessons.
Targeted audiences:
Teachers, instructional designers, staff, parents, or all stakeholders
Content:
Digital Equity
Equitable access to digital technology and resources
Adaptive and assistive technologies for diversified learners
Using digital tools to contribute to positive social change
For more see the Standard 5.1 (see above)
Digital Learning for Safe, Healthy, Legal, and Ethical Uses
Information filtering, netiquette, cyberbullying, acceptable use, fair-use, diversity, global issues, global awareness, online cross-cultural learning etc.
For more see the Standard 5.3 & 5.4 (see above)
Diversity, Cultural Understanding, and Global Awareness
Diversities learners, cultural understanding, and global awareness
Digital-age communication and collaboration tools
Cross-cultural, collaborative learning in global communities
Engaging in civil discourse online
For more see the Standard 5.4 (see above)
Podcast/Audio content
Create a companion podcast or audio for each topic to address universally designed content.
Length: 5+ minutes each
Optional:
Transcript
Upload to any podcast platform
Any relevant digital learning content to your school/school district to reflect the local community needs and culture.
Online technologies:
Integrate at least THREE or more online technologies to deliver your instructions and
to support the online "Constructivist" instructional strategies.
Discussion board; online chat, online conferencing systems, blogs/Journals, social network tools.
Be sure you include online discussion instructions, guidelines, and criteria.
Be sure to include online discussion facilitation guideline, if applicable.
Note: Online technologies here are referred to "the delivery technologies" that you intend to integrate to delivery your online instructions, not the online technologies that you intend to teach, if applicable.
Online instructional strategies:
Integrate these THREE required online instructional strategies:
Online collaboration
OCoP
Online social presence (OSP), Social & Emotional Learning (SEL), and/or Culturally Responsive Teaching (CRT)
Integrate three or more other online Constructivist instructional strategies.
Integrate "effective" online Constructivist instructional strategies. (Read online Constructivist instructional strategies) (You may integrate "Instructivist" instructional strategies but Constructivist instructional strategies should be your main key design.)
Apply a model of online learning from our readings to guide you to select effective online instructional strategies and to design interactive online activities.
Ally's a model of online learning
The attributes of learning & six forms of online interaction
Online collaborative learning (OCL)
Online community of practice (OCoP)
Community of Inquiry (CoI)
Online activities:
In each lesson, you should have at least one or more "interactive" online activities to promote active online learning.
Design online "interactive" learning activities by integrating online Constructivist instructional strategies.
The activities must be completely online ones since this is an online PD.
HINTS: High level of online learner-learner interaction, high level of interactive online collaboration, high level of interactive online community etc.
All online activities should integrate online technologies AND include online activity instructions, guidelines, and assessments etc. to assist the targeted audiences to complete the activities.
Assessments
All online learning activities should be accessed.
Both formative and summative assessment should be integrated, particularly formative ones.
Assessment rubric(s) or other methods for each online activity that measures the achievement of the unit objectives.
Integrated assessment methods are able to assess
Content,
Integrated online technologies (Discussion board; online conferencing systems, blogs, social network tools etc.)
Integrated online constructivist instructional strategies and online activities (Online collaboration, OCoP, Learner interaction etc.).
School/Organization policy & procedures Page: Include
A link(s) and/or a page(s) to contain the school's or the school district's online learning or digital/computing policies and procedures.
A report of the research AND evaluation on the school's or district's online learning policy and procedures. Particularly examine and evaluate the areas of safe, legal, and ethical use of digital information and technology resources for online learning, or digital/computing policy and procedures.
A draft policy/procedures based on your research and evaluation finding.
Communication & Collaboration Tool Page
Integrate at least one or more online communication collaboration tools to facilitate the online communication and collaboration among your community members.
The integrated communication and collaboration tools can be related to your unit technologies.
Online Resource Page
At least 25 shared bookmarked resources all together.
The resources should be categories by the content or others, such as netiquette, cyberbullying, acceptable use, fair-use, online social presence, SEL, CRT etc.
Optional
Use Diigo to organize online resources.
Include your social tagging architecture for social bookmarking tools (Diigo) to provide and to allow your community members to contribute their learning resources.
Contact Page:
Include the contacts allowing the community to research you.
Notes:
All listed pages above should be completed but not limited.
Be sure you have full or most access to the selected LMS. Certain free trial periods may run out before the end of the class.
Q: Our group members are in adult learning field. Is the project to be in K-12 context?
A: Your final PD can be in any adult learning context, such as higher education, corporate, policy, military, or government fields. Global Digital Citizenship theme should remain the same.
Q: Can we approach our final PD as an online micro-credential learning?
A: Yes. Excellent idea. Go for it.
Q: Can we integrate the design of digital badges?
A: Yes. Great idea. I am looking forward to seeing the design.
Would like to learn from previous students' final units?
See: https://www.diigo.com/profile/chihhsiungtu?query=ETC645
Search the keyword/tag: PD
Not all units are available at time you access. Some units may require password or the groups have taken them down. The requirements for final PD may vary from semester to semester.)
Final PD presentation recordings from the previous students.
If you have taken ETC567, you have some knowledge on Global Digital Citizenship. You can use the knowledge you gained from ETC567 to help you to help you to develop your ETC645 final unit.
Grading criteria/rubrics
Assignment 5: Final Project Presentation
Assignment 5: Final Project Presentation
Grade: 6 points
Group Project
Easy (1) - Challenging (5): 3.5 (Previous students' ratings)
Submission:
Using NAU Zoom to record the presentation.
A group member should submit the project's URL and Zoom recording to both
the Canvas Lesson 7 AND
Lesson 8 discussion areas, allowing the class to access and engage in peer evaluation.
Instructions
Part I: Record your group PD presentation on Blackboard Collaborate Ultra
Goals:
To promote your PD.
To assist your PD audiences, the classmates, and the instructor to understand your online PD and to provide a grand tour to your online PD.
Length: 10-12 Minutes Maximum
Use NAU Zoom to record your group presentations.
Format:
Presentation or showcasing format.
Use your live PD website demo/tour and/or screenshots of your PD websites to show and tell.
Presentation slides not required but encouraged, if applicable.
The presentations should include these sections:
Overview (1 minute):
What is your PD about?
What are the applied learning theories, the applied model of online learning, and the design framework or model etc.?
How did UX design guide your team design process?
Instructional strategies (5.5 minutes):
Highlight your online instructional strategies (Online collaboration; Learner interaction; OCoP; Learner Resources; Learner planning & Publishing; & any other instructional strategies.).
Discuss what and where online instructional strategies that you apply and what online technologies you integrate to deliver your online learning activities.
EX: Where/What activities that you integrate OCoP?
Be sure to apply different models from our readings to support your designs.
Be sure to cover all required online instructional strategies and highlight other important integrated online instructional strategies.
Assessments (1 minute):
Discuss how you assess your online PD.
Be sure to discuss how you assess all three components, content, online technology, and online instructional strategies.
School/Organization policy & procedures (.5 minute)
Briefly discuss/highlight your revised school/organization policy
Align to the standards (0.5 minute)
Discuss in what ways your PD address the aligned standards (Be specific; just say your PD align to the standards is not acceptable):
2.5. Learning Communities - Locate, develop, and disseminate models of local and global learning communities and support implementation through local infrastructure and policies (In other words, ensure your online PD must integrate online technology to support the local and global learning communities.)
5.1. Digital Equity-Facilitate equitable access to digital tools and resources, including adaptive and assistive technologies, to support the diverse needs of all learners (In other words, ensure your unit integrates online technologies, and resources to support the diversified learners.)
5.2. Policies for Safe, Healthy, Legal, and Ethical Use-Lead district in the collaborative development, dissemination, and evaluation of district policies and procedures designed to guide, in the least restrictive manner possible, the safe, legal, and ethical use of digital information and technology resources among all stakeholders (In other words, ensure to research and to evaluate your school's or districts' online learning policies and procedure.)
5.3. Programs for Safe, Healthy, Legal, and Ethical Use- Develop and implement programs to educate all stakeholders about the safe, legal, and ethical use of information and technology resources, including district policies and procedures regarding filtering, netiquette, cyberbullying, acceptable use, and fair-use guidelines for online resources (In other words, design and develop an online PD unit to teach colleagues, parents, or the community about these listed topics.)
5.4. Diversity, Cultural Understanding, and Global Awareness- Model and promote diversity, cultural understanding, and global awareness by providing and supporting digital-age communication and collaboration tools for students and faculty to engage in cross-cultural, collaborative projects addressing global issues (In other words, ensure your units model and promote these listed topics.)
Conclusions (2 minute)
Discuss the strengths and weaknesses of your draft/revised policy.
5.2. Policies for Safe, Healthy, Legal, and Ethical Use-Lead district in the collaborative development, dissemination, and evaluation of district policies and procedures designed to guide, in the least restrictive manner possible, the safe, legal, and ethical use of digital information and technology resources among all stakeholders (In other words, ensure to research and to evaluate your school's or districts' online learning policies and procedure.)
Discuss in what way your OCoP design in the PD is "sustainable?"
Discuss in what way you PD may facilitate/moderate effective electronic communication and discussions etc.
Using NAU Zoom to record the presentation.
Part II: Share your PD information and PD presentation recording with the class
Lesson 7 Discussion Board
One person from your team post your group project title, URL, project descriptions, and PD presentation recording URL to Canvas discussion board; therefore, our classmates know where to visit your project and the presentation recording.
Be sure to provide the classmates a guest/student account to review your PD on the LMS. If needed, provide log in instruction. In some cases, one guest account could be provided to all classmates.
Notes:
You must use NAU Zoom to present and to show your actual PD on LMS/CMS so others have better ideas what your PD may look alike.
PowerPoint is not required but recommended.
Bonus: Participate the course evaluations
Bonus: Participate the course evaluations
Grade: 1 point
Due: See Assignment Page
Individual Assignment
Submission:
See what previous students said why we should participate the course evaluation.
Be sure to click "Send Proof" to notify the instructor. By default, student names are confidential. A student may elect to “Send Proof” to the instructor, if the instructor requests, by going to nau.edu/course_evals > Surveys > Completed Surveys > Send Proof link in the Date/Time Completed column.
Want to learn more: See Student evaluation survey.
Page updated
Google Sites
Report abuse