A primary goal in learning analytics (LA) research and development is to advance the understanding of student learning within authentic educational settings and to provide data-driven insight to learners, educators, and administrators to improve learning and learning design (LD). The intersection between LA and LD enables the development of meaningful insights for educators and learners that can inform learner feedback and the refinement of course design. In a review of the literature, Law and Liang (2020) identified two key features of efforts that are successful in making the LA/LD connection: (1) the existence of a tight coupling between the intended learning outcomes, task sequence, analytics used, and feedback generated; and (2) that the LA to be adopted and the feedback to be given to the learners are part of the LD process rather than a separate post-design activity. To scaffold the deployment of design-appropriate LA requires LD frameworks and technology platforms that enable the operationalisation of these frameworks.
Over time there have been several efforts to develop taxonomies and frameworks to bridge the LA/LD gap. For example, the Cognitive OPeration framework for Analytics (COPA) (Gibson, Kitto & Wills, 2014) bridges LA and LD through integrating LD-related constructs relating to learner cognition into the LA taxonomy. Seufert et al. (2019) further highlighted the importance of connecting LA objectives to pedagogical concerns when proposing a 2-dimensional framework for the categorisation of analytic objectives: the context of learning (individual vs. social), and the analytic purpose (providing prediction vs. supporting reflection). Building on insight from existing literature, Law & Liang (2020) proposed a multilevel LA taxonomy that connects parameters in five aspects of LA decisions (measures, data type, functionality, techniques, and stakeholders) to three different levels of pedagogical decision making (course, curriculum component, and task levels).
In other emerging work, patterns of performance have been proposed to help educators map how the LD can help learners to achieve certain learning goals, and learners to understand how they are progressing in the trajectory of their learning (Milligan et al., 2020). This can allow learners to monitor and manage their own learning, while also providing educators with feedback on whether the LD is effective in enabling student attainment of learning goals. The models of progression that underpin the analysis of these patterns of performance are strongly influenced by the educator’s LD decisions relating to tasks, sequence, and assessment measures. The models are premised on a clear developmental focus which provides learners with an opportunity to demonstrate growth as well as to adapt to the context and content of the course.
The DesignLAK22 workshop will explore how these different LD and learner progress models can be combined with LA measures to be translated into meaningful visualisations of feedback for learners and educators. The workshop session will involve a series of activities on the process of developing these visualisations in practice supported through the use of the Learning Design Studio (LDS) platform (Law et al., 2017). The design of the LDS was informed by Goodyear’s (1999) four level framework of pedagogical decision making and Alexander, Ishikawa and Silverstein’s (1977) model of a design pattern language to encapsulate the hierarchically embedded granularities in design. The platform provides a learning task taxonomy that reflects the pedagogical orientation of each of the selected tasks and a task setting template to record associated social, technological and assessment settings. LDS enables the exploration of learners’ behaviour and outcomes by based on LD principles and assumptions. Educators can also use the LA tool to specify LA visualisations.
Over the past six years, the DesignLAK series of workshops have explored the intersection between learning analytics and learning design from a number of different perspectives. DesignLAK workshops have previously focused on key concepts around feedback processes (Authors, 2016), assessment design (Authors, 2017), and validity of assessment measures (Authors, 2019). In 2018, the DesignLAK workshop showcased different LA/LD tools from around the world (Authors, 2018), and in 2021 (our first online workshop) we explored and used MIT’s DIVE prototyping tool to allow participants to rapidly prototype LA visualisations with reference to LD (Authors, 2021). DesignLAK22 will provide an opportunity to bring together key elements from these previous workshops, through the application of models of LA/LD and learner progression that have been inspired by the outcomes of the conversations and sharing of practice through these previous events.
The objectives of the DesignLAK22 workshop are to explore (1) the alignment between intended learning outcomes, pedagogy, task sequence, and assessment design with LA, (2) the ways that models of learner progression can be developed from a LD to inform how feedback can be visualised, and (3) how the resulting LD-informed LA visualisations can be interpreted and utilised for feedback to learners and educators/designers. The workshop is designed for a wide audience including learning analytics researchers and practitioners, as well as learning designers interested in the use of LA to inform their research/practice (i.e., LAK attendees).
Alexander, C., Ishikawa, S., & Silverstein, M. (1977). A Pattern Language: Towns, Buildings, Construction. OUP: USA.
Gibson, A., Kitto, K., & Willis, J. (2014). A cognitive processing framework for learning analytics. Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ʼ14), 24–28 March 2014, Indianapolis, IN, USA (pp. 212–216). New York: ACM. https://doi.org/10.1145/2567574.256761
Goodyear, P. (1999). Pedagogical frameworks and action research in open and distance learning. European Journal of Open, Distance and E-Learning, 2(1).
https://old.eurodl.org/?p=archives&year=1999&article=35
Law, N., Li, L., Farias Herrera, L., Chan, A., & Pong, T. C. (2017). A Pattern Language Based Learning Design Studio for an Analytics Informed Inter-Professional Design Community. Interaction Design and Architecture(s), 33, 92 - 112.
Law, N., & Liang, L. (2020). A Multilevel Framework and Method for Learning Analytics Integrated Learning Design. Journal of Learning Analytics, 7(3), 98-117. https://doi.org/10.18608/jla.2020.73.8
Milligan, S. K., Luo, R., Hassim, E., & Johnston, J. (2020). Future-proofing students: What they need to know and how to assess and credential them. Melbourne Graduate School of Education, the University of Melbourne: Melbourne.
Seufert, S., Meier, C., Soellner, M., & Rietsche, R. (2019). A pedagogical perspective on big data and learning analytics: A conceptual model for digital learning support. Technology, Knowledge and Learning, 24(4), 599-619. https://doi.org/10.1007/s10758-019-09399-5