Research Projects
Research Projects
University of Missouri - Columbia
Evaluation of Mizzou Cloud & DevOps Using Learning eXperience Design Methods (January 2023 - January 2024)
“Mizzou Cloud DevOps” (MCD) platform, an online self-service platform for students to learn Cloud DevOps-related concepts, tools, and technologies. The MCD platform offers free and easy access to learning modules with hands-on exercises that leverage different cloud infrastructures i.e., public clouds such as Amazon Web Services (AWS), and Google Cloud Platform (GCP). In addition, the MCD platform features an “application-inspired learning” methodology, which teaches students about popular Cloud DevOps concepts such as e.g., Kubernetes, VMWare Tanzu, KubeEdge, Jenkins, Ansible, Kubeflow, Prometheus, Grafana based on the integration of real-world application use cases (e.g., cybersecurity, healthcare, video analytics). It also features a flexible and expandable catalog of eight learning modules and is scalable for training any number of students. A workshop series for instructors and students from various universities in the United States has been conducted. In addition, a usability study with students with non-computer science background, and a series of focus groups with MCD platform users has been conducted to evaluate the learner experience design of the platform.
Role: I had been directly involved as a UX researcher in the user-centered design consultation and evaluation of the MCD platform, including design recommendations, building research protocol, and conducting expert review, heuristic evaluation, usage study, and user satisfaction surveys.
Publications:
Li, S., Glaser, N., Singh, K., Gu, Y., Neupane, R., Prasad, C., Songjie, W., Neupane, K., Pandey, A., Xiyao, C., Gafurov, D., & Yeddulapalli, H.S. (2023). Evaluating a Cloud and DevOps Online Learning Platform using Learner Experience Design Evaluation Methods. Presented at the 2023 Association of Educational and Communications Technology, Orlando, FL.
Neupane, R. L., Calyam, P., Wang, S., Neupane, K., Pandey, A., Cheng, X., Gafurov, D., Yeddulapalli, H. S., Glaser, N., Singh, K., Li, S., Gu, Y., & Sharan, S. (2024). Online Self-Service Learning Platform for Application-Inspired Cloud Development and Operations (DevOps) Curriculum. IEEE Transactions on Learning Technologies. DOI: 10.1109/TLT.2024.3428842
Active learning (AL) is a range of instructional design strategies to enhance teaching and learning experiences. On this continuum, the highest level is active student-centered learning (versus active but still teacher-centered) (Chi, 2009). To enhance faculty’s instructional opportunities (instructors, professors) and apply AL strategies, redesigning physical spaces may be a key factor (Krathwohl, 2002). Flexible spaces may affect their abilities, ideas, and course redesigns toward AL and higher-order thinking skills (Krathwohl, 2002). The purpose of this research was to study how instructors use the newly developed redesigned technology-enabled spaces in colleges, and how their adoption of the new spaces leads to AL pedagogies and influences the learning experience of students.
Role: I was directly involved in this project as a project leader. This study was conducted in the Spring of 2022 at the Business School at MU by the learning experience (LX) researchers. I was directly responsible for project management, participant recruitment, study design, and data collection and analysis.
Publications:
Singh, K., Glaser, N., Alzoubi, D., Schmidt, M., Li, S., & Gu, Y. (Accepted, 2023). Optimizing Learning Experiences: Integrating Learning Experience Design Principles with Technology-Enabled Active Learning. Transdisciplinary Learning Experience Design: Futures, Synergies, and Innovation. Springer
Li, S., Alzoubi, D., Glaser, N., Mendoza, K. R., Schmidt, M., & Singh, K. Formative Design and Active Learning Spaces: Enhancing Engagement through Learning Experience Design Principles and Technology Integration. (Submitted to Journal of Formative Design in Learning)
This campus cyberinfrastructure research project included two cases of data-intensive computational applications, i.e., bioinformatics and health informatics. The design and User eXperience (UX) evaluations of the applications were conducted at the Information Experience Lab from January 2019 to May 2021. The goal of this research project was to analyze the issues with the existing process of accessing scientific data. Followed by, iteratively designing, developing, and evaluating the user experience of technologies.
Role: I was assigned as a student project leader for both cases. I worked as a UX researcher with cross-disciplinary teams, including computer scientists, domain scientists, biology researchers, and healthcare practitioners. In addition, my role as a project leader included research design, investigation, methodology, and project management. In this project, I got an opportunity to learn multiple human-centered design methods and research concepts for designing and developing robust, user-friendly systems. To evaluate the user experience of the simulated models of the systems, I used multiple methods, such as semi-structured interviews, surveys, task-based usability tests, think-aloud, and observations which involved qualitative and quantitative analysis. In addition, I used Participatory Action Research (PAR) approach to understand the entanglement of cross-disciplinary UX research collaborations in the field of cyberinfrastructure development.
KBCommons is a cloud-based prescriptive system that is part of larger cyberinfrastructure. The goal of the system is to provide bioinformatics users with an easy-to-use web-based platform integrated with ready-made data analysis workflows and an ability to predict optimum computation solutions for users based on their preferences of performance, agility, and cost. I have investigated bioinformatics users' current usage, perceptions, and preferences around using scientific data computational technologies. Informed by this work, in my current work, I designed, built, and evaluated domain scientists personalized data-intensive computational technologies to support research and learning.
The honest broker-based healthcare data access system is part of a larger campus cyberinfrastructure. One of the aims of the honest broker system was to decrease the amount of effort required of data providers and users to access healthcare data in a secure way while still ensuring compliance with healthcare data security standards. I have investigated healthcare researchers' and practitioners' current usage, perceptions, and preferences around using scientific data computational technologies. Informed by this work, in my current work, I designed, built, and evaluated domain scientists personalized data-intensive computational technologies to support research and learning.
Publications:
Singh, K. (2024). A Systematic Literature Review of The Application of User Experience Studies in Cyberinfrastructure for Scientific Research. Quality and User Experience. Quality and User Experience. [DOI coming soon].
Singh, K., Jahnke, I., & Calyam, P. ( 2024). Entangled Collaborations: Tensions in Cross-Disciplinary User Experience Studies in Cyberinfrastructure Projects. Behaviour and Information Technology. DOI: 10.1080/0144929X.2024.2315325
Singh, K., Li, S., Jahnke, I., Alarcon, M., Mosa, A., & Calyam, P. (2023). Improving Big Data Governance in Healthcare Institutions: User Experience Research for an Honest Broker based Application to Access Healthcare Big Data. Behaviour & Information Technology. DOI: 10.1080/0144929X.2023.2196596
Singh, K., Jahnke, I., Mosa, A., & Calyam, P. (2021). The Winding Road of Requesting Healthcare Data for Analytics Purposes – Using the ‘One Interview Mental Model’ Method for Improving Services of Health Data Governance and Big Data Request Processes. Journal of Business Analytics, 1-18. DOI: 10.1080/2573234X.2021.1992305
Singh, K., Li, S., Jahnke, I., Pandey, A., Lyu, Z., Joshi, T., & Calyam, P. (2020, December). A Formative Usability Study to Improve Prescriptive Systems for Bioinformatics Big Data. In 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Seoul, South Korea, (pp. 735- 742). IEEE. DOI: 10.1109/BIBM49941.2020.9313097
The objective of the CAFNR project was to develop and evaluate the user and learning experience of online learning modules, created as a part of an online course for the U.S. risk-based inspection process for food imports. The overall learning goal of the online course was to facilitate an understanding of the logic and implementation of the U.S. Risk-based Inspection process for Food Imports (RIFI) and identify priorities for improving the inspection process of food imports for international officials, China, and South Africa. An iterative design, development, and evaluation method was adopted for this project. The project was completed and delivered in May 2021.
Role: I worked with the UX research team to design and develop the online mobile microlearning modules of approximately 60 minutes each to educate Chinese officials about the U.S. risk-based inspection of food imports. We collaborated with the content experts at CAFNR to design the online learning modules by conducting a literature review for developing interactive learning. We conducted requirement gathering and analysis through semistructured interviews with CAFNR officials, followed by designing and developing two online modules for interactive online learning using interaction design techniques. First, we conducted a heuristics evaluation of the online course interface by utilizing sociotechnical-pedagogical heuristics. Next, we evaluated the user experience of the modules with 10 participants, including learning effectiveness, efficiency, and appeal using system-oriented methods, such as task-based usability tests, user satisfaction surveys, and questionnaires, semi-structured interviews, and learning evaluation surveys. For the learner experience, we measured the learning growth of a sample of 46 participants through pre- and post-test surveys. As a part of the CAFNR project, I have co-authored a journal paper published in Food Control journal and a book chapter for AECT Springer which is under review.
Screenshot of Learning Activity 1 in Module 1.1
Screenshot of a True/False Activity in Module 1.2
Publications:
Li, S., Singh, K., Yu, F., Riedel, N., & Jahnke, I. (2022). Digital Learning Experience Design and Research of a Self-Paced Online Course for Risk-Based Inspection of Food Imports. Food Control. DOI: 10.1016/j.foodcont.2021.108698
Jahnke, I., Li, S., Singh, K., Yu, F., & Riedel, N. (2023). Combining User Experience and Learning Efficacy in Design and Redesign. In: Spector, M.J., Lockee, B.B., Childress, M.D. (eds) Learning, Design, and Technology. Springer, Cham. DOI: 10.1007/978-3-319-17727-4_179-1
The objective was to conduct a user experience and usability evaluation for four selected courses at the Univeristy of Missouri Extension. These courses were a mix of online and blended classrooms. A longitudinal study to evaluate the user experience was conducted using expert reviews, a user experience study with external users, and in-depth semistructured interviews of course participants (three phases of interviews per course). The project was conducted from October 2018 to May 2019 at the IE Lab. I joined the project as a co-investigator in January 2019.
Role: I worked in a team to plan and design data collection methods to evaluate two out of four courses. We evaluated the user experience of the courses by collecting data using multiple methods, such as semi-structured interviews, think-aloud protocol, surveys, questionnaires, and performing quantitative and qualitative analysis of the collected data. We wrote a technical report on the sociotechnical pedagogical usability of the courses and provided design improvements as suggested by the participants. As a part of the project, I have co-authored two journal papers and a book chapter published in 2020-2021.
Publications:
Jahnke, I., Riedel, N., Singh, K., & Moore, J. (2021). Advancing Sociotechnical Pedagogical Heuristics for the Evaluation of Online Courses. Online Learning, 25(4), 337-360. DOI: 10.24059/olj.v25i4.2439
Pham, M., Singh, K., & Jahnke, I (2021). Socio-Technical-Pedagogical Usability of Online Courses for Older Adult Learners. Interactive Learning Environments, 1-17. DOI: 10.1080/10494820.2021.1912784
Jahnke, I, Schmidt, M., Pham, M., & Singh, K. (2020). Sociotechnical-Pedagogical Usability for Evaluating Learner Experience in Technology-Enhanced Learning Environments. In Schmidt, M., Tawfik, A., Jahnke, I., & Earnshaw, Y. (Eds.), Learner and user experience research: An introduction for the field of learning design & technology. EdTech Books. [Silver Book Award recognizing authors who have created works that have been accessed at least 10,000 times]
The assisted living center at the University of Missouri uses sensors to detect the biometrics of its residents. These sensors sent notifications to the clinical professionals to alert them of any different behaviors of the residents. The objective of this research project was to explore the user behavior of clinical professionals (e.g., nurses) with their work practices associated with the sensor alert systems and understand the potential issues of adoption of the alert system. The project was delivered in May 2020.
Role: As a team, we conducted a Sociotechnical Walkthrough (STWT) to explore and analyze the gaps between creators' intention and the users’ implementation (mainly nursing staff and social workers) of the sensor alert system in the assisted living center for the elderly. We used qualitative methods, such as semi-structured interviews and focus groups to collect data. Modeling instruments (SeeMe modeling) were applied for data analysis to examine the different clinical workflows versus the expectations of the system development team. Our results showed different mental models of the socio-technical practice which led to a mismatch of nurse care coordination, social practices, and technology use.
Publication:
Jahnke, I., Riedel, N., Popescu, M., Skubic, M., & Rantz, M. (2021). Social practices of nurse care coordination using sensor technologies–Challenges with an alert system adoption in assisted living communities for older adults. International Journal of Nursing Sciences, 8(3), 289-297.
This research project was a qualitative study funded by the Donald W. Reynolds Journalism Institute (RJI) at the University of Missouri-Columbia to conduct in-depth interviews with traditional and digital news readers. The objective of the study was to study readers' motivation and behavior with newspaper reading of digital and print newspaper readers. This study was performed as a groundwork for future surveys and effects studies by the RJI. In phase 1, IE Lab conducted 12 in-depth face-to-face interviews for news consumption by print news readers using a semi-structured interview protocol and participant observation of reading a traditional newspaper.
Role: In phase 2, I was directly involved through the IE Lab to conduct 12 in-depth face-to-face interviews for news consumption by digital news readers. I also observed the participants for their news reading behavior using a digital device which participants typically use for reading news, e.g., smartphone, laptop, tablet.
Publication:
Zhou, S., Kiesow, D., & Guo, L. (2023) The Values of Print: Affordances and Sensemaking for Newspaper Consumers. Journalism Practice, 17(2), 336-353, DOI: 10.1080/17512786.2021.1910984 (User Experience Study conducted by the Information Experience Lab under the supervision of Dr. Isa Jahnke).
This research project was conducted in collaboration with the Reynolds Journalism Institute (RJI) at the University of Missouri-Columbia to create an ongoing spreadsheet to monitor and analyze growth and interaction of subscribers to a news text service initiated by the client to help Latino immigrants. The client used a SMS platform to provide a 2-way local news and information service as structured inquiries (text messages), where the participants could receive a text and could text back.
Role: I worked in a team to analyze user engagement with approximately 400 users using quantitative methodologies. We collected subscriber interaction and engagement data through spreadsheet. Our data analysis focused on how those numbers were impacted by factors, such as issue and content or question type of the news.