Background
Background
Embedded Files
Artificial intelligence (AI) and machine learning (ML) are increasingly being integrated into healthcare research and systems for applications such as clinical decision support, predictive analytics, workflow optimization, and more.1 Nursing organizations and educators continue to emphasize the importance of preparing future nurse scientists to engage with data science and AI-related concepts, as these technologies become more integrated into healthcare and research environments.2, 3 While recent nursing literature suggests that AI/ML methods are increasingly being applied across a wide range of nursing-relevant problems4, highlighting the growing relevance of AI literacy within nursing research training, integration of AI and data science content within doctoral nursing curricula remains variable.5,6 Recent recommendations emphasize the need for foundational AI literacy among nurses and nurse scientists, including an understanding of analytic workflows, model interpretation, and sources of algorithmic bias.5 In response, this teaching-as-research (TAR) project evaluates a brief educational micro-module designed to help first-year PhD nursing students connect foundational statistical methods with emerging AI/ML applications in nursing research. The intervention was implemented within an existing doctoral-level statistics course and emphasized critical evaluation of bias and exploration of how familiar statistical approaches may extend into broader AI/ML applications in nursing and health research.
TAR Question
TAR Question
Does embedding a brief AI micro-module in an advanced statistics course increase doctoral nursing students' perceived understanding of and self-efficacy with the connections between advanced statistical methods and contemporary AI and ML approaches in health research?
Hypothesis: Doctoral nursing students who complete the micro-module will demonstrate higher perceived understanding of and self-efficacy with AI concepts in nursing research.
Hypothesis: Doctoral nursing students who complete the micro-module will demonstrate higher perceived understanding of and self-efficacy with AI concepts in nursing research.
Methods
Methods
This micro-module was embedded in NURS631 Advanced Statistics for the Health Sciences, a required course in the PhD program at the University of Arizona College of Nursing. This course builds on prior graduate-level statistics training and supports doctoral students in developing the analytic skills needed to conduct quantitative nursing research. Course content includes descriptive and inferential statistics, linear and logistic regression, exploratory factor analysis, and structural equation modeling, with applied statistical work conducted in STATA. The course was taught by Drs. Graciela Silva and Chiyoung Lee. The micro-module was designed to extend the course's existing statistical content by introducing connections between traditional statistical methods and AI/ML applications in nursing research and practice.
Design and Data Collection: The micr0-module was delivered asynchronously and used a pre/post evaluation design to assess perceived understanding and self-efficacy related to AI literacy. Students completed a 10-item Likert-type survey and five open-ended questions immediately before and after the micro-module. Likert items used a five-point novice-to-expert scale to assess perceived confidence connecting traditional statistical methods with AI workflows, translate nursing research questions into expanded analytic strategies, recognize sources of bias, evaluate the added value of AI-based approaches, critically interpret ML-driven findings, and communicate AI/ML concepts in interdisciplinary research settings. Open-ended questions were more reflective and explored how students conceptualized the relationship between statistics, data science, and AI; how they envisioned extending methods relevant to their own research using AI/ML approaches; their understanding of the analytic workflow from raw data to clinical application; potential risks and limitations of AI-driven health research; and areas of confidence or uncertainty related to applying data science concepts in their own work.
Data Analysis: Due to small cohort size, analyses were done with attention to preserving the interpretative context of responses while minimizing deductive identifiability. Quantitative survey data were summarized descriptively, with item-level pre/post means and overall pre/post change in mean score reported. Open-ended responses were reviewed using a structured, deductive approach organized around the module's core content areas. Analytic notes documenting interpretive decisions throughout the narrative synthesis were maintained. Analysis focused on whether post-module responses reflected greater specificity in AI/ML terminology, evidence of analytic pipeline reasoning, and recognition of potential biases. Discussion board responses were assessed similarly; however, topic-specific details are not reported to protect participant confidentiality. All statistical analyses and visualizations were done in STATA/MP 19.0 and R, respectively.
This project was reviewed and approved by the University of Arizona IRB as exempt, no-greater-than-minimal-risk research under IRB submission ID STUDY00007671. All participants provided consent prior to participation.
Design and Data Collection: The micr0-module was delivered asynchronously and used a pre/post evaluation design to assess perceived understanding and self-efficacy related to AI literacy. Students completed a 10-item Likert-type survey and five open-ended questions immediately before and after the micro-module. Likert items used a five-point novice-to-expert scale to assess perceived confidence connecting traditional statistical methods with AI workflows, translate nursing research questions into expanded analytic strategies, recognize sources of bias, evaluate the added value of AI-based approaches, critically interpret ML-driven findings, and communicate AI/ML concepts in interdisciplinary research settings. Open-ended questions were more reflective and explored how students conceptualized the relationship between statistics, data science, and AI; how they envisioned extending methods relevant to their own research using AI/ML approaches; their understanding of the analytic workflow from raw data to clinical application; potential risks and limitations of AI-driven health research; and areas of confidence or uncertainty related to applying data science concepts in their own work.
Data Analysis: Due to small cohort size, analyses were done with attention to preserving the interpretative context of responses while minimizing deductive identifiability. Quantitative survey data were summarized descriptively, with item-level pre/post means and overall pre/post change in mean score reported. Open-ended responses were reviewed using a structured, deductive approach organized around the module's core content areas. Analytic notes documenting interpretive decisions throughout the narrative synthesis were maintained. Analysis focused on whether post-module responses reflected greater specificity in AI/ML terminology, evidence of analytic pipeline reasoning, and recognition of potential biases. Discussion board responses were assessed similarly; however, topic-specific details are not reported to protect participant confidentiality. All statistical analyses and visualizations were done in STATA/MP 19.0 and R, respectively.
This project was reviewed and approved by the University of Arizona IRB as exempt, no-greater-than-minimal-risk research under IRB submission ID STUDY00007671. All participants provided consent prior to participation.
Micro-Module Components
Micro-Module Components
First, students reviewed a peer-reviewed article4 on decoding ML in nursing research and a one-page handout mapping statistical approaches covered in the course (e.g., regression, exploratory factor analysis, multilevel modeling, survival analysis) to potential AI/ML extensions and nursing applications. Second, students listened to a pre-recorded 15-minute lecture introducing a six-stage analytic pipeline spanning raw data collection, data cleaning and pre-processing, statistical modeling, AI/ML methods, interpretation (e.g., performance metrics), and clinical application. Emphasis was placed on structural and algorithmic bias, including how bias may emerge and propagate across stages of the analytic process. Third, students participated in a structured discussion board activity in which they applied the module’s concepts to their own research interests by identifying a statistical method relevant to their work, proposing a potential AI/ML extension, and reflecting on related ethical and bias considerations.
Results
Results
Sample: Participants were four first-year doctoral nursing students (3 female, 1 male) ranging in age from 35 to 47 years. All held advanced practice degrees prior to enrollment into the program (3 Doctor of Nursing Practice, 1 Master's in Nursing Education) and reported extensive clinical experience (range 13-15 years). Self-rated familiarity with technology ranged from 2 to 4 on a 5-point scale, suggesting moderate baseline comfort with technology across the cohort.
Quantitative Results: Pre-module self-efficacy was 2.38 (range: 1.0-3.4) on a 1-5 scale, reflecting an Advanced Beginning level across the cohort. Post-module, the mean self-efficacy score increased to 3.13 (range: 2.0-4.4), reaching the Competent threshold (Figure 1). Three of four participants showed composite gains of 10 or more points, with one one participant demonstrating the largest relative change moving from the Novice to Advanced Beginner level. One participant showed a marginal decline (mean Δ= -0.10), though their completed the post-module survey much later than the others following additional prompts. This could introduce temporal confounding that limits direct comparison. A preliminary matched-pairs effect size of d=1.32 was observed. This estimate should be interpreted cautiously given the pilot nature of the study (N=4). At the item level, improvement across all 10 competency domains were noted, with the largest improvements in evaluating AI's added value beyond standard statistical methods (mean Δ= +1.25), recognizing sources of bias (mean Δ= +1.00), and critically interpreting AI/ML-driven findings (mean Δ= +1.00)
All findings are descriptive and preliminary/precludes inferential conclusions.
Qualitative Results: Overall, results suggest that analytic pipeline reasoning became more explicit, and bias and data quality concerns became more specific after the module. Before the module, open-ended responses varied in how clearly students described movement from data collection to analysis to clinical application. Post-module responses more consistently described multi-step workflows that included research question development, data preparation, statistical modeling, and AI/ML extension, interpretation, and application. Discussion boarded responses also showed greater evidence of applied understanding. Although not forms of potential bias were not consistently identified across participants, responses suggest nascent awareness of how different forms of bias may arise at multiple stages in the research process.
Quantitative Results: Pre-module self-efficacy was 2.38 (range: 1.0-3.4) on a 1-5 scale, reflecting an Advanced Beginning level across the cohort. Post-module, the mean self-efficacy score increased to 3.13 (range: 2.0-4.4), reaching the Competent threshold (Figure 1). Three of four participants showed composite gains of 10 or more points, with one one participant demonstrating the largest relative change moving from the Novice to Advanced Beginner level. One participant showed a marginal decline (mean Δ= -0.10), though their completed the post-module survey much later than the others following additional prompts. This could introduce temporal confounding that limits direct comparison. A preliminary matched-pairs effect size of d=1.32 was observed. This estimate should be interpreted cautiously given the pilot nature of the study (N=4). At the item level, improvement across all 10 competency domains were noted, with the largest improvements in evaluating AI's added value beyond standard statistical methods (mean Δ= +1.25), recognizing sources of bias (mean Δ= +1.00), and critically interpreting AI/ML-driven findings (mean Δ= +1.00)
All findings are descriptive and preliminary/precludes inferential conclusions.
Qualitative Results: Overall, results suggest that analytic pipeline reasoning became more explicit, and bias and data quality concerns became more specific after the module. Before the module, open-ended responses varied in how clearly students described movement from data collection to analysis to clinical application. Post-module responses more consistently described multi-step workflows that included research question development, data preparation, statistical modeling, and AI/ML extension, interpretation, and application. Discussion boarded responses also showed greater evidence of applied understanding. Although not forms of potential bias were not consistently identified across participants, responses suggest nascent awareness of how different forms of bias may arise at multiple stages in the research process.
Quantitative Results:This figure shows the mean item-level change in self-efficacy scores (post - pre) across 10 competency domains. All items showed positive mean change, ranging from +0.50 to +1.25. The largest gain was observed for evaluating AI's added value beyond standard statistical approaches (Q5: mean Δ = +1.25), followed by recognizing sources of analytic bias (Q6: mean Δ= +1.00) and critically interpreting AI/ML-driven findings (Q9: mean Δ= +1.00) . Q6 is notable in that all four participants improved by exactly one scale point (SD=0). Items related to analytic pipeline description (Q4), interpreting AI/ML study claims (Q9), and interdisciplinary communication (Q10) also showed moderate mean gains (mean Δ= +0.75). The smallest gains (mean Δ= +0.50 each) were observed for items addressing AI use cases (Q3), analytic strategy translation (Q2), data quality assessment (Q7), and generative AI in reasoning (Q8).
Abbreviations: AI, artificial intelligence; DS, data science; ML, machine learning; RQ, research question
Qualitative Results:This figure shows words clouds from pre- and post-module open-ended responses. Common words (e.g., feel, think, things) and topic-specific terms were excluded prior to analysis. Terms appearing four or more times across responses were retained. Pre-module responses generally described AI/ML in broad terms, largely emphasizing pattern recognition. Several responses reflected limited engagement, including simple "not sure" responses. All post-module responses were more substantive and framed AI/ML as connected to, and dependent on, statistical reasoning, data quality, and appropriate interpretation. For example, one post-module response described AI/ML as relying on "sound statistical principles and high-quality data." Post-module responses were more consistently described multi-step workflows, suggesting more analytic pipeline reasoning. Bias concerns were also most specific in post-module responses. One response noted that AI/ML may "amplify existing biases" when data are incomplete; another emphasized that findings may "reinforce existing inequities rather than improve care."
Qualitative responses also offered insight beyond quantitative findings. In one case, qualitative responses reflected a shift from values-based concerns about AI's limitations ("I'd be worried about the risk of oversimplifying complex human experience") to more technically grounded risk language ("AI can amplify existing biases if the underlying data re incomplete or unrepresentative of the population being studied"), despite a slight decreased in self-rated confidence/self-efficacy.
Discussion board responses also showed evidence of applied understanding. All participants described multi-step analytic workflows that aligned with the module's pipeline framework. Participants identified appropriate statistical techniques relevant to their developing research question and described plausible AI/ML extensions, including complex approaches like natural language processing. Due to the size of the cohort and the potential for deductive identification, responses cannot be shared.
Qualitative responses also offered insight beyond quantitative findings. In one case, qualitative responses reflected a shift from values-based concerns about AI's limitations ("I'd be worried about the risk of oversimplifying complex human experience") to more technically grounded risk language ("AI can amplify existing biases if the underlying data re incomplete or unrepresentative of the population being studied"), despite a slight decreased in self-rated confidence/self-efficacy.
Discussion board responses also showed evidence of applied understanding. All participants described multi-step analytic workflows that aligned with the module's pipeline framework. Participants identified appropriate statistical techniques relevant to their developing research question and described plausible AI/ML extensions, including complex approaches like natural language processing. Due to the size of the cohort and the potential for deductive identification, responses cannot be shared.
Discussion:An interesting finding that emerged was that perceived confidence and conceptual development may not always progress in parallel when learners are introduced to complex methodological content. While this micro-module appeared to support emerging AI literacy reflected in more specific qualitative responses, self-efficacy gains were variable. This distinction is important for formative curriculum development because self-efficacy scores alone may not capture early shits in metacognitive awareness or methodological reasoning.7 Health professions education literature similarly cautions that self-reported confidence should be interpreted as an adjunct to evidence of learning or competence.8 Thus, a strength of this project was its coupling of survey and qualitative findings, which showed how students' conceptual reasoning may have developed even when perceived confidence/self-efficacy did not increase uniformly.
Limitations: This project was conducted in a single course with a small cohort, and results are not generalizable. Because of the size of the cohort, results had to be summarized in aggregate, minimizing identifying details. This limits the extent to which individual learning trajectories could be fully described. Furthermore, the evaluation relied on self-reported self-efficacy, and the open-ended responses were collected before and after the module; thus, findings do not capture objective competency or sustained learning.
Limitations: This project was conducted in a single course with a small cohort, and results are not generalizable. Because of the size of the cohort, results had to be summarized in aggregate, minimizing identifying details. This limits the extent to which individual learning trajectories could be fully described. Furthermore, the evaluation relied on self-reported self-efficacy, and the open-ended responses were collected before and after the module; thus, findings do not capture objective competency or sustained learning.
Reflection
Reflection
What worked well? What challenges did you have? What surprised you?Despite a prolonged IRB approval process, the actual project was completed quickly. The micro-module was designed to take less than 2 hours, which was helpful for this cohort (full-time students with full-time jobs). Placing the micro-module at the end of the semester (after all coursework was presented) was unhelpful to participants, as they were busy finishing up the semester. Moving forward, the micro-module can be re-designed to be introduced in the middle of the semester. Thankfully, the study still received a 100% participation rate, with only one participant needing an additional prompt to complete the post-module survey. In regard to the data, I was surprised to see how helpful it was to integrate quantitative and qualitative data. I am a quantitative researcher, and this was my first experience working with qualitative data. It was helpful to see how reflective responses can offer greater insight to students' learning and capture shifts that strict pre/post surveys can potentially misrepresent.
How has this project informed how you will teach in the future? What have you learned as an educator?This project allowed me to go to the literature and learn about how students' perceived confidence does not always equate to competence. In health professions education, students may feel confident after a learning activity, but their performance may not demonstrate competence in a particular skill. Measuring students' learning is complex. Self-efficacy data are useful, but they are insufficient on their own as proof of learning. We need to be careful in how we determine if an educational innovation is truly effective. As a future educator, I will incorporate objective and subjective measures to assess learning and competence, and I will be cautious in my interpretation of findings.
What do you want to continue learning? What excites you?I would like to continue learning about how learning unfolds over time. As in this project, an increase in perceived confidence may not be an initial sign of growth. Instead, the deeper awareness of the topic may initially translate to decreased confidence. However, the awareness itself is a valuable marker of growth that is critical for developing competence. I also excited to see how this micro-module can be expanded and possibly incorporated into future doctoral nursing curricula.
Photo from Unsplash (contributed by Beth Stevenson)
How has this project informed how you will teach in the future? What have you learned as an educator?This project allowed me to go to the literature and learn about how students' perceived confidence does not always equate to competence. In health professions education, students may feel confident after a learning activity, but their performance may not demonstrate competence in a particular skill. Measuring students' learning is complex. Self-efficacy data are useful, but they are insufficient on their own as proof of learning. We need to be careful in how we determine if an educational innovation is truly effective. As a future educator, I will incorporate objective and subjective measures to assess learning and competence, and I will be cautious in my interpretation of findings.
What do you want to continue learning? What excites you?I would like to continue learning about how learning unfolds over time. As in this project, an increase in perceived confidence may not be an initial sign of growth. Instead, the deeper awareness of the topic may initially translate to decreased confidence. However, the awareness itself is a valuable marker of growth that is critical for developing competence. I also excited to see how this micro-module can be expanded and possibly incorporated into future doctoral nursing curricula.
Photo from Unsplash (contributed by Beth Stevenson)
Stefanie L. Boyles, BSN, BA, RN
Stefanie L. Boyles, BSN, BA, RN
Stefanie Boyles is a PhD Candidate at the University of Arizona College of Nursing under the mentorship of Dr. Elise Erickson, PhD, CNM, FACNM. Her research focuses on understanding pregnancy as a dynamic process of physiological adaptation rather than a static state of health or disease. She integrates continuous physiologic data from wearable devices with other bio/social measures to characterize how maternal systems adapt across gestation. A central goal of her work is to identify early physiologic and contextual signatures of suboptimal adaptation, particularly those that precede hypertensive disorders of pregnancy. Her work combines clinical experience with data driven analyses and sits at the intersection of maternal health, precision science, and complex systems. She aims to develop a program of research that develops more personalized, mechanism-informed approaches to prediction, prevention, and care.
She is a Graduate Research Associate for an NIH-funded study (5R01HD111125) on oxytocin sensitivity and postpartum hemorrhage: testing genetic and epigenetic biomarkers for improving maternal morbidity (PARTO: Predicting Adequate Response to Oxytocin Study). Her dissertation is under the BioBAYB2 Study (Biological Rhythms Before and After Your Birth II), which is actively enrolling participants. She is a Jonas Philanthropies Scholar and QuidelOrtho Global Health Scholar. You can connect with her on LinkedIn.
She is a Graduate Research Associate for an NIH-funded study (5R01HD111125) on oxytocin sensitivity and postpartum hemorrhage: testing genetic and epigenetic biomarkers for improving maternal morbidity (PARTO: Predicting Adequate Response to Oxytocin Study). Her dissertation is under the BioBAYB2 Study (Biological Rhythms Before and After Your Birth II), which is actively enrolling participants. She is a Jonas Philanthropies Scholar and QuidelOrtho Global Health Scholar. You can connect with her on LinkedIn.
Acknowledgements
Thank you to Drs. Galyen and Winet for their continual encouragement and support. As a remote doctoral student, I appreciate the intention and effort to create an option that allowed me to be included. Also many thanks to Drs. Silva and Lee for allowing me to conduct this project in their course, and to the participants for generously sharing their time and thoughtful reflections. Acknowledgements
AI Use Disclosure: ChatGPT was used to generate visual elements for this website, including the header image, project timeline graphic, and sample characteristics graphic. The author provided all data used to create these visuals and reviewed each image to ensure accuracy and alignment with the project.
References
References
Clancy, T. R. (2020). Artificial intelligence and nursing: The future is now. JONA: The Journal of Nursing Administration, 50(3), 125–127. https://doi.org/10.1097/NNA.0000000000000855
Shea, K. D., Brewer, B. B., Carrington, J. M., Davis, M., Gephart, S., & Rosenfeld, A. (2019). A model to evaluate data science in nursing doctoral curricula. Nursing Outlook, 67(1), 39–48. https://doi.org/10.1016/j.outlook.2018.10.007
Buchanan, C., Howitt, M. L., Wilson, R., Booth, R. G., Risling, T., & Bamford, M. (2020). Predicted influences of artificial intelligence on the domains of nursing: Scoping review. JMIR Nursing, 3(1), e23939. https://doi.org/10.2196/23939
Choi, J., Lee, H., & Kim-Godwin, Y. (2025). Decoding machine learning in nursing research: A scoping review of effective algorithms. Journal of Nursing Scholarship, 57(1), 119–129. https://doi.org/10.1111/jnu.13026
Hoelscher, S. H., & Pugh, A. (2025). N.U.R.S.E.S. embracing artificial intelligence: A guide to artificial intelligence literacy for the nursing profession. Nursing Outlook, 73(4), Article 102466. https://doi.org/10.1016/j.outlook.2025.102466
Schneidereith, T. A., & Thibault, J. (2023). The basics of artificial intelligence in nursing: Fundamentals and recommendations for educators. Journal of Nursing Education, 62(12), 716–720. https://doi.org/10.3928/01484834-20231006-03
Stanton, J. D., Sebesta, A. J., & Dunlosky, J. (2021). Fostering metacognition to support student learning and performance. CBE-Life Sciences Education, 20(2), Article fe3. https://doi.org/10.1187/cbe.20-12-0289
Gabbard, T., & Romanelli, F. (2021). The accuracy of health professions students’ self-assessments compared to objective measures of competence. American Journal of Pharmaceutical Education, 85(4), Article 8405. https://doi.org/10.5688/ajpe8405
Page updated
Report abuse