Prompt, But Verify

An asynchronous, five-module curriculum for medical trainees that applies evidence-based medicine principles to the critical appraisal of AI-generated clinical information.

The Gap

Medical trainees are already using large language models for clinical reasoning, yet no widely available curriculum teaches them to verify what these tools produce.

LLM outputs present with high linguistic confidence regardless of accuracy, can cite fabricated references, and may amplify the historical and algorithmic biases that threaten health equity.

At the same time, patients are increasingly arriving at clinical encounters with AI-generated health information that clinicians have no established framework to evaluate.

Curricular Response

Prompt, But Verify addresses this gap by treating AI appraisal as a clinical reasoning skill rather than a technology literacy topic.

The course gives learners a structured, repeatable process: verify AI-generated claims against primary sources, audit outputs for algorithmic bias, communicate transparently with patients about AI's role, and document the entire process for reproducibility.

The result is a set of habits that integrate into existing EBM practice rather than existing alongside it.

Designed for Learners Across the Medical Education Continuum

The course is designed for deployment as an asynchronous elective or required curricular supplement within Canvas LMS. Each module takes approximately 90–120 minutes and can be completed on any schedule. The course assumes no prior AI expertise from learners or facilitators and is built with Universal Design for Learning principles to support diverse learner populations across UME, GME, and CME contexts.

This course is ideal for:

MS3 / MS4 Clerkship Students
Residents
Medical Educators & Faculty

The course's five modules include vignettes that follow Dr. Maya Patel as she learns about the challenges of integrating AI into clinical practice in an evidence-based way.

Design Principles

Several deliberate design choices distinguish this course from existing AI literacy offerings in medical education, including:

AI Appraisal as an EBM Skill

Rather than teaching AI as a standalone technology topic, the course positions AI output appraisal within existing EBM frameworks. Learners apply PICO formulation, GRADE-style evidence rating, and claim-by-claim source verification to AI-generated content — the same rigor they apply to published literature.

Equity as a Structural Requirement

An entire module addresses algorithmic bias identification and mitigation using the Representation / Design / Deployment audit framework and counterfactual stress-testing. An equity guardrail requires learners to document at least one equity risk and mitigation strategy to pass the capstone.

Patient Communication Integration

A dedicated module addresses an emerging clinical need with limited curricular precedent: responding to patients who present with AI-generated health information. Learners develop communication strategies grounded in shared decision-making and health literacy principles.

Course Module Design

Curriculum Overview

Each module is organized around a Grand Question and three Sub-Questions, anchored by a clinical vignette that advances a single intern's week. The Challenge-Based Learning cycle within each module builds toward the five-phase capstone portfolio.

Capstone Assessment: AI-Verified Clinical Evidence Package

The summative capstone is a five-phase portfolio in which learners construct a complete, provenance-logged clinical evidence package for an authentic clinical question. Each phase aligns to a module's learning objectives and is scored with analytic rubrics that serve double duty for self-assessment and instructor feedback, consistent with a Master Adaptive Learner approach. Total: 100 points across the five phases.

Program Capstone Design

A Thoughtful Approach to Learning about AI

Pedagogical Model

Example of one of our course vignettes:

Course Developer

Daniel A. Novak, PhD
Associate Professor & Director of Student Scholarly Activities
UCR School of Medicine · Department of Social Medicine, Population & Public Health

Dr. Novak is a learning scientist whose research focuses on the design of scalable, equity-centered curricula at the intersection of emerging technology and clinical education. He holds a PhD in Learning Sciences from the University of Washington, and is a 2025–26 Harvard Macy Institute scholar.

Report abuse