Key takeaways
- Clinical reasoning is among the most consequential skills a physician develops. It is also one of the hardest to assess consistently. Faculty at major medical schools routinely read and score dozens of student essays per exam cycle, with limited mechanisms to ensure objectivity across cohorts, campuses, or years.
- Grade inflation in preceptor evaluations is a widely shared problem. DDx by Sketchy's rubric-based assessment framework surfaces competency-level data across history-taking, differential reasoning, and management planning. That granularity is not available through standard preceptor assessments.
- Two medical schools, the University of South Dakota Sanford School of Medicine and the University of Missouri School of Medicine (Mizzou), implemented DDx through the Clinical Reasoning Catalyst Program. USD reached a 94% completion rate across 70 students in their clerkship year. Mizzou reached a 60% voluntary completion rate across approximately 400 learners, before any formal completion requirement was in place.
- Both schools position DDx as a structured complement to faculty-led clinical education, not a replacement for in-person clinical training. The platform reduces faculty assessment burden by delivering consistent, rubric-based data at a scale and frequency that preceptor evaluations and OSCEs cannot match.
- The Clinical Reasoning Catalyst Program is active at 12 medical schools, spanning MD, DO, and GME programs. Scholarship on implementation and outcomes is underway at multiple institutions.
At Mizzou, faculty have been administering written clinical reasoning exams for 20 years. Each exam cycle, individual faculty members read and score up to 128 student essays on clinical reasoning. The process is time-intensive, variable year to year, and difficult to standardize across regional campuses. It is also, by most measures, the state of the art.
"From year to year, from block to block, how are we really ensuring that they are being objectively measured and assessed versus subjectivity?" asked Dr. Colleen Hayden, EdD, MS, Associate Dean for Program Evaluation and Assessment at Mizzou Med, during the DDx by Sketchy webinar, "AI-Enabled Simulation in Medical Education: Early Observations from Implementations."
That question is not unique to Missouri. It is the defining challenge of clinical reasoning education. How do you assess a cognitive process consistently, at scale, and without placing the full burden on faculty time?
Clinical reasoning is hard to assess consistently, and the burden falls on faculty
Clinical reasoning is taught. It is also notoriously difficult to assess with objectivity. Preceptor evaluations are subject to grade inflation. OSCEs are expensive to run and logistically demanding to standardize across sites. Written exams require significant faculty time to score and are difficult to calibrate across cohorts.
Dr. Alan Sazama, MD, inaugural Department Chair for Emergency Medicine at the University of South Dakota Sanford School of Medicine (USD), described the pattern directly: "We struggle mightily, as I'm sure a lot of schools do, with grade inflation with our assessments. It's hard to distinguish from student to student who's excelling and who's struggling -- and what the areas are where they can really focus in."
USD runs a 12-month longitudinal integrated clerkship across seven core disciplines. Students are distributed across rural South Dakota. Faculty availability is limited. Running OSCEs after every quarter, as Dr. Sazama noted he would prefer, is financially and operationally out of reach.
DDx’s rubric-based simulation brings performance gaps to the forefront
Both USD and Mizzou Med joined the DDx by Sketchy Clinical Reasoning Catalyst Program. The grant initiative is currently active at 12 medical schools. It pairs institutions with DDx's competency-based simulation cases, a shared rubric-based clinical reasoning assessment framework, and a peer community of practice spanning MD, DO, and GME programs.
The assessment framework was built collaboratively. DDx convened a faculty group to develop a clinical reasoning rubric grounded in published literature, specifically the ART and IDEA scales, and adapted it for use in virtual simulation environments.
Faculty at both institutions found the DDx platform's data more granular than what preceptor evaluations typically generate. The system surfaces individual student performance across discrete competencies: history-taking, differential reasoning, and management planning. It also captures behavioral data, including time spent per case, hint requests, and question scores, giving educators a second layer of information alongside performance scores.
"We’ve found it to be a really valuable tool,” Dr. Sazama said. “The feedback that the clinical reasoning assessment is spitting out is a lot more valuable and specific to where students are excelling and struggling than our standard faculty preceptor assessments.”
"You can sort by pre-clerkship or clinical clerkship. It really helps to differentiate what the level of your learner is. That's been extremely helpful,” Dr. Hayden said.
Dr. Hayden noted that behavioral signals are already proving useful, even before formal outcomes data is available. “We are really interested and excited to already see these early outputs that we're getting from the data. It's showing us where individual students are struggling,” she said.
Students who engaged less with DDx cases, spending little time and moving quickly from start to finish, tended to score lower on end-of-course written clinical reasoning exams a few weeks later. "It's early," she said, "but it is interesting to look at student behavior and performance."
DDx’s AI-enabled simulation enables consistent access and comparability across distributed clinical programs
For programs with students and preceptors spread across large geographic areas, the consistency problem is structural, not just operational.
USD places students at clinical sites across rural South Dakota. Mizzou is expanding to a regional campus in Springfield and transitioning from a 2x2 curriculum model to an 18-month pre-clerkship structure. Among USD's Catalyst Grant peers, the University of Washington's WAMI consortium serves students across multiple states with a largely volunteer clinical faculty, making standardized assessment a persistent challenge.
For these programs, the value of AI-enabled simulation is not that it replaces in-person clinical education. Both Dr. Sazama and Dr. Hayden were direct on that point. "This is just a part of the curriculum," Dr. Sazama said. "I don't think it would be wise to make it the only part of your curriculum."
"I will say I've been very impressed when I go through the [DDx] cases. It’s much more robust than I anticipated when I started to go through it," he added.
The value is consistency. A student at a rural South Dakota clinical site and a student rotating at a main campus hospital can complete the same case, assessed against the same rubric, with the same feedback structure. That comparability is what LCME accreditation requires and what distributed programs consistently struggle to document.
Student completion rates and engagement data: Early indications at USD and Mizzou
USD enrolled all 70 of its Pillar 2 students, assigning 7 DDx cases per clerkship discipline alongside 7 cases from its previous platform. Campus coordinators track completion through the platform. As of the webinar, USD had reached a 94% completion rate, with 100% expected because completion is a clerkship requirement.
Student engagement above the requirement has also been notable. Dr. Sazama observed that students are completing cases beyond what is assigned. Preceptors have reported that students are bringing DDx cases into clinical conversations during rotations.
At Mizzou Med, DDx cases are assigned across 8 course blocks, totaling 24 to 32 cases in the first two years, plus 5 cases in the internal medicine clerkship. Cases are not yet tied to a formal completion requirement. The voluntary completion rate is 60%. Fourth-year students, post-match with no academic stake, proactively contacted Dr. Hayden to request case assignments to use independently before starting residency.
The DDx by Sketchy case library currently includes approximately 300 cases, organized by training level from pre-clerkship through residency. Faculty can request new cases on specific topics, with a standard turnaround of approximately 2 days.
What USD and Mizzou are measuring next
Both programs are in the early stages of formal scholarship. USD is developing a student survey, currently in IRB review, that will ask about perceived clinical reasoning improvement, relevance to clinical rotations, and satisfaction compared to the previous platform. Dr. Sazama's research goal is to document whether AI-enabled simulation can address faculty sufficiency gaps in rural and distributed programs and build the evidence base needed to expand use across campus sites.
Dr. Hayden's near-term priority is closing the feedback loop with students. Her plan includes a pre-brief to set expectations before students begin cases and a debrief process that shares cohort-level findings with learners.
"'You said it, we did it' really does go a long way," Dr. Hayden noted, making clear that data used to improve curriculum sustains engagement over time.
Both schools are also beginning to address faculty development as the next implementation layer. Students have adapted quickly. Faculty comfort with interpreting platform assessment data is the next step.
A decades-long problem worth revisiting
Written clinical reasoning exams, preceptor evaluations, and OSCEs have been the core of clinical assessment for decades. They work, imperfectly, within the constraints of faculty time, geographic distribution, and institutional resources. The gap between what those methods can tell educators and what educators need to know has always existed. That limitation has been accepted, until now.
What the Catalyst Grant program is beginning to document across 12 schools, three years of training, and hundreds of learners, is that a different kind of data is now available and actionable.
Frequently Asked Questions
What is the DDx by Sketchy Clinical Reasoning Catalyst Program? The Clinical Reasoning Catalyst Program is a grant initiative that provides medical schools with access to DDx by Sketchy at no cost in exchange for participation in implementation research and scholarship. The program is currently active at 12 medical schools, spanning MD, DO, and GME programs. Participating institutions receive competency-based simulation cases, a shared rubric-based assessment framework, and access to a peer community of practice.
Does DDx by Sketchy replace in-person clinical education or OSCEs? No. Both implementing institutions described DDx as a structured complement to faculty-led clinical education. It is designed to reduce faculty assessment burden while providing scalable, consistent, rubric-based practice and assessment that OSCEs and preceptor evaluations cannot deliver at the same frequency or geographic reach. Dr. Sazama noted: "I don't think it would be wise to make it the only part of your curriculum."
How does DDx by Sketchy address grade inflation in clinical assessments? Standard preceptor evaluations are subject to grade inflation and vary significantly by individual faculty. DDx generates rubric-based scores across discrete competencies, including history-taking, differential reasoning, and management planning, using a framework grounded in published literature. That data gives educators a consistent, objective measure that sits alongside, and can contextualize, preceptor evaluations.
How does DDx by Sketchy support programs with students at multiple clinical sites? Students at different locations complete the same cases, assessed against the same rubric, with the same feedback structure. That comparability does not depend on which preceptor a student is assigned to or which campus they are rotating through. For LCME accreditation purposes, it provides documented evidence of consistent assessment across distributed training environments.
What does the DDx by Sketchy case library include? The library currently includes approximately 300 cases, organized by training level from pre-clerkship through residency. Cases cover clinical reasoning, communications, routine care, and clinical skills, including EKG interpretation, chest x-ray reading, and point-of-care ultrasound. Faculty can request new cases on specific topics, with a standard turnaround of approximately 2 days.
Is DDx by Sketchy built on generative AI? How does it prevent hallucinations? DDx uses a large language model to power case interactions, but the AI operates within a tightly defined structure grounded in case data and scoring rubrics. Each case is built by physicians and reviewed by human faculty, creating a guardrail of human review rather than relying on unconstrained generation. The AI is then instructed to act as a standardized patient or standardized attending within the bounds of a predefined case. For example, it knows the patient's history but not the diagnosis, mirroring what a real patient would know.
What student engagement data has been reported from early implementations? At USD, 70 Pillar 2 students reached a 94% completion rate, with 100% expected because completion is a clerkship requirement. Students are also completing cases beyond what is required, and preceptors have noted that students are raising DDx cases in clinical conversations. At Mizzou Med, approximately 400 learners participated with a 60% voluntary completion rate before any formal requirement was in place. Fourth-year students, post-match, independently requested additional case assignments to prepare for residency.
What research is coming out of the Catalyst Program? Multiple Catalyst institutions are developing formal scholarship. USD is currently in IRB review for a student survey measuring perceived clinical reasoning improvement, platform satisfaction, and relevance to clinical rotations. Mizzou Med is analyzing correlations between DDx engagement behavior and summative clinical reasoning exam scores. The broader program goal is to publish findings on implementation strategies and educational outcomes across diverse program types and geographic settings.
Watch the Full Webinar On Demand
Dr. Sazama and Dr. Hayden share additional implementation details, lessons learned, and audience Q&A in the full session recording. Watch the webinar on demand: AI-Enabled Simulation in Medical Education: Early Observations from Implementations.
DDx by Sketchy is a clinical readiness platform built by physicians and medical educators. It develops reasoning, communications, and technical skills through immersive cases, competency-based assessments, AI-powered insights, and personalized feedback. The Clinical Reasoning Catalyst Program is active at 12 medical schools.
