Simulation-based learning, such as the use of standardized patients or skills labs, has long been a cornerstone of medical education. These experiences give students the chance to practice clinical skills in a safe, structured environment. Despite their importance, access to high-quality simulation is often limited. The costs, logistics, and faculty time required to run in-person simulations can restrict both the frequency and availability of these opportunities, leaving many students limited practice time before entering clinical rotations.
In recent years, generative AI has emerged as a promising tool to help bridge this gap. AI-powered patient simulations offer the potential for scalable, flexible, and accessible clinical practice. However, this technology is not without its own challenges, with concerns surrounding the accuracy of medical information and the variability of AI-generated responses.
Pooja Jethani, MD, MS, Sketchy's Senior Content Strategist - AI, discusses some of these challenges and why they matter when developing AI-driven simulations:
“Studies show that, even for the most advanced models, around 30% of responses are not supported by LLMs, meaning they can't consistently back their claims, and this is risky when it comes to dealing with medical tasks or guiding decision making. Another challenge we've seen is the variability of responses. It makes it extremely challenging to standardize the experience across the board.”
At Sketchy, we were mindful of these limitations when developing DDx, our AI-driven, interactive case simulation platform. Our goal was to harness the strengths of generative AI while addressing its pitfalls, creating cases that are as engaging as they are educationally sound.
How does DDx choose which cases to build?
Every DDx case starts with a question: What kind of clinical scenario will best serve our learners right now?
We begin by identifying high-yield, educationally rich scenarios that are appropriate for the target level of training. We choose cases based on three key criteria: clinical relevance (is this a condition students are likely to see on exams or in real life?), educational value (does this case allow learners to apply clinical reasoning, integrate multiple systems, and synthesize information in a meaningful way?), and level appropriateness (will the case challenge learners at their current stage without overwhelming them?).
We strive for a balance between bread-and-butter diagnoses and more complex or nuanced presentations that demand deeper critical thinking.
Who writes DDx cases and how?
Once we've settled on a case concept, it's time to bring it to life. This part of the process is handled by our Subject Matter Experts (SMEs) — experienced physicians who bring deep clinical knowledge and a passion for teaching from a wide range of specialties including Internal Medicine, Neurology, General Surgery, Pediatrics, Emergency Medicine, Family Medicine, and Psychiatry.
Ben Muller, MD, Chief Content Officer at Sketchy, highlights why partnering with experienced medical educators is essential:
“Having a medical educator think about the level of the student and what their goals are, and what they're trying to learn at any given moment is extremely helpful in helping curate that sort of informational underpinning that is the foundation of these patient interactions.”
When writing a case, our SMEs focus on mirroring the real-life thought processes that doctors use every day — from taking a patient history and performing a physical exam through generating a differential, interpreting labs and imaging, and deciding on management. Once the initial draft is created, it is peer-reviewed by another SME. This script then serves as the foundation that allows the AI model to act as a realistic conversational partner while maintaining accuracy and comprehensiveness.
How does DDx wire a case into an AI simulation?
Once the written case is finalized, it moves into the hands of our content team — made up of in-house physicians — who take the clinical content and translate it into an interactive, AI-powered experience. We build the case within our platform and create tailored prompts to simulate realistic interactions between the learner, the patient, and the attending physician. Our cases are designed to support a wide variety of learner inputs. Rather than a script or multiple-choice format, students can type their own thoughts, ask open-ended questions, or suggest diagnoses — and the simulation responds dynamically.
How does DDx ensure case quality and accuracy?
Before any case goes live, it goes through a multi-step quality assurance process including SME review (confirming that the AI interactions make sense, the differential is appropriate, and the management decisions align with current guidelines) and student testing (providing feedback on functionality, challenge level, engagement, and learning outcomes). We take this feedback seriously and often make multiple rounds of revisions based on what we learn.
To ensure that our cases remain accurate, relevant, and aligned with current best practices, we regularly review and update them per the latest clinical guidelines. In addition, each case includes an opportunity for learners to provide feedback at the end of the simulation — an essential part of our iterative process for identifying areas for improvement.
Frequently asked questions
How does DDx ensure clinical accuracy in AI-generated patient simulations?
Every DDx case is developed by subject matter experts, practicing physicians across multiple specialties, and peer-reviewed by a second clinician before any AI interaction is built around it. The expert-authored script forms the foundation that constrains the AI model's behavior, preventing the hallucination and variability that affect open-ended generative AI tools.
What types of medical specialties does DDx cover?
DDx cases span a wide range of specialties including Internal Medicine, Emergency Medicine, Neurology, General Surgery, Pediatrics, Family Medicine, and Psychiatry. Cases are mapped across clinical settings and learner levels, from pre-clinical students just beginning to learn disease presentations to residents and advanced practice learners sharpening diagnostic and management skills.
How are DDx cases calibrated for different levels of medical training?
Cases are tagged by clinical discipline, difficulty level, and phase of training (didactic vs. clinical). Some cases are designed for pre-clinical students encountering disease concepts for the first time; others are built for clinical students or residents who are sharpening diagnostic prioritization and management planning. Faculty can also choose to truncate cases, limiting exposure to only the portions relevant to a student's current stage and expand access as training progresses.
