Simulation-based learning, such as the use of standardized patients or skills labs, has long been a cornerstone of medical education. These experiences give students the chance to practice clinical skills in a safe, structured environment. Despite their importance, access to high-quality simulation is often limited. The costs, logistics, and faculty time required to run in-person simulations can restrict both the frequency and availability of these opportunities, leaving many students limited practice time before entering clinical rotations.
In recent years, generative AI has emerged as a promising tool to help bridge this gap. AI-powered patient simulations offer the potential for scalable, flexible, and accessible clinical practice. However, this technology is not without its own challenges, with concerns surrounding the accuracy of medical information and the variability of AI-generated responses.
Pooja Jethani, MD, MS, Sketchy’s Senior Content Strategist - AI, discusses some of these challenges and why they matter when developing AI-driven simulations:
“Studies show that, even for the most advanced models, around 30% of responses are not supported by LLMs, meaning they can’t consistently back their claims, and this is risky when it comes to dealing with medical tasks or guiding decision making. Another challenge, we've seen is the variability of responses. It makes it extremely challenging to standardize the experience across the board.”
At Sketchy, we were mindful of these limitations when developing DDx, our AI-driven, interactive case simulation platform. Our goal was to harness the strengths of generative AI while addressing its pitfalls, creating cases that are as engaging as they are educationally sound. We designed our case development process to support learners as they build clinical reasoning skills, explore different diagnostic pathways, make mistakes, and ask questions, all without the pressures of real-life patient care.
Here’s a behind-the-scenes look at how we bring our DDx cases to life—from the initial concept to the interactive experiences you see on the platform today.
1. Conception: Choosing the Right Cases for the Right Learners
Every DDx case starts with a question: What kind of clinical scenario will best serve our learners right now?
We begin by identifying high-yield, educationally rich scenarios that are appropriate for the target level of training. That means some cases are geared toward pre-clinical students just beginning to learn about diseases and symptoms, while others are designed for clinical students and residents who are sharpening their diagnostic and management skills.
We choose cases based on a few key criteria:
- Clinical Relevance: Is this a condition that students are likely to see on exams or in real life? Is it common enough to warrant inclusion, or rare but high-stakes?
- Educational Value: Does this case allow learners to apply clinical reasoning, integrate multiple systems, and synthesize information in a meaningful way?
- Level Appropriateness: Will the case challenge learners at their current stage without overwhelming them?
We strive for a balance between bread-and-butter diagnoses and more complex or nuanced presentations that demand deeper critical thinking.
2. Case Writing: Where Expertise Meets Educational Design
Once we’ve settled on a case concept, it’s time to bring it to life. This part of the process is handled by our Subject Matter Experts (SMEs)— experienced physicians who bring deep clinical knowledge and a passion for teaching.
Our SMEs come from a wide range of specialties, including:
- Internal Medicine
- Neurology
- General Surgery
- Pediatrics
- Emergency Medicine
- Family Medicine
- Psychiatry
These physicians don’t just know medicine—they know how to teach medicine. Many of them have worked extensively with medical students, residents, and fellows, and they understand the unique challenges and pressures of medical education.
Ben Muller, MD, Chief Content Officer at Sketchy, highlights why partnering with experienced medical educators is essential for our case development process:
“Having a medical educator think about the level of the student and what their goals are, and what they're trying to learn at any given moment is extremely helpful in helping curate that sort of informational underpinning that is the foundation of these patient interactions.”
When writing a case, our SMEs focus on mirroring the real-life thought processes that doctors use every day. This includes:
- Taking a patient history: What are key questions to ask to better understand a patient’s presentation? What clues are hiding in their symptoms, social history, or family background?
- Performing a physical exam: What findings are most relevant? How do subtle signs guide your differential?
- Generating a differential diagnosis: How do you prioritize possibilities? When should you keep casting a wide net vs. narrowing in on a likely cause?
- Interpreting labs and imaging: How do diagnostic tests confirm or challenge your initial assumptions?
- Deciding on management: What is the appropriate next step in care? What treatments are evidence-based, practical, and safe?
Once the initial draft for a case is created, it is peer-reviewed by another SME. This process ensures the creation of a high-quality patient script that encompasses the key learning points for that particular case. This script serves as the foundation that allows the AI model to act as a realistic conversational partner while maintaining the accuracy and comprehensiveness of the information. The result is a case that feels like a real patient encounter—full of nuance, complexity, and teachable moments.
3. Case Wiring: Bringing the Simulation to Life with AI
Once the written case is finalized, it moves into the hands of our content team, which is made up of in-house physicians, who take the clinical content and translate it into an interactive, AI-powered experience.
We build the case within our platform and create tailored prompts to simulate realistic interactions between the three main characters:
- The Learner (you!) – who asks questions, proposes differentials, and makes decisions
- The Patient – who shares their story, responds to questions, and reveals new details
- The Attending Physician – who guides, challenges, and mentors the learner throughout the case
Our cases are designed to support a wide variety of learner inputs. That means you're not locked into a script or multiple-choice format. Instead, you can type in your own thoughts, ask open-ended questions, or suggest diagnoses—and the simulation responds dynamically.
This dynamic interaction is what sets DDx apart. It pushes learners to think like clinicians in real time, test their assumptions, and engage in active rather than passive learning.
4. Quality Assurance: Testing for Realism, Accuracy, and Educational Value
Before any case goes live on the DDx platform, it goes through a multi-step quality assurance process. Our goal is to ensure that each case is not only accurate and medically sound, but also intuitive, realistic, and enjoyable to work through.
This QA process includes:
SME Review
The original case writer—often alongside a peer reviewer—goes through the entire simulation to confirm that:
- The AI interactions make sense and reflect the intended clinical reasoning flow
- The differential diagnosis is appropriate and evidence-based
- The orders and management decisions align with current guidelines and best practices
- The tone, pacing, and complexity are appropriate for the learner's level
Student Testing
Next, we test the case with real medical students, who provide feedback on:
- Functionality – Does the case flow as intended? Are there any issues obtaining the necessary information?
- Challenge level – Is the case too easy, too hard, or just right?
- Engagement – Does the simulation feel realistic and immersive?
- Learning outcomes – Do learners come away with a better understanding of the disease process and diagnostic approach?
We take this feedback seriously and often make multiple rounds of revisions based on what we learn.
5. Publishing: Launching Cases Into the Real World
Once a case has passed QA, it’s ready for publication on the DDx platform. The finished product is a rich, interactive learning experience that students can explore on their own time, at their own pace. Cases are tagged by the tested organ system and clinical discipline, allowing educators and students to easily discover cases relevant to their curriculum.
6. Regular case review: keeping cases up to date
To ensure that our cases remain accurate, relevant, and aligned with current best practices, we regularly review and update them per the latest clinical guidelines. Our team of SMEs monitors changes in medical standards and incorporates new evidence into existing cases as needed. In addition, each case includes an opportunity for learners to provide feedback at the end of the simulation. This student feedback is an essential part of our iterative process—helping us identify areas for improvement, clarify confusing content, and enhance the overall learning experience. Through these ongoing updates, we strive to keep DDx cases both clinically current and learner-centered.
The Future of Clinical Learning: What’s Next?
We’re constantly building, refining, and expanding our case library to meet the evolving needs of medical learners. We're also exploring new ways to leverage emerging technologies to engage students through innovative case formats and interactive learning experiences.
Curious to learn more? Check out our recent webinar, where we dive deeper into our case creation process and explore how you can use LLMs to build your own interactive clinical cases.
