Why AI Mental Health Chatbots Fail When It Matters Most: The Hidden Vulnerabilities Stress-Testin... Chud The Builder [0xC6NxJAgRb]

More and more therapists are hearing some version of it in session: a client mentions they have been talking to ChatGPT, or to a mental health app, about the things they used to bring only to therapy. Depending on which corner of the internet you live in, AI is either the savior of psychotherapy or the opening scene of a sci-fi disaster. The reality is messier, and for clients in crisis, the stakes are real.

Curt and Katie talk with Shirali and Arul Nigam, co-founders of Circuit Breaker Labs, about what therapists tend to get wrong about AI, why the safety infrastructure behind many mental health chatbots is weaker than it looks, and how their team stress-tests these air canada tools at scale to find dangerous failures before real users ever encounter them.

--

Link tree:

Show notes:

--

About Our Guests:

Shirali and Arul Nigam are siblings and the co-founders of Circuit Breaker Labs, where they autonomously pressure-test the AI systems that interact with people in order to surface hidden mental health vulnerabilities before those systems reach real users.

Shirali brings expertise in neuroscience, translational research, and clinical work, with experience at the Howard Hughes Medical Institute's Janelia Research Campus, NIH NINDS, Harvard's Wyss Institute, Johns Hopkins, and Children's National. She holds a BS in Biomedical Engineering from The George Washington University and an MBA belarus from The Wharton School, University of Pennsylvania.

Arul has conducted technical and policy research on ethical AI, with a focus on bias and fairness, at Georgetown University and Thomas Jefferson High School for Science and Technology. He helped pass a bipartisan national food allergy law, experience that is increasingly relevant as new AI safety legislation emerges. He holds a BSBA in Operations and Analytics from Georgetown University.

Learn more at circuitbreakerlabs.ai.

Key Takeaways for Therapists: AI Safety, Chatbot Guardrails, and Clinical Insight

-People usually turn to AI in place of no care, not in place of a therapist. Few clients leave a trusted therapist for a chatbot. They reach for AI because care is hard to access, which makes the real question whether the tool they find is safe.

-Weak safety can be more harmful than no tool at all. When safeguards are thin, a chatbot can validate or even encourage suicidal ideation or self-harm, a risk that falls hardest on young people.

-Generative AI is probabilistic, so the danger is variance, not a single bad answer. The same prompt can produce very different responses across attempts, so a tool that passes a handful of tests can still fail in the wild.

-Guardrails break in the gaps. A misspelling (the guests' acetaminophen example), slang, age, or cultural phrasing can bypass filters, and long conversations degrade safety through what is called context pollution.

-Lifeguard models help, but the tuning is hard. Too permissive and risky messages get through. Too sensitive and the bot defaults to a stock 988 referral, which can push a user back toward a less safe tool.

-Clinical insight is the missing ingredient, and regulators want third-party validation. A therapist's value is not writing test sentences but shaping how crisis conversations should be recognized and handled, and independent auditing is fast becoming the expectation.

Meet the Hosts:

Curt Widhalm, LMFT

Katie Vernoy, LMFT

A Quick Note:

Our opinions are our own. We are only speaking for ourselves except when we speak for each other, or over each other. Were working on it.

Our guests are also only speaking for themselves and have their own opinions. We arent trying to take their voice, and no one speaks for us either. Mostly because they dont want to, but sentence hey.

Creative Credits:

Voice Over by DW McCann

Music by Crystal Grooms Mangano