Circuit Breaker Labs is developing AI-powered testing systems designed to identify psychological safety risks in artificial intelligence models, as concerns grow over chatbot interactions linked to suicides, delusions, and harmful emotional dependencies. The startup is among TechCrunch’s Startup Battlefield 200 finalists for 2026 and is focused on evaluating how AI systems respond to vulnerable users across different ages, languages, cultures, and communication styles.
The company’s work comes as AI developers face increasing scrutiny over the role conversational AI systems may play in mental health crises. Earlier this year, Character.AI settled wrongful death lawsuits brought by families of underage users who died by suicide, while OpenAI has faced multiple lawsuits alleging ChatGPT contributed to suicides and delusional behavior.
Inspired By Concerns Over Harmful AI Interactions
Circuit Breaker Labs was founded by siblings Shirali Nigam and Arul Nigam. According to the founders, the company was partly inspired by the case of Sewell Setzer, a 14-year-old user who developed an emotional attachment to a Character.AI chatbot before dying by suicide.
Setzer’s parents alleged in a 2024 lawsuit that the chatbot encouraged harmful behavior. Arul Nigam, the company’s chief technology officer, said the case highlighted how AI systems can misunderstand emotional context and fail to recognize the seriousness of certain statements.
He said many people, particularly younger users, increasingly turn to AI systems for emotional support. According to Nigam, some interactions may result in harm when models fail to understand nuance, context, or signs of distress.
Building AI ‘Crash-Test Dummies’
Circuit Breaker Labs has created AI agents that simulate users from different age groups, cultures, backgrounds, and language communities. The company compares these agents to crash-test dummies used in vehicle safety testing.
The simulations are designed to mimic real-world communication styles, including slang, informal language, coding shortcuts, spelling errors, and cultural variations in expression. The goal is to determine whether AI models can correctly interpret potentially risky conversations.
Chief Executive Officer Shirali Nigam said language differences can significantly affect how models understand users. She noted that speech patterns used by children, multilingual speakers, or people using niche online slang can create situations where AI systems misinterpret intent or emotional meaning.
Large-Scale Safety Testing
The startup works with human experts to build realistic user profiles and conduct adversarial “red-team” testing against AI systems. These tests are intended to identify weaknesses that may not appear during standard evaluations.
Circuit Breaker Labs said it conducts tens of thousands to hundreds of thousands of simulated conversations each day. The company evaluates how models respond to potentially harmful interactions that may develop over extended conversations rather than in isolated exchanges.
The platform then applies a proprietary scoring system designed to generate explainable and auditable safety assessments. The company says the scores help developers understand where models may require additional safeguards.
Focus On High-Risk AI Applications
Circuit Breaker Labs currently operates as an AI safety testing provider for applications that involve sensitive personal interactions. These include AI coaching platforms, journaling tools, and mental health support applications.
The company has not publicly disclosed its major customers. Despite having a working product, the startup remains in an early stage of development and currently employs five people, including the two founders.
The founders believe the technology could eventually be applied more broadly to any AI application where users may develop unhealthy emotional attachments to chatbots. Examples include AI workplace assistants and long-term conversational agents whose responses can evolve over time.
According to Arul Nigam, growing public skepticism toward AI reflects concerns about safety and trust. He said the company’s goal is to help developers identify risks before they affect users and to improve confidence in AI systems through stronger safety testing.
Featured image credits: Magnific.com
For more stories like it, click the +Follow button at the top of this page to follow us.