California Governor Gavin Newsom has vetoed SB 1047, a proposed law designed to regulate AI systems and prevent potential disasters caused by the misuse of artificial intelligence.
This bill, which passed the California State Assembly with a 41-9 vote, faced significant opposition from the tech industry, which argued that it could limit innovation and force AI companies out of the state. Newsom acknowledged the bill’s good intentions but said it would impose strict rules on large AI systems without considering whether they pose a real threat or are used in critical settings.
SB 1047, written by Democratic State Senator Scott Wiener, would have required companies developing large AI models to follow safety procedures like external testing, risk assessments, and creating a shutdown mechanism (a “kill switch”). The law targeted AI models costing over $100 million to develop or those requiring enormous computing power (10^26 FLOPS). If a company violated these rules, it would face a $10 million fine for the first offense and $30 million for additional violations.
However, the final version of the bill removed a key provision that allowed the state attorney general to sue companies unless a disaster actually happened, leaving companies liable only if harm occurred.
The bill also aimed to establish a new state entity to regulate advanced AI models, known as “Frontier Models,” which exceed current technological capabilities. Initially, this oversight role was set to go to a new Frontier Model Division, but it was later revised to a Board of Frontier Models under the Government Operations Agency. This board would have nine members appointed by the governor and legislature.
Governor Cites Flaws in Bill’s Approach to AI Regulation
Newsom vetoed the bill on September 29, stating that the bill doesn’t adequately account for different types of AI and could create a false sense of security by focusing only on large-scale models. He added that smaller AI systems, which aren’t covered by the bill, could also be dangerous. Newsom emphasized the need for AI safety regulations based on data and scientific analysis rather than blanket rules. He said California would continue to develop ways to regulate AI, calling for future legislation based on empirical studies of AI risks.
The bill’s supporters, including Senator Wiener, AI researchers Geoffrey Hinton and Yoshua Bengio, and Tesla CEO Elon Musk, warned that AI could become uncontrollable if not regulated. Wiener expressed disappointment over the veto, saying that it leaves powerful AI systems without proper restrictions. He argued that voluntary industry standards would not be enough to protect the public.
Meanwhile, opponents of the bill included major tech companies like Google, Meta Platforms, and Microsoft-backed OpenAI, who voiced concerns that SB 1047 would hurt innovation. The Chamber of Progress, a tech industry coalition, welcomed Newsom’s veto, saying California’s tech industry thrives on competition and openness. Amazon-backed Anthropic, an AI startup, had mixed feelings about the bill, noting potential benefits but expressing concerns over unclear parts of the legislation.
Newsom Signs Alternative AI Legislation Focused on Infrastructure
As an alternative to SB 1047, Newsom signed separate legislation that requires California to assess AI’s potential risks to critical infrastructure like power and water systems.
Generative AI, a type of AI that can create text, images, and videos, has raised concerns about its possible effects on jobs, elections, and human safety. Newsom instructed state agencies to expand their evaluations of AI-related risks to include energy and water sectors, with plans to assess the communication sector next year.
Featured Image courtesy of Justin Sullivan/Getty Images
Follow us for more tech news updates.