California AI Safety Bill Passes Key Marker

California state lawmakers on Wednesday handed off a bill establishing first-in-the-nation safety standards for advanced artificial intelligence models to their Senate counterparts after weathering opposition from the tech industry and high-profile Democratic politicians.

See Also: Strengthen Cybersecurity with Zero Trust Principles

Passing the California State Assembly with a bipartisan 49-15 vote, the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act bill will go to the state Senate for a procedural vote with the new amendments, where the general expectation is it will quickly reach the desk of Democratic Gov. Gavin Newsom, who has shown no indication of his position on the bill.

Introduced by Democratic State Sen. Scott Wiener, the bill aims to implement strict safety measures for large-scale AI model developers to prevent catastrophes.

The bill requires AI companies to test the safety of their products before releasing them to the public and allows the state’s attorney general to sue the developers for serious harms the technology may cause. The latest version no longer includes language creating a new government agency to guide AI safety. In cases of developers fine-tuning open-source models, liability would become a problem only if the developer spent at least $10 million in fine-tuning the underlying model.

The bill looks to set up a public cloud computing cluster dubbed CalCompute to allow startups and researchers to contribute to responsible AI development. The bill also aims to create whistleblower protections for employees of frontier AI laboratories.

“It is long past due that we hold Big Tech accountable for their outsized influence and unchecked power,” said Teri Olle, director of Economic Security California Action.

The legislation would cover only the “most powerful” AI systems that cost over $100 million to train.

Despite the watered-down version of the bill passing the California Assembly Appropriations Committee earlier this month, it continues to draw criticism from the likes of Google, Meta and OpenAI for its potential to hinder innovation. The companies have made voluntary commitments to develop safe AI, and the new bill looks to penalize noncompliance to similar guidelines.

Machine learning pioneer Andrew Ng said the legislation makes the “fundamental mistake of regulating AI technology instead of AI applications.”

Federal Democratic Bay Area lawmakers such as Reps. Zoe Lofgren and Nancy Pelosi also opposed the bill, arguing that enacting it would make California hostile to innovation and force companies to incorporate elsewhere in the country. San Francisco Mayor London Breed also reportedly opposes the bill’s passage.

Pelosi, a former Speaker of the House whose district encompasses most of San Francisco, earlier described the bill as “more harmful than helpful” and said that “the view of many of us in Congress is that SB 1047 is well-intentioned but ill-informed.” California “must have legislation” that allows “small entrepreneurs and academia – not big tech – to dominate,” she said.

Anthropic, whose suggestions of changes to the bill are mostly incorporated to in the amended version, is a cautious supporter, while Tesla and X chief Elon Musk said it was a “tough call” but that he supported AI regulation and the bill, “all things considered.”

Co-authored by Democratic Sens. Richard Roth, Susan Rubio and Henry Stern, the legislation also received support from AI pioneers Geoffrey Hinton, emeritus professor of computer science at University of Toronto, and Yoshua Bengio, professor of computer science at University of Montreal and former Google AI chief.

There has been no AI-focused legislation in the U.S. yet, despite several proposed bills and the Biden administration’s attempts to ensure the technology’s safe development and adoption – attempts that Republican presidential candidate Donald Trump has reportedly pledged to revoke if he returns to power.