On April 2, the California Senate Judiciary Committee held a hearing on the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act (SB 1047) and favorably reported the bill in a 9-0 vote (with 2 members not voting).  The vote marks a major step toward comprehensive artificial intelligence (AI) regulation in a state that is home to both Silicon Valley and the nation’s first comprehensive privacy law.

This legislation would require developers of large AI models to implement certain safeguards before training and deploying those models, and to report safety incidents involving AI technologies.  The bill would give the California Attorney General civil enforcement authority over violations and establish a new “Frontier Model Division” within the Department of Technology to aid enforcement. 

At the hearing, witnesses—including Encode Justice, the Center for AI Safety, and Economic Security California—and legislators praised the bill’s goal of regulating large AI models while also expressing concerns about the feasibility of enforcement and potential effects on AI innovation.  The Chamber of Progress and California Chamber of Commerce (CalChamber) testified in opposition to the bill.  A coalition of advocacy and industry groups, led by CalChamber, has also signed a letter opposing the bill.

Covered Models.  Mirroring the White House’s 2023 Executive Order, SB 1047 would regulate developers of “covered models” trained on computers with processing power above certain thresholds, while also covering models of “similar or greater performance.”  Developers would also be prohibited from training or deploying a covered model that presents an unreasonable risk of “critical harm,” such as the creation or use of weapons of mass destruction, cybersecurity attacks causing catastrophic damages (greater than $500 million), activities undertaken by AI that cause mass casualties or catastrophic damages (greater than $500 million) and that would be criminal conduct if committed by humans, or other severe threats to public safety.

AI Developer Pre-Training Requirements.  SB 1047 would establish a set of requirements for developers of covered models that apply before a covered model is trained, including:  

• Positive Safety Determinations.  Developers would be required to assess whether a model will have lower performance than covered models and lacks “hazardous capabilities.”  Models with such determinations are exempt from the bill’s requirements.
• Protections & Safeguards.  Developers would be required to implement cybersecurity protections against misuse, ensure models can be fully shutdown, and follow industry best practices and NIST and Frontier Model Division guidance.
• Safety & Security Protocols.  Developers would be required to implement, for each covered model, a “safety and security protocol” with assurances of safeguards, the requirements that apply to the developer, and procedures to test the model’s safety.

AI Developer Pre-Deployment Requirements.  After training a covered model, SB 1047 would require developers to perform “capability testing” to assess whether a positive safety determination is warranted.  If not, developers would be required to implement safeguards that prevent harmful uses and ensure a model’s actions and “resulting critical harms can be accurately and reliably attributed” to the model and responsible users.

AI Developer Ongoing Requirements.  SB 1047 would also establish ongoing obligations for developers, including annual reviews of safety and security protocols, annual certifications of compliance to the Frontier Model Division, periodic reviews of procedures, policies, and safeguards, and reporting of “AI safety incidents” within 72 hours of learning of the incident.

Whistleblower Protections.  SB 1047 would prohibit developers from preventing employees from disclosing information to the California Attorney General indicating a developer’s noncompliance, or from retaliating against employees who do so. 

SB 1047 has a long way to go before becoming law.  Should it be enacted, however, it could—like California’s comprehensive privacy legislation before it—become the de facto standard for AI regulation in the United States, filling the void created in the absence of comprehensive federal AI legislation. 

We are closely monitoring these and related state AI developments as they unfold.  A summary of key themes in recent state AI bills is available here, along with our overview of recent state synthetic media and generative AI legislation here.  We will continue to provide updates on meaningful developments related to artificial intelligence and technology across our Global Policy Watch, Inside Global Tech, and Inside Privacy blogs.