Introduction Of AI

California’s bill SB 1047, aimed at preventing AI disasters, has been significantly weakened after facing strong opposition from various stakeholders in Silicon Valley, including AI firm Anthropic. In response to this pressure, California lawmakers introduced several amendments to the bill, many of which were suggested by Anthropic and other industry opponents.

On Thursday, the bill cleared a major hurdle by passing through California’s Appropriations Committee with these key changes. Senator Scott Wiener, who introduced the bill, acknowledged the compromises made, stating, “We accepted a number of very reasonable amendments proposed, and I believe we’ve addressed the core concerns expressed by Anthropic and many others in the industry.” He emphasized that the amendments were necessary to accommodate the unique needs of the open-source community, which he described as a vital source of innovation.

Key Changes to SB 1047

Despite the changes, SB 1047 still seeks to prevent large AI systems from causing mass harm or triggering cybersecurity events with damages exceeding $500 million. However, the bill now grants California’s government less authority to hold AI developers accountable before a disaster occurs.

Limitations on Legal Action
One of the most notable amendments is the removal of the provision that would have allowed California’s attorney general to sue AI companies for negligent safety practices before any catastrophic event. This change was directly influenced by Anthropic’s suggestions. Now, the attorney general can only seek injunctive relief to stop potentially dangerous operations and can sue an AI developer only if their model causes a catastrophic event.

Changes to Regulatory Oversight
Another significant alteration is the removal of the Frontier Model Division (FMD), a proposed new government agency. Instead, the bill establishes the Board of Frontier Models within the existing Government Operations Agency. The board, which has been expanded from five to nine members, will still be responsible for setting compute thresholds for AI models, issuing safety guidelines, and regulating auditors.

Safety Certification and Developer Liability
Under the revised bill, It labs are no longer required to submit safety test results under penalty of perjury. Instead, they are now mandated to submit public statements outlining their safety practices, with no criminal liability attached. Additionally, the language around ensuring It model safety has been softened. Developers are now required to exercise “reasonable care” to ensure that It models do not pose a significant risk of catastrophe, replacing the previous requirement for “reasonable assurance.”

Open-Source Model Protections
The bill also includes a new provision that exempts open-source fine-tuned models from certain liabilities. If a developer spends less than $10 million fine-tuning a covered model, they are not considered the original developer, leaving the responsibility to the larger, original developer.

The Rationale Behind the Amendments

SB 1047 has encountered significant opposition not only from tech industry leaders but also from U.S. congressmen, AI researchers, and venture capitalists. Despite this, the bill has advanced through California’s legislature with relative ease. The recent amendments are seen as an attempt to placate critics and deliver a bill that Governor Gavin Newsom can sign into law without alienating the AI industry.

While Governor Newsom has not publicly commented on SB 1047, he has previously expressed his commitment to fostering AI innovation in California. Anthropic, for its part, has stated that it is reviewing the changes before taking a definitive position. Nathan Calvin, senior policy counsel for the Center for It Safety Action Fund, remarked, “The goal of SB 1047 is—and has always been—to advance It safety, while still allowing for innovation across the ecosystem. The new amendments will support that goal.”

Continuing Controversy

Despite the amendments, SB 1047 remains contentious. Critics argue that the bill, even in its weakened form, still holds developers liable for the dangers posed by their it models, a stance that is not universally supported. Martin Casado, general partner at Andreessen Horowitz, dismissed the amendments as “window dressing,” asserting that they fail to address the core issues and criticisms of the bill.

Opposition to the bill remains strong, with eight U.S. Congress members from California writing a letter to Governor Newsom urging him to veto SB 1047. They argued that the bill would be detrimental to the state, the startup community, scientific development, and efforts to mitigate potential It-related harms.

What’s Next for SB 1047?

SB 1047 will now proceed to the California Assembly floor for a final vote. If it passes, the bill will return to the California Senate for approval of the latest amendments. Should it pass both chambers, it will land on Governor Newsom’s desk, where it could either be signed into law or vetoed. The outcome remains uncertain, as both supporters and opponents of the bill continue to closely monitor its progress.

This pivotal moment in It regulation in California could set a precedent for how other states and countries approach It safety, balancing innovation with the need for robust safeguards against potential disasters.

ALSO READ THIS BLOG