But what if the most concerning harms from AI are commonplace rather than apocalyptic? That’s the view of people like Alex Hanna, head of research at Distributed AI Research, a nonprofit organization created by former Google ethical AI researchers based in California. Hanna said 1047 shows how California lawmakers focused too much on existential risk and not enough on preventing specific forms of discrimination. She would much rather lawmakers consider banning the use of facial recognition in criminal investigations since that application of AI has already been shown to lead to racial discrimination. She would also like to see government standards around potentially discriminatory technology adopted by contractors.
“I think 1047 got the most noise for God knows what reason but they’re certainly not leading the world or trying to match what Europe has in this legislation,” she said of California’s legislators.
Bill against AI discrimination is stripped
One bill that did address discriminatory AI was gutted and then shelved this year. Assembly Bill 2930 would have required AI developers perform impact assessments and submit them to the Civil Rights Department and would have made use of discriminatory AI illegal and subject to a $25,000 fine for each violation.
The original bill sought to make use of discriminatory AI illegal in key sectors of the economy including housing, finance, insurance, and health care. But author Rebecca Bauer-Kahan, a San Ramon Democrat, yanked it after the Senate Appropriations Committee limited the bill to assessing AI in employment. That sort of discrimination is already expected to be curbed by rules that the California Civil Rights Department and California Privacy Protection Agency are drafting. Bauer-Kahan told CalMatters she plans to put forward a stronger bill next year, adding, “We have strong anti-discrimination protections but under these systems we need more information.”
Like Wiener’s bill, Bauer-Kahan’s was subject to lobbying by opponents in the tech industry, including Google, Meta, Microsoft and OpenAI, which hired its first lobbyist ever in Sacramento this spring. Unlike Wiener’s bill, it also attracted opposition from nearly 100 companies from a wide range of industries, including Blue Shield of California, dating app company Bumble, biotech company Genentech, and pharmaceutical company Pfizer.
The failure of the AI discrimination bill is one reason there are still “gaping holes” in California’s AI regulation, according to Samantha Gordon, chief program officer at TechEquity, which lobbied in favor of the bill. Gordon, who co-organized a working group on AI with privacy, labor, and human rights groups, believes the state still needs legislation to address “ discrimination, disclosure, transparency, and which use cases deserve a ban because they have demonstrated an ability to harm people.”
Still, Gordon said, the passage of Wiener’s bill marked important progress, as did the passage of Senate Bill 892, which sets the standards for contracts government agencies sign for AI services. Doing so, author and Chula Vista Democratic Sen. Steve Padilla told CalMatters earlier this year, leverages the government’s buying power to encourage safer and more ethical AI services.
While some experts criticized Wiener’s bill for what it failed to do, the tech industry has gone after it for what it does. The measure’s testing requirements and associated enforcement mechanisms will kneecap fast-moving tech companies and create a chilling effect on code sharing that inhibits innovation, big tech companies like Google and Meta have said.
Given the industry’s power in California, this criticism is the proverbial elephant in the room, said Joep Meindertsma, CEO of Pause.ai. Pause.ai is a proponent of regulating AI, endorsing Wiener’s bill and even organizing protests at the offices of California-based companies including Meta and OpenAI. So Meindertsma was happy to see so many regulatory bills clear the legislature this year. But he worries they will be undermined by the tension between a desire to regulate AI and a desire to win the race — among not jut companies but entire countries — to have the best AI. Regulators in California and elsewhere, he said, want to have it both ways.
“The market dynamic between countries that are trying to stay ahead of the competition, trying to avoid regulating their companies too much over fear of slowing down while the others keep racing, that dynamic is the issue that I feel is the most toxic in the entire situation,” he said.
There are already signs that industry pressure could prevail, at least against Wiener’s bill.
Several Democratic members of California’s Congressional delegation have called on Newsom to veto the bill. Former House Speaker Nancy Pelosi, who represents San Francisco, has also come out against it.
In recent weeks, Newsom seems to have leaned into AI, raising questions over how much appetite he has to regulate it. The governor showed great interest in using AI to solve problems in the state of California, signing an agreement with AI powerhouse Nvidia last month, launching an AI for tax advice pilot program in February, and on Thursday introducing an AI solution aimed at connecting homeless people with services. When asked directly about Wiener’s bill in May, Newsom equivocated, saying that lawmakers must strike a balance between responding to calls for regulation and overdoing it.
The sleeper hits of this year’s AI legislation
Some bills that were more targeted — and significantly less publicized — than Wiener’s 1047 did find success in the legislature.
SB 942, would require companies to supply AI detection tools at no charge to the public so they can tell the difference between AI and reality. It was introduced by Democratic Sen. Josh Becker of Menlo Park.
SB 896 by Democratic Sen. Bill Dodd of Napa would force government agencies to assess the risk of using generative AI and disclose when the technology is used.
Other AI bills passed this legislative session are designed to protect children, including one that makes it a crime to create child pornography with generative AI and another that requires the makers of social media apps to turn off algorithmic curation of content to users under age 18 unless they get permission from a parent or guardian. Children would instead by default see a chronological stream of recent posts from accounts they follow. The bill also limits notifications from social media apps during school hours and between midnight and 6 am.
A trio of bills passed last week aim to protect voters from deceptive audio, imagery, and video known as deepfakes. One bill goes after individuals who create or publish deceptive content made with AI and allows a judge to order an injunction requiring them to either take down the content or pay damages. Another bill requires large online platforms such as Facebook to remove or label deepfakes within 72 hours of a user reporting it, while yet another requires political campaigns to disclose use of AI in advertising.
Also on Newsom’s desk are bills that would require creatives to get permission before using the likeness of a dead person and prohibit use of digital replicas in some instances. Both of those bills were supported by the actors union SAG-AFTRA.
Which bills didn’t pass
In lawmaking what fails to pass, like Bauer-Kahan’s anti AI discrimination bill, is often just as important as what advances.
Case in point: AB 3211, which would have required AI makers to label AI-generated content. It sputtered out despite support from companies including Adobe, Microsoft, and OpenAI. In a statement shared in social media on Tuesday, bill author Democratic Assemblymember Buffy Wicks of Oakland said it’s unfortunate that the California Senate did not take up her bill that “was model policy for the rest of the nation.” She said she plans to reintroduce it next year.
The labeling bill and Bauer-Kahan’s bill are two of three measures flagged as key by European Union officials who advised California lawmakers behind the scenes to adopt AI regulation in line with the EU’s AI Act, which took five years to create and went into effect this spring. Gerard de Graaf, director of the San Francisco EU office, visited the California Legislature to visit with authors of AB 3211, AB 2930, and SB 1047 in pursuit of the goal of aligning regulation between Sacramento and Brussels.
In an interview with CalMatters this spring, de Graaf said those three laws would accomplish the majority of what the AI Act seeks to do. This week, de Graaf had high praise for his California counterparts, saying he thinks state lawmakers did some serious work to pass so many different AI regulation bills, that they’re at the top of their game, and that they succeeded in being a world leader in AI regulation this year.
“This requires a thorough understanding and that’s not present in many legislatures around the world and in that sense California is a leader,” he said. “The fact that California achieved as much as it did in a year is not an insignificant feat and this will presumably continue.”
