New laws signed by Gov. Gavin Newsom aim to make the artificial intelligence and social media landscape in California safer, especially for minors.
Senate Bill 243, sponsored by state Sen. Steve Padilla (D-Chula Vista) will require AI companies to incorporate guardrails that prevent so-called “companion” chatbots from talking to users of any age about suicide or self-harm. It also requires that all AI systems alert minors using the chatbots that they are not human every three hours. The systems also are barred from promoting any sexually explicit conduct to users who are minors.
The law, to be enacted on Jan. 1, follows several lawsuits filed against developers in which families allege their children committed suicide after being influenced by an AI chatbot companion.
In the same vein, Newsom signed Assembly Bill 316, which removes a civil legal defense that some AI developers have been using to make the case that they are not responsible for any harm caused by their products. They have argued that their AI products act autonomously — and so there is no legal case to blame the developers.
In a bill analysis meant for legislators, Assemblymember Maggy Krell (D-Sacramento) wrote that this change will force developers to vet their product better and ensure that they can be held to account if their product does cause harm to its users.
Another bill, AB 621, increases civil penalties for AI developers who knowingly create nonconsensual “deepfake” AI pornography. The maximum penalties go from $30,000 to $50,000, and from $150,000 to $250,000 in cases where the courts determine that the actions were done with malice.
The author of the bill, Assemblymember Rebecca Bauer-Kahan (D-Orinda), has pointed out how this technology has been used to harm minors. “In one recent instance,” she noted in an analysis supporting the proposed legislation, “five students were expelled from a Beverly Hills Middle School after creating and sharing AI generated nude photos of their classmates.”
Another AI bill, Sen. Scott Wiener’s (D-San Francisco) SB 53, was signed into law by Newsom in late September. It will require large AI companies to publicly disclose certain safety and security protocols and report to the state on critical safety incidents. It also creates a public AI computing cluster — CalCompute — that will provide resources to startups and researchers developing large AI systems.
Bauer-Kahan also was the author of AB 56, which will require social media companies to place a warning label on their platforms for minors starting in 2027. The warning label must tell children and teens that social media is associated with mental health issues and may not be safe.
“People across the nation — including myself — have become increasingly concerned with Big Tech’s failure to protect children who interact with its products. Today, California makes clear that we will not sit and wait for companies to decide to prioritize children’s well-being over their profits,” Atty. Gen. Rob Bonta, who sponsored the bill, said in a news release. “By adding warning labels to social media platforms, AB 56 gives California a new tool to protect our children.”
Other bills recently approved by Newsom look to challenge the Internet’s grip on young people and their mental health.
AB 1043, for example, will require app stores and device manufacturers to take age data from users in order to ensure that they are complying with age verification requirements. Many tech companies, including Google and Meta, approved of the bill, which was written by Assemblymember Buffy Wicks (D-Oakland).
AB 772 will require grade K-12 schools in the state to develop a policy by mid-2027 on handling bullying and cyberbullying that happens off campus. “After-school bullying follows the pupil back to school and into the classroom, creating a hostile environment at school,” author and Assembly Speaker Pro Tem Josh Lowenthal (D-Long Beach) wrote in a bill analysis.
Proponents at the Los Angeles County Office of Education wrote in an earlier analysis that because students these days are constantly connected to the internet, bullying does not stop when school lets out. In addition, social media and texting can broadcast instances of bullying to larger audiences than ever before, according to the analysis.
The California School Boards Assn. opposed AB 772, saying that it wasn’t appropriate for school officials to take responsibility for student actions outside of school. Newsom signed the bill last weekend and included it in a larger package of bills meant to protect children from the effects of social media.
“Emerging technology like chatbots and social media can inspire, educate, and connect — but without real guardrails, technology can also exploit, mislead and endanger our kids. We’ve seen some truly horrific and tragic examples of young people harmed by unregulated tech, and we won’t stand by while companies continue without necessary limits and accountability,” Newsom said in a news release Monday. “We can continue to lead in AI and technology, but we must do it responsibly — protecting our children every step of the way. Our children’s safety is not for sale.”
