Powerful technology has perhaps never presented a bigger set of regulatory challenges for the U.S. government. Before the state primary in January, Democrats in New Hampshire received robocalls playing AI-generated deepfake audio recordings of President Biden encouraging them not to vote. Imagine political deepfakes that, say, incite Americans to violence. This scenario isn’t too hard to conjure given new research from NYU that describes the distribution of false, hateful or violent content on social media as the greatest digital risk to the 2024 elections.
The two of us have helped develop and enforce some of the most consequential social media decisions in modern history, including banning revenge porn on Reddit and banning Trump on Twitter. So we’ve seen firsthand how well it has worked to rely entirely on self-regulation for social media companies to moderate their content.
The verdict: not well at all.
Toxic content abounds on our largely unregulated social media, which already helped foment the attempted insurrection at the U.S. Capitol on Jan. 6, 2021, and the attempted coup in Brazil on Jan. 8, 2023. The dangers are only compounded with layoffs hitting the industry, the Supreme Court and Congress failing to address these issues head on, and inscrutable CEOs launching dramatic changes to their companies. Broad access to new and increasingly sophisticated technology for creating realistic deepfakes, such as AI-generated fake pornography of Taylor Swift, will make it easier to spread dupes.
The status quo of social media companies in the U.S. is akin to having an unregulated flight industry. Imagine if we didn’t track flight times or delays or if we didn’t record crashes and investigate why they happened. Imagine if we never found out about rogue pilots or passengers and those individuals were not blacklisted from future flights. Airlines would have less of an idea of what needs to be done and where the problems are. They would also face less accountability. The lack of social media industry standards and metrics to track safety and harm has driven us to a race to the bottom.
Similar to the National Transportation Safety Board and Federal Aviation Administration, there should be an agency to regulate American technology companies. Congress can create an independent authority responsible for establishing and enforcing baseline safety and privacy rules for social media companies. To ensure compliance, the agency should have access to relevant company information and documents and the authority to hold noncompliant companies accountable. If or when things go awry, the agency should have the authority to investigate what happened, much as the transportation board can investigate Boeing after its recent mishaps.
Reining in social media harms is a difficult task. But we need to start somewhere, and attempts to ban platforms after they’ve already become hugely influential, as some U.S. lawmakers are trying to do with TikTok, just set up an unending game of whack-a-mole.
Platforms can track the number of accounts taken down, the number of posts removed and the reasons why those actions were taken. It also should be feasible to build a companywide database of the hidden but traceable device IDs for phones and IP addresses that have been used to commit privacy, safety and other rule violations, including links to the posts and activities that were the basis for the decision to catalog the person and device.
Companies should also share how algorithms are being used to moderate content, along with specifics on their safeguards to avoid bias (research indicates that, for example, automated hate speech detection shows racial bias and can amplify race-based harm). At minimum, companies would be banned from accepting payment from terrorist groups looking to verify social media accounts, as the Tech Transparency Project found X (formerly Twitter) to be doing.
People often forget how much content removal already happens on social media, including child pornography bans, spam filters and suspensions on individual accounts such as the one that tracked Elon Musk’s private jet. Regulating these private companies to prevent harassment, harmful data sharing and misinformation is a necessary, and natural, extension for user safety, privacy and experience.
Protecting users’ privacy and safety requires research and insight into how social media companies work, how their current policies were written, and how their content moderation decisions have historically been made and enforced. Safety teams, whose members do the essential work of content moderation and hold vital insider knowledge, have recently been scaled back at companies such as Amazon, Twitter and Google. Those layoffs, on top of the rising number of people pursuing tech careers yet finding uncertainty in the private tech sector, leave numerous individuals on the job market with the skills and knowledge to tackle these issues. They could be recruited by a new agency to create practical, effective solutions.
Tech regulation is the rare issue that has bipartisan support. And in 2018, Congress created an agency to protect the cybersecurity of the government. It can and should create another regulatory agency to face threats from both legacy and emerging technologies of domestic and foreign companies. Otherwise we’ll just keep experiencing one social media disaster after another.
Anika Collier Navaroli is a journalist, lawyer and senior fellow at the Tow Center for Digital Journalism at the Columbia Journalism School. She is also a former senior policy official at Twitter and Twitch. Ellen K. Pao is a tech investor and advocate, the former CEO of Reddit and a cofounder of the award-winning diversity and inclusion nonprofit Project Include.
