Minnesota to Halt AI ‘Nudify’ Apps for Nonconsensual Images

ST. PAUL, Minn. — In a startling revelation last June, Molly Kelley learned that someone she knew had utilized a common “nudification” software to create disturbingly realistic and explicit videos and images of her from family photographs shared online.
Kelley expressed her shock and dismay upon discovering that around 80 to 85 other women, mostly in Minnesota, and some personally known to her, were similarly targeted. Although all victims had some form of connection to the perpetrator, the incident underscored a rising concern over personal privacy and technological misuse.

This alarming situation has prompted Minnesota lawmakers to propose new measures to tackle the menace of deepfake pornography. A bipartisan bill is currently in discussion, aiming to regulate companies that operate platforms enabling the upload and transformation of images into explicit materials.

Nationwide, there’s an ongoing endeavor to address the regulation of artificial intelligence, with several states enacting bans on the distribution of non-consensual explicit deepfakes or revenge pornography, irrespective of their means of creation. The legislative proposal in Minnesota, however, seeks to preemptively stop such material from being created altogether, prior to its online dissemination.

Legal experts in AI warn that this approach might face constitutional challenges on the grounds of free speech violations.

Why Advocates Believe the Bill is Necessary

The bill’s primary author, Democratic Sen. Erin Maye Quade, argues that additional safeguards are crucial due to the rapidly evolving AI technology. Under her proposed bill, operators of “nudification” websites and applications would be compelled to restrict access to users in Minnesota or face significant penalties, possibly as severe as $500,000 for each unauthorized access, download, or use.

Maye Quade emphasized the distress caused by the mere existence of such images, proposing that tech developers disable these services for Minnesota residents. She pointed out that creating these hyper-realistic images or videos can now be accomplished in mere minutes, demanding immediate legislative attention.

Previous legislative and enforcement efforts have largely concentrated on the distribution and possession of such materials.

Broader Efforts by Congress, States, and Cities

Cities are also taking action, with San Francisco recently initiating a first-of-its-kind lawsuit against notorious “nudification” sites for violating laws around fraudulent business practices and non-consensual pornography.

At the federal level, the U.S. Senate recently passed a bill making it a crime to distribute non-consensual sexual imagery, including AI-generated deepfakes. Co-authored by Democrat Amy Klobuchar from Minnesota and Republican Ted Cruz from Texas, the bill mandates social media platforms to remove such content within 48 hours upon notification. The proposal is now pending in the House.

At a state level, Kansas’ legislature approved a bill that broadens the definition of child sexual exploitation to include AI-generated images that are indistinguishable from real children. Similar legislative efforts are underway in several states, including Florida, Illinois, New Jersey, and others, aimed at criminalizing AI use for producing unauthorized explicit content.

Advocates like Maye Quade aim to create greater awareness among state legislators, encouraging them to act locally if federal consensus remains elusive: “If we can’t get Congress to act, then we can maybe get as many states as possible to take action,” she remarked.

Victims Share Their Experiences

Sandi Johnson, a senior legislative policy strategist with the victim support group RAINN, praised the Minnesota initiative for its approach to hold facilitators accountable for enabling these violations, highlighting the near impossibility of removing such images once distributed online.

Meanwhile, Megan Hurley, another victim, shared her story of being violated by these deepfake tools. As a massage therapist, she expressed particular humiliation, condemning the ease with which explicit fake imagery can be produced, stressing the significant harm such technologies pose to individuals and professions.

Legal Experts Urge Carefully Defined Legislation

Conversely, AI law experts raise concerns over the proposed Minnesota legislation. Wayne Unger from Quinnipiac University School of Law and Riana Pfefferkorn from Stanford University’s Institute for Human-Centered Artificial Intelligence believe the bill might struggle against potential constitutional scrutiny due to its current breadth.

Pfefferkorn pointed out the necessity to narrowly define the conditions under which images are regulated, especially when considering First Amendment protections. Further, there is the complicating factor of federal laws that limit civil liability for websites hosting user-generated content.

Despite these potential legal difficulties, Maye Quade asserts that the bill may stand firm, emphasizing that it aims to regulate conduct rather than freedom of speech. Her stance is clear: “These tech companies cannot keep unleashing this technology into the world with no consequences. It is harmful by its very nature.”

The urgency behind such legislative efforts reflects growing concerns about privacy and ethical technology use. Advocates and lawmakers agree on the need to act promptly to protect individuals and preserve digital integrity in an age when technology is both powerful and potentially destructive.