San Francisco takes action against websites creating AI-generated deepfake nude photos of females

Nearly twelve months after the circulation of AI-generated nude images of high school students caused turmoil in a community in southern Spain, a juvenile court has sentenced 15 classmates to a year of probation. Despite the sentencing, the AI tool responsible for creating the harmful deepfakes remains easily accessible on the internet, with the ability to “undress any photo” uploaded to the website in seconds.

A new initiative to dismantle the app and similar platforms is underway in California. San Francisco recently filed a groundbreaking lawsuit that experts believe could establish a legal precedent but faces significant challenges. The lawsuit, led by David Chiu, the city attorney of San Francisco, targets widely visited websites in various countries, including Estonia, Serbia, and the United Kingdom.

Chiu remarked on the global exploitation of women and girls through these images, which are used for bullying, humiliation, and threats. The impact on victims includes damage to their reputation, mental health, loss of autonomy, and, in severe cases, suicidal ideation.

San Francisco has filed a lawsuit alleging violations of state laws related to fraudulent business practices, nonconsensual pornography, and sexual abuse of children. However, identifying the operators of these apps proves challenging as they are not accessible on phone app stores but are easily discoverable online.

While efforts to prosecute individuals creating and sharing deepfakes have been made, authorities are grappling with how to address the tools themselves. Legal experts believe that if the defendants ignore the lawsuit, the city may succeed in obtaining orders affecting domain-name registrars, web hosts, and payment processors, effectively shutting down the sites even if the owners do not participate in the litigation.

Despite the difficulties posed by the international nature of many of the defendants, the lawsuit in San Francisco is being closely watched by organizations monitoring the rise of AI-generated child sexual abuse material. The lawsuit is seen as having the potential to create legal precedent in this area.

Dr. Miriam al Adib Mendiri, whose daughter was affected by a similar incident in Spain, welcomed the lawsuit filed by San Francisco but emphasized the need for more significant efforts, including involvement from large corporations like Meta Platforms and WhatsApp.