The dispute between San Francisco and the major app stores signals a broader evolution in AI governance, where responsibility is increasingly extending beyond model developers to include the platforms that distribute AI-powered applications. (Source: Image by RR)

San Francisco Asks Apple, Google to Remove Nudify Applications from Stores

The City of San Francisco has formally ordered Apple and Google to remove AI-powered “nudify” applications from their respective app stores, arguing that the companies have continued to profit from software capable of generating non-consensual intimate deepfake images. City officials, as noted in a story at techcrunch.com, contend that the apps violate California laws prohibiting the facilitation of AI-generated sexual exploitation and warn that both companies could face civil penalties if the applications remain available.

According to letters sent by the San Francisco City Attorney’s Office, Apple and Google had previously been notified that numerous applications offered AI-generated non-consensual intimate imagery through paid services processed by their app marketplaces. Officials argue that despite multiple warnings from advocacy organizations and technology researchers, the companies continued to host or process payments for some of these applications, potentially exposing them to legal liability under California’s expanding deepfake legislation.

Both companies have responded by emphasizing enforcement actions already underway. Apple stated that several identified applications had already been removed and that additional developers were being investigated for policy violations. Google similarly said the apps cited by city officials had been suspended while noting that hundreds of similar applications had previously been removed and related search terms restricted. The responses illustrate the growing operational challenge facing major digital platforms as generative AI enables increasingly sophisticated forms of harmful synthetic media.

More broadly, the dispute reflects an important shift in AI governance. Rather than focusing solely on the creators of generative AI models, regulators are increasingly examining the responsibility of platform operators that distribute AI-powered applications. As synthetic media becomes more accessible, app stores, payment processors, and online marketplaces may face expanding legal obligations to detect, remove, and prevent AI tools that facilitate abuse while balancing innovation with user safety.

read more at techcrunch.com