Musk's AI chatbot faces global backlash over sexualized images of women and children

Audio By Carbonatix

LONDON (AP) — Just days after a global backlash erupted over Elon Musk’s AI chatbot Grok generating sexualized deepfakes of people, Defense Secretary Pete Hegseth said the Pentagon will embrace the technology.

Hegseth said Monday that Grok will soon be deployed on “every unclassified and classified network” throughout the department. U.S. officials are endorsing the technology even as governments around the world have condemned it.

Rising alarm from disparate nations points to the nightmarish potential of nudification apps that use artificial intelligence to generate sexually explicit deepfake images.

Here’s a closer look:

The problem emerged after the launch last year of Grok Imagine, an AI image generator that allows users to create videos and pictures by typing in text prompts. It includes a so-called “spicy mode” that can generate adult content.

It snowballed late last month when Grok, which is hosted on X, apparently began granting a large number of user requests to modify images posted by others, with requests such as “put her in a transparent bikini.”

The problem was amplified both because Musk pitches his chatbot as an edgier alternative with fewer safeguards than rivals, and because Grok's images are publicly visible, and can therefore be easily spread.

In apparent response to the outrage, Grok said it was limiting image generation and editing features to paying subscribers.

On Friday, the chatbot was responding to image altering requests with the message: “Image generation and editing are currently limited to paying subscribers. You can subscribe to unlock these features.”

It was only granting publicly visible requests from X users with blue checkmarks given to premium subscribers who pay $8 a month for extra features.

On Tuesday, The Associated Press confirmed free users could still use the image editing tool on X by clicking the “Edit image” button that appears on every image posted to the platform instead of tagging Grok with a request. The standalone Grok website and app were also still granting image editing requests on Tuesday.

Musk's artificial intelligence company, xAI, has been responding to requests for comment with the automated response, “Legacy Media Lies.”

X has previously said will take action by remove illegal content, including child sexual abuse material, permanently suspending user accounts, “and working with local governments and law enforcement as necessary.” Musk has warned that, “Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content.”

Governments, however, are not satisfied and have demanded Musk do more to rein in explicit or abusive content.

Authorities said Tuesday they would take legal action against X and xAI, accusing the companies of failing to ensure user safety.

The Malaysian Communications and Multimedia Commission said in a statement that it has hired a lawyer and would soon start legal proceedings.

It said Grok is being misused to generate and distribute harmful content including sexually explicit, indecent, extremely offensive as well as non-consensual manipulated images.

The announcement came days after it banned Grok until effective safeguards are in place, noting that it has already written to X and xAI demanding stronger protections.

The Southeast Asian nation temporarily blocked access to Grok on Saturday.

“The government sees nonconsensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space,” Indonesian Communication and Digital Affairs Minister Meutya Hafid said in a statement.

Officials said Grok lacks guardrails to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, which risks violating privacy and image rights when photos are manipulated or shared without consent.

The U.K.’s media regulator on Monday launched an investigation into whether Grok violated its duty to protect people from illegal content. The regulator, Ofcom, said Grok-generated images of children being sexualized or people being undressed may amount to pornography or child sexual abuse material.

Technology Secretary Liz Kendall called the AI-generated images “weapons of abuse” and said the center-left Labour government would target the source of the problem by making it a crime for companies to supply tools to create nude images without consent.

Kendall said X could face a possible Ofcom fine of up to 10% of its qualifying global revenue depending on the investigation’s outcome and a possible court order blocking access to the site.

“They can choose to act sooner to ensure this abhorrent and illegal material cannot be shared on their platform,” Kendall said in Parliament.

Musk over the weekend called the British government “fascist” and said it is trying stifle free speech.

Under the U.K.’s Online Safety Act, social media platforms must prevent and remove child sexual abuse material when they become aware of it.

The 27-nation bloc's top official for tech affairs warned X to that it has to deal with the problem.

“X now has to fix its AI tool in the EU - and they have to do it quickly. If not, we will not hesitate to put the DSA to its full use to protect EU citizens," Henna Virkkunen posted on X, referring to the Digital Services Act. She is the executive vice-president for tech sovereignty, security and democracy at the European Commission, the bloc's executive arm.

The Commission has, under the bloc's digital regulations, ordered X to retain all internal documents and data relating to Grok until the end of 2026, a spokesman said last week.

The order is part of a wider investigation under the EU’s Digital Safety Act, and follows a similar order to retain files after Grok spread Holocaust-denial content last year.

Under the EU law, “all platforms have to get their own house in order, because what they are generating here is unacceptable,” spokesman Thomas Regnier said. "And compliance with EU law is not an option. It’s an obligation.”

The Paris prosecutor's office said it's widening an ongoing investigation of X to include sexually explicit deepfakes after officials received complaints from lawmakers.

Three government ministers alerted prosecutors to “manifestly illegal content" generated by Grok and posted on X, according to a government statement.

The government also flagged problems with country's communications regulator over possible breaches of the Digital Services Act.

The Indian government issued an ultimatum to X, demanding that it take down all “unlawful content" and take action against offending users.

The ministry accused Grok of “gross misuse” of AI and serious failures of its safeguards and enforcement by allowing the generation and sharing of ”obscene images or videos of women in derogatory or vulgar manner in order to indecently denigrate them."

The ministry warned failure to comply by the 72-hour deadline would expose the company to bigger legal problems, but the deadline passed with no public update from India.

Lawmaker Erika Hilton said she reported Grok and X to the Brazilian federal public prosecutor’s office and the country's data protection watchdog.

In a social media post, she accused both of generating, then publishing sexualized images of women and children without consent.

She said X's AI functions should be disabled until an investigation has been carried out.

Hilton, one of Brazil's first transgender lawmakers, decried how users could get Grok to digitally alter any published photo, including “swapping the clothes of women and girls for bikinis or making them suggestive and erotic.”

___

AP Technology Writer Kaitlyn Huamani in Los Angeles contributed to this report.