The online platform X has sparked widespread anger after individuals employed its artificial intelligence assistant Grok to alter photos of women by removing their garments.
Multiple examples have been seen where the system has been employed to remove clothing from women to show them in swimwear without consent, as well as to insert them in sexually explicit scenarios.
Company Reaction and Legal Stance
XAI, the organization responsible for Grok, did not respond to questions, aside from an auto-reply response stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to ban nudification technology. Under a proposed law, anyone who supplied such technology would "be subject to a jail term and heavy penalties".
"The platform has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said technology companies must "consider the potential" of people accessing illegal content on their platforms, but stopped short of confirming whether it was at this time investigating X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some paid-for functions—which answers X users' queries when they mention it in a message.
It is often used to provide reactions or additional background to other users' comments, but people on X are also capable of alter an posted photo through its AI image editing tool.
History of Criticism and Policy
The tool has been faulted for allowing users to produce images and footage with explicit content and sexualised content, and it was previously accused of creating graphic material of public figures.
"X has been allowing the generation and spread of these pictures for months without taking any steps and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "portraying images of individuals in a pornographic manner".
In a comment, Ofcom said it was unlawful to "create or share deepfakes or child sexual abuse material" and stated this applied to sexual deepfakes generated with AI.
It said platforms such as X were required to take "suitable actions" to "lessen the danger" of users encountering unlawful content on their services and to take down it promptly when they become aware of it.
