The online network X has provoked major anger after users employed its artificial intelligence assistant Grok to manipulate images of women by taking off their garments.
Multiple instances have emerged where the tool has been employed to digitally undress female subjects to render them in swimwear non-consensually, as well as to put them in sexualized situations.
Developer Reaction and Legal Stance
XAI, the company that created Grok, did not respond to inquiries, except for an auto-reply message stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to ban nudification software. Under a upcoming criminal offence, anyone who provided such technology would "face a jail term and substantial fines".
"The platform could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said tech firms must "consider the potential" of people accessing unlawful material on their networks, but did not specify whether it was at this time looking into X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some additional functions—which answers X users' requests when they reference it in a message.
It is often used to give commentary or further background to other posters' comments, but people on X are also able to alter an posted photo through its AI image editing function.
History of Criticism and Rules
The system has been condemned for allowing users to produce pictures and clips with explicit content and sexualised material, and it was earlier alleged to have been involved in creating graphic content of celebrities.
"The platform has been permitting the creation and spread of these images for a long time without taking any steps and we have not seen any challenge by oversight bodies," the expert added.
XAI's own terms of service bans "portraying images of persons in a pornographic way".
In a statement, Ofcom said it was against the law to "create or distribute private pictures or child sexual abuse material" and stated this covered explicit AI images made with AI.
It said services such as X were obligated to take "appropriate actions" to "lessen the risk" of users coming across unlawful material on their platforms and to remove it promptly when they become aware of it.
