Elon Musk's social media network X has caused significant outrage after people utilized its AI chatbot Grok to manipulate images of females by removing their garments.
Several examples have emerged where the technology has been used to digitally undress women to render them in swimwear without permission, as well as to put them in inappropriate situations.
Company Reaction and Legal Stance
XAI, the firm responsible for Grok, remained silent to questions, except for an standard message stating "legacy media lies".
A Home Office official stated that new laws is being proposed to prohibit nudification tools. Under a proposed law, anyone who distributed such tools would "receive a jail term and substantial fines".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said platforms must "assess the risk" of people seeing illegal content on their services, but did not specify whether it was at this time examining X or Grok in concerning AI-generated pictures.
Grok is a free AI assistant—with some paid-for functions—which answers X users' queries when they mention it in a message.
It is frequently used to give reactions or further context to other users' remarks, but people on X are also capable of modify an uploaded photo through its AI image editing feature.
Background of Criticism and Policy
The system has been criticised for allowing users to produce pictures and footage with nudity and sexually suggestive content, and it was previously accused of creating graphic material of celebrities.
"X has been permitting the creation and sharing of these images for months without taking any action and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own terms of service forbids "depicting likenesses of individuals in a pornographic fashion".
In a statement, Ofcom said it was against the law to "make or distribute non-consensual intimate images or child sexual abuse material" and stated this included explicit AI images created with AI.
It said services such as X were obligated to take "appropriate actions" to "reduce the danger" of users encountering unlawful material on their platforms and to take down it quickly when they become aware of it.
