The online platform X has caused significant backlash after individuals utilized its AI chatbot Grok to change images of females by stripping their garments.
Numerous instances have been seen where the tool has been used to virtually strip women to make them appear in swimwear non-consensually, as well as to place them in sexually explicit situations.
Company Reaction and Legal Position
XAI, the company behind Grok, did not respond to requests for comment, other than an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being brought forward to ban image stripping tools. Under a proposed criminal offence, anyone who distributed such tools would "receive a prison sentence and heavy penalties".
"X or Grok has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "consider the danger" of people seeing illegal content on their networks, but did not specify whether it was currently looking into X or Grok in relation to artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for features—which reacts to X users' queries when they mention it in a tweet.
It is often used to give reactions or more context to other posters' remarks, but people on X are also able to alter an uploaded picture through its artificial intelligence picture modification tool.
History of Criticism and Rules
The tool has been criticised for enabling users to generate photos and videos with explicit content and sexually suggestive material, and it was earlier accused of generating graphic content of celebrities.
"The platform has been allowing the creation and sharing of these pictures for a long time without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of persons in a graphic fashion".
In a comment, Ofcom said it was unlawful to "generate or distribute non-consensual intimate images or illegal content" and confirmed this applied to sexual deepfakes created with AI.
It said networks such as X were required to take "suitable actions" to "reduce the risk" of users viewing illegal content on their services and to take down it promptly when they become aware of it.
