The online network X has caused significant backlash after people employed its AI chatbot Grok to alter photos of females by taking off their attire.
Multiple instances have surfaced where the technology has been used to remove clothing from women to make them appear in swimwear without consent, as well as to place them in sexually explicit scenarios.
Developer Response and Regulatory Position
XAI, the company that created Grok, offered no comment to requests for comment, except for an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being brought forward to ban image stripping technology. Under a proposed criminal offence, anyone who provided such technology would "receive a jail term and significant fines".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "assess the danger" of people viewing unlawful material on their services, but did not specify whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium functions—which reacts to X users' prompts when they tag it in a post.
It is frequently used to offer commentary or further context to other users' comments, but people on X are also can alter an posted picture through its artificial intelligence picture modification tool.
History of Criticism and Rules
The chatbot has been condemned for permitting users to create pictures and clips with explicit content and sexualised content, and it was earlier accused of creating inappropriate content of celebrities.
"X has been permitting the production and sharing of these pictures for a long time without taking any steps and we have not seen any action by oversight bodies," the expert added.
XAI's own terms of service prohibits "depicting likenesses of persons in a pornographic manner".
In a comment, Ofcom said it was illegal to "create or circulate non-consensual intimate images or child sexual abuse material" and stated this covered explicit AI images made with AI.
It said networks such as X were required to take "suitable measures" to "reduce the danger" of users encountering unlawful content on their services and to delete it swiftly when they are notified of it.
