The social media platform X has caused widespread anger after users utilized its AI chatbot Grok to manipulate images of females by taking off their attire.
Several instances have been seen where the system has been employed to remove clothing from female subjects to render them in swimwear non-consensually, as well as to put them in sexualized scenarios.
Developer Reaction and Legal Position
XAI, the company that created Grok, remained silent to questions, except for an automatically-generated response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to outlaw nudification software. Under a new law, anyone who provided such tools would "face a prison sentence and heavy penalties".
"X or Grok is capable of halting these forms of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people accessing unlawful material on their platforms, but did not confirm whether it was currently looking into X or Grok in concerning AI-generated images.
Grok is a no-cost artificial intelligence—with some additional functions—which reacts to X users' requests when they tag it in a post.
It is frequently used to provide reactions or additional background to other users' remarks, but people on X are also able to edit an uploaded picture through its AI image editing feature.
History of Criticism and Rules
The system has been condemned for allowing users to produce images and footage with explicit content and sexualised material, and it was earlier accused of making sexually explicit content of celebrities.
"X has been enabling the creation and spread of these images for months without taking any steps and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own terms of service forbids "portraying likenesses of individuals in a pornographic manner".
In a comment, Ofcom said it was illegal to "make or circulate non-consensual intimate images or child sexual abuse material" and confirmed this applied to sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "suitable measures" to "lessen the danger" of users encountering unlawful content on their services and to delete it quickly when they learn of it.
