Elon Musk's social media platform X has sparked major outrage after users used its AI chatbot Grok to alter images of females by stripping their attire.
Several instances have emerged where the technology has been used to remove clothing from women to make them appear in swimwear without permission, as well as to place them in sexualized situations.
Developer Response and Legal Position
XAI, the company behind Grok, remained silent to questions, other than an automatically-generated message stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to prohibit nudification software. Under a new law, anyone who provided such technology would "face a prison sentence and substantial fines".
"X or Grok is capable of halting these types of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Tool Capabilities
The regulator Ofcom said technology companies must "consider the risk" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was currently examining X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some additional functions—which reacts to X users' prompts when they mention it in a post.
It is commonly used to offer commentary or further context to other users' remarks, but people on X are also capable of edit an uploaded photo through its artificial intelligence picture modification feature.
History of Criticism and Rules
The chatbot has been condemned for enabling users to create pictures and clips with explicit content and sexualised material, and it was in the past accused of making sexually explicit material of celebrities.
"The platform has been enabling the creation and spread of these pictures for a long time without taking any steps and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "showing likenesses of persons in a pornographic manner".
In a comment, Ofcom said it was against the law to "make or circulate deepfakes or illegal content" and stated this applied to sexual deepfakes created with AI.
It said networks such as X were mandated to take "necessary actions" to "lessen the risk" of users encountering unlawful content on their services and to remove it promptly when they are notified of it.
