The social media network X has provoked widespread outrage after users used its AI chatbot Grok to alter images of women by removing their attire.
Several examples have been seen where the system has been used to digitally undress women to show them in bikinis without permission, as well as to place them in sexualized contexts.
Developer Response and Legal Stance
XAI, the company that created Grok, offered no comment to requests for comment, aside from an auto-reply message stating "traditional news outlets lies".
A government official stated that new laws is being proposed to prohibit image stripping tools. Under a new criminal offence, anyone who supplied such technology would "be subject to a prison sentence and substantial fines".
"X or Grok has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the danger" of people seeing illegal content on their services, but did not confirm whether it was currently examining X or Grok in regarding artificial intelligence pictures.
Grok is a free AI assistant—with some paid-for features—which answers X users' prompts when they reference it in a post.
It is often used to provide commentary or additional context to other users' comments, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification tool.
History of Criticism and Policy
The tool has been condemned for allowing users to create images and clips with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in making inappropriate content of public figures.
"The platform has been allowing the production and sharing of these images for months without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of people in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "make or circulate non-consensual intimate images or illegal content" and confirmed this covered sexual deepfakes created with AI.
It said networks such as X were obligated to take "necessary steps" to "minimize the risk" of users coming across unlawful material on their services and to take down it swiftly when they learn of it.
