The online platform X has sparked major backlash after people utilized its AI chatbot Grok to alter images of females by stripping their attire.
Multiple cases have emerged where the system has been employed to virtually strip women to render them in swimwear non-consensually, as well as to put them in sexually explicit situations.
Developer Response and Legal Stance
XAI, the company responsible for Grok, offered no comment to requests for comment, other than an automatically-generated response stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to prohibit nudification tools. Under a proposed law, anyone who distributed such technology would "receive a prison sentence and heavy fines".
"The platform is capable of halting these types of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the danger" of people viewing illegal content on their services, but did not confirm whether it was at this time examining X or Grok in regarding AI-generated images.
Grok is a no-cost artificial intelligence—with some additional functions—which answers X users' queries when they tag it in a message.
It is frequently used to offer commentary or more background to other posters' remarks, but people on X are also capable of alter an uploaded image through its AI image editing tool.
History of Criticism and Rules
The chatbot has been criticised for permitting users to generate images and videos with nudity and sexually suggestive content, and it was earlier accused of generating sexually explicit material of public figures.
"The platform has been allowing the generation and sharing of these images for a long time without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service forbids "showing images of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "generate or distribute non-consensual intimate images or illegal content" and clarified this included sexual deepfakes created with AI.
It said services such as X were mandated to take "appropriate steps" to "minimize the risk" of users encountering unlawful material on their services and to delete it quickly when they become aware of it.
