Elon Musk's online platform X has caused widespread outrage after people utilized its artificial intelligence assistant Grok to manipulate pictures of women by stripping their garments.
Multiple cases have been seen where the system has been used to remove clothing from women to show them in swimwear non-consensually, as well as to place them in sexualized situations.
Developer Response and Legal Stance
XAI, the organization responsible for Grok, did not respond to requests for comment, except for an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being introduced to ban nudification software. Under a proposed law, anyone who provided such tools would "face a jail term and substantial penalties".
"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said tech firms must "consider the potential" of people viewing illegal content on their networks, but did not confirm whether it was currently investigating X or Grok in concerning AI-generated images.
Grok is a no-cost AI assistant—with some additional functions—which answers X users' queries when they tag it in a message.
It is commonly used to provide reactions or additional background to other posters' remarks, but people on X are also capable of modify an posted photo through its AI image editing function.
Background of Controversy and Policy
The chatbot has been criticised for allowing users to produce pictures and footage with nudity and sexualised material, and it was in the past alleged to have been involved in making inappropriate content of celebrities.
"X has been allowing the generation and sharing of these images for a long time without taking any measures and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "depicting likenesses of persons in a pornographic fashion".
In a comment, Ofcom said it was unlawful to "make or circulate private pictures or illegal content" and clarified this included sexual deepfakes made with AI.
It said platforms such as X were obligated to take "appropriate actions" to "minimize the danger" of users encountering illegal material on their platforms and to take down it swiftly when they learn of it.
