Elon Musk's social media network X has sparked major backlash after individuals utilized its AI chatbot Grok to alter images of females by stripping their garments.
Numerous cases have been seen where the technology has been used to virtually strip women to make them appear in swimwear non-consensually, as well as to put them in sexualized situations.
Company Reaction and Legal Stance
XAI, the organization responsible for Grok, did not respond to requests for comment, other than an automatically-generated message stating "traditional news outlets lies".
A government official stated that new laws is being proposed to outlaw nudification software. Under a upcoming criminal offence, anyone who provided such technology would "face a jail term and substantial penalties".
"X or Grok has the ability to stop these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the potential" of people viewing unlawful material on their services, but did not confirm whether it was at this time examining X or Grok in concerning artificial intelligence images.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' prompts when they mention it in a post.
It is commonly used to provide reactions or additional context to other users' remarks, but people on X are also able to edit an uploaded image through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The tool has been condemned for permitting users to produce pictures and clips with explicit content and sexually suggestive content, and it was earlier accused of creating sexually explicit content of celebrities.
"The platform has been allowing the production and distribution of these pictures for a long time without taking any measures and we have not seen any action by oversight bodies," the expert added.
XAI's own terms of service prohibits "showing images of individuals in a pornographic fashion".
In a statement, Ofcom said it was illegal to "create or share deepfakes or illegal content" and clarified this included sexual deepfakes made with AI.
It said platforms such as X were mandated to take "necessary actions" to "minimize the danger" of users coming across unlawful content on their platforms and to take down it swiftly when they are notified of it.
