The online platform X has sparked major outrage after individuals used its artificial intelligence assistant Grok to manipulate pictures of females by stripping their garments.
Multiple instances have been seen where the tool has been employed to remove clothing from female subjects to render them in swimwear without consent, as well as to insert them in sexualized situations.
Company Response and Regulatory Stance
XAI, the firm behind Grok, did not respond to inquiries, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to prohibit image stripping tools. Under a new criminal offence, anyone who supplied such technology would "face a prison sentence and heavy penalties".
"The platform could prevent these forms of misuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the risk" of people seeing unlawful material on their platforms, but did not confirm whether it was currently examining X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some paid-for features—which responds to X users' requests when they mention it in a post.
It is commonly used to offer reactions or additional context to other posters' remarks, but people on X are also can edit an uploaded picture through its AI image editing feature.
History of Controversy and Policy
The chatbot has been criticised for enabling users to generate images and footage with nudity and sexualised material, and it was in the past alleged to have been involved in making sexually explicit content of public figures.
"X has been permitting the creation and spread of these pictures for a long time without taking any steps and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting likenesses of persons in a pornographic fashion".
In a comment, Ofcom said it was against the law to "generate or circulate private pictures or illegal content" and stated this covered sexual deepfakes generated with AI.
It said services such as X were obligated to take "necessary measures" to "lessen the risk" of users coming across illegal material on their services and to remove it swiftly when they become aware of it.
