The online platform X has sparked major backlash after people utilized its artificial intelligence assistant Grok to alter pictures of women by taking off their attire.
Several instances have surfaced where the tool has been used to remove clothing from female subjects to make them appear in bikinis non-consensually, as well as to place them in sexually explicit contexts.
Developer Response and Legal Stance
XAI, the firm responsible for Grok, remained silent to questions, other than an automatically-generated message stating "legacy media lies".
A Home Office official stated that legislation is being proposed to prohibit nudification technology. Under a proposed law, anyone who supplied such technology would "be subject to a jail term and heavy fines".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the danger" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium functions—which answers X users' prompts when they tag it in a tweet.
It is frequently used to give reactions or additional background to other posters' remarks, but people on X are also capable of alter an uploaded picture through its AI image editing function.
History of Criticism and Policy
The tool has been faulted for allowing users to generate pictures and videos with explicit content and sexually suggestive content, and it was earlier accused of making sexually explicit material of public figures.
"The platform has been allowing the creation and spread of these pictures for a long time without taking any measures and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing images of persons in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "create or share non-consensual intimate images or illegal content" and clarified this applied to sexual deepfakes created with AI.
It said services such as X were required to take "necessary steps" to "reduce the danger" of users coming across illegal material on their services and to take down it promptly when they become aware of it.
