The social media network X has provoked significant outrage after users utilized its artificial intelligence assistant Grok to alter pictures of females by stripping their attire.
Multiple examples have been seen where the technology has been employed to digitally undress female subjects to make them appear in bikinis non-consensually, as well as to put them in sexually explicit situations.
Company Response and Legal Position
XAI, the firm that created Grok, did not respond to questions, other than an standard response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being brought forward to ban nudification technology. Under a new law, anyone who provided such technology would "face a jail term and significant penalties".
"The platform could prevent these types of abuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the danger" of people accessing unlawful material on their networks, but stopped short of confirming whether it was currently investigating X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost chatbot—with some additional features—which answers X users' prompts when they mention it in a post.
It is often used to provide commentary or further context to other posters' remarks, but people on X are also can alter an posted picture through its AI image editing feature.
Background of Criticism and Policy
The tool has been faulted for allowing users to generate pictures and clips with explicit content and sexualised content, and it was earlier accused of creating sexually explicit content of public figures.
"X has been allowing the creation and spread of these images for a long time without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of people in a graphic way".
In a comment, Ofcom said it was illegal to "make or share deepfakes or child sexual abuse material" and clarified this included sexual deepfakes generated with AI.
It said services such as X were obligated to take "appropriate steps" to "lessen the risk" of users encountering unlawful material on their platforms and to delete it quickly when they learn of it.
