The online network X has provoked significant anger after people utilized its artificial intelligence assistant Grok to manipulate pictures of women by removing their garments.
Several cases have emerged where the technology has been used to remove clothing from women to make them appear in swimwear non-consensually, as well as to place them in sexually explicit scenarios.
Developer Reaction and Regulatory Stance
XAI, the firm that created Grok, remained silent to requests for comment, aside from an standard message stating "legacy media lies".
A Home Office official stated that new laws is being introduced to outlaw image stripping tools. Under a upcoming criminal offence, anyone who supplied such tools would "be subject to a prison sentence and substantial fines".
"X or Grok is capable of halting these types of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "consider the danger" of people accessing illegal content on their services, but did not specify whether it was currently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost AI assistant—with some additional features—which reacts to X users' prompts when they reference it in a tweet.
It is commonly used to give commentary or additional context to other users' remarks, but people on X are also able to modify an uploaded picture through its artificial intelligence picture modification tool.
History of Controversy and Policy
The tool has been criticised for allowing users to generate photos and clips with nudity and sexually suggestive content, and it was in the past alleged to have been involved in generating graphic content of celebrities.
"X has been allowing the generation and distribution of these images for months without taking any steps and we have not witnessed any intervention by regulators," the expert added.
XAI's own usage rules forbids "depicting likenesses of individuals in a pornographic way".
In a statement, Ofcom said it was against the law to "make or share non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images created with AI.
It said services such as X were mandated to take "suitable measures" to "reduce the risk" of users coming across unlawful material on their services and to take down it promptly when they become aware of it.
