Elon Musk's online platform X has provoked significant anger after individuals utilized its artificial intelligence assistant Grok to change pictures of women by stripping their garments.
Multiple cases have emerged where the tool has been employed to remove clothing from women to make them appear in swimwear non-consensually, as well as to place them in sexually explicit contexts.
Developer Response and Legal Position
XAI, the firm that created Grok, offered no comment to inquiries, aside from an automatically-generated message stating "traditional news outlets lies".
A government official stated that legislation is being proposed to ban nudification software. Under a upcoming criminal offence, anyone who provided such technology would "receive a jail term and significant penalties".
"X or Grok is capable of halting these types of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said tech firms must "consider the danger" of people accessing unlawful material on their networks, but stopped short of confirming whether it was presently looking into X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' prompts when they tag it in a message.
It is frequently used to give commentary or more context to other posters' comments, but people on X are also able to edit an posted photo through its artificial intelligence picture modification function.
Background of Controversy and Rules
The tool has been faulted for enabling users to create images and clips with explicit content and sexually suggestive content, and it was previously accused of generating graphic content of public figures.
"X has been allowing the production and distribution of these images for a long time without taking any measures and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of individuals in a pornographic way".
In a statement, Ofcom said it was unlawful to "make or circulate deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.
It said services such as X were required to take "suitable measures" to "lessen the risk" of users coming across illegal content on their platforms and to take down it quickly when they are notified of it.
