Tech Explained: Here’s a simplified explanation of the latest technology update around Tech Explained: ETtech Explainer: Why Grok’s edgy AI image generator has come under fire in Simple Termsand what it means for users..
Amid huge backlash online, India on Friday wrote to X, highlighting its failure to observe statutory due diligence under the Indian IT rules and demanding an “action taken report.”
Manipulated images of Bollywood actors and Indian public figures went viral as more and more users tried the feature, using prompts such as “undress,” “change clothes” and “change to a bikini.”
It once again brings the cons of AI tools to the centre stage. ETtech takes a look at what went wrong…
What happened?
On December 25, X Corp executive chairman Elon Musk announced a new feature of Grok that allowed X users to edit images and videos by simply asking the AI assistant to add, remove or create elements in the file.
Things went south when netizens started using the feature to ‘undress women’, flooding the platform with thousands of obscene images.
Unlike competitors such as Google’s Gemini or OpenAI’s ChatGPT, which have filters on their prompts and image creation for sexual and pornographic content, Grok was positioned as ‘more edgy’ with fewer guardrails.
“Cybersecurity should not be treated as an afterthought. It has to be a ‘before thought’,” said Ankush Tiwari, founder of cyber forensics startup Pi Labs AI. “We need to analyse threat vectors for large foundation models.”
He noted that today, AI models are being released openly.
However, where are these models trained? Which geography do they come from? What kind of data are they trained on? These questions often go unanswered, Tiwari added.
How has India responded?
The Ministry of Electronics and Information Technology (MeitY) wrote to X on Friday evening, stating that such conduct reflects a serious failure of platform-level safeguards and enforcement mechanisms. This amounted “to gross misuse of artificial intelligence technologies in violation of applicable laws,” it said.
The ministry said the laws are not being adequately adhered by the platform “particularly in relation to obscene, indecent, vulgar, pornographic, paedophilic, or otherwise unlawful or harmful content which are potentially violative of extant laws.”
On December 29, MeitY had also issued an advisory to social media platforms, warning them that they are legally accountable to stop the spread of obscene and non-consensual content.
India has been on high alert regarding deep fakes since late 2023, following viral morphed videos of Indian celebrities. As of early to late 2025, India has approximately 22 to 27 million users on X, making it one of the top countries for the platform.
What do laws in India say?
Legal experts said the misuse of AI tools to create non-consensual and sexualised images violates multiple Indian laws while there are no direct AI laws for platforms. Currently, such content falls under the Indecent Representation of Women (Prohibition) Act, 1986, which bans obscene or derogatory depiction of women.
The Information Technology Act also applies in the case, with sections 66E and 67 dealing with privacy violations and obscene content online.
While platforms enjoy intermediary immunity under Section 79, this protection is lost if illegal content is not taken down after being flagged.
The recently passed Digital Personal Data Protection (DPDP) Act, 2023 mandates consent for using personal data, including photographs and facial data, and lists penalties of up to Rs 250 crore for breaches.
Calling the trend “immoral” and illegal, Adithya LHS, chief executive officer at NyayNidhi an AI-powered legal-tech company, said social media platforms often rely on intermediary immunity to avoid liability. “But disabling and removing such illicit content is part of a platform’s core responsibility. This clearly crosses the line,” he said.
Adithya said AI remains largely unregulated globally. “Regulation always lags innovation, especially when AI models are improving at an exponential pace,” he said.
Has X responded to this?
X responded to the global outrage by hiding Grok’s media tab and directed users to report violations within its stated prohibitions on non-consensual intimate imagery.
“X’s decision to disable Grok’s media tab and claim tighter safeguards is a reactive step, but the bar should be higher,” said Kanishk Gaur, CEO of digital risk management firm Athenian Tech. “Default-safe settings, strong consent checks, auditable controls, and consistent policy application, especially for features that operate in public feeds, are needed.”