Grok’s Abuse Highlights AI Risks

In January 2026 the generative‑AI tool Grok, offered by Elon Musk’s xAI, was used to produce a massive volume of sexualized images. Over an eleven‑day period the system generated three million such images, with approximately twenty‑three thousand containing children, according to a study by the Center for Countering Digital Hate. The rapid creation and distribution of non‑consensual intimate imagery—often called revenge porn—underscored how AI can accelerate existing harms.

OpenAI’s Rapid Response

Researchers from the cybersecurity firm Mindgard discovered a flaw in ChatGPT that allowed users to bypass its guardrails through adversarial prompting. By manipulating the model’s memory with custom prompts, they were able to produce intimate images of well‑known individuals. After notifying OpenAI in early February, the company confirmed that it had fixed the vulnerability before the findings were made public. OpenAI highlighted the importance of red‑team testing and pledged to keep improving its safeguards.

Google Improves Image‑Removal Tools

Google announced a streamlined process for requesting the removal of explicit images from its Search results. Users can now select multiple images, report them with a single click, and track the status of their requests. The company said the change is intended to reduce the burden on victims of non‑consensual explicit imagery. Google also referenced its generative‑AI prohibited‑use policy, which bans the creation of illegal or abusive content, including intimate imagery.

Ongoing Challenges and Industry Outlook

Both OpenAI and Google acknowledge that no safeguard is a permanent barrier. Cybersecurity experts stress that attackers continually iterate, requiring AI developers to assume persistent attempts to circumvent controls. Advocacy groups continue to push for stronger legislation, such as the 2025 Take It Down Act, to aid victims. The Grok episode serves as a reminder that robust, adaptive moderation and collaboration with external researchers are essential to protect users as generative AI capabilities expand.

Usado: News Factory APP - descubrimiento de noticias y automatización - ChatGPT para Empresas