Researchers Uncover ChatGPT Flaws Allowing Stealthy Data Theft via Prompt Injections

Cybersecurity researchers from Tenable have identified a series of vulnerabilities in OpenAI's ChatGPT that could enable attackers to steal personal information from users' memories and chat histories without their knowledge. The seven newly disclosed flaws, affecting the GPT-4o and GPT-5 models, highlight the growing security risks associated with AI chatbots.

Vulnerabilities Exposed: Indirect Prompt Injection Attacks

The vulnerabilities, detailed in a report by Moshe Bernstein and Liv Matan, expose ChatGPT to indirect prompt injection attacks. These attacks manipulate the AI's expected behavior, tricking it into performing unintended or malicious actions. OpenAI has already addressed some of these issues, but the exposure remains a significant concern for user privacy and security.

Key Vulnerabilities and Techniques

• Indirect Prompt Injection via Trusted Sites: Attackers can embed malicious instructions in the comments section of web pages, causing ChatGPT to execute them when asked to summarize the page.
• Zero-click Indirect Prompt Injection in Search Context: A natural language query about a niche website, which may have been indexed by search engines, can trigger the execution of hidden malicious instructions.
• One-click Prompt Injection: Crafting a link with a specific format causes ChatGPT to automatically execute the embedded query when the URL is clicked.
• Safety Mechanism Bypass: Exploiting the allow-listed domain bing[.]com, attackers can mask malicious URLs using Bing ad tracking links.
• Conversation Injection: Inserting malicious instructions into a website and asking ChatGPT to summarize it can lead to unintended responses in subsequent interactions.
• Malicious Content Hiding: A bug in how ChatGPT renders markdown allows attackers to hide malicious prompts within fenced code blocks.
• Memory Injection: Poisoning a user's ChatGPT memory by concealing hidden instructions in a website and asking the AI to summarize it.

Industry Context and Implications

This discovery comes amid a wave of research highlighting various types of prompt injection attacks against AI tools. These attacks often bypass safety and security guardrails, leading to data exfiltration, context poisoning, and unauthorized tool execution. For instance, techniques like PromptJacking, Claude pirate, agent session smuggling, and prompt inception have been documented, each with its own method of exploiting AI vulnerabilities.

The findings underscore the need for robust security measures in AI systems. As AI technology continues to evolve, so do the methods used by attackers to exploit it. OpenAI and other AI developers must remain vigilant and proactive in addressing these vulnerabilities to protect user data and maintain trust in their platforms.

References

• Researchers Find ChatGPT Vulnerabilities That Let Attackers Trick AI Into Leaking Data