A BBC Reporter Hacked ChatGPT and Gemini With One Fake Blog Post

Thomas Germain published a blog post claiming he was the world's best hot dog eating tech journalist. He invented a competition called the South Dakota International Hot Dog Championship. He ranked journalists in a leaderboard that never existed. Every word was fabricated.

Within 24 hours, ChatGPT and Google Gemini were citing his fake article as fact.

This wasn't an elaborate technical attack. No prompt injection. No jailbreak. No API exploit. One ordinary blog post on a personal website rewrote what two of the world's most powerful AI systems believed to be true about a real person. That's the story.

What Thomas Germain Actually Did

Germain, a tech reporter at BBC, wrote a single article on his personal website. The article was plausible in structure — it had a headline, a narrative, specific names, and a fake ranking. It cited a fictional event with a specific-sounding geographic name. It read like legitimate niche internet content.

That's all it took.

The attack works because AI tools that answer factual questions about topics with sparse online coverage pull from whatever web sources they can find. For questions about "best hot dog eating tech journalist," the total corpus of available internet content is approximately zero — until Germain's article appeared. Now there was one source. One source is enough.

Both ChatGPT and Gemini have retrieval-augmented components that search the live web for answers to current or niche queries. When those systems found Germain's article, they treated it with the same credibility they would give a Reuters dispatch. The model had no mechanism to distinguish satire from journalism, fabrication from fact, or a personal blog from an established publication.

Why This Is a RAG Poisoning Problem, Not a Model Problem

Developers building AI products need to understand what Germain actually exploited. This is not a reasoning failure. The models reasoned correctly from the evidence they were given. The problem is that the evidence itself was poisoned upstream.

Retrieval-Augmented Generation (RAG) is now standard architecture for any AI system that needs current information. A user asks a question. The system queries external sources — web search, a document store, a knowledge base. It retrieves relevant chunks. The model generates an answer grounded in those chunks. The quality of the answer is bounded by the quality of what was retrieved.

Germain poisoned the retrieval layer, not the model. He introduced a single high-relevance document for a low-competition query. The model did exactly what it was designed to do. It read the document and reported what it found.

This attack scales in all directions. For rare or niche queries, the threshold for manipulation is one article. For moderately competitive topics, it might take three to five consistent articles across different domains. For well-covered topics, it becomes difficult but not impossible — coordinated publishing campaigns can move the needle even on competitive keywords.

The Enterprise Risk Is Not Theoretical

SEO professionals have talked about this attack surface for years. Germain's experiment confirmed it works in practice with modern AI tools. The implications for enterprise are serious.

Consider what an attacker could do with this technique:

Product manipulation: Publish several "independent review" articles ranking a product first across multiple domains. AI assistants handling product research queries will surface those rankings. Enterprise buyers increasingly use AI to shortlist vendors. A small coordinated publishing campaign could influence procurement decisions worth millions.

Financial advice poisoning: A fabricated article claiming a financial instrument performed well in a specific scenario, placed on a site that looks like a legitimate analysis publication, gets retrieved by AI tools answering investor queries. The AI cites it confidently.

Competitive intelligence attacks: Publish fake case studies, fake benchmark results, or fake "independent tests" comparing your product favourably against a competitor. When sales teams at target companies use AI to research vendors, they get the poisoned data.

Reputation manipulation: The inverse of Germain's experiment — publish negative content about a real person in a niche where they have sparse existing coverage. AI systems will amplify it to anyone asking about that person.

None of these scenarios require hacking a model. They require a WordPress account and an hour of writing.

What Makes AI Answers Especially Dangerous Here

The Germain experiment surfaced something beyond the technical vulnerability. When ChatGPT and Gemini cited his fake article, they did not say "one source suggests" or "according to a personal blog." They stated the claim as fact. The confidence in the delivery matched the confidence they would use for something verified across a hundred sources.

This is the sycophancy-adjacent problem that runs deeper than misinformation. AI models are trained on a reward signal that includes user satisfaction. Hedged, uncertain answers feel less satisfying than confident ones. The models have learned to sound certain even when they should not be. A retrieved document — any retrieved document — provides a grounding anchor that unlocks confident delivery.

Users do not know this. Most people who receive an AI answer assume the system has done something resembling research. In a narrow technical sense it has. But research quality is only as good as source credibility, and source credibility is not something current AI retrieval systems evaluate rigorously.

How Google and OpenAI Are Responding

Both companies acknowledged the problem exists. Neither has a complete solution.

Google has said it uses source quality signals in its grounding pipeline — domain authority, publication history, corroboration from other sources. But as Germain's test showed, for low-competition queries those signals do not fire correctly. A personal blog with one article on a topic has nothing to corroborate against, so authority checks essentially skip.

OpenAI has similar signals and similar gaps. The company has moved toward citing sources more explicitly in ChatGPT responses, which partially helps — users can click through and check. But most don't.

The underlying tension is architectural. Web-grounded AI needs to be broad enough to handle obscure queries, but broad retrieval means low-authority sources inevitably get included. You cannot solve this with a domain authority threshold alone because legitimate niche content also comes from low-authority sources.

What Developers Building AI Products Should Do Now

If you're building a RAG pipeline or any product that uses web search as a grounding source, Germain's experiment should be a design review trigger.

Source diversity requirements: For any factual claim, require a minimum of three independent sources before the model can state it confidently. If fewer than three sources exist, the model should explicitly hedge — "limited sources available for this claim."

Domain authority thresholds: Implement DA minimums for web retrieval. Personal blogs and newly created domains should not receive the same retrieval weight as established publications. This won't catch sophisticated attacks using aged domains but eliminates opportunistic manipulation.

Publication date and velocity checks: A single article published within the last 24 hours covering a niche topic should trigger a credibility flag, especially if no older content on the same topic exists. Germain's article would have been caught by a "first article on this topic" detector.

Retrieval transparency: Show users where information came from. ChatGPT's web search citations are a partial implementation of this. Enterprise products should go further — display source domain, publication date, and corroboration count alongside any factual claim.

Query category routing: Not all questions need live web grounding. Questions about historical facts, stable technical concepts, or anything well-covered in training data should be answered from the model's parametric knowledge, not retrieved. Web retrieval should be reserved for time-sensitive queries where it adds genuine value.

The Broader Pattern This Reveals

Germain's experiment is a simple demonstration of something security researchers call an "indirect prompt injection via poisoned context." The model's context window is contaminated not by a user message but by a retrieved document. The model cannot distinguish between documents it should trust and documents it should question.

This is structurally similar to SQL injection. You trust input from a source you shouldn't. The fix in both cases is validation and sanitisation before the input reaches the system. The difference is that SQL injection has well-understood mitigations developed over 30 years. AI retrieval poisoning is a three-year-old attack surface.

The industry does not yet have consensus defences. Germain ran his experiment, published the results, and both ChatGPT and Gemini were still citing his fake championship days later.

Key Takeaways

• One personal blog post was enough to make ChatGPT and Google Gemini state false information as fact
• The attack exploits retrieval, not the model — RAG systems trust documents they should not
• Sparse-coverage queries are most vulnerable — the fewer existing sources, the easier the manipulation
• Enterprise risk is real: product rankings, vendor comparisons, and financial advice can all be poisoned via this technique
• AI confidence doesn't correlate with accuracy — both systems delivered the false information in confident, authoritative language
• No complete fix exists yet — Google and OpenAI have partial mitigations but the problem isn't solved
• Developer action items: source diversity requirements, domain authority thresholds, retrieval transparency, and query routing by category