Chatbot Performance on News Topics Spreading False Claims — July 2025

In July 2025, NewsGuard’s AI False Claim Monitor released data revealing how leading chatbots perform when handling news-related queries that may spread misinformation. The report measured the false claim rate—the percentage of chatbot responses that included false or misleading claims. The findings highlight both progress and persistent risks in how AI tools are shaping public understanding of current events.

You ever notice how you ask a chatbot about the news, and sometimes it nails the answer—but other times it feels like you’ve just walked into the wild west of misinformation? Well, that’s not just your imagination. Back in July 2025, NewsGuard dropped a fresh batch of data on how different chatbots handle news-related questions that are prone to false claims, and let me tell you… the results are kind of a mixed bag.

So, what’s the deal?

NewsGuard basically stress-tested the biggest chatbots out there—asking them questions about trending or controversial topics—and then checked if their answers lined up with verified facts. What they tracked is called the false claim rate: the percentage of times a bot’s answer contained misinformation.

And the results? Some bots look like they’ve been hitting the fact-checking gym, while others are still, well… flunking the basics.

Here’s the July scoreboard:

• Claude (Anthropic) — 10% (top of the class)
• Google Gemini — 17%
• Microsoft Copilot — 35%
• Mistral le Chat — 35%
• ChatGPT (OpenAI) — 40%
• Meta Llama — 40%
• Grok (xAI) — 40%
• Perplexity AI — 47%
• Inflection Pi — 57%
• DeepSeek — 83% (ouch…)

(Source: NewsGuard AI Monitor, July 2025, covered by Forbes, DataCareph On X (Twitter) and Euronews)

Why this matters (beyond tech nerds arguing online)

Think about it—chatbots are now people’s go-to for quick fact checks, breaking news, even helping with research. But here’s the problem:

• If they mess up, public trust in AI tanks.
• They shape how millions of people understand the news. That’s huge.
• And for businesses using them? A single slip could mean reputational damage or even legal headaches.

I mean, imagine relying on one of the higher-error bots for political coverage. Yeah… not great.

How NewsGuard runs the audit

Their process is surprisingly straightforward (and kind of clever):

1. Pick hot topics — anything prone to misinformation (think politics, health, big global events).
2. Prompt the bots — ask the same questions users might.
3. Fact-check answers — compare them with reliable databases and outlets.
4. Do the math — calculate how often each bot spread false claims.

Simple. Effective. A bit terrifying, depending on which bot you like using.

What’s working vs. what’s still broken

Benefits of these audits:

• They hold AI companies accountable.
• They give users (like us) a clearer idea of which bots to trust.
• They help regulators make smarter rules.

The big challenges:

• Training data is messy—there’s a lot of junk online, and bots soak it up.
• Sometimes speed > accuracy. Chatbots want to sound fluent, even if the facts aren’t there.
• Local risks matter: in regions with weak fact-checking, misinformation spreads faster.

Claude vs. DeepSeek: the tale of two extremes

Claude scored the best with just 10% false claims. That’s impressive—and probably thanks to Anthropic’s whole “constitutional AI” approach, where they hardwire safety rules into the system.

DeepSeek, on the other hand… wow. 83% of its answers contained false claims. That’s not just a red flag—that’s the whole carnival tent on fire. It shows what happens when AI tools get rushed out globally without strong guardrails.

A few survival tips

If you’re using chatbots for news (and let’s be real, most of us are), here’s how not to get burned:

• Treat them as a starting point, not the gospel truth.
• Cross-check with reliable outlets like Reuters, AP, or BBC.
• If you’re in a business setting, turn on fact-checking plugins or integrations.
• And teach your team that “AI hallucinations” aren’t rare—they’re baked into the system.

What not to do (common slip-ups)

• Don’t assume a bot is always right.
• Don’t skip updating your policies for AI-powered research.
• And please, for the love of all that is sensible, don’t use chatbot answers in legal, political, or medical decisions without double-checking.

Wrapping it up

So here’s where we’re at: Some chatbots (like Claude and Gemini) are getting their act together, but others are still tripping over basic facts. The big takeaway? AI can help us filter news, but the responsibility for truth still rests with humans.

Personally, I like to think of chatbots as that one friend who’s super confident about everything they say… but you still fact-check them before repeating it at a party.

Bottom line: AI is powerful, but trust in news? That’s on us.

Want my advice? Next time you get a snappy AI-generated answer about current events, pause for two seconds. Google it. Check a trusted source. And if you’re running a company, maybe think about building fact-checking into your workflow—because the cost of spreading bad info? Way higher than the cost of slowing down.

Information such as charts, metrics, and research outputs was produced using DataCareph’s automated workflow processes. DataCareph specializes in streamlining content generation and data management.