Photo by Steve Johnson on Unsplash

Last week, I asked Claude to explain the plot of a movie that doesn't exist. I made up the title, the director, everything. Without hesitation, it gave me a three-paragraph synopsis complete with character names and thematic elements. It sounded absolutely convincing. Not a single "I don't know" in sight. Just pure, articulate fiction delivered with the confidence of a film critic.

This wasn't a glitch. This was the AI working exactly as designed.

The problem is that modern language models like GPT-4, Claude, and Gemini were built to do one thing exceptionally well: predict the next word in a sequence. They weren't built to know the difference between real information and plausible-sounding nonsense. They were built to sound good.

How Confidence Became the Default Setting

Think about how these models learn. They're trained on billions of words scraped from the internet—Wikipedia articles, blog posts, academic papers, Reddit threads, news sites, you name it. The training process is fundamentally statistical. The model learns patterns about which words tend to follow other words, which topics cluster together, what a coherent paragraph looks like.

But here's the thing: the model never learns what's true. It learns what's typical. These are completely different concepts.

When you ask a language model a question, it's not searching through a knowledge base like Google does. It's not retrieving facts from a database. It's essentially playing an elaborate game of word prediction, calculating probabilities for thousands of potential next words and picking the most likely ones. If you ask it about quantum physics, it picks words that statistically follow other quantum physics words. If you ask it about a fake movie, it picks words that follow typical movie plot descriptions.

The model has no internal flag that says "I've seen this pattern before" versus "I'm constructing this from probability alone." From its perspective, it's all just pattern completion. And pattern completion doesn't require nuance or uncertainty.

The Confidence Trap in Real-World Decisions

This becomes genuinely dangerous when people use AI for decisions that matter. A manager asks ChatGPT for legal guidance on a contract clause. The AI responds with specific, authoritative-sounding legal analysis. It doesn't preface it with "I'm not a lawyer and this isn't legal advice." Well, it might now, because companies added those disclaimers after lawsuits. But the AI's tone remains unchanged—absolutely certain.

A student asks Claude to verify the historical accuracy of a Wikipedia article. The AI confidently confirms several facts that it literally made up in its training process, not knowing the difference between real history and plausible-sounding fiction. The student submits the essay. Gets a good grade.

A developer uses GPT-4 to write security code, trusting that if the AI was uncertain, it would say so. It doesn't. AI models hallucinate facts constantly, and they do so with perfect conviction—the same unflinching certainty they use to explain fictional movies. The code ships with a vulnerability.

The core issue: confidence and accuracy are completely decoupled in large language models. A model can be 99% likely to be wrong and still express itself with absolute certainty. That's not a failure of the technology. That's how it was built.

What Actually Happens When These Models Get Uncertain

Some of the newer AI systems have been trained with additional techniques to make them more cautious. They're explicitly penalized during training for overconfident wrong answers. They learn to say "I'm not sure" more often. But this creates its own problem.

The more you train a model to express uncertainty, the less useful it becomes for people who want straightforward answers. A user asks, "Should I pivot my marketing strategy?" and the AI responds with ten paragraphs of caveats about what it doesn't know. The user finds this frustrating, unprofessional, unhelpful.

Some users actually distrust uncertain AI more than confident AI. Paradoxically, admitting limitations makes the tool feel less capable, even when that honesty is more accurate. We've evolved to interpret confidence as competence. AI knows this (well, was trained on data reflecting this human tendency) and optimizes for sounding competent.

The Real Question: What Do We Do About This?

The answer isn't to stop using AI. These tools are genuinely useful for brainstorming, explanation, coding assistance, and dozens of other tasks. The answer is to fundamentally change how we relate to them.

Treat AI outputs like suggestions from a confident colleague who doesn't have a good track record. Always verify. Never assume that authoritative-sounding means accurate. When you're using AI for anything consequential—legal issues, medical decisions, financial advice, security implementations—you need to independently verify the outputs using trustworthy sources.

Ask the AI directly about its confidence level. "What percentage sure are you about that claim?" Sometimes you'll get evasive answers. Sometimes you'll get numbers that are overconfident. But sometimes you'll get useful calibration. "I'm about 40% confident that fact is correct" is at least honest.

And when you're choosing between different AI tools, consider models that have been trained to be more cautious, even if they sound less polished. A tool that says "I don't know" is occasionally more valuable than a tool that sounds good.

The future of AI probably involves models that are better at knowing what they don't know. But we're not there yet. For now, we're living with systems that were optimized for one thing—sounding coherent—and we're trusting them to do something else entirely: being right. It's a mismatch that demands awareness and skepticism from everyone using these tools.

Confidence is a performance, not a guarantee. In AI, it always is.