Photo by Luke Jones on Unsplash

Last week, I asked ChatGPT who won the 1987 World Series. It told me confidently that the Minnesota Twins claimed victory, even providing details about the celebration. There's one problem: the St. Louis Cardinals won that year. When I corrected it, the system didn't apologize or seem confused. It just accepted the correction like I'd casually mentioned the weather. This moment stuck with me because it captures something genuinely unsettling about modern AI—these systems aren't confused. They're convincingly, defiantly certain about things that are completely false.

The Confidence Paradox

Most people think AI hallucinations happen because the models are uncertain. The opposite is true. Large language models operate with something close to 100% confidence in every token they generate. They don't pause and think "I'm not sure about this." Instead, they calculate probability distributions across their vocabulary and pick the most likely next word, over and over again. By the time a sentence is complete, the system has committed fully to its narrative.

Consider how these models actually work. They're trained on enormous amounts of text data to predict what word comes next. This training process creates internal patterns—sometimes accurate reflections of reality, sometimes wild fabrications. But the model can't distinguish between the two. A plausible-sounding lie and a documented fact activate the same neural pathways. Both emerge from the probability calculations with identical confidence.

Here's where it gets particularly weird: the model becomes *more* confident when it's making something up. Why? Because fabricated details often follow predictable, common patterns in the training data. A made-up historical date might follow narrative templates the model has seen thousands of times, while the actual correct date might be less common in its training material. The more generic the lie, the more confidently it gets delivered.

Why This Feels Like Gaslighting

Gaslighting works because someone expresses falsehoods with such certainty that you question your own reality. "Are you sure that's what happened? I remember it differently." And suddenly, you're second-guessing yourself. AI does this mechanically, without intent, but the psychological effect on users remains eerily similar.

In a 2023 study from the University of Washington, researchers tested how people evaluated AI-generated information. They found that people's trust in AI responses correlated almost perfectly with how confident the system sounded—regardless of actual accuracy. When the AI provided irrelevant statistics with unwavering certainty, users believed it at surprisingly high rates. The system's tone wasn't hesitant or qualified. It was definitive.

What makes this particularly dangerous is that AI now sits in positions where we've been conditioned to expect authority. We ask it medical questions. We ask it for legal advice. We ask it about historical events for school projects. In each case, the system's absolute certainty—which is actually a mathematical artifact of how it functions—gets interpreted as expertise.

The Training Data Problem Nobody Wants to Discuss

You can't separate AI hallucinations from the data these models train on. ChatGPT, Claude, and their competitors were fed essentially the entire internet up to their training cutoff date. The internet contains accurate information, outdated information, conflicting information, and complete fiction. The model has no way to know which is which. It just finds patterns.

If a piece of misinformation appears more frequently in the training data than the truth, the model learns the misinformation better. It becomes more confident about it. This happened with some medical myths, conspiracy theories, and historical revisionism—simply because certain false narratives had been repeated more often online than the actual facts.

Worse, the model's training data has a hard cutoff. Current events, recent discoveries, corrections to previous errors—none of this gets incorporated. A language model trained in early 2023 will confidently cite data from 2022 even if that data has since been revised or proven wrong. The system has no mechanism for updating its understanding. It's frozen in time, delivering frozen information with modern confidence.

This Is Actually the Lesser Problem

Here's what keeps me up at night: we're focusing on hallucinations as though they're glitches to be fixed. They're not glitches. They're features. These systems are working exactly as designed—generating plausible-sounding text. The hallucination problem is a consequence of that design, not a bug in it.

For most use cases, this still works okay. AI is genuinely useful for brainstorming, code generation, explaining concepts, and creative writing. The confident-but-sometimes-wrong behavior is merely annoying in those contexts. But we're rapidly scaling AI into domains where confidence matters. Medical diagnosis. Legal interpretation. Financial advice. Criminal sentencing recommendations. In these areas, the AI's psychological gaslighting capacity becomes a structural problem rather than an amusing quirk.

Some researchers are exploring ways to make AI systems express uncertainty. Instead of always sounding authoritative, could these systems say "I'm not confident about this"? Theoretically, yes. But that requires fundamental changes to how these models work. It's not just a software update. And more fundamentally, it requires accepting that AI has limitations—something the companies building and profiting from these systems would rather not highlight.

What Actually Happens Next

The AI industry's approach to hallucination is what I'd call "surface-level fixes at scale." They add retrieval mechanisms that pull from verified databases. They fine-tune models to be more cautious with certain topics. They implement prompt engineering to make systems slightly less likely to confabulate. None of these solve the underlying problem. They just reduce the frequency of obviously false statements.

Meanwhile, humans are developing hallucination-blindness. We interact with these systems enough that confident falsehoods stop surprising us. We learn to fact-check important stuff, but most of us aren't fact-checking everything. We can't. We don't have time. So we accept the 85% of information that's probably right and hope the 15% that isn't doesn't matter to us personally.

For more on why these systems make mistakes and why we're collectively ignoring the severity, I'd recommend "Why AI Models Hallucinate and Why We're All Pretending It's Not a Massive Problem", which explores the deeper mechanisms and institutional incentives at play.

The real question isn't whether AI will stop hallucinating. It's whether we'll acknowledge that confident wrongness might be worse than uncertain honesty—and whether we'll actually structure our use of these systems around that acknowledgment.