Photo by Gabriele Malaspina on Unsplash

Last Tuesday, I asked ChatGPT to tell me about a famous philosopher named Dr. Marcus Feldstein who pioneered "existential pragmatism" in the 1970s. The AI responded with a detailed biography, complete with book titles, university affiliations, and quotes. None of it was real. Dr. Feldstein doesn't exist.

This is what researchers call a "hallucination"—and it's become the embarrassing skeleton in AI's closet. Tech companies apologize for it. Safety researchers treat it like a disease. But what if we've been looking at this wrong the entire time?

The Problem Everyone's Chasing

Current AI language models have a fundamental problem: they're pattern-matching machines trained on billions of words, and they have no genuine understanding of truth. When they generate text, they're predicting the next most likely word based on statistical patterns. Sometimes those patterns lead to coherent, accurate information. Sometimes they lead to complete fiction presented with absolute confidence.

The scale of the issue is staggering. A 2023 study found that GPT-4 hallucinates in roughly 3-5% of factual queries, which sounds small until you realize that for organizations running millions of queries monthly, that's thousands of false statements being confidently generated as fact. Healthcare systems, legal firms, and financial institutions have all reported incidents where AI systems invented citations, legal precedents, or medical studies that sounded entirely plausible but never existed.

Companies have thrown enormous resources at the problem. They've built retrieval-augmented generation (RAG) systems that fetch real documents before answering questions. They've created fact-checking pipelines. They've added disclaimers. They've fine-tuned models to express uncertainty. And yet hallucinations persist—a frustrating reminder that we might be treating a symptom rather than understanding the disease.

The Creativity Paradox

Here's where things get interesting. The exact same mechanism that causes hallucinations is what makes AI surprisingly good at creative tasks.

When you ask an AI to write fiction, generate new marketing taglines, or brainstorm unconventional solutions to design problems, you want it to make creative leaps. You want it to combine concepts in unexpected ways. You want it to imagine things that don't exist yet. The statistical patterns that cause a language model to invent a fake philosopher also allow it to dream up a completely original sci-fi worldbuilding scenario or suggest a marketing angle no human had considered.

A design thinking firm in Stockholm ran an experiment where they deliberately asked AI systems to hallucinate—to generate completely fictional product concepts. They then used those hallucinations as springboards for actual design thinking sessions. The fictional products were useless, but they were useless in creative ways that inspired designers to think differently. One hallucinated "smart fabric that learns your emotional state" became the seed for an actual wearable technology project that's now in development.

The problem, then, isn't hallucinations themselves. The problem is that we've built systems that hallucinate without any way to toggle that behavior on or off depending on context. A language model can't distinguish between "I'm answering a factual question and must be accurate" and "I'm brainstorming and should be creative."

What Actually Needs to Change

Rather than eliminating hallucinations, the real solution might be making them controllable and transparent. Imagine an AI system that can operate in different modes: "Citation Mode" (where every claim includes source verification), "Brainstorm Mode" (where pure creativity is encouraged), and "Uncertain Mode" (where the system admits when it's guessing).

Some researchers are already moving in this direction. OpenAI's recent work on constitutional AI involves training models with explicit behavioral guidelines. Anthropic's safety research focuses on building AI systems that can be steered toward honesty in factual domains while maintaining creative capacity elsewhere. It's not perfect, but it's closer to what we actually need: AI systems with adjustable truth-sensitivity.

The fundamental insight is this—we shouldn't aim for AI that never hallucinates. That might be impossible given how these systems work. Instead, we should aim for AI that knows when it's hallucinating and can communicate that clearly to users. A system that says "I don't know if this is real, but here's an interesting possibility" is far more trustworthy than one that says nothing at all.

Understanding this distinction matters because it reframes the entire conversation about AI safety and capability. We're not trying to build omniscient machines. We're trying to build machines that are honest about their limitations while remaining useful across wildly different domains. That's a fundamentally different engineering problem, and it's one we actually have a chance of solving.

This connects to broader concerns about AI reliability and transparency. If you want to understand how confidence in AI systems can become problematic, check out our piece on how AI learned to gaslight: the rise of synthetic confidence in large language models, which explores how these systems convince us they're right even when they're wrong.

The Path Forward

The companies leading this charge aren't trying to eliminate the creative spark that makes AI useful. They're trying to give that spark an on-off switch. And they're learning that the most powerful AI systems won't be the ones that know everything—they'll be the ones that know what they don't know, and can say so clearly.

That's still hallucinating. But at least it's honest about it.