![Cover image for [AI Guide #2] Why AI Hallucinates: How to Spot and Handle It](/assets/images/posts/c6145b61-e8d2-4983-b19d-795cb10e3577/ai-hallucination-1.jpg)
I asked AI to research sources, and it gave me a polished list with paper titles and authors. A search revealed that the papers did not exist. If you have used AI for a while, you have probably had a similar experience. This phenomenon, where AI presents false information as fact, is called hallucination. This installment explains why hallucinations occur, which answers to question, and how to reduce them.
Hallucinations are a side effect of how AI works, not a bug
As we saw in Part 1, AI is not a machine that finds the correct answer; it is a machine that generates a plausible next word. But plausibility and truth are different standards.
After the sentence “A representative paper on OO is,” an author’s name and a plausible paper title naturally follow. AI faithfully follows that natural pattern. If the paper exists, that is fortunate; but when accurate information is missing or vague in the training data, it creates a fake entry with perfect formatting. It has seen countless citation formats, so the format looks real while the content is fabricated.
The attitude is even more troublesome. People hedge when they speak about something they do not know, saying “perhaps” or “I’m not sure,” but AI states invented content in confident sentences. Because confidence and accuracy operate independently, tone cannot filter out hallucinations.
Efforts to reduce hallucinations are ongoing. Training AI to say it does not know when appropriate is also becoming more common. Still, it is hard to say that newer models are always less wrong. In evaluations published by developers, newer models have sometimes fabricated answers to questions about people and facts more often than earlier models. What reduces hallucinations more reliably than model generation is providing supporting sources, which we cover later. Either way, assuming that today’s AI does not make mistakes is the most dangerous approach.
4 types of answers to question
You cannot live by questioning every answer, so it is practical to know where hallucinations commonly occur.
1. Specific proper nouns and numbers. Paper titles, book page numbers, legal provision numbers, statistics, and case names. Their format may look plausible, but their existence is another matter. Sources are especially likely to be fabricated when you ask, “Can you provide the source?”
2. Recent information. Because of the knowledge cutoff discussed in Part 1, recent issues answered without web search may be outdated or fabricated. Prices, schedules, and personnel information are typical examples.
3. Narrow, obscure topics. Topics with abundant training data, such as famous history and widely used technologies, are relatively accurate. With topics that have little data, such as local information, internal terminology, and obscure people, AI is more likely to fill gaps with imagination.
4. Leading premises. If a question contains a false premise, such as “Tell me about the incident where King Sejong threw a smartphone,” AI may invent a story that fits the premise instead of correcting it. The more confidently a question is phrased, the more AI plays along. Today’s models tend to correct famous examples like this, but they still play along with less familiar topics.
How to respond: lower the cost of verification
The basic strategy for handling hallucinations is not “never trust AI,” but verify only information where being wrong would have serious consequences.
First, enable web search. Modern AI services offer a web search mode. Having AI answer based on search results greatly reduces hallucinations. More importantly, you can click the source link to check whether it leads to a real page. But an open link is not enough. You must also check whether the page actually says what the answer claims. Sometimes the link is real but the summary is wrong.
Second, provide the source material yourself. For example, attach the contract file when asking AI to summarize it. Making AI answer from material you provide, rather than its own memory, is the most reliable way to reduce hallucinations. We cover this method in depth in Part 4.
Third, challenge its confidence. For an important answer, ask, “Are you sure? If you are not certain, say you do not know.” If it is a hallucination, AI sometimes corrects itself. But this is not perfect either; it may retract a correct answer. Treat it only as a supporting measure.
Fourth, adjust verification to the use case. Hallucinations have limited impact in brainstorming or drafting. In areas where mistakes cause harm, such as contracts, medicine, law, and investment, use AI answers only as a starting point and always verify them against the original and with an expert.
Summary
- Hallucinations are a side effect of the principle of generating plausible sentences, not a malfunction.
- A confident tone is not evidence of accuracy.
- Be especially skeptical of proper nouns, numbers, sources, recent information, and obscure topics.
- Add evidence through web searches and attachments, and verify only important information.
The conclusion of the hallucination discussion is not “so do not use AI.” Understanding hallucinations shows where you can trust AI and where you cannot. Within those boundaries, the key to improving output quality is how you ask questions—in other words, prompting. The next installment covers four fundamentals of prompting that can make the same question produce different answers.