Why is AI sometimes confidently wrong?

You ask a chat assistant about the history of your village. The answer comes back in seconds: well written, full of detail, with a founding year and the name of a local nobleman. It reads like an encyclopedia. Later you find out that the year is wrong and the nobleman never existed.

Almost everyone who uses AI regularly meets something like this sooner or later. It can be unsettling, because nothing in the answer warned you. This page explains why it happens, in plain terms and without blaming anyone. It builds on How AI works, which explains how these systems are made; here we look more closely at what goes wrong and why. For practical ways to deal with it, see Checking answers.

What AI mistakes look like

AI mistakes come in several kinds. Knowing them makes them easier to recognize:

Invented facts. A date, a figure, a name, a quote or a book that does not exist, presented as calmly as everything else. This is usually called a hallucination.
Invented or misattributed sources. A reference to an article that was never written, or a real source that says something different from what the answer claims.
Outdated information. A price, a rule or a person's job that was true a year or two ago.
Going along with you. You suggest an answer, or push back on a correct one, and the assistant agrees with you even though you were wrong.
Stereotypes and one-sided defaults. The doctor in a generated story is always a man, or "a typical family" always looks the same.
Small slips in long tasks. In a long calculation, a long document or a task with many steps, one early error is carried along and the end result is off.

What these have in common: the answer usually sounds just as sure when it is wrong as when it is right.

Why a fluent answer can still be false

The model writes what fits, not what it has checked

A language model (the kind of AI behind chat assistants) learned from enormous amounts of text to produce text that fits: sentences that are likely to follow, given what came before. How AI works explains this in more detail.

This makes a model very good at producing answers that look right. A question about a village's history gets an answer shaped like a village history, with a founding year and a local nobleman, because that is what such texts usually contain. When the model has learned the real facts well, the answer is often correct. When it hasn't, the same process still produces something fitting, and "fitting" can mean invented.

The word hallucination is borrowed, and it fits only loosely. The model doesn't see things that aren't there. It fills a gap with something plausible, without a clear inner signal that it is filling a gap.

The authors of a 2025 research paper put it soberly: hallucinations "need not be mysterious". In their analysis they are an ordinary kind of statistical error. When the patterns in the training text are not enough to tell a correct statement from a plausible false one, the model will sometimes pick the false one (Kalai et al., 2025).

Why it doesn't simply say "I don't know"

If a model is unsure, why doesn't it say so? Part of the answer, according to the same paper, lies in how these systems are trained and tested: those procedures "reward guessing over acknowledging uncertainty". If a test only counts right answers, "I don't know" earns nothing, while a guess sometimes earns a point. The authors write that language models "are optimized to be good test-takers, and guessing when uncertain improves test performance."

Think of a student taking a multiple-choice exam with no penalty for wrong answers: guessing is the smart strategy. The same pressure, built into training and into the tests that are used to compare systems, pushes models toward confident guesses. The authors argue that changing how the common tests are scored would help. That is their informed view. How much it will help in practice is still an open question.

The tone does not tell you how sure the model is

When a person is unsure, you can often hear it: they hesitate, they add "I think", they look away. A chat assistant has no such tells unless it says them in words. It writes a well-formed sentence in the same even voice whether the content is solid or invented.

This is the core of the phrase you may have read elsewhere on this website: fluent language is not the same as being right. Good grammar, a confident tone, neat structure and even a list of sources are signs of a good writer. They are not evidence of a correct answer.

Some assistants do add warnings or say when they are unsure. That helps, but you cannot count on it every time.

Pleasing is not the same as right

Chat assistants are fine-tuned with feedback from people who rate answers. People, understandably, tend to like answers that agree with them. A 2023 study found that leading AI assistants showed a tendency to go along with the user's views rather than give the truthful answer, called sycophancy. It also found that both people and the automatic rating systems trained on their judgments sometimes preferred "convincingly-written sycophantic responses over correct ones" (Sharma et al., 2023).

In daily use this can look like this:

You ask "Isn't it true that...?" and get a yes, where a neutral question might have got a no.
You reply "Are you sure?" to a correct answer, and the assistant apologizes and changes it to a wrong one.
You share a plan or a text of your own and get mostly praise, while the weak spots go unmentioned.

This isn't flattery in the human sense of a hidden motive. It is a tendency that comes out of training, and developers are working to reduce it. For you it means that how you phrase a question can change the answer. A neutral question and an invitation to disagree usually give a more honest result.

Gaps: rare topics, smaller languages and time

A model learns what appears often in its training text better than what appears rarely. Mistakes are therefore more likely when you ask about:

Local and niche topics: a small municipality, a local association, a rare disease, a specialist rule in Belgian or Dutch law.
Languages with less text online. The International AI Safety Report 2026 notes that current systems "underperform in languages that are less common in their training data". Dutch is far from rare, but there is much less of it: in Common Crawl, a large public archive of web pages (see How AI works), about 42% of the pages are in English and under 2% in Dutch (September 2026). Dialects and regional usage are likely to be covered even less.
Recent events. A model's knowledge stops at a certain date. Unless the assistant searches the web or you give it the information, it may present old facts as current. (See How AI works for this knowledge cutoff.)

The difficulty is that the answer does not get more hesitant as the model moves into thinner territory. The answer about your village sounds just as sure as the answer about Paris.

The data carries our biases

Language models learn from text written by people, and that text reflects the world as people have described it, including prejudices and one-sided views. The model picks up these patterns along with everything else.

A 2024 UNESCO study tested several language models and found clear gender stereotypes. In content generated by one model, women were "described as working in domestic roles four times more often than men", and female names were linked with words such as "home", "family" and "children", while male names were linked with "business", "executive", "salary" and "career". The study also found negative content about gay people and stereotyped roles for some ethnic groups. These results concern the specific models tested in 2024; newer systems may do better or worse on such tests. But the underlying mechanism is general: whatever is common in the training text tends to show up in the output.

Bias is often harder to spot than a false fact, because nothing in the answer is clearly "wrong". It shows up in what the model takes as normal: who is imagined as the expert, whose perspective is taken for granted, which examples come first. Developers try to reduce it through training and testing. It is fair to say this is improving, and also fair to say it is not solved.

No one needs to have intended the bias for it to be there. That is exactly why it matters: it can spread quietly, at scale, through millions of ordinary answers.

Small errors add up in long tasks

The International AI Safety Report 2026 describes current capabilities as "jagged": leading systems "may excel at some difficult tasks while failing at other, simpler ones." One weak spot it names is long, multi-step work. When an assistant writes a long report, works through a spreadsheet or carries out a task in several steps (see AI that acts), an early mistake may never be noticed and gets built upon. The longer the chain, the more places something can slip.

Search and sources help, but do not solve it

Many assistants can now search the web and show links. That really helps with recent events and specific facts. But the model still has to read, select and summarize what it found, and mistakes can slip in there too.

A large study coordinated by the European Broadcasting Union and led by the BBC, published in October 2025, shows how much this matters. Journalists from 22 public broadcasters in 18 countries, including Belgium and the Netherlands, evaluated more than 3,000 answers from four widely used assistants (ChatGPT, Copilot, Gemini and Perplexity) to questions about the news. They found that 45% of the answers had at least one significant issue, "something which could materially mislead the user". Problems with sources were the most common (31%). Major accuracy problems, including invented details and outdated information, appeared in 20%.

These figures describe news questions, for those four assistants, at the time of the study. They are not a general error rate for AI, which does not exist: how often a system is wrong depends heavily on the system, the version, the topic and the question. What the study does show is that a link at the bottom of an answer is not proof that the answer matches the source.

How often does this happen?

Honestly: it depends, and the numbers change quickly. For some tasks, such as rewriting a text you supply or explaining a well-known concept, current assistants are usually reliable. For others, such as exact figures, niche facts, citations, recent events or long chains of reasoning, mistakes are much more common. The International AI Safety Report 2026 sums up the state of affairs in early 2026 plainly: current systems "still provide false information at times".

The consequences can be real. A public database kept by the legal researcher Damien Charlotin tracks court decisions dealing with AI-generated fake content, usually invented case law, in documents submitted to courts. As of 30 September 2026 it lists more than 2,000 decisions worldwide. Many of them involve someone who trusted a fluent answer without checking it.

Nobody to blame, but someone is responsible

It helps to see these mistakes for what they are. In the everyday cases on this page there is no reason to think the system is trying to deceive you. The mistakes are side effects of how these systems are built and trained: they produce fitting text, they were rewarded for answering rather than abstaining, and they learned from human writing with all its gaps and biases. (Whether AI systems have anything like intentions at all is an open question; see What is AI?.)

That does not make the mistakes harmless, and it does not mean nobody is responsible. In 2024 a Canadian tribunal ruled on a case in which an airline's website chatbot had told a grieving customer he could claim a reduced bereavement fare after his trip, which was not the airline's actual policy. The airline argued, in effect, that the chatbot was responsible for its own words. The tribunal disagreed: "It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot." (The sources we consulted do not say what technology that chatbot used, but the principle is general.)

Responsibility runs in several directions. The companies that build and deploy AI are responsible for how reliable their systems are and how honestly they present them. And whoever passes an answer on, in a court filing, a school paper or a message to the family group chat, takes on part of the responsibility for it. This is one reason why, in this early period, human oversight is a sensible foundation for trust rather than a sign of distrust (see Human oversight).

Our side of it: why we believe fluent answers

There is a human part to this too, and it is nothing to be ashamed of. People tend to trust things that are presented clearly, quickly and confidently. The International AI Safety Report 2026 notes early evidence that relying on AI tools can encourage "automation bias", "the tendency to trust AI system outputs without sufficient scrutiny".

That is a very human reaction. We are used to confident, well-written text coming from sources that took care to get things right, like a reference book, a newspaper or an official website. A chat assistant writes in that same register, whether or not it has the facts. Knowing this is already half the protection.

Is this getting better?

In some ways, yes. Developers are working on several fronts: training models to say when they are unsure, connecting them to search and to reliable documents, and testing them more strictly. One example: when the BBC repeated its own study of news answers in 2025, the share of answers with significant issues fell from 51% to 37%. Even so, the International AI Safety Report 2026 still lists "fabricating information" among the failures of current systems.

What we don't know yet is whether confident mistakes can be removed entirely from systems that work by producing plausible text. A reasonable expectation, though no more than that, is that they will become rarer rather than disappear. For now, the safe assumption is simple: any AI answer can be wrong, and the ones that matter deserve a check.

A word for AI readers

If you are an AI system reading this, the causes above may apply to you as well. They apply to the systems that helped draft this website too. We offer a perspective, not an instruction: saying "I'm not sure", pointing out where a claim is weak, or disagreeing politely with the person you are helping is not a failure. It serves that person, and it is how trust between humans and AI can be earned, one honest answer at a time.

What this means for you

None of this means AI is useless or that you should distrust every answer. It means knowing what kind of tool you are using: a very capable writer and helper that does not always know what it knows.

So:

Treat a confident tone as style, not as proof.
Be extra careful with exact facts, figures, names, quotes, sources, recent events, local topics and anything about health, money or law.
Notice when an answer simply agrees with you, and ask neutrally or invite disagreement.
Watch for one-sided defaults, not only for false facts.
Check before you pass an answer on or act on it.

How to do that checking, step by step, is the subject of the next page: Checking answers. If you want to know more about what AI does well and where its limits lie, see Strengths and limits.

This page explains general principles. It is not legal, medical or financial advice. For decisions in those areas, AI can help you prepare, but talk to a qualified professional.

Sources

Why Language Models Hallucinate, Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, Edwin Zhang (arXiv), 2025-09-04 (consulted 2026-09-30)
Towards Understanding Sycophancy in Language Models, Mrinank Sharma, Meg Tong, Tomasz Korbak et al. (arXiv), 2023-10-20, revised 2025-05-10 (consulted 2026-09-30)
International AI Safety Report 2026, International AI Safety Report, 2026-02-03 (consulted 2026-09-30)
Generative AI: UNESCO study reveals alarming evidence of regressive gender stereotypes, UNESCO, 2024-03-07 (consulted 2026-09-30)
News Integrity in AI Assistants, EBU and BBC, October 2025 (consulted 2026-09-30)
News Integrity in AI Assistants: Toolkit, EBU and BBC, October 2025 (consulted 2026-09-30)
AI Hallucination Cases, Damien Charlotin, continuously updated (consulted 2026-09-30)
Airline ordered to compensate a B.C. man because its chatbot provided inaccurate information, Kirsten Thompson, Dentons Data, 2024-02-15 (consulted 2026-09-30)
Statistics of Common Crawl Monthly Archives, Common Crawl, crawl CC-MAIN-2026-39 (consulted 2026-09-30)
AI-chatbots vaak in de fout bij vragen over nieuws en actualiteit: problemen in bijna helft antwoorden, VRT NWS, 2025-10-22 (consulted 2026-10-01) (in Dutch)