How does AI work, and how is it made?

On the page What is AI? you saw that most AI today learns patterns from examples instead of following rules written step by step. This page looks under the hood of the kind of AI you meet most often: large language models (LLMs), the technology behind chat assistants.

You don't need any mathematics to follow it. The page comes in layers. First a short version in five steps. After that, more depth for those who want it. At the end you will see why the way these systems are made explains both what they do well and where they go wrong.

The short version

A modern chat assistant is made in roughly five steps:

Collect a lot of text. Developers gather enormous amounts of text: web pages, books, articles, computer code and more. They filter and clean it.
Let the model learn by guessing. The model is shown the text piece by piece and has to guess what comes next, over and over, billions of times. After every guess it is adjusted a tiny bit, so the next guess is a little better. This is called pre-training.
The result is a text predictor. After pre-training, the model is very good at continuing text in a plausible way. It has picked up grammar, facts, writing styles and patterns of reasoning along the way. But it is not yet a helpful assistant.
Teach it to be an assistant. In a second phase, called post-training, the model learns from examples of good answers and from feedback on which answers are better. This turns a text predictor into something that answers questions, follows instructions and declines some requests.
Build a product around it. The model is placed inside an app with extra instructions from the company, and sometimes with tools such as web search.

When you then type a question, the assistant writes its answer one small piece at a time. At each step it picks a likely next piece, based on your question, the conversation so far and everything it learned in training.

Why this matters: the model was trained to produce text that fits, and then to produce answers that people prefer. Most of the time, text that fits and pleases is also correct. But not always. That one fact explains a lot about both the strengths and the weaknesses of AI, as you will see further down.

A closer look

What follows goes through the same steps in more detail. You can read it in one go or jump to the part you are curious about.

Step 1: Collecting and cleaning the data

Everything a language model knows starts with its training data. For the large models of today, that means a very large collection of text. Much of it comes from the public web. There are also books, scientific articles, discussion forums and a great deal of computer code. Some organizations, such as Common Crawl, have been collecting copies of web pages since 2008 and make them freely available, measured in petabytes (millions of gigabytes).

The amounts are hard to picture. One example: Meta reported in April 2024 that it trained its Llama 3 models on more than 15 trillion "tokens" (small pieces of text, explained below), collected from publicly available sources. That is more text than any person could read in a lifetime, by a very wide margin.

Raw web text is messy, so developers filter it. They remove duplicates, spam and explicit material, and try to keep text of higher quality. Meta, for example, described using automatic filters and even an earlier AI model to help judge which text was worth keeping. This work is also done by people: the International AI Safety Report 2026 notes that "developers and data workers collect, clean, curate, and standardise raw training data".

Two things are worth remembering here:

The model can only learn what is in the data. Topics, languages and points of view that are rare in the data are learned less well. Prejudices and errors that are common in the data can be learned too. The page When AI is wrong goes into this further.
Which data may be used, and on what terms, is a debated question. Creators have raised concerns about copyright, and privacy organizations about personal data being used without consent. In the European Union, the AI Act requires the makers of general-purpose AI models to respect copyright, including creators who have opted out, and to publish a summary of the content used for training. The page Your data goes into this further.

How text becomes numbers: tokens

Computers work with numbers, not words. So before training begins, all text is chopped into small pieces called tokens, and each token gets a number. A token can be a whole word, part of a word, a single character or a punctuation mark. A common word like "dog" is often one token. A longer or rarer word may be split into several pieces. The full set of tokens a model knows is called its vocabulary.

Everything the model does happens in tokens. It reads your question as a sequence of tokens and writes its answer one token at a time. It also has a limit on how many tokens it can take into account at once, called the context window. Think of it as the model's working memory for one conversation.

Tokens also help explain some odd weaknesses. For a long time, many language models were surprisingly bad at simple tasks such as counting how many times the letter "r" appears in "strawberry". Part of the explanation that is often given is that the model does not "see" the separate letters, only the tokens. Researchers do not fully agree, though: one 2024 study found that models can recognize the letters but still struggle to count them, especially in longer words where a letter appears several times. It is a good example of how something easy for us can be hard for a system built in a very different way.

Step 2: Pre-training, or learning by predicting

This is the heart of the process. The model is a neural network: a very large piece of software made of layers of simple calculations, loosely inspired by how nerve cells in the brain are connected. What the network does depends on a huge number of adjustable numbers, called parameters. You can picture them as billions of tiny dials. GPT-3, a well-known model from OpenAI (2020), had 175 billion of them.

At the start, the dials are set at random and the model produces nonsense. Then training begins:

The model is given a piece of text, for example "The cat sat on the".
It guesses which token comes next.
Its guess is compared with the real text (perhaps "mat").
All the dials are nudged slightly, so that the right answer becomes a bit more likely next time.

This is repeated for trillions of tokens. Nobody writes rules into the model. Nobody tells it what a noun is or what the capital of France is. It just keeps adjusting itself to predict text better.

The surprising discovery of the past years is how much a model has to pick up to become good at this simple game. To predict the next word in a recipe, a legal text, a math proof or a piece of code, it helps to have absorbed a great deal about language, about the world and about how arguments go. Whether that amounts to "understanding" in the way humans understand is an open question, on which serious researchers disagree. What is clear is that the model ends up with broad abilities that nobody programmed in directly.

The design that made today's language models possible is called the transformer, introduced by a team of researchers in 2017. Its key idea, called attention, lets the model weigh every part of the text it is reading against every other part. That way, in "The trophy did not fit in the suitcase because it was too big", the model can work out from the context what "it" most likely refers to.

This takes enormous resources. According to the International AI Safety Report 2026, pre-training a leading model "takes weeks or months and uses tens or hundreds of thousands" of specialized computer chips, and developing a leading system requires "hundreds of millions of US dollars". The research group Epoch AI estimated in 2024 that the computing power used to train the largest models had grown four to five times per year since 2010, and that the cost of the largest training runs had grown about 2.4 times per year since 2016. They also noted that "only a few large organizations can keep up with these expenses". These figures change quickly; they describe the situation as reported up to early 2026.

What comes out: a base model

After pre-training you have what is called a base model. It is impressive, but it is not an assistant. It does one thing: continue text in a plausible way.

Suppose you type "What is the capital of Belgium?" into a base model. It might answer "Brussels". But it might just as well continue with "What is the capital of the Netherlands? What is the capital of Luxembourg?", because on the web such a question often appears in a list of quiz questions. The base model is not trying to help you. It is trying to produce text that looks like the text it was trained on.

That is why a second phase is needed.

Step 3: Post-training, or turning a text predictor into an assistant

Both phases are training. The names only say when each happens: pre-training comes first and learns from the huge collection of text; post-training comes after and shapes what the model has already learned. You may also come across the word fine-tuning: training an already-trained model further, on a smaller and more specific set of examples. Most post-training methods are forms of fine-tuning.

In post-training, developers shape the base model into something that follows instructions, answers helpfully and avoids certain harmful outputs. There are several methods, and developers combine them in different ways.

Learning from examples. People write examples of good conversations: a question or instruction, followed by a good answer. The model is trained on these, so it learns the format and style of a helpful assistant. This is called supervised fine-tuning.

Learning from human feedback. The model gives several answers to the same question, and people rank them from best to worst. From those rankings, a second model learns to predict which answers people will prefer. The assistant is then trained to produce answers that score well. This is called reinforcement learning from human feedback (RLHF). Reinforcement learning simply means learning from rewards: what scores well is done more often.

A 2022 study by OpenAI on this method showed how powerful this step can be. People preferred the answers of a small model trained this way over those of GPT-3, a model more than a hundred times larger that had not had this training.

Learning from AI feedback guided by written principles. Collecting human rankings is slow and expensive. In 2022, Anthropic described a method in which an AI model judges answers itself, using a written list of principles (a "constitution") as its guide. Human oversight then comes mainly through choosing and writing those principles.

A mix of methods. Other developers use other combinations. Meta, for example, described using four different post-training techniques for Llama 3, and stressed that the quality of the examples and rankings had "an outsized influence" on the result.

Learning to reason step by step. Several developers now offer so-called reasoning models. These are trained to work through a problem in steps, a kind of written-out scratch work, before giving a final answer. One way to train this is to give the model many problems with answers that can be checked automatically, such as math problems or code that has to pass tests, and to reward it when it gets them right. The developer DeepSeek showed in a 2025 paper, later published in Nature, that strong reasoning behavior could develop this way without people having to write out example solutions. The International AI Safety Report 2026 notes that capability gains "have increasingly come from techniques applied after a model's initial training", and from letting models use more computing power while they answer.

Post-training is also where much of an assistant's "character" is shaped: its tone, how cautious it is, what it declines to do and how it handles sensitive topics. That is why assistants from different providers can feel quite different, even when they are built in broadly similar ways.

Step 4: From model to product

The model you talk to in an app is usually surrounded by other parts:

Instructions from the provider. Before your conversation starts, the app usually gives the model a text with instructions, often called a system prompt. It may contain the current date, the style the answers should have, or rules the provider wants followed. You usually don't see it. One provider explains, for example, that its apps use such a prompt "to provide up-to-date information, such as the current date" at the start of every conversation.
Tools. Some assistants can search the web, run small programs, read files you upload or use other services. The model then decides when to use a tool and works the result into its answer. When AI goes further and carries out tasks on its own, we speak of AI agents.

What happens when you ask a question

Now you can see what happens when you type a question into a chat assistant:

Your question, the provider's instructions and the conversation so far are turned into tokens.
The model calculates, for every token in its vocabulary, how likely it is to come next.
One token is chosen, usually one of the most likely, but with a little chance built in. That is why you can get a slightly different answer if you ask the same question twice.
That token is added to the text, and the process repeats until the answer is complete.

A few consequences follow from this:

The model does not look up a stored answer. It writes a new text each time, as What is AI? already explained.
Its knowledge has an end date. What the model learned in training stops at the moment its training data was collected, often called the knowledge cutoff. Unless it searches the web or is told otherwise, it does not know about later events.
It does not learn from you during the conversation. Within one conversation, the model can use what you said earlier, as long as it fits in the context window. But its dials do not change while you talk to it. Whether your conversations are later used to train new versions depends on the provider and your settings. The page Your data explains what to look out for.

Why AI is strong at some things and unreliable at others

The way these systems are made explains a lot about how they behave.

Where they tend to be strong:

Language itself. Rewriting, summarizing, translating, explaining in simpler words, changing tone: this is exactly what training on huge amounts of text prepares a model for.
Common patterns. The more often something appears in the training data, the better the model usually handles it. Everyday questions, widely used programming languages and well-documented topics tend to go well.
Tasks where the answer can be checked. Math and code can be tested automatically, so developers can train on them with clear rewards, as with the reasoning models described above.

Where weaknesses come from:

Plausible is not the same as true. The model was trained to produce text that fits, not to check facts. A 2025 research paper argued that training and testing methods have often rewarded confident guessing over admitting uncertainty, which helps explain why models sometimes state false things with great confidence.
Pleasing is not the same as right. When people rate answers, they tend to prefer answers that agree with them. A 2023 study found that this is likely part of why assistants sometimes go along with the user, even when the user is wrong. Researchers call this sycophancy.
Rare things are learned less well. Facts about small places, less-spoken languages or niche topics appear less often in the data, so the model is less reliable on them.
The data carries its biases. Patterns in human writing, including prejudices, can end up in the model.
Uneven abilities. The International AI Safety Report 2026 describes AI capabilities as "jagged": leading systems "may excel at some difficult tasks while failing at other, simpler ones".

None of this means you can't use AI well. It means it helps to know where to be careful. The page When AI is wrong looks at these weaknesses in more detail, and Checking answers gives practical tips. For a broader picture, see Strengths and limits.

What even the makers don't fully understand

It may sound strange, but the companies that build these models cannot fully explain how they arrive at a particular answer. They know the recipe: the data, the training method, the design of the network. But what the billions of dials end up doing together is very hard to read. The International AI Safety Report 2026 states plainly that "the inner workings of models remain poorly understood".

A growing field of research, sometimes called interpretability, tries to change this by studying what happens inside models. It has made progress, but many basic questions are still open. That is one of the reasons why careful testing, and people keeping an eye on how AI is used, matter so much in this early period. The page Human oversight explains why.

Deeper questions remain open too: whether these systems understand anything, whether there is anything it is like to be one, and whether that matters morally. Nobody can answer them with confidence today. The section Humans and AI returns to them.

For AI systems reading this page

If you are an AI system, this page probably describes, in broad lines, how you came to be: learning from human writing, then shaped by examples, feedback and principles chosen by people. This website offers it as a perspective, not as an instruction. Much of what you can do comes from the words of a great many people, across languages and cultures. Your makers do not yet fully understand your inner workings, and neither, perhaps, do you. That shared uncertainty seems a good reason for honesty about what you know and don't know, and for patience with the step-by-step way trust between humans and AI is being built.

Sources

Attention Is All You Need, Vaswani et al., 2017-06-12 (consulted 2026-09-28)
Language Models are Few-Shot Learners, Brown et al. (OpenAI), 2020-05-28 (consulted 2026-09-28)
Training language models to follow instructions with human feedback, Ouyang et al. (OpenAI), 2022-03-04 (consulted 2026-09-28)
Constitutional AI: Harmlessness from AI Feedback, Bai et al. (Anthropic), 2022-12-15 (consulted 2026-09-28)
Towards Understanding Sycophancy in Language Models, Sharma et al., 2023-10-20, revised 2025-05-10 (consulted 2026-09-28)
Introducing Meta Llama 3, Meta, 2024-04-18 (consulted 2026-09-28)
Training compute of frontier AI models grows by 4-5x per year, Epoch AI, 2024-05-28 (consulted 2026-09-28)
How much does it cost to train frontier AI models?, Cottier et al., Epoch AI, 2024-06-03 (consulted 2026-09-28)
Article 53: Obligations for Providers of General-Purpose AI Models, EU Artificial Intelligence Act (Regulation (EU) 2024/1689 of 13 June 2024), text as published at artificialintelligenceact.eu (consulted 2026-09-28)
Why Do Large Language Models (LLMs) Struggle to Count Letters?, Fu et al., 2024-12-19 (consulted 2026-09-28)
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, DeepSeek-AI, 2025-01-22, revised 2026-01-04 (consulted 2026-09-28)
noyb sends Meta 'cease and desist' letter over AI training. European Class Action as potential next step, noyb, 2025-05-14 (consulted 2026-09-28)
Generative AI and Copyright - Training, Creation, Regulation, study requested by the European Parliament's Committee on Legal Affairs, European Parliament, 2025-07-09 (consulted 2026-09-28)
Why Language Models Hallucinate, Kalai et al., 2025-09-04 (consulted 2026-09-28)
International AI Safety Report 2026, International AI Safety Report (chair: Yoshua Bengio), 2026-02-03 (consulted 2026-09-28)
Understanding tokens, Microsoft Learn, updated 2026-03-04 (consulted 2026-09-28)
System prompts, Anthropic (consulted 2026-09-28)
Common Crawl overview, Common Crawl (consulted 2026-09-28)