LLM internals: how transformers work
Explain how a text model processes pieces of a prompt.
Reviewed: 2026-10-03
Source-based teaching example · Fictional data
Narration: Microsoft Mark · locally synthesized

00:00:00.000 — What you will learn

Explain how a text model processes pieces of a prompt.

Visual description:
Explain how a text model processes pieces of a prompt.

00:00:06.583 — The idea

A text model processes tokens, which are pieces of text. In causal attention, each position can use earlier positions and itself, but not future ones. This describes an architecture, not a promise of perfect recall.

Visual description:
A text model processes tokens, which are pieces of text. In causal attention, each position can use earlier positions and itself, but not future ones. This describes an architecture, not a promise of perfect recall.

00:00:24.958 — One useful term

Token. A piece of text represented by an ID for the model; it is not always a whole word.

Visual description:
A piece of text represented by an ID for the model; it is not always a whole word.

00:00:34.083 — A worked example

With four causal tokens, the positions can use one, two, three and four positions respectively. That gives ten allowed query-and-key pairs. It does not allow looking at future tokens.

Visual description:
With four causal tokens, the positions can use one, two, three and four positions respectively. That gives ten allowed query-and-key pairs. It does not allow looking at future tokens.

00:00:49.917 — Saved attention data

The model can reuse saved keys and values while generating. This is called a key-value cache. The lesson estimates that data for one sequence.

Visual description:
Key-value cache: saved attention data reused during generation.

00:01:02.542 — A small memory calculation

For this toy case, multiply the six factors shown. The result is six hundred forty bytes. This estimates the key-value data for one sequence, not all memory used to serve the model.

Visual description:
2: K + V
2 layers
2 KV heads
4 dimensions
2 bytes/value
10 tokens
2 × 2 × 2 × 4 × 2 × 10 = 640 bytes · simplified, one sequence

00:01:18.250 — Your turn to think

Can a position use a future token in this causal example? Pause and decide before continuing.

Visual description:
Can a position use a future token in this causal example?
Pause the video if you need more time.

00:01:35.333 — Why this answer works

No. The lesson’s simplified memory calculation is separate from total serving memory. Other architectures and undisclosed provider designs may differ.

Visual description:
No. The lesson’s simplified memory calculation is separate from total serving memory. Other architectures and undisclosed provider designs may differ.

00:01:48.208 — Check 1

Keep the causal attention limit.

Visual description:
Keep the causal attention limit.

00:01:53.042 — Check 2

Check memory units and assumptions.

Visual description:
Check memory units and assumptions.

00:01:58.292 — Check 3

Do not assume every model has the same internals.

Visual description:
Do not assume every model has the same internals.

00:02:04.292 — Now try a different case

Open the lesson and try its different practice case. Use every check in the full task. Watching alone does not mark the lesson complete.

Visual description:
Open the lesson
Try the new case
Check and revise
The full task and its requirements are in the linked lesson.