Watch the explanation. Practice the skill.
LLM internals: how transformers work
Explain how a text model processes pieces of a prompt.
Captions appear here during playback.
LLM internals: how transformers work
Explain how a text model processes pieces of a prompt.
Video playback is unavailable. Read the descriptive transcript below, go directly to practice, or open the video file.
Source-based teaching example · Fictional data · Reviewed 2026-10-03. Microsoft Mark · locally synthesized. On a small screen, use fullscreen for diagram details or read the transcript.
Read the descriptive transcript
0:00 · What you will learn
Explain how a text model processes pieces of a prompt.
What appears on screen
Explain how a text model processes pieces of a prompt.
0:06 · The idea
A text model processes tokens, which are pieces of text. In causal attention, each position can use earlier positions and itself, but not future ones. This describes an architecture, not a promise of perfect recall.
What appears on screen
A text model processes tokens, which are pieces of text. In causal attention, each position can use earlier positions and itself, but not future ones. This describes an architecture, not a promise of perfect recall.
0:24 · One useful term
Token. A piece of text represented by an ID for the model; it is not always a whole word.
What appears on screen
A piece of text represented by an ID for the model; it is not always a whole word.
0:34 · A worked example
With four causal tokens, the positions can use one, two, three and four positions respectively. That gives ten allowed query-and-key pairs. It does not allow looking at future tokens.
What appears on screen
With four causal tokens, the positions can use one, two, three and four positions respectively. That gives ten allowed query-and-key pairs. It does not allow looking at future tokens.
0:49 · Saved attention data
The model can reuse saved keys and values while generating. This is called a key-value cache. The lesson estimates that data for one sequence.
What appears on screen
Key-value cache: saved attention data reused during generation.
1:02 · A small memory calculation
For this toy case, multiply the six factors shown. The result is six hundred forty bytes. This estimates the key-value data for one sequence, not all memory used to serve the model.
What appears on screen
2: K + V 2 layers 2 KV heads 4 dimensions 2 bytes/value 10 tokens 2 × 2 × 2 × 4 × 2 × 10 = 640 bytes · simplified, one sequence
1:18 · Your turn to think
Can a position use a future token in this causal example? Pause and decide before continuing.
What appears on screen
Can a position use a future token in this causal example? Pause the video if you need more time.
1:35 · Why this answer works
No. The lesson’s simplified memory calculation is separate from total serving memory. Other architectures and undisclosed provider designs may differ.
What appears on screen
No. The lesson’s simplified memory calculation is separate from total serving memory. Other architectures and undisclosed provider designs may differ.
1:48 · Check 1
Keep the causal attention limit.
What appears on screen
Keep the causal attention limit.
1:53 · Check 2
Check memory units and assumptions.
What appears on screen
Check memory units and assumptions.
1:58 · Check 3
Do not assume every model has the same internals.
What appears on screen
Do not assume every model has the same internals.
2:04 · Now try a different case
Open the lesson and try its different practice case. Use every check in the full task. Watching alone does not mark the lesson complete.
What appears on screen
Open the lesson Try the new case Check and revise The full task and its requirements are in the linked lesson.
Watching explains the example. Complete the lesson practice to check what you can do. Explore all visual lessons →