Cost, latency, and trade-offs
Measure response time and usage without hiding retries or unknowns.
Reviewed: 2026-10-03
Source-based teaching example · Fictional data
Narration: Microsoft Mark · locally synthesized

00:00:00.000 — What you will learn

Measure response time and usage without hiding retries or unknowns.

Visual description:
Measure response time and usage without hiding retries or unknowns.

00:00:07.042 — The idea

Record what work was performed and how long it took. Include repeated attempts. Use the declared rate and units, and do not treat missing usage as zero.

Visual description:
Record what work was performed and how long it took. Include repeated attempts. Use the declared rate and units, and do not treat missing usage as zero.

00:00:20.375 — One useful term

Latency. The time taken for a response or task, measured from a defined starting point.

Visual description:
The time taken for a response or task, measured from a defined starting point.

00:00:29.583 — A worked example

The exercise uses an invented rate card. A retry repeats work even if the first answer was unusable. Ten timing samples also cannot reliably describe very rare slow responses.

Visual description:
The exercise uses an invented rate card. A retry repeats work even if the first answer was unusable. Ten timing samples also cannot reliably describe very rare slow responses.

00:00:44.917 — Your turn to think

Does a failed answer mean the first attempt used no resources? Pause and decide before continuing.

Visual description:
Does a failed answer mean the first attempt used no resources?
Pause the video if you need more time.

00:01:02.250 — Why this answer works

No. Record performed work and any unknown usage. Keep toy calculations separate from actual provider bills.

Visual description:
No. Record performed work and any unknown usage. Keep toy calculations separate from actual provider bills.

00:01:13.292 — Check 1

Include retries in measured work.

Visual description:
Include retries in measured work.

00:01:18.458 — Check 2

State the rates, units and sample size.

Visual description:
State the rates, units and sample size.

00:01:24.583 — Check 3

Compare models with the same surrounding workflow.

Visual description:
Compare models with the same surrounding workflow.

00:01:30.500 — Now try a different case

Open the lesson and try its different practice case. Use every check in the full task. Watching alone does not mark the lesson complete.

Visual description:
Open the lesson
Try the new case
Check and revise
The full task and its requirements are in the linked lesson.