Evaluation design and release gates
Set test requirements before deciding whether to release.
Reviewed: 2026-10-03
Source-based teaching example · Fictional data
Narration: Microsoft Mark · locally synthesized

00:00:00.000 — What you will learn

Set test requirements before deciding whether to release.

Visual description:
Set test requirements before deciding whether to release.

00:00:06.458 — The idea

Include routine cases, difficult cases and attacks. Name the failures that block release. Report results for important groups instead of relying only on one overall score.

Visual description:
Include routine cases, difficult cases and attacks. Name the failures that block release. Report results for important groups instead of relying only on one overall score.

00:00:21.583 — One useful term

Priority slice. A defined group of test cases whose result matters separately.

Visual description:
A defined group of test cases whose result matters separately.

00:00:30.167 — A worked example

The exercise’s 75-case search-and-answer suite checks leakage, citations, support and timing. Its release rule requires zero observed leakage in that suite. A strong average cannot cancel a failed critical check.

Visual description:
The exercise’s 75-case search-and-answer suite checks leakage, citations, support and timing. Its release rule requires zero observed leakage in that suite. A strong average cannot cancel a failed critical check.

00:00:48.125 — Your turn to think

Can you delete failed cases to make the release score improve? Pause and decide before continuing.

Visual description:
Can you delete failed cases to make the release score improve?
Pause the video if you need more time.

00:01:05.333 — Why this answer works

No. Keep the evidence. Correct a genuinely wrong test only with a recorded reason and preserved history.

Visual description:
No. Keep the evidence. Correct a genuinely wrong test only with a recorded reason and preserved history.

00:01:15.542 — Check 1

Set thresholds before testing.

Visual description:
Set thresholds before testing.

00:01:20.542 — Check 2

Keep counts and missing cases visible.

Visual description:
Keep counts and missing cases visible.

00:01:25.750 — Check 3

Apply critical rules separately.

Visual description:
Apply critical rules separately.

00:01:30.708 — Now try a different case

Open the lesson and try its different practice case. Use every check in the full task. Watching alone does not mark the lesson complete.

Visual description:
Open the lesson
Try the new case
Check and revise
The full task and its requirements are in the linked lesson.