Vendor evaluation that doesn't get gamed
Choose candidates using critical rules before average scores.
Reviewed: 2026-10-03
Source-based teaching example · Fictional data
Narration: Microsoft Mark · locally synthesized

00:00:00.000 — What you will learn

Choose candidates using critical rules before average scores.

Visual description:
Choose candidates using critical rules before average scores.

00:00:06.833 — The idea

Define important failure rules before testing vendors. Keep every failed critical case visible. A candidate that passes the task test can still need operational evidence.

Visual description:
Define important failure rules before testing vendors. Keep every failed critical case visible. A candidate that passes the task test can still need operational evidence.

00:00:21.000 — One useful term

Critical gate. A required rule whose failure prevents an unconditional recommendation.

Visual description:
A required rule whose failure prevents an unconditional recommendation.

00:00:29.667 — A worked example

A answers 18 of 20 cases correctly but invents both critical deadlines. B answers 17 correctly and declines to guess those deadlines. A fails the predeclared rule.

Visual description:
A answers 18 of 20 cases correctly but invents both critical deadlines. B answers 17 correctly and declines to guess those deadlines. A fails the predeclared rule.

00:00:44.458 — Your turn to think

Should A win because its average is higher? Pause and decide before continuing.

Visual description:
Should A win because its average is higher?
Pause the video if you need more time.

00:01:00.292 — Why this answer works

No. B is only a conditional candidate while review effort and operational evidence remain unknown.

Visual description:
No. B is only a conditional candidate while review effort and operational evidence remain unknown.

00:01:08.875 — Check 1

Apply gates before scores.

Visual description:
Apply gates before scores.

00:01:13.750 — Check 2

Retain critical failures.

Visual description:
Retain critical failures.

00:01:18.542 — Check 3

Show missing lifecycle evidence.

Visual description:
Show missing lifecycle evidence.

00:01:23.625 — Now try a different case

Open the lesson and try its different practice case. Use every check in the full task. Watching alone does not mark the lesson complete.

Visual description:
Open the lesson
Try the new case
Check and revise
The full task and its requirements are in the linked lesson.