Watch the explanation. Practice the skill.
ML and deep learning: enough to be dangerous
Judge a classifier by its errors as well as its accuracy.
Captions appear here during playback.
ML and deep learning: enough to be dangerous
Judge a classifier by its errors as well as its accuracy.
Video playback is unavailable. Read the descriptive transcript below, go directly to practice, or open the video file.
Source-based teaching example · Fictional data · Reviewed 2026-10-03. Microsoft Mark · locally synthesized. On a small screen, use fullscreen for diagram details or read the transcript.
Read the descriptive transcript
0:00 · What you will learn
Judge a classifier by its errors as well as its accuracy.
What appears on screen
Judge a classifier by its errors as well as its accuracy.
0:06 · The idea
Count correct and incorrect predictions against known labels. Different mistakes can have different consequences. Training progress alone does not decide whether a system is suitable for use.
What appears on screen
Count correct and incorrect predictions against known labels. Different mistakes can have different consequences. Training progress alone does not decide whether a system is suitable for use.
0:21 · One useful term
Classifier. A model or rule that assigns an input to a category.
What appears on screen
A model or rule that assigns an input to a category.
0:29 · A worked example
The toy example has four cases: two correct predictions and two mistakes. Accuracy is one half. One missed positive and one false alarm can still matter differently to the task.
What appears on screen
The toy example has four cases: two correct predictions and two mistakes. Accuracy is one half. One missed positive and one false alarm can still matter differently to the task.
0:44 · Your turn to think
Does the same accuracy score mean two systems have the same risk? Pause and decide before continuing.
What appears on screen
Does the same accuracy score mean two systems have the same risk? Pause the video if you need more time.
1:01 · Why this answer works
No. Inspect which errors occurred and what each would cause. Keep test data separate from data used to fit the model.
What appears on screen
No. Inspect which errors occurred and what each would cause. Keep test data separate from data used to fit the model.
1:12 · Check 1
Check counts and the totals you divide by.
What appears on screen
Check counts and the totals you divide by.
1:17 · Check 2
Prevent information leaking from test data into training.
What appears on screen
Prevent information leaking from test data into training.
1:24 · Check 3
Judge error costs for the actual task.
What appears on screen
Judge error costs for the actual task.
1:29 · Now try a different case
Open the lesson and try its different practice case. Use every check in the full task. Watching alone does not mark the lesson complete.
What appears on screen
Open the lesson Try the new case Check and revise The full task and its requirements are in the linked lesson.
Watching explains the example. Complete the lesson practice to check what you can do. Explore all visual lessons →