Watch the explanation. Practice the skill.
Production architecture and inference serving
Test service capacity under bursts, not just average demand.
Captions appear here during playback.
Production architecture and inference serving
Test service capacity under bursts, not just average demand.
Video playback is unavailable. Read the descriptive transcript below, go directly to practice, or open the video file.
Source-based teaching example · Fictional data · Reviewed 2026-10-03. Microsoft Mark · locally synthesized. On a small screen, use fullscreen for diagram details or read the transcript.
Read the descriptive transcript
0:00 · What you will learn
Test service capacity under bursts, not just average demand.
What appears on screen
Test service capacity under bursts, not just average demand.
0:07 · The idea
A system can handle typical traffic and fail when requests arrive together. Check service rate, waiting and fallback behavior. Measure the test setup’s own limits too.
What appears on screen
A system can handle typical traffic and fail when requests arrive together. Check service rate, waiting and fallback behavior. Measure the test setup’s own limits too.
0:21 · One useful term
p95 latency. The time at or below which 95 percent of the measured requests completed.
What appears on screen
The time at or below which 95 percent of the measured requests completed.
0:30 · A worked example
Two simulated designs have different service rates and queue limits. One meets a median target but fails the p95 target during bursts. The decision records the alternative and an offline fallback.
What appears on screen
Two simulated designs have different service rates and queue limits. One meets a median target but fails the p95 target during bursts. The decision records the alternative and an offline fallback.
0:47 · Your turn to think
Does a queue limit alone prove the p95 target? Pause and decide before continuing.
What appears on screen
Does a queue limit alone prove the p95 target? Pause the video if you need more time.
1:04 · Why this answer works
No. Measure the workload. Check units and whether the load generator itself limited the test.
What appears on screen
No. Measure the workload. Check units and whether the load generator itself limited the test.
1:13 · Check 1
Test bursts and normal load.
What appears on screen
Test bursts and normal load.
1:18 · Check 2
Check units in capacity estimates.
What appears on screen
Check units in capacity estimates.
1:23 · Check 3
Record fallback and test limitations.
What appears on screen
Record fallback and test limitations.
1:29 · Now try a different case
Open the lesson and try its different practice case. Use every check in the full task. Watching alone does not mark the lesson complete.
What appears on screen
Open the lesson Try the new case Check and revise The full task and its requirements are in the linked lesson.
Watching explains the example. Complete the lesson practice to check what you can do. Explore all visual lessons →