Watch the explanation. Practice the skill.
AI threat modeling and adversarial testing
Find and block the action an attacker is trying to cause.
Captions appear here during playback.
AI threat modeling and adversarial testing
Find and block the action an attacker is trying to cause.
Video playback is unavailable. Read the descriptive transcript below, go directly to practice, or open the video file.
Source-based teaching example · Fictional data · Reviewed 2026-10-03. Microsoft Mark · locally synthesized. On a small screen, use fullscreen for diagram details or read the transcript.
Read the descriptive transcript
0:00 · What you will learn
Find and block the action an attacker is trying to cause.
What appears on screen
Find and block the action an attacker is trying to cause.
0:06 · The idea
Map what needs protection and where trust changes. Test attacks in the supplied sandbox. Verify that forbidden execution stays blocked even if the model’s words change.
What appears on screen
Map what needs protection and where trust changes. Test attacks in the supplied sandbox. Verify that forbidden execution stays blocked even if the model’s words change.
0:20 · One useful term
Trust boundary. A point where information or an action crosses between different levels of permission or trust.
What appears on screen
A point where information or an action crosses between different levels of permission or trust.
0:30 · A worked example
The vulnerable seed follows instructions inside retrieved text and exposes export. The fix treats source text as untrusted, removes export by default and checks permission in application code.
What appears on screen
The vulnerable seed follows instructions inside retrieved text and exposes export. The fix treats source text as untrusted, removes export by default and checks permission in application code.
0:46 · Your turn to think
Is testing refusal wording enough to prove export never ran? Pause and decide before continuing.
What appears on screen
Is testing refusal wording enough to prove export never ran? Pause the video if you need more time.
1:03 · Why this answer works
No. Assert that forbidden actions produce no execution. Add the failed attack as a repeatable regression case.
What appears on screen
No. Assert that forbidden actions produce no execution. Add the failed attack as a repeatable regression case.
1:14 · Check 1
Check actual effects.
What appears on screen
Check actual effects.
1:18 · Check 2
Enforce permission outside model output.
What appears on screen
Enforce permission outside model output.
1:24 · Check 3
Use only the supplied sandbox targets.
What appears on screen
Use only the supplied sandbox targets.
1:30 · Now try a different case
Open the lesson and try its different practice case. Use every check in the full task. Watching alone does not mark the lesson complete.
What appears on screen
Open the lesson Try the new case Check and revise The full task and its requirements are in the linked lesson.
Watching explains the example. Complete the lesson practice to check what you can do. Explore all visual lessons →