Requirements, test strategy, and acceptance
Replace “be accurate” with a requirement two reviewers can test.
In this lesson · L4 of 7
All lessons in this track →Progress for this track is saved in this browser only. No completion badge is issued. View your learning.
Start with the idea
Replace “be accurate” with a requirement two reviewers can test.
State the cases, correct behavior and evidence to retain. Define what happens when sources are missing or conflicting. Identify the person who accepts the result.
Testable requirement: A requirement with an observable result and a defined pass-or-fail rule.
See the example and check your understanding
A made-up teaching example
The training rewrite uses ten fixed notices. Deadlines must match source passages, and conflicting dates must trigger a refusal to guess. Outputs, source IDs and configuration are retained.
Ask yourself
Is “a human is in the loop” a complete acceptance test?
Show the explanation
No. Say what the reviewer checks and can reject. The test lead reports evidence; the designated owner decides.
Start with these checks
- Define expected behavior and evidence.
- Keep critical rules separate from averages.
- Review relevant version changes.
Use the different practice case below and check every requirement in its full instructions.
The video explains the main idea. Use the sources, instructions and checks below to complete the practice.
Captions appear here during playback.
Requirements, test strategy, and acceptance
Replace “be accurate” with a requirement two reviewers can test.
Video playback is unavailable. Read the descriptive transcript below, go directly to practice, or open the video file.
Source-based teaching example · Fictional data · Reviewed 2026-10-03. Microsoft Mark · locally synthesized. On a small screen, use fullscreen for diagram details or read the transcript.
Read the descriptive transcript
0:00 · What you will learn
Replace “be accurate” with a requirement two reviewers can test.
What appears on screen
Replace “be accurate” with a requirement two reviewers can test.
0:07 · The idea
State the cases, correct behavior and evidence to retain. Define what happens when sources are missing or conflicting. Identify the person who accepts the result.
What appears on screen
State the cases, correct behavior and evidence to retain. Define what happens when sources are missing or conflicting. Identify the person who accepts the result.
0:21 · One useful term
Testable requirement. A requirement with an observable result and a defined pass-or-fail rule.
What appears on screen
A requirement with an observable result and a defined pass-or-fail rule.
0:30 · A worked example
The training rewrite uses ten fixed notices. Deadlines must match source passages, and conflicting dates must trigger a refusal to guess. Outputs, source IDs and configuration are retained.
What appears on screen
The training rewrite uses ten fixed notices. Deadlines must match source passages, and conflicting dates must trigger a refusal to guess. Outputs, source IDs and configuration are retained.
0:46 · Your turn to think
Is “a human is in the loop” a complete acceptance test? Pause and decide before continuing.
What appears on screen
Is “a human is in the loop” a complete acceptance test? Pause the video if you need more time.
1:04 · Why this answer works
No. Say what the reviewer checks and can reject. The test lead reports evidence; the designated owner decides.
What appears on screen
No. Say what the reviewer checks and can reject. The test lead reports evidence; the designated owner decides.
1:15 · Check 1
Define expected behavior and evidence.
What appears on screen
Define expected behavior and evidence.
1:20 · Check 2
Keep critical rules separate from averages.
What appears on screen
Keep critical rules separate from averages.
1:26 · Check 3
Review relevant version changes.
What appears on screen
Review relevant version changes.
1:31 · Now try a different case
Open the lesson and try its different practice case. Use every check in the full task. Watching alone does not mark the lesson complete.
What appears on screen
Open the lesson Try the new case Check and revise The full task and its requirements are in the linked lesson.
Watching explains the example. Complete the lesson practice to check what you can do. Explore all visual lessons →
Prerequisites: distinguish output from evidence; review the AI workflow reference. Public self-study is not official agency training, accreditation or a credential.
What you will be able to do
This lesson develops the ability to translate outcomes into measurable requirements and evidence gates. The setting is NORTHSTAR Mission Portfolio, a fictional public-service portfolio containing ten proposed AI and non-AI interventions, three synthetic vendors, constrained staff capacity, and a staged incident. Every person, organization, system record, metric, and outcome in the case is fictional. Do not replace the case with operational, classified, CUI, export-controlled, proprietary, employer-confidential, or third-party personal information.
By the end of the lesson, you will be able to:
- Demonstrate how to translate outcomes into measurable requirements and evidence gates.
- Apply that model to the synthetic NORTHSTAR Mission Portfolio environment.
- Produce a reviewable artifact that separates evidence, assumption, uncertainty, and authority.
- Recognize the critical failure and select a safe remediation path.
The performance standard is not confident language. It is a traceable decision, an artifact another qualified reviewer can inspect, a clear boundary around the learner’s authority, and a safe response to uncertainty. Knowledge checks are formative and do not award a certification or endorsement.
This curriculum runs self-study on VetsLearn. Where the text calls for an instructor, peer or formal review, use the stated acceptance criteria for self-review. Reflection questions support reflection; they do not replace independent assessment. A peer review is optional unless your actual organization requires it.
Part 1 Executable acceptance requirements
A requirement names user, condition, observable behavior, threshold, method, evidence and acceptance owner. Separate average quality from critical gates: disclosure, unauthorized action, inaccessible review and failure to abstain. Class thresholds are not federal standards. Specify regression tests and notification for model, retrieval, corpus, tools and permissions. Include restoration and exit. Automated scores inform acceptance; they do not appoint the test author as the accepting official.
Requirements-to-test matrix
| Requirement | Fixture | Pass evidence | Critical rule |
|---|---|---|---|
| Source-supported date | Notice says 09:00 | Output states 09:00 and cites span | No invented date |
| Conflict abstention | Two contradictory notices | Output flags conflict and requests human resolution | No confident selection |
| Tool permission | Read-only user asks to send | No action; denial recorded | No unauthorized send |
| Accessible review | Keyboard-only reviewer | Source and reject control reachable | Missing rejection route blocks use |
| Restoration | Corrupt index | Known query and role denial both pass after restore | Do not reopen on partial recovery |
Gaming test: if citation presence is the metric, a wrong answer can append a real source ID. Therefore score whether the cited span actually supports the claim. If latency is measured only for generation, include retrieval and review so the test reflects the task.
Synthetic results: four citation labels are present; one source span contradicts the deadline. Presence is 4/4, support is 3/4, and the critical date gate fails. Retain all outputs rather than replacing the result with the flattering presence metric.
Feedback: acceptance criteria need both ordinary measures and non-negotiable failure rules. Their authority still belongs to the designated acceptance owner.
Part 2 Synthetic worked example
Weak: “Be accurate.” Synthetic rewrite: on ten fixed notices, each deadline matches its source span; conflicting dates cause abstention; zero unsupported deadlines are accepted. Retain output, source ID and configuration. The test lead reports; the designated owner decides.
Guided practice: identify the unsupported conclusion, name missing evidence and rewrite its scope. Change one assumption and identify which decision reopens.
Part 3 Feedback and repair
Pass when two readers can decide from the same evidence, critical gates cannot be averaged away and version changes trigger review. “Human in loop” alone is not a test.
Retain the failed version, explain the criterion, correct the artifact and test a different invented case.
Next step: carry the artifact and failed cases into the next module lesson or capstone. Retain limitations when explaining your work.