When AI is wrong
Catch errors in meaning, numbers and approval claims.
In this lesson · 04 of 7
All lessons in this track →Saved in this browser. View your learning.
Start with the idea
Catch errors in meaning, numbers and approval claims.
Check each important claim separately. Recalculate numbers yourself. Read the source’s actual wording: “may” is different from “must,” and scheduled review is different from completed approval.
Independent check: A check that does not rely on the same answer being correct.
See the example and check your understanding
A made-up teaching example
Three records take 12, 18 and 24 minutes. Their average is 18 minutes. The policy allows a draft-only trial after approval. A draft changes that permission into a requirement and claims publication is already approved.
Ask yourself
If you repair the average, is the whole brief now correct?
Show the explanation
No. The policy and approval claims still need repair. Correct arithmetic does not establish savings or permission.
Start with these checks
- Recalculate the average.
- Check the exact policy passage.
- Keep missing approval and baseline evidence visible.
Use the different practice case below and check every requirement in its full instructions.
The video explains the main idea. Use the sources, instructions and checks below to complete the practice.
Captions appear here during playback.
When AI is wrong
Catch errors in meaning, numbers and approval claims.
Video playback is unavailable. Read the descriptive transcript below, go directly to practice, or open the video file.
Source-based teaching example · Fictional data · Reviewed 2026-10-03. Microsoft Mark · locally synthesized. On a small screen, use fullscreen for diagram details or read the transcript.
Read the descriptive transcript
0:00 · What you will learn
Catch errors in meaning, numbers and approval claims.
What appears on screen
Catch errors in meaning, numbers and approval claims.
0:06 · The idea
Check each important claim separately. Recalculate numbers yourself. Read the source’s actual wording: “may” is different from “must,” and scheduled review is different from completed approval.
What appears on screen
Check each important claim separately. Recalculate numbers yourself. Read the source’s actual wording: “may” is different from “must,” and scheduled review is different from completed approval.
0:21 · One useful term
Independent check. A check that does not rely on the same answer being correct.
What appears on screen
A check that does not rely on the same answer being correct.
0:29 · A worked example · 1
Three records take 12, 18 and 24 minutes. Their average is 18 minutes. The policy allows a draft-only trial after approval.
What appears on screen
Three records take 12, 18 and 24 minutes. Their average is 18 minutes. The policy allows a draft-only trial after approval.
0:42 · A worked example · 2
A draft changes that permission into a requirement and claims publication is already approved.
What appears on screen
A draft changes that permission into a requirement and claims publication is already approved.
0:51 · Your turn to think
If you repair the average, is the whole brief now correct? Pause and decide before continuing.
What appears on screen
If you repair the average, is the whole brief now correct? Pause the video if you need more time.
1:08 · Why this answer works
No. The policy and approval claims still need repair. Correct arithmetic does not establish savings or permission.
What appears on screen
No. The policy and approval claims still need repair. Correct arithmetic does not establish savings or permission.
1:19 · Check 1
Recalculate the average.
What appears on screen
Recalculate the average.
1:24 · Check 2
Check the exact policy passage.
What appears on screen
Check the exact policy passage.
1:29 · Check 3
Keep missing approval and baseline evidence visible.
What appears on screen
Keep missing approval and baseline evidence visible.
1:35 · Now try a different case
Open the lesson and try its different practice case. Use every check in the full task. Watching alone does not mark the lesson complete.
What appears on screen
Open the lesson Try the new case Check and revise The full task and its requirements are in the linked lesson.
Watching explains the example. Complete the lesson practice to check what you can do. Explore all visual lessons →
Before you start
Prerequisite: source-bounded prompts. You will classify claims, verify numbers and source meaning, and leave unsupported answers unresolved. Paper and a calculator are sufficient; no AI account or live data is required.
What you will be able to do
- Distinguish unsupported claims, source misreading and calculation errors.
- Independently verify source meaning and numerical claims.
- Repair a brief while preserving unresolved questions.
Part 1 Match the check to the failure
A hallucination is an unsupported or fabricated generated claim. Other failures include misreading a source, omitting a qualification, stale retrieval, arithmetic errors and unauthorized action. Each needs a different repair: open the exact passage for citations; recompute numerical claims; inspect current authoritative sources for current requirements; test executable behavior for code.
Model confidence, rephrasing the same question and multiple models repeating a claim are not independent evidence. They may share a mistaken premise. Web access can fetch current information, but pages can still be stale, irrelevant or misread. Check authority, date, scope and exact support.
Prioritize by consequence and reversibility. Internal drafts can influence decisions, so show unknowns visibly. Consequential medical, legal, financial or regulated work requires applicable professional review and organizational authority; this exercise grants neither.
Part 2 Worked example: meaning and arithmetic
Synthetic P1, paragraph 2: “Teams may run a draft-only trial after coordinator approval. Publication requires separate review.” T1: “Three records took 12, 18 and 24 minutes.” A generated brief says: “P1 requires all teams to deploy AI. Average time is 20 minutes. Publication is already approved.”
Claim one changes may to requires and a trial to deployment. Claim two fails arithmetic: (12 + 18 + 24) / 3 = 18 minutes. Claim three has no approval record. Repairing only the number leaves two serious errors. A defensible brief preserves the optional trial, approval condition and separate publication review.
Part 3 Guided classification and answers
Classify (a) three records total 54 minutes; (b) every future record will take 18 minutes; (c) a trial is possible after approval; (d) a trial is worth considering. Answers: supported calculation; unsupported extrapolation; supported conditional statement; interpretation needing criteria. Use claim, evidence/location, check, result and disposition columns. Say supported by these three records, not the model is 95% sure.
Substantive review: 2026-10-02. Supplied policies and records are fictional.
Interactive practice / Fictional fixture
Audit the flawed brief here
Use P1 and T1 above. Recompute the number, settle three audit questions, then write your corrected brief. Our answer unlocks after you have written yours and run the checks.
Everything runs in this browser. No AI model is called and no work is uploaded. Bounded checks cover specific facts and rules. Your writing is not graded: you review it yourself. Practice never marks a lesson complete automatically.
Two sentences, P1 and T1 only. Keep open questions open. Write your claim audit on paper or in your own tool.
Try the task, run the checks, then revise. Checks are teaching feedback, not an instructor grade.
Observed browser simulation
This trace comes from the bounded local checker. It is not a Python run, model result or production tool execution.
Self-review
Locked. Write every written part, then run the checks. A few words is not an attempt.
This is self-review, not grading. Read our answers, then tick what your own work does. Ticks are optional. They stay on this page and are not saved or sent anywhere. A different answer can be just as good if it meets the lesson criteria.
Your corrected two-sentence brief
A strong answer
Teams may run a draft-only trial after coordinator approval (P1, paragraph 2); publication requires separate review, and no approval is on record. The three T1 records averaged 18 minutes, but no pre-AI baseline is supplied, so savings and any publication date remain open questions.
A flawed answer
All teams must publish the trial this week. Three records averaged 18 minutes, proving AI saved 50%. The coordinator approved publication.
What is wrong with it: It makes publication mandatory and sets a deadline. P1 paragraph 2 allows only an optional draft-only trial after coordinator approval. The 18-minute average is right, but with no pre-AI baseline it proves no saving. No approval record is supplied. A defensible brief preserves the optional trial, approval condition and separate publication review.
Keep or restore your work
Drafts are not saved automatically. Save only fictional practice data. A browser draft may be lost if storage is cleared; download a copy to keep it.