Same model, different harness
Explain how tools, permissions and checks change a workflow.
Reviewed: 2026-10-03
Illustrated teaching example • Fictional data
Narration: Microsoft Mark · locally synthesized

00:00:00.000 — The surrounding system changes the work.

Two systems can use the same model and still behave differently. Context selection, tools, permissions, retries, and verification all affect the workflow. This illustrated comparison uses constructed behavior. It is not a measured comparison of provider quality or a live model benchmark.

Visual description:
Constructed comparison · same model assumption

00:00:22.708 — Configuration A: drafting only.

In the first configuration, the system receives supplied text and drafts an answer. It has no file tools and cannot create a workbook. If asked to save a spreadsheet, it can explain a plan or produce text, but it must not claim to have written a file.

Visual description:
Available: Supplied text + generated answer
Unavailable: File reads, writes and execution
An instruction does not create a missing capability.

00:00:41.917 — Configuration B: bounded tools.

In the second configuration, the harness exposes an approved read tool and a narrowly scoped output tool. It validates arguments, enforces access, and checks the resulting artifact. These capabilities add useful work and new failure modes. The model still cannot legitimately bypass the application’s permission rules.

Visual description:
Model request
Validate arguments
Authorize scope
Execute tool
Check artifact
Tool availability is not unrestricted authority.

00:01:06.167 — Observe the actual action.

A request to write a report is not proof that the report exists. Inspect the tool result and the output path, then check the artifact’s content. A tool error must remain an error in the final response. Describe partial completion accurately rather than converting it into a success claim.

Visual description:
REQUEST: write_report: approved output path
CHECK: arguments and scope accepted
RESULT: artifact produced in fixture
VERIFY: content checked against source
Simulated trace · fictional fixture

00:01:27.833 — Tools do not remove the need for verification.

Suppose the file exists but contains an invented savings number. The write succeeded while the task still failed. Verify both the action and the content. A correct artifact needs supporting evidence, and a correct summary must distinguish successful steps from unresolved requirements.

Visual description:
File exists → therefore the task passed?
Execution: File created (pass)
Content: Invented savings (fail)
Decision: Repair before completion (warning)

00:01:49.250 — Compare complete workflows.

For a real comparison, hold the task and source fixed and record model configuration, tool permissions, checks, retries, latency, cost, and human correction. Use representative repeated cases. Do not attribute all differences to the model when the surrounding system changed.

Visual description:
Inputs: Same task and source
Controls: Tools and permissions recorded
Evidence: Artifact and failure checks
Costs: Tools, retries and human review

00:02:11.042 — Run the offline workflow lab.

Use the downloadable offline lab in the curriculum. Compare the drafting policy with the bounded tool policy, inspect denial and failure cases, and explain what each setup can actually produce. The lab is a teaching simulator. It is not a production sandbox or a live same model benchmark.

Visual description:
What changed because of the harness?
Permitted actions.
Visible tool results.
Independent artifact checks.
Your evidence: a configuration comparison with limitations.