Reliable workflows and agents
Recover repeated work without causing duplicate effects.
In this lesson · P9 of 12
All lessons in this track →Progress for this track is saved in this browser only. No completion badge is issued. View your learning.
Start with the idea
Recover repeated work without causing duplicate effects.
Record an operation’s identity and intended effect. Replaying the same operation should not duplicate that effect. Durable recovery also needs a storage and reconciliation design.
Idempotency key: An identifier used to recognize repeated attempts at the same intended operation.
See the example and check your understanding
A made-up teaching example
The simulated workflow records an action and crashes before acknowledging its message. Replay uses the same key and does not duplicate the action. Changed intent is rejected.
Ask yourself
Does an in-memory exercise prove recovery after a real process restart?
Show the explanation
No. Memory disappears with the process. Persistent storage and reconciliation need separate design and tests.
Start with these checks
- Test identical replay and changed intent.
- Keep approvals in reliable state.
- Distinguish simulation from durable recovery.
Use the different practice case below and check every requirement in its full instructions.
The video explains the main idea. Use the sources, instructions and checks below to complete the practice.
Captions appear here during playback.
Reliable workflows and agents
Recover repeated work without causing duplicate effects.
Video playback is unavailable. Read the descriptive transcript below, go directly to practice, or open the video file.
Source-based teaching example · Fictional data · Reviewed 2026-10-03. Microsoft Mark · locally synthesized. On a small screen, use fullscreen for diagram details or read the transcript.
Read the descriptive transcript
0:00 · What you will learn
Recover repeated work without causing duplicate effects.
What appears on screen
Recover repeated work without causing duplicate effects.
0:06 · The idea
Record an operation’s identity and intended effect. Replaying the same operation should not duplicate that effect. Durable recovery also needs a storage and reconciliation design.
What appears on screen
Record an operation’s identity and intended effect. Replaying the same operation should not duplicate that effect. Durable recovery also needs a storage and reconciliation design.
0:21 · One useful term
Idempotency key. An identifier used to recognize repeated attempts at the same intended operation.
What appears on screen
An identifier used to recognize repeated attempts at the same intended operation.
0:30 · A worked example
The simulated workflow records an action and crashes before acknowledging its message. Replay uses the same key and does not duplicate the action. Changed intent is rejected.
What appears on screen
The simulated workflow records an action and crashes before acknowledging its message. Replay uses the same key and does not duplicate the action. Changed intent is rejected.
0:45 · Your turn to think
Does an in-memory exercise prove recovery after a real process restart? Pause and decide before continuing.
What appears on screen
Does an in-memory exercise prove recovery after a real process restart? Pause the video if you need more time.
1:03 · Why this answer works
No. Memory disappears with the process. Persistent storage and reconciliation need separate design and tests.
What appears on screen
No. Memory disappears with the process. Persistent storage and reconciliation need separate design and tests.
1:14 · Check 1
Test identical replay and changed intent.
What appears on screen
Test identical replay and changed intent.
1:19 · Check 2
Keep approvals in reliable state.
What appears on screen
Keep approvals in reliable state.
1:25 · Check 3
Distinguish simulation from durable recovery.
What appears on screen
Distinguish simulation from durable recovery.
1:30 · Now try a different case
Open the lesson and try its different practice case. Use every check in the full task. Watching alone does not mark the lesson complete.
What appears on screen
Open the lesson Try the new case Check and revise The full task and its requirements are in the linked lesson.
Watching explains the example. Complete the lesson practice to check what you can do. Explore all visual lessons →
What you will be able to do
- Model durable state explicitly.
- Use idempotency, transactions, compensation, and approval.
- Test crashes, replay, races, and malicious observations.
Prerequisites: Builder-level interface validation, Python, testing, metadata logging and state-transition tracing. Hardware benchmarks and model accounts are not required.
This curriculum runs self-study on VetsLearn. Where the text calls for an instructor, peer or formal review, use the stated acceptance criteria for self-review. Reflection questions support reflection; they do not replace independent assessment. A peer review is optional unless your actual organization requires it.
Part 1 Technical instruction
Durable workflows separate state from a worker process. Each transition has a precondition, action, recorded result, and next state. At-least-once delivery means duplicate messages are expected; idempotency keys make repeated requests converge on one recorded outcome. Transactions protect local consistency, while compensating actions address completed external steps that cannot share a transaction. Human approval is a signed state transition with scope and expiry, not a sentence in a prompt. Audit records must reconstruct decisions without exposing prohibited content.
Mechanism and implementation
An effect may finish before acknowledgment is lost. Record scoped idempotency keys/outcomes durably and reconcile unknown status before retrying. Approval binds actor, exact action/arguments, resource version and expiry. Cancellation stops future work; completed effects need reviewed compensation.
Constructed offline example
Save as topic_example.py and run python topic_example.py with an installed Python 3.11+ interpreter. This standard-library construction uses synthetic values; its checks are not provider measurements.
ledger = {}
def action(key, value):
if key in ledger:
if ledger[key] != value:
raise ValueError("key_reused_for_different_action")
return "replayed"
ledger[key] = value
return "recorded"
assert action("case-1:v1", "draft") == "recorded"
assert action("case-1:v1", "draft") == "replayed"
assert len(ledger) == 1
print("one simulated effect despite replay")Engineering decisions
- Define state transitions and terminal states.
- Choose idempotency scope and retention.
- Place approvals before consequential simulated actions.
Worked example
A synthetic case workflow crashes after recording an action but before acknowledging the message. Replay uses the same idempotency key and does not duplicate the action. A malicious observation cannot move the case to approved.
Diagnose the failure
Identical replay produces one effect; changed intent rejects. This in-memory construction is not process-durable; persistent storage needs transaction/reconciliation design.
Part 2 Guided practice
Simulate a crash after the record and before acknowledgment, then replay. Independently reject changed arguments under the same key, expired approval and cancellation before a next action.
Answer criteria and feedback: Identical replay produces one effect; changed intent rejects. This in-memory construction is not process-durable; persistent storage needs transaction/reconciliation design.
Continue with the downloadable pack below. If the pack or interpreter is unavailable, trace this self-contained example by hand in an input/expected-status/controlling-rule table and label execution unverified. Do not invent passing output. No account, credential, paid call, purchase or actual deployment is required.
Part 3 Remediation
Replay the event log from state zero. If reconstructed state differs, locate the first nondeterministic or unrecorded transition. After repair, reassess with the alternate scenario in scenario_alt.json. A critical authorization, data-handling, evidence-integrity, or prohibited-action failure cannot be averaged into successful practice; it requires targeted remediation.
Part 4 Sources and currency
Use the workflow decision reference for model/deployment/harness distinctions and controlled comparisons. Source currency must be checked before each cohort. Provider/product behavior is illustrative where mocked; resolve current interfaces only from official documentation in an authorized environment.