Production capstone, incident exercise, and board

What you will be able to do

1. Operate a production-like synthetic service against SLOs and gates.
2. Respond to an unannounced incident inject.
3. Defend design, evidence, residual risk, and operating ownership.

Module capstone. This lesson pulls together every skill in the module. Budget real time for it.

Prerequisites: Builder-level interface validation, Python, testing, metadata logging and state-transition tracing. Hardware benchmarks and model accounts are not required.

This curriculum runs self-study on VetsLearn. Where the text calls for an instructor, peer or formal review, use the stated acceptance criteria for self-review. Reflection questions support reflection; they do not replace independent assessment. A peer review is optional unless your actual organization requires it.

Part 1 Technical instruction

The Production capstone combines architecture, data governance, authorization, threat evidence, evaluation, observability, reliability, supply chain, delivery, and incident response. The team declares SLOs, failure rules, ownership, and change plan before observation. An inject tests detection and decision quality, not theatrical speed. The board reviews immutable evidence, asks what changed, challenges assumptions, and requires an explicit statement of what the training artifact does not prove. The sandbox is not an ATO, clearance qualification, contracting authority, or independent-expert designation.

Mechanism and implementation

A board asks what evidence supports and which decisions remain human. Integrate only task-needed components and record alternatives. Incident injects test decisions after change rather than the initial score. Course evidence, authorization scope and operational readiness are separate.

Constructed offline example

Save as topic_example.py and run python topic_example.py with an installed Python 3.11+ interpreter. This standard-library construction uses synthetic values; its checks are not provider measurements.

evidence = {"normal": True, "denied_access": True, "rollback": False,
            "recovery_verified": False}
release = all(evidence.values())
assert not release
print("no-go: repair rollback and demonstrate recovery")

Engineering decisions

• Freeze scope and decision rights.
• Predeclare release, containment, and rollback gates.
• Assign legitimate owners for residual risk and follow-up.

Worked example

The seed service passes initial gates, then an inject changes an authorization mapping and degrades p95 latency. The team contains the route, preserves logs, rolls back, verifies negative-access and latency recovery, and adds both cases to CI.

Diagnose the failure

Absent rollback or recovery blocks the claimed readiness scope. Preserve inject timing. Self-review supports learning but cannot confer an ATO, independent certification or job readiness.

Part 2 Guided practice

Create a requirement map, then inject authorization drift and outage into a local mock. Independently hold a decision board with named owners, unresolved evidence and no-go option; solo learners can write evidence-based questions/answers.

Answer criteria and feedback: Absent rollback or recovery blocks the claimed readiness scope. Preserve inject timing. Self-review supports learning but cannot confer an ATO, independent certification or job readiness.

Continue with the downloadable pack below. If the pack or interpreter is unavailable, trace this self-contained example by hand in an input/expected-status/controlling-rule table and label execution unverified. Do not invent passing output. No account, credential, paid call, purchase or actual deployment is required.

Part 3 Remediation

Use the reset manifest and incident controller. If the environment becomes inconsistent, preserve the event log, restore the last known-good snapshot, and document the gap rather than reconstructing favorable evidence. After repair, reassess with the alternate scenario in scenario_alt.json. A critical authorization, data-handling, evidence-integrity, or prohibited-action failure cannot be averaged into successful practice; it requires targeted remediation.

Part 4 Sources and currency

• https://csrc.nist.gov/pubs/sp/800/61/r3/final
• https://csrc.nist.gov/pubs/sp/800/218/final
• https://airc.nist.gov/Docs/Generative_AI_Profile.pdf

Use the workflow decision reference for model/deployment/harness distinctions and controlled comparisons. Source currency must be checked before each cohort. Provider/product behavior is illustrative where mocked; resolve current interfaces only from official documentation in an authorized environment.