Data preparation, provenance, and records
Prepare data without losing where it came from.
In this lesson · A3 of 8
All lessons in this track →Progress for this track is saved in this browser only. No completion badge is issued. View your learning.
Start with the idea
Prepare data without losing where it came from.
Keep the original source and record your changes. Use only fields needed for the task. Made-up training records are different from real records with names removed.
Provenance: A record of where information came from and how it changed.
See the example and check your understanding
A made-up teaching example
The training task counts roles and shifts. Three invented rows give two A/day records and one B/night record. IDs and personal notes are not needed for the counts.
Ask yourself
Is a real table with names removed automatically synthetic data?
Show the explanation
No. Synthetic here means invented. Other fields can still reveal information about real people.
Start with these checks
- Keep the original source available.
- Record each change and its reason.
- Justify every retained field.
Use the different practice case below and check every requirement in its full instructions.
The video explains the main idea. Use the sources, instructions and checks below to complete the practice.
Captions appear here during playback.
Data preparation, provenance, and records
Prepare data without losing where it came from.
Video playback is unavailable. Read the descriptive transcript below, go directly to practice, or open the video file.
Source-based teaching example · Fictional data · Reviewed 2026-10-03. Microsoft Mark · locally synthesized. On a small screen, use fullscreen for diagram details or read the transcript.
Read the descriptive transcript
0:00 · What you will learn
Prepare data without losing where it came from.
What appears on screen
Prepare data without losing where it came from.
0:05 · The idea
Keep the original source and record your changes. Use only fields needed for the task. Made-up training records are different from real records with names removed.
What appears on screen
Keep the original source and record your changes. Use only fields needed for the task. Made-up training records are different from real records with names removed.
0:19 · One useful term
Provenance. A record of where information came from and how it changed.
What appears on screen
A record of where information came from and how it changed.
0:26 · A worked example
The training task counts roles and shifts. Three invented rows give two A/day records and one B/night record. IDs and personal notes are not needed for the counts.
What appears on screen
The training task counts roles and shifts. Three invented rows give two A/day records and one B/night record. IDs and personal notes are not needed for the counts.
0:41 · Your turn to think
Is a real table with names removed automatically synthetic data? Pause and decide before continuing.
What appears on screen
Is a real table with names removed automatically synthetic data? Pause the video if you need more time.
0:58 · Why this answer works
No. Synthetic here means invented. Other fields can still reveal information about real people.
What appears on screen
No. Synthetic here means invented. Other fields can still reveal information about real people.
1:08 · Check 1
Keep the original source available.
What appears on screen
Keep the original source available.
1:13 · Check 2
Record each change and its reason.
What appears on screen
Record each change and its reason.
1:18 · Check 3
Justify every retained field.
What appears on screen
Justify every retained field.
1:23 · Now try a different case
Open the lesson and try its different practice case. Use every check in the full task. Watching alone does not mark the lesson complete.
What appears on screen
Open the lesson Try the new case Check and revise The full task and its requirements are in the linked lesson.
Watching explains the example. Complete the lesson practice to check what you can do. Explore all visual lessons →
Before you start
Prerequisite: A2 workflow package and O2 data permission. Offline/no-cost route: a notebook, calculator and supplied synthetic facts. Optional tools must already be authorized. No real workplace data, paid API, new account or external action is required.
What you will be able to do
- Create inventory, dictionary, and flow map.
- Apply minimization, redaction, and synthetic substitution.
- Route privacy, records, accessibility, and IP decisions.
Part 1 Data preparation creates evidence and new data paths
Provenance records origin, owner, permission/license, collection date, source version and transformations. A dictionary gives field meaning, type, allowed values and missing-value convention. Minimize fields to the declared purpose before building prompts or retrieval indexes. Rare combinations and free text can still identify people after names are removed. Record each destination: originals, extracts, embeddings, logs, outputs and backups. Retention and deletion belong to the applicable records/data authority; do not invent universal time limits. Accessible source formats reduce extraction errors and support equivalent review routes.
Part 2 Worked synthetic example
Synthetic rows: R01/role A/day/“trained teammate”; R02/role B/night/“rare qualification Q7”; R03/role A/day/blank. Purpose is count by role/shift. Keep role and shift, exclude notes and IDs in the aggregate input. Counts: A/day=2, B/night=1. The rare note is unnecessary. Provenance says invented training rows, version 1, no real people. Do not call a redacted real copy synthetic.
Part 3 Guided practice and feedback
For each field ask whether removing it prevents the aggregate task. IDs and notes do not affect counts; remove them. Record transformation as projected columns then grouped counts. A deletion test on an invented duplicate shows that copy was removed, not that every log/backup or provider copy was erased.
Review and next step
Self-review is practice, not independent certification or operational approval. A4 uses provenance and source scope to build retrieval evidence.
Substantive review: 2026-10-02. Use the configuration guide when selecting tools. NIST AI RMF is voluntary risk-management guidance and grants no course or operational authorization.