The migration that passes every test and changes every number
Green pipeline, different answer. Both true. Nothing in a normal stack is built to notice.
The pipeline is green. The number is different. Both are true, and nothing in a normal stack is built to notice.
I’ve watched this play out on three migrations now and it goes the same way every time. Row counts match. Schemas match. Every test passes, because every test asserts on structure. Nulls, uniqueness, referential integrity, freshness. Then someone in Finance opens the new report, the exposure figure is 8% lower than last month, and nobody can say whether that’s the migration or the business.
The reason is simple enough to be uncomfortable. Your tests assert on the shape of the data. What changed was the meaning, and no artifact in your stack was holding it. The rule lived in a WHERE clause that got rewritten by someone who didn’t know status IN (‘A’,‘P’) was a regulatory exclusion and not a filter somebody once found convenient.
Late binding is the structural fix
A metric that names a physical table has bet its meaning on that table never moving. In a migration, the table always moves. A metric that names a concept, with the binding as a separate, reviewable, dated fact, survives the move. Better: it makes the move visible. The binding changed, and the binding is an object with a diff.
That’s what turns “what breaks if I change this column?” into a question you can answer before the pull request merges, instead of during the incident review. The column isn’t an anonymous string in a repo. It’s bound to concepts, which carry claims, which carry controls, which hold a release. The blast radius is computable, so it can be refused.
Reconciliation shouldn’t be a spreadsheet that was correct on the day it was signed. It should be a control that runs tonight, and every night after.
Why this got urgent this year
Because the rate of change went up and the rate of review didn’t. When an agent can refactor forty queries in an afternoon, the traditional safeguard, a human noticing something looks off in a diff, stops scaling. It was never a strong control anyway. It was a slow one, and the slowness was doing the work.
The answer isn’t to forbid the agent. It’s to make the guardrail machine-readable, so whatever generates the change hits the same closed door: a contractor, a junior, or a model at three in the morning. An agent that quietly changes a regulated number should fail on a bound control, not on somebody’s vigilance.
The practice
What this replaces is migration assurance as a consulting deliverable: a beautiful reconciliation pack, correct on the day it was signed, decaying from that afternoon onward. Same rigour, same domain expertise, but generated from the model, versioned with it, and re-runnable every night by nobody at all.