Your claims are already an eval suite. Stop writing them twice.
The best way to tell an agent what “correct” means isn’t a better prompt. It’s the set of checks that have to hold, which you already wrote, in week one.
The best way to tell an agent what “correct” means isn’t a cleverer prompt. It’s the set of things that have to be true. And you already wrote those, in week one, in the requirement, before anyone opened a notebook.
There’s a line in Google’s SDLC paper I keep coming back to: tests and evals become the primary mechanism for communicating intent to an agent. A well-written eval suite tells the AI what correct means, and gives you an automated way to check it. Write the tests before you generate the code, because together they’re the contract.
Now look at what a clarified requirement actually contains. The measure sits at contract-account grain. Flat-rate accounts are excluded. The window is twelve months, calendar-consecutive. Executed move-outs inside 30 days drop out. Those aren’t notes. Every one is a testable proposition, and together they’re a more precise statement of intent than any prompt you could write.
Two artifacts, one sentence
We keep writing this twice. Once as a requirement, in prose, for humans. Then again, months later, as data quality rules, in code, by someone who wasn’t there. The second version is reverse-engineered from the first version’s implementation, which means we test the SQL against itself and call it assurance.
A claim and a control are the same sentence written for two audiences. Write it once and let both of them read it.
The mechanical version: once a requirement is a set of typed claims, the checks come off the model instead of being invented against it. Grain implies a uniqueness check. An exclusion implies a check that nothing excluded appears. A window implies freshness and completeness. You’re not building a quality backlog. You’re reading one.
What it changes about the conversation
The eval framing does something the DQ framing never managed: it makes the checks interesting to the business. Nobody in Finance wants a meeting about referential integrity. Everybody in Finance wants a meeting about whether flat-rate accounts should be in the number, and that meeting is the check being written. Same hour, same room, two outputs.
It also fixes what an agent is aiming at. Handed a bag of tables and a paragraph, an agent will produce something plausible and you’ll be reviewing prose. Handed a claim set, it has a bar to clear, and you’re reviewing against a standard you already agreed. That’s the difference between reading output and judging it.
How to run it
The prize isn’t tidier governance. It’s that “what does correct mean here?” stops being a thing people argue about after the fact, and becomes the artifact the work started from.