Data Product Agentic Engineering
Software engineering spent two years working out how to build with agents without shipping garbage. It landed on a name, agentic engineering, and a discipline underneath it. Data hasn’t done this yet. Analytics is still vibe coding, and calling it a copilot.
Generation is solved. Verification, judgment and direction are the new craft.
That’s Google’s line, about code. Now say it about a number. Generating SQL was never the hard part of analytics. Agreeing what the number means was, and proving it held was. AI just removed the only part of the job that was ever easy, and left the whole of the difficult part exposed.
Frame and figures throughout adapted from The New SDLC With Vibe Coding: From ad-hoc prompting to Agentic Engineering, by Osmani, Saboo & Kartakis, Google, May 2026. The remapping to data products is ours.
Vibe analytics
Karpathy’s phrase for prompting your way to code you never read has an exact analogue in our world, and it has been shipping into steering packs for two years. Ask a question. Get a number. Accept it because it looks about right. The difference is that a wrong number, unlike broken code, doesn’t crash. It gets presented.
The differentiator was never whether you use AI. It’s how the output gets verified, and in data, the output is a number that carries a decision.
Two evals: what it built, and how it got there
Agentic engineering checks the artifact and the trajectory: the sequence of steps and tool calls that produced it. Because a fluent output that skipped its verification steps is more dangerous than one with a visible error. Data has no equivalent of this, and needs one more badly than code does.
The check everybody already knows how to imagine. Row counts, nulls, referential integrity, a threshold, a reconciliation against a known total. Necessary. Nowhere near sufficient. This is exactly the set of tests that passes on the morning your migration silently changed every figure on the page.
Did it aggregate at the contract-account grain that was ruled on, or quietly at meter? Did it apply the flat-rate exclusion? Did it take the join path somebody signed off, with the cardinality they expected? A number can be correct this month and arrive by a route that will be wrong next month. Only one of these two evals catches that.
You don’t have a model problem. You have a harness problem.
Google’s estimate is that the model is roughly 10% of a working agent and the harness, meaning tools, context, sandboxes, guardrails and observability, is the other 90%. On one public benchmark a team moved an agent from outside the top 30 into the top 5 by changing only the harness. Same model. That is where the leverage lives, and in data almost nobody is building it.
Read that right-hand column end to end and it describes a product. That product is PYX. We didn’t set out to build a harness for data agents. We set out to make meaning survive a hand-off, but the two turn out to be the same object.
Your output stops being the data product. It becomes the system that makes them.
A factory manager doesn’t assemble every widget by hand. They design the line and hold the quality bar. The same move is available to a data team, and it is the single biggest change in what the job is.
Specification quality is the new bottleneck
When implementation collapses to minutes, the constraint moves to how precisely you can say what you want. In data that means grain, exclusions and the blocking questions nobody wanted to answer. The queue doesn’t disappear. It moves to the front, where it’s cheap.
Give success criteria, not steps
Agents do better against a stated bar than a stated procedure. In data the bar is the claim set: what must hold for this number to be right. Write those first and you’ve written the spec, the tests and the controls in one pass.
Architecture stays human
Trade-offs depend on business context a model can’t reach. Whether the regulator counts at business partner or contract account isn’t in your schema. It’s in a licence condition and somebody’s head. That decision was always the job. Now it’s the whole job.
Vibe analytics is cheap the way a payday loan is cheap
Low upfront cost, punishing running cost. The bill doesn’t arrive as tokens. It arrives as the reconciliation meeting, the third rebuild of the same measure, and the quarter where two teams both prove they’re right.
No reusable semantics, so the next question re-derives the same meaning slightly differently. No stated grain, so the reconciliation tax comes due every quarter. No retained evidence, so an audit is a reconstruction project. It compounds, quietly, and it never shows up on the line item that bought it.
Agree the meaning once, as a machine-readable artifact, and the marginal cost of the next data product against the same estate drops hard. The controls are already bound. The concepts already exist. The argument has already been had, and it’s written down.
In code the last 20% is hard. In data it’s invisible.
An agent gets you 80% of a feature and struggles with the edge cases. Same in analytics, except the failures are conceptual rather than syntactic: a wrong assumption about business logic, a missing exclusion, an unasked clarification. Code that’s 80% right throws an exception. A number that’s 80% right gets a slide.
Wrong assumption about the business
The agent picked a grain. It didn’t know there was a decision to make, so it didn’t flag one.
No clarification sought
It answered instead of asking. Nothing in the output records what it assumed to get there.
Looks right, passes the tests
Structure is intact. Meaning isn’t. Every check you own is aimed at the half that didn’t break.
The lifecycle is the harness
Every stage exists to do one of the jobs the discipline demands: specify precisely, constrain the agent, verify both the output and the path, and keep the evidence.
Structure scales. Vibes don’t.
AI amplifies the data culture it lands in: a team with stated grain and bound controls gets dramatically more out of it than one without, because AI multiplies your weaknesses just as fast as your strengths. And the human role isn’t shrinking, it’s changing name. The people who can specify precisely, judge output critically and design the constraints are the agentic data analysts, and they’re worth more than they have ever been.