Ship one change on a quiet Tuesday afternoon, watch checkout completion tick up half a point over the next few hours, and you have something you can almost believe. The release has a timestamp. The metric has a before and an after. Nothing else went out that day. The line that rises after your commit is, as much as a line ever is, about your commit. That is the whole promise of tying a number to a deploy: the change you made is the change you can read.
Now ship two things twenty minutes apart. A pricing tweak at 2:10, a checkout rewrite at 2:30. An hour later checkout completion is up a full point — a genuinely good afternoon. And you cannot say whose afternoon it was. The lift sits on top of both releases at once. Maybe the pricing tweak did all the work and the rewrite did nothing. Maybe it’s the reverse. Maybe each did half. Every one of those stories fits the same rising line exactly, and the line has no way to tell you which one is true. The window holds two changes, and the number in it belongs to both of them and to neither.
The tempting arithmetic
The instinct is to split the difference. Two deploys, one point of lift — call it half a point each and move on. It reads as fair. It is also invented. Nothing in the data supports a fifty-fifty split; you chose it because it’s tidy, not because it’s true. The pricing tweak might have been worth the whole point and the rewrite a wash, in which case your neat halving just handed a failing change half a medal and docked a winning one half its due.
Worse, the split looks like a measurement. Once “checkout rewrite: +0.5pt” is sitting in the readout with a number next to it, nobody remembers it was a guess dressed as a reading. It gets quoted in standup. It goes in the deck. Someone greenlights three more rewrites on the strength of a figure that was never anything but arithmetic performed on an unanswerable question. A wrong number with a decimal point does more damage than an honest blank, because the decimal point is a claim of precision the situation never had.
Two changes in one window don’t give you two small signals. They give you one signal you can’t divide. The lift is real; its owner is unknown; and no amount of arithmetic on the total will conjure an author the data doesn’t contain.
Overlap is the default, not the exception
You might think this is a corner case — that most changes ship alone into clean air. They don’t. A team that ships well ships often, and often means close together. Deploys cluster around the start of the day, bunch up before a Friday freeze, pile on top of each other during a push. The quiet Tuesday with exactly one release in it is the rare one. Any tool that only tells the truth when a change ships in isolation is a tool that tells the truth on the days you needed it least.
And it isn’t only your own deploys that crowd the window. A marketing email goes out at ten. A competitor’s site goes down at noon. Payday lands on the first of the month and everyone’s cart gets a little braver. The window around your change is full of other people’s changes, most of which never touch your repo and none of which announce themselves. A baseline drawn from the trend before you shipped catches some of that — the slow, standing stuff. It does nothing for the other deploy that landed inside the very same hour, because that one wasn’t part of the before. It’s part of the after, right alongside yours.
What the honest tool does instead
So code-results does the unglamorous thing: when two changes share a window, it declines to hand either of them a clean number. It draws the measurement window around each deploy — the stretch it would need to watch to read that change fairly — and when two of those windows overlap, it marks the pair confounded and holds the figure. Not a zero. Not a shrug. A specific, legible state that says: a real thing happened here, and it cannot yet be assigned. The lift is shown. The ownership is withheld. The readout would rather post a blank it can defend than a decimal it made up.
Held is not the same as lost. A confounded window is a question waiting for the noise to clear, and most clear on their own. Ship nothing else for a day and the slower metric finally reads against one change alone. Revert the pricing tweak and the checkout rewrite is suddenly alone in its window with a clean before and after. Let the two changes recur on different days — the tweak again on Thursday, the rewrite again next week — and their effects start to separate across the repetitions, the way two overlapping voices come apart when you hear each one alone somewhere else. The tool’s job in the meantime is not to fake an answer. It’s to remember the question, keep the window open, and settle the number the moment the window earns a single owner.
Confounded is a reading, not a failure
It would be easy to treat “confounded” as the tool coming up empty — a gap where a metric should be. It’s the opposite. Learning that two of your changes can’t be told apart is information, and often the useful kind. It tells you your release cadence is out-running your ability to read your own results: you are shipping faster than you can learn, which means some of those changes are flying blind and you didn’t know it. The fix isn’t a cleverer estimator. It’s twenty minutes of daylight between the two deploys you actually care about reading — cheap, once you know which pair is stepping on each other’s line.
That is the same discipline that makes a confirmed null worth shipping and a guardrail worth watching: the readout is only worth trusting on its good days if it refuses to overclaim on its hard ones. A tool that invents an owner for every lift is easy to like and impossible to believe. The one that says I can’t split this yet, and here’s exactly why is the one you can still believe six months later, when the number it finally does hand you is standing alone in a window it earned by itself.
The lift was real. It had two possible authors and the window couldn’t name one. So code-results didn’t name one either — it held the pen, kept the page open, and waited for the afternoon that would.