It’s easy to look up what an engineering team shipped last quarter. The changelog and merged pull requests provide a record. Finding out whether those changes improved activation or retention takes more work, and the answer may still be hard to isolate.
Development tools track tests, reviews, and deployments closely. Production monitoring tells us when a release is broken or slow. But those records don’t usually tell the team whether a change had the business effect they intended.
What shipping metrics leave out
Merged pull requests and deploys per day are already logged, which makes them convenient progress measures. They show how much a team shipped. To evaluate the work, we also need to look at the numbers the changes were supposed to improve.
A busy quarter can leave sign-ups, activation, revenue, and retention unchanged. If we only count releases, we miss that result and lose the chance to reconsider what we’re building.
Every deploy is a hypothesis
Suppose we redesign onboarding because we expect it to improve activation. That gives us a specific question to revisit after the change reaches users: did activation improve?
By the time the numbers settle, the team may already be working on another feature. Unless someone returns to that question, the assumption behind the work goes unchecked. We wanted an easier way to look back at a specific deploy and see what followed it.
Putting deploys on the metric timeline
code-results puts dated deploys alongside the business metrics a company watches. The team can compare a metric before and after a release, and see which changes might be worth investigating.
A deploy and a metric moving at the same time is a correlation, not proof of causation. Revenue might climb because of a holiday spike. A useful feature might arrive in the same week a competitor cuts its prices, leaving the metric flat. code-results shows the timing; the team still has to investigate other explanations before attributing a result to the release.
Why a tiny studio cares about this
We wanted this for the products we operate ourselves. When we make a change, we want to come back a few weeks later and check whether it helped. The tools we used made that harder than finding the release itself.
Like ganttchart.ai, which came from needing a plan before a meeting, code-results began with a question we kept needing to answer. Keeping the deploy and the metric together gives us a place to start.