Part One: Strategy · Deciding what to build
Shipping feels like the finish, and the next piece of work is waiting. Whether the change helped anyone can only be seen afterward.
Releasing something is satisfying, and it is what gets counted. The plan marks the item done, and the people who built it are moved to the next thing. But shipping was never the goal. The goal was that somebody's trouble would go away. Those are two separate events, and the second may not have happened.
The feature may not have been found. It may have been found and misunderstood. It may be used and not help. Without looking you cannot tell which, and you build the next thing on a guess.
So the work is finished when the people who built it have checked, and not before. The item stays open on the plan, with their names on it, until they have looked. That takes two things written down ahead of time: how things stand today, and what should be different, for whom. Without the first there is nothing to compare with. A few weeks after release, go back to the people whose trouble it was. Has the workaround gone? Counts help, and where there are enough people you can give the change to half of them and compare. A count says how many. A visit says why.
A site-reporting app for building contractors adds photos to the daily report, so that foremen can stop texting pictures to the office. Before the change, one office manager counted the photos that arrived by text in a week. A month after, she counted again, and little had moved. Two site visits explained it. On a weak signal the upload failed and said nothing. A text at least queued.
What you find is a new problem. Give it a limit of its own, as in DECIDE WHAT IT'S WORTH (21), or take the feature out.
Therefore:
Before you build, write down how things stand and what should change, for whom. A few weeks after shipping, go and look. If nothing changed, set a limit on fixing it or take it out.
In practice. On the day you ship, put a date in the calendar four weeks out: "Did it help?" Add today's number and the names of two customers to ask.
Goes with. What to count is in MEASURE PROGRESS, NOT ATTENTION (29), and the way to look is WATCH SOMEONE USE IT (59). If the answer is no, TAKE THINGS AWAY (27).