What did a smooth launch actually prove?

“It went fine last time” leaves out quite a lot.

Annie Duke, the decision-making author and former poker player, calls it “resulting” when we use the outcome to decide whether the original choice was a good one. A good result makes the choice look sensible, even when we haven't really examined it.

Imagine a team releasing a new customer portal. They decide to skip testing it at the traffic level expected from the launch emails. The portal stays up, everyone is pleased, and skipping the test starts to look like a useful saving for the next release.

But the emails were delayed. The surge of customers they expected never arrived.

The portal has shown it can handle the traffic it received. That tells us something, but it hasn't answered the question the test was meant to answer.

We'd want the team to look at what they knew when they dropped the test, and why they thought it was reasonable. They can use the launch results too, once they're clear about what actually happened.

This is part of the psychology behind the Delivery Leadership System. Success can make us less curious about how we achieved it.

Next time, those emails may go out exactly when they're supposed to.