We are consistently bad at estimation and are unapologetic about it. While tasks and projects overrun their estimates by 81% and 122% respectively, engineers often respond by saying that accurate estimation is impossible or not even valuable in the first place. We even come up with half-jokes such as the ninety-ninety rule to portray our inability as a natural law. However, accurate estimation can help a lot with planning, resource allocation, risk management, as well as minimizing resource waste and work in progress. To get better at estimation, we first need to start taking it seriously as a skill that has to be learned and practiced.
Why Is Estimation Hard?
Before we look at solutions, let’s identify the problem. There are several things that work against us when estimating the amount of time a task will take. I break them down in the following sections.
Cognitive Biases
Daniel Kahneman and Amos Tversky coined the metaphor of two systems operating concurrently in our brains: system 1 operates automatically and quickly, with little or no effort and no sense of voluntary control, whereas system 2 is much slower and allocates attention to the effortful mental activities that demand it, including complex computations. The interplay of the two systems minimizes effort and optimizes performance. System 1 however has numerous biases and produces systematic errors. It is the part of us that jumps to conclusions, focuses on the wrong problem, or gets thrown off by unimportant details and it’s especially active when we try to estimate uncertain tasks.
The most relevant to our discussion is the planning fallacy, which is a widespread tendency to produce overly optimistic plans that are unrealistically close to the best-case scenarios. It is the reason why we consistently overestimate how much we can do on a weekend, for example. Its main cause is basing forecasts on the readily accessible information, while ignoring the “unknown unknowns” as well as the statistics about the past durations of similar projects. In practice this means that when planning, we focus on the parts of the project that are familiar and extrapolate from them, overestimate our level of focus and commitment, and don’t allow for the inevitable disruptions. To make it worse, we are either unaware of the duration and success rate of similar projects, or choose to ignore this information because “this time is different.”
Another important cognitive bias is anchoring. I frequently observe it when a manager asks their subordinate a question like “Will you be able to get this done in N days?” The number that is mentioned in the question biases the rest of discussion and will tend to produce estimates that are close to it. Questions about estimates should always be open-ended if you want the best chance of getting an accurate answer.
Misaligned Incentives
Sometimes getting an accurate estimation is not an actual goal of the estimation process. For example, when estimation is part of bidding, there is a strong tendency to produce a lower estimate with the goal of raising it after the bid is won. Conversely, when estimation is for an internal project, there is an incentive to play it safe and overestimate, since finishing a project ahead of schedule is seen as a positive. There are many other cases—for example if a person is being micromanaged, their defense mechanism can be to artificially pad the estimates in order to avoid unwanted attention.
Then, there is the case of lack of trust and poor communication between engineers and management. For example, in projects with unrealistic deadlines, engineers can unintentionally “sign up” for unlimited overtime just by producing a wrong estimate. Tracy Kidder illustrates this in The Soul of A New Machine:
There was, it appeared, a mysterious rite of initiation through which, in one way or another, almost every member of the team passed. The term that the old hands used for this rite—West invented the term, not the practice—was “signing up.” By signing up for the project you agreed to do whatever was necessary for success. You agreed to forsake, if necessary, family, hobbies, and friends—if you had any of these left (and you might not if you had signed up too many times before). From a manager’s point of view, the practical virtues of the ritual were manifold. Labor was no longer coerced. Labor volunteered.
This “signing up” can happen unintentionally when an engineer carelessly gives an estimate which then morphs into a binding commitment. Unsurprisingly, engineers come up with all sort of tricks in order to avoid accidentally “signing up,” such as providing vague estimations or using story points that cannot be translated into anything tangible.
Not Taking Estimation Seriously
We spend a lot of time learning and practicing skills such as software design, writing tests, incident response, or interviewing. However we don’t give estimation the same level of importance. I rarely see retrospectives on estimation, different methods of estimation being discussed, senior engineers coaching or giving feedback on estimation technique and accuracy. At best, we have superficial discussions about planning poker and story points. Estimation is a skill just like any other one that can be measured, learned, practiced and improved.
Improving Your Estimations
Luckily, with some practice you can not only improve the accuracy of your estimation, but also improve consistency and even quantify the level of uncertainty you have about estimations. In fact, almost anything can be estimated and measured, including task duration and complexity, probability of project success, software quality, team cohesion, etc.
Measure and Improve
The first step is to recognize estimation as an important skill and an essential part of project work. Explain the value of estimation to your team, collect data on how accurate your historical estimates have been, decide on a target condition, brainstorm on how to get there, iterate, recognize mistakes, and celebrate achievements. A good way to put some structure around this is to setup a continuous improvement process that follows a plan-do-check-act cycle. Getting better at estimation will give the team a sense of mastery and pride and help build trust between engineering and business.
Make Uncertainty Explicit
Estimates are by definition uncertain. This means that there are multiple possibilities for how long a task might take. Say, if you are really lucky, you can get it done in one day, worst case it will take five days, but you are pretty confident it will be somewhere in this range. In the beginning, when there are lots of unknowns, the range might be very large; with more information and more certainty, it shrinks.
Thinking in terms of ranges forces us to consider both the best-case and worst-case scenarios and helps mitigate the optimism bias and planning fallacy. It also makes it clear what the biggest unknowns about the work are. Gaining more information about these unknowns (e.g., though prototyping, experiments, or research) helps shrink the range and improve the accuracy of the estimate.
Decompose It
One of such ways to reduce uncertainty is decomposition. It involves figuring out how to compute something very uncertain from its constituent parts that are less uncertain or are easier to measure. An obvious way is to break down a task into smaller parts and estimate them individually. You can also model the total estimate as a function of multiple variables, for example, ”total time = X * number of APIs to build + Y * teams to coordinate with + Z * clients to migrate” and then estimate individual variables. Once you have estimated probability distributions of individual components, you can use a Monte Carlo simulation to estimate the complete task.
Start With Historical Data
How would you estimate a task that you know little about? The best you can do is find comparable tasks that you or someone else completed and use their durations as the baseline estimation range. Once more information is available, it can be used to refine the initial estimate. This is essentially Bayesian reasoning applied to estimation.
Stick to Commitments
Once you make uncertainty of your estimations explicit, you’ll be less likely to get pressured into accepting an unrealistic deadline. However, if you do estimate and commit to getting something done by a certain date, treat meeting this commitment as a matter of professional honor. Being consistent helps build trust and makes it clear that your estimates are a result of serious thought and they cannot be changed arbitrarily, just by asking you to try harder.
Conclusion
Estimation is an important part of our engineering toolbox, it’s not that hard to learn and it can provide a lot of value if done well. Of course, it can also be taken too far, where unnecessarily detailed plans are created too far into future. The appropriate effort depends on the project context, level of uncertainty, and the value gained by reducing risks. In my experience, teams that neglect estimation end up with missed deadlines, late discovered risks, and poor trust with business stakeholders. We can improve this by approaching estimation with the same level of rigor we approach other engineering tasks and investing more in the areas where it matters.