Estimating, what is it good for?
We just had a very interesting discussion about estimates. We started this discussion with re-assessing the weight of our story points, but it lead us to talking about why we need estimates.
I think the ideal goal of any framework or methodology, as for managers, is to disappear and lead to anarchy (but not chaos). Therefore, I look for any occasion to get rid of iterations, estimates, follow-up, or any other method or tool. I’d like to get to a minimum process focusing on communication, and then make it disappear as it becomes natural.
Estimates can be useful, but they are expensive. And we weren’t clear about where they were useful. So we tried to think out of the box, and rebooted our thinking about the reasons why we needed estimates.
In a recent post about transitioning to agile, Dean Stevens said:
[We can] explain some benefits of an Agile Transformation this way:
- Business wants predictable and improved throughput.
- Customers want quick lead times for quality product.
- Teams want a good working environment where they can contribute and succeed.
I thought the ultimate goal of estimating was to come up with a long term planning (few months) with some confidence. From my experience, customers demand some planning for their requirements.
But I was explained that getting to that end to sell something was a commercial failure. Long term planning is never reliable. It is not when it comes from the provider, and it is not when it comes from the customer! I know you rarely deliver exactly what you planned a few months ago. And does your customer still exactly needs what he requested 6 months ago? Is he still ready to use it now that you’re delivering him?
Negotiating such plannings in a precise way is a poker game, where everybody looses.
You can still have a high level long term planning, but be clear on the fact that it is nothing more than an unclear wish-list based on the knowledge you have today. Don’t constrain your development process for this objective. Manage long-term planning outside of the development process, using tools like portfolio management or epic/theme backlog, where you estimate your epic/themes in comparison with other [done] epic/themes, and not development items like user stories.
The agile community agrees on the use of the discussion around estimates. I totally agree with it. Estimating is a great way to make sure we have a discussion. It’s a great way to detect comprehension gaps between team members (devs, qa, business,…). It also allows devs to agree on smaller chunks of the story (tasks) to run in parallel in order to swarm. By having this discussion, and the objective of a consensus around estimates, we have a better chance to align our comprehension of the why, the what, and the how.
But estimating is only one way to get there. We could get to the same point with a different objective, like colorful or timely drawings, a list or a number of tests, mock-ups,… So again, if the goal of the estimate is only the discussion, if it’s a game among others to run during the iteration planning, we should not make the rest of the process heavier for it.
What about the process? Through estimates, you get a throughput. By making this throughput predictable, you make the process and the backlog predictable. By optimizing your throughput, you optimize your whole process. By detecting deviations in the throughput, you detect exceptions to your standard process (i.e. deviant deviations), and thus optimize it. And so on…
But there are other ways, like following up the cycle time, the WIP, and thus bottlenecks. You can monitor your throughput as a number of items, and not a number or story points. If your items are within a reasonable size, the standard deviation of your throughput and cycle time should be acceptable (compared to estimating error or your constraints). Agreeing whether an item is of a “reasonable size” is estimating. But I believe it is a lot lighter than formal methods like planning poker.
It leaves us to the short-term planning. Estimates are useful to know what you’ll do in the next few weeks. Like for long-term planning, I’m not clear on the goal of this. Making sure you respect due dates is definitely a fair objective. Motivating teams with an objective to reach is another one. Prioritizing considering constraints and slack is also important. Having a prioritized backlog where teams can pull work without bothering a PO is great. Fine, we’ll suppose you need to know what you’ll be able to produce in the next few weeks. And drum roll…
BUT is estimating useful to have a prioritized development backlog and a short-term planning? It is if your throughput and cycle time standard deviations are too important. If they are, you need to sort your items by categories where these standard deviations are acceptable.
As a conclusion, what you need is buckets where cycle time and bandwidth occupation are close enough for all items. If you only have one bucket, it’s perfect. If you have more than 4 or 5 buckets, you should really think about it. Estimating is knowing which bucket fits your item, or splitting an item so that all sub-items can fit one of your buckets.
For different levels of plannings, you can have different buckets. For example, you may have development buckets for the development backlog, and release buckets for the long-term backlog. Those buckets are not of the same nature. Neither are the items they contain. Don’t try to fit development items in long-term backlogs, or long-term items (epics/themes) in the development backlog, without further analysis.
As a conclusion conclusion, monitor your throughput, your cycle times, and their standard deviations, identify your buckets, and make them the least possible.
What do you think?
AAAgile 
I’m agree with you. Except that this is possible with a team who’s mature enough to do so. It is also usefull to have estimates to get commitment from the team. If you can reach it with the cycle time process, your in business.
Thanks for the comment Martin,
Given our addiction to estimating, I agree that giving up estimates is probably only for mature teams. But thinking about it, I wonder if immature teams can estimate. And I wonder if stories of a reasonable size wouldn’t be a good point to start with, before estimating. I might be dreaming, but who knows.
Scrum’s commitment is not a thing I’ve really seen working on the long term, apart from labs isolated from real life, for a few iterations. And again, it’s not trustworthy from immature teams. I’d like to see it working, but I’ve not yet (for many reasons, I won’t elaborate here, it could be a whole article and it’s already discussed in other blogs). But velocity, or average throughput, as a number of story points or a number of stories, works as soon as it’s predictable.
Finally, what the business wants is that we respect our commitments (i.e. due dates), and that we can deliver often and quickly (i.e. cycle time), in a predictable way. So what I’m thinking about is: why do we focus on indirect metrics like estimates when we can easily get the metrics we’re looking for?
My point is probably idealistic, and you perfectly showed it in your comment. I need to experiment it a bit more, but I think I have something worth trying.
This debate has been had many times before. What you’re describing is right-sizing, and right sizing is still estimating — it’s just a different type of estimating.
Yep, I agree with you Charles. I’m just looking for ways of making it as quick and efficient as possible. Using data about what we did yesterday definitely helps. And as I said at the beginning of this post, the discussion was also the perfect occasion to think about it from scratch. This discussion showed us we weren’t clear about why we made estimates, and I’m sure it’s case for most teams/organizations. I hate taking things for granted, and I love controversy. It helps us think and innovate.
Story points also uses data about what we did yesterday(historical velocity), and has other added advantages. You mention that estimating is expensive. I would suggest that it is only expensive if you let it be expensive. The best estimate facilitators know that there is a point past which further discussion about estimates doesn’t lead to more accurate estimates. The same is true of all estimating techniques, whether it be planning poker or right sizing. As such, right sizing can be just as time consuming and wasteful as any other estimation technique.
Once again I completely agree with you, thanks for this discussion. It’s just another way of doing it. The idea behind this article is to challenge and understand estimates (or right sizing) more deeply, and getting back to what really matters (like cycle time). And my hope with a technique like right sizing is to get slowly from scrum’s planning poker to kanban’s one size for all. Another way to say it is I hope to have only 1 story point stories. Anyway, I hope to have more data in the future and dig a bit deeper.
Fair enough, though I will note that Planning Poker is not a Scrum practice. It is an XP originated practice that has been used by many Scrum practitioners.
As an aside, how exactly does cycle time matter? In software development, I have misgivings about all metrics, because I haven’t yet found many that are easily and correctly consumable by wide audiences. If a software metrics expert uses them, I’m less concerned. The problem is that there aren’t many software metrics experts out there in the biz.
Btw, I have read David Anderson’s Kanban book, so you can assume that I have that knowledge.
Charles,
I never doubted you had all the knowledge, not only by the book 😉 And anyway, I don’t think our discussions require a diploma, other than experience and introspection. Everyone is welcome. Even I can talk!
I consider cycle time as a metric that matters cause it’s supposedly correlated to lead time, which is what the customer expects when he asks for a fix or feature. Or at least, it’s the part of the lead time that we can optimize as developers.
It’s true that it highly depends on organizations, and that the one I’m working in now is quite special: like a start-up, we are just a lab at a software vendor, just starting to get actual users. That’s why I don’t put throughput aside, cause it’s a prerequisite to estimate a release date for a larger bunch of output, like a version or a theme.
I also realized when I finished this article that the vocabulary was quite technical. If I knew more meaningful words for things like throughput, lead/cycle time, velocity, story points, backlog items,… that I wouldn’t need to explain every time, I would definitely use them.
I think you should be very careful about optimizing cycle time(as you have defined it here). Trying to optimize cycle time can lead to poor code quality if you don’t have rigid quality standards in place. Do you have rigid quality standards in place? If so, what are they?
It can also lead to people trying to increase speed so much that they fail to be creative and focus more on delivering value. To me, this is the inherent danger of trying to optimize cycle time.
Regarding the vocabulary thing, I think you used appropriate vocabulary — it’s just that the vocabulary is still a bit ambiguous in this space — esp the lean space. See here for an example of that:
http://www.isixsigma.com/methodology/lead-time-vs-cycle-time/
I’m a blogger too, so I can tell you, once you get to a certain level of expertise, AND you communicate a lot via prose (email, blogging), it will be important to understand definitions because not understanding them can lead to a last of wasted time in discussions. This issue also comes up often on development teams. 🙂
About vocabulary: my problem is less being precise than understood by non experts. When I use technical terms, like in this post, I try to use them in an appropriate way, though I sometimes miss my goal, as you pointed out. But as you underlined about being misunderstood by non-experts, I’d prefer using terms that everyone can understand without further explanations.
About quality: so far, if we had issues with quality, it wasn’t due to external pressure. We are careful about not trading quality (code reviews, careful testing, design evolution) for the pride of pretty figures. Metrics must remain an indicator among others. You’re right, it is a real danger.
Sounds good. If you’re disciplined about quality, and you realize that metrics are sometimes not terribly useful, then I think you’ll be fine. Also, remember that external pressure is not the only thing that can affect behavior. Sometimes internal team behavior, trying to meet a goal (reducing cycle time, for instance), can also end up with the same “deadline behavior.”