Hans Samios' Personal Lean-Agile Knowledge Base

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Why Do People Overcommit? ======

And why is over committing a problem?

One of the base Agile Principles is that we try to set things up so that Teams operate at a sustainable pace. When Teams hear about this there is general excitement that it might be possible, that they might be able to restore some kind of work / life balance. 

Then there is the belief that “they won’t let us do that around here.” This is a rational concern in many organizations as Management often believes that being busy is a sign of being productive and useful, and that management tools like “stretch goals” and “mandatory overtime” will result in increased delivery of value.

Coaches are aware of this, and so coach management that they need to be very positive about how they want to have a sustainable pace. Management (often though coaching) understand that they have an interest in a plan based on reality to improve the predictability of work delivered. They begin to understand that to improve how much is delivered (throughput) they cannot overload the Team as this increases the amount of work-in-progress. They begin to realize that one reason the organization are annoying their customers is that we say we will do something, and then fail to meet that commitment simply because there is too much work to get done. In other words Management has a reason to be positive about the sustainable pace and can be authentic in their delivery. They are looking forward to the day when the Team makes and meets commitments more often than not.

But still it inevitably happens. All the work is now visible so we know Teams are trying to take on more than is realistically possible. We try to help the Team, but they don’t seem to be able to do anything about it.

Perhaps we leave it this time. “They’ll learn when they don’t make it.” But the next time it happens, and the next time, and the next time. But now it’s worse. The original excitement that Agile will deliver a work / life balance is gone since the Team has to work hard to meet the commitments they made.

I’ve seen this pattern happen over and over again. I once ran a survey of 65 Teams and found that the average overcommit of the Teams was 42% and that about 69% of Teams over-committed on a regular basis. 

====== What Factors Contribute to Teams Over Committing? ======

The question we have to ask ourselves is “why?” After all, no one is forcing the Team to do this, right? Management thinking is correct - if we don’t address the problem we will remain unpredictable in our delivery, and have reduced throughput due to high work in progress. Over time, we will further reduce throughput due to a reduction in engagement of the Team. Worse, our customers will see no improvement in our ability to make and meet commitments.

My view is that there are a number of factors which contribute at the individual and Team level:

  * Image management: We want to look good, both as an individual and as part of a super Team. We therefore want to be seen to be doing more, to provide more value.
  * Not understanding how to say “not now”: Many organizations (especially IT organizations) have a policy to say “yes” to all work. This history has not taught Teams how to say “not now” to their customers and stakeholders. Even if they know they cannot do the work.
  * Optimism bias: People overestimate the likelihood of positive outcomes when they look into the future. There is benefit in having a positive outlook, but in this case we are letting this attitude adversely affect our ability to make rational decisions.
  * The retrospective bet: The last retrospective was very positive and the Team determined a problem and a solution. The Team bets on this result. This means that we assume we will get better before we actually get better.
  * A genuine want to help: Knowledge workers really want to help and so when someone says “I just need this ...” they will take it on themselves to get it done, especially if they know the person asking.
  * Hero culture: The organization has traditionally rewarded (one could even say developed) the heroes of an organization. This is so ingrained that people are not even aware they are operating this way.
  * Uncontrolled work intake system: Many Teams, especially when starting, do not have control of their work intake system. This means they are unable to plan and predict the work, and are impacted more than expected by “break ins”, for example.
  * Invisible work: The Team thinks it has a view of all the work, but in reality a lot of work is hidden. This hidden work usually impacts the ability of the Team to deliver on their commitments.
  * Poor estimations: Early in a Team’s life the Team will not have a good understanding of what it takes to deliver value which often results in over-committing.
  * Return work: When work is completed, there are often issues associated with the initial delivery, and time must be taken by the Team to address these issues. This is often not factored into the capacity of the Team.

And sometimes Management do not help the situation:

  * Management talks, but does not walk, Agile: For example, there is a conversation on over-committing, but no real action. Message to Team “situation normal: just keep saying yes.” Or management says “Thank-you Brent for taking this on.” Message to Team “we will continue to reward heroes.”
  * Management implies: More insidious, because it is harder to see, is when management say things that they intend to be received as "do what is realistically possible" but it comes across to the Team as "we want you to do this, no matter what". Phrases like "we challenge the team to" and "please schedule this effort into your upcoming sprints" are often seen as requests to do something regardless, especially in organizations where this kind of thinking was the norm, irrespective of the intent of management.
  * Management unwillingness to make the hard calls: Sometimes management are presented with a problem and are unable to make a decision. Even though the Team has evaluated various positions management response is “If we just did it like this ... we can do both of these ...”. Spoken like a person who doesn’t actually have to do the work.
  * Management unwillingness to protect the Team from the consequences of “not now”: While Teams make the call, and Management agree, you will often see Management fold when their customer or stakeholder complains about the decision. Don’t get me wrong, there is often a need to change plans, but if that discussion does not include a discussion about taking work off the Team, all management is really saying is “you have to do it.” 

In many ways these are all examples of short term thinking that has huge consequences in the long term.

This assumes that management understands the issue and want to do something about it. Organizations sometimes have a culture that simply doesn't have an interest in what is actually possible, where the date and what we want by that date is seen as the only driving factor. This type of situation is often seen where the development function is treated as a feature factory. Demands come in from the business or product management to the development function but since there is no relationship between the amount of the demand and what can actually get done, the development function is doomed for failure. Rather than work to address the issue, people demand more as, after all, they didn't get what they wanted last time, repeating the cycle and creating a hostage-like environment. (Note: Sadly, in my snarkier moments, I picture a screaming spoilt child demanding more and more while parents struggle to accomodate. Not a worthy picture.)

====== What Can We Do To Reduce Need to Overcommit? ======

The question is “what can we do to improve the situation?”

Assuming management wants to engage and actually address the issue, there are some obvious things we can do. Firstly, we can help our Teams understand that these drivers exist and that we often create problems for ourselves. Sometimes a simple discussion raising awareness will help. This is true of both the above individual and environmental factors which lead to over committing. For example:

  * Track data: Assuming you don't have this information, start tracking the relevant data to make the problem visible. Depending on need, track demand vs capacity and the say / do ratio (make / meet commitments) and ensure people can see what is going on.
  * Optimism bias: Many people are unaware of the optimism bias we all have. You can make people aware of this bias by having a discussion around the [[https://www.ispot.tv/ad/7Jim/prudential-the-prudential-magnets-experiment|Prudential Ad About Thinking Positive in the Future]]
  * Hero Culture: You can talk about the Hero culture in the context of books like the [[https://www.amazon.com/Phoenix-Project-DevOps-Helping-Business/dp/0988262592|“Phoenix Project”]]. 
  * Yesterday's Weather Forecasting: We could talk about using “[[https://www.martinfowler.com/bliki/YesterdaysWeather.html|Yesterday’s Weather]]” approach to determining how much work to take on instead of assuming we can do more. 
  * Capacity Buffer: A capacity buffer could be added or increased to:
    * Reflect the unknowns in the work intake system. The idea here is to force the Team to commit to less “planned” items until the intake system stabilizes.
    * Reflect the fact of that not everything will go to plan even if the system is relatively stable. This type of thinking goes back to [[https://a.co/d/bDLu7cF|"Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency" by Tom DeMarco]] and, more recently to [[https://a.co/d/8x9gKvO|"Goldratt's Rules of Flow" by Efrat Goldratt-Ashlag]].
  * Retrospective: Run a single subject retrospectives on the problem, and discuss things like estimates, to improve the Team’s understanding of the work, impact of not meeting commitment, etc..
  * Visibility: Team Members could ensure that “everything is on the board” to reduce the impact of hidden work.
  * Value Stream Mapping: Teams could do a value stream mapping exercise to understand percentage of work that is accurate and complete, and so work to reduce the amount of work that returns.
  * Management authenticity: Management could show awareness of their fallibility by admitting to past errors and discussing how they will address going forward.

We also might try a more “culture oriented” approach. For example, we could adopt a mantra like:

> Under-commit; Over-deliver.

We would set the cultural expectation that it is OK to under-commit so long as we meet the resultant expectation. Some Management will worry than this will mean that Teams will slack off. The data above shows that we do not have this problem in reality. And wouldn’t you love to be in the room when a Team reports to the customer saying “we were able to complete the committed items and, since we had spare capacity, we also delivered this high priority item you were wanting.” BTW: I have actually seen this happen.

Often these approaches have been tried and you still find a pattern of over commitment. In these situations the Team might want to set up a single subject Retrospective to discuss approaches to improve. This might be a place to discuss the issue, raise awareness of factors, and review possible approaches. I suspect that when you ask the Team, there will be other, more specific factors and potential approaches to take.

For example, one factor that we often see is that while overcommit data is available, it is also ignored. It’s just a number on the screen. One idea to make this more visceral is to hand out tokens which represent capacity and as people plan work, take the appropriate number of tokens away. Watch for an interesting discussion to develop when people run out of tokens and still have more demand (“we have to get this done!”)

And finally we could also consider other more direct and experimental approaches. For example:

  * 1/2 Velocity experiment: Team sets up an experiment by saying “Why don’t we set up an experiment where we plan to half our normal velocity as an artificial limit just to see if it helps.” The idea here is that by reducing the commitment the Team will actually produce more and feel better about the result.
  * 1/2 Iteration experiment: Team sets up an experiment by saying “Why don’t we set up an experiment where we plan to half an Iteration just to see if it helps.” The idea here is that it easier to plan a short term future than a long term one.

Note: These ideas can be applied at all levels of Iteration - a Team 2 week cadence, a Program quarterly cadence. 

The bottom line is that over commitment is a problem that needs to be addressed if we really want to achieve a sustainable pace for the Teams, manage the expectations of our customers, ensure good work / life balance for our people increasing their engagement, and increase throughput. 

====== Want to Know More? ======

  * [[our_estimates_are_terrible|Our Estimates are Terrible!]]
  * [[https://www.amazon.com/Phoenix-Project-DevOps-Helping-Business/dp/0988262592|“Phoenix Project”]]
  * [[https://www.martinfowler.com/bliki/YesterdaysWeather.html|Yesterday’s Weather]] 
  * [[https://a.co/d/bDLu7cF|"Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency" by Tom DeMarco]] 
  * [[https://a.co/d/8x9gKvO|"Goldratt's Rules of Flow" by Efrat Goldratt-Ashlag]]


{{tag>FAQ commit WIP behavior Team}}