How Do We Assign Business Value to PI Objectives?

Premise

Seems to be a lot of question the use of “value” with PI Objectives, around the process of setting the value of PI Objectives, and how they are worked during the Program Increment.

Background

Been a need for a while but 2018-04-10-SuperSet(SDM)InspectAndAdapt brought it to the fore.

Paired with 2018-04-30-WritingPIObjectives.

What Problems Have You Seen With the Business Value Assignment Process?

The general literature calls for the Business Owner to go from team to team during a PI Planning event, and assign values in the range of 10 (high value) to 1 (low value) on all the team's PI Objectives on a relative scale.

Sounds easy, doens't it? But there seems to be a lot of different approaches to assigning Business Value to PI Objectives, the process seems to cause a lot of confusion, not to mention misunderstanding, and it often leads to people abandoning the process all together. Some problems I have seen, based on a particular approach to generating the Business Value include:

The Business Owners have some kind of an absolution scale in their head for business value. Perhaps the “things that support this strategic initiative that is near and dear to my heart automatically get a 10”. What this means is that the team that is doing that “strategic” work automatically gets a number of 10's whereas all the other teams get 1's. Does this mean the work of the other teams is seriously less important that that 1 team? No matter how hard you try to explain otherwise, those teams with the lessor scores are not going to be motiviated.

The Business Owners wonder how they will “compare” the work of the different teams. They often feel like they do not have sufficent context to determine whether PI Objective 3 of Team 1 is more valuable than PI Objective 2 of Team 2. Lets face it, while there is a lot of value information around, the PI Objectives are seen for the first time by the Business Owners at the PI Planning event. This leaves the Business Owner in a pretty poor place - they feel like they are making arbitrary (and often indefensible) judgements based on limited information in front of a whole room of people. Is it any wonder that people question the basis and value of the Business Value?

Whether because of the arbitary nature of the process, or because of an approach where the Business Owner has an absolute scale in their heads, you will often see Business Values for a team become pretty undifferentiated - all 10's, or all 1's, or, because “I don't really know” all 5's. None of this adds value to the joint understanding of the work.

How Do We Assign Business Values to PI Objectives?

How do we make the process smoother and more useful? One thing to note in the above “why” discussion is that the Business Value associated with a PI Objective is a team level discussion. The purpose is:

Ensure common understanding of the commitment being made.

Enable local decision making for the team.

Allow us to create and track a metric which shows a team's ability to make and meet commitments.

What does this mean? It means we don’t really have to try to establish a Train / Program concept of Business Value but simply use a team-by-team approach. The approach I've found to work well is:

Business Owner reviews a specific Team's PI Objectives

Business Owner selects the most valuable of those PI Objectives and assigns it a value of 10.

Business Owner selects the next most valuable PI Objective and determines “in comparison with that 10 valued PI Objective, this PI Objective is an X”.

Repeat 3 until done.

This process allows us to have the discussions we need, while reducing the “arbitariness” of the process, at the same time providing the data to use in the Program Increment:

Teams can use the relative values to improve their (local) decision making processes.

Teams can create the metric to show how they are making and meeting commitments. This is called the “predictability metric” and is percentage calculated by comparing the sum of the Business Values planned by the team in comparison tho the sum of the Business Values actually delivered by the team.

Probably the biggest problem with the assignment of Business Value is when the PI Objectives do not clearly articulate the the value, or is written in such a way that the business is not able to understand what is being done. It is therefore important that PI Objectives are “good” - see references below for more information.

Another problem that we sometimes see is when people confuse what they are providing. For example some people start to think “if I have PI Objective 1 as a 10, and PI Objective 2 as an 8, then this means that the 10 item will be worked first.” Business Value is a value, not a priority. It might be used to make prioritization decisions but it is not showing the priority of the work.

Note that as a side benefit, the train / program level metric for predictability percentage calculated by comparing the sum of the Business Values planned by the teams on the train in comparison tho the sum of the Business Values actually delivered by the teams on the train. At both the team and program level the metric is presented as “are we making and meeting commitments to deliver value?”

How Do We Work with PI Objective Business Values During the Program Increment?

At the end of the PI Planning event we have a set of business values. And and the end of this PI we would like to have a set of values that are updated to reflect the value of what was actually delivered so we can improve on our ability to deliver value. In other words the updated values are used to calculate the predictability metric and so is an input into the Inspect and Adapt).

One mistake a lot of Release Train Engineers make is to wait until the end of the Program Increment to gather up the values. The thinking is that we don’t want to waste people’s (especially these senior Business Owner types) on updating these values so we should wait for the next big event.

As said, this is a mistake:

The timing is such that the most time (say a quarter) has occurred between the initial commitment, the demonstration of value and the assignment of updated business values. Even with the best intention in the world, it is difficult to ask the Business Owner to remember all context that went into each of these steps especially when they are now being asked to update all the numbers at once.

The teams will often feel slighted when, from their perspective suddenly a value is reduced when their numbers go down. While this might actually be valid (now that the delivery is seen, the value could be less than expected) it’s pretty hard on a team who then has no recourse to ask why there is a change and / or perhaps point out considerations the business owner is not seeing.

The entire Train is wasting time while the business owner makes decisions about the value they are seeing. This seems like a poor trade-off of time - 1 or 2 business owners time vs 100 people on the Train.

The Train is constantly delivering value as represented by demonstrations of deliverable functionality. If we only get people to comment on value at the end of the PI this gives the false impression that value only is delivered at the end. We want to encourage discussions about delivering value continuously, not on some arbitrary time-box and so want to make sure everyone, especially senior people, know that value is always delivered.

A better approach is to have the business owner update the values as they are delivered. Typically as a result of a System Demonstration the business owner (or their proxies) will have seen the value provided. The team(s) are responsible for providing the link between the feature, the objectives and the work (stories) they have completed so there is a clear understanding of the value being provided.

If the Business Owner attends the System Demonstration, we could get an updated value by saying to the Business Owner “ … at PI Planning you said this was an 8. Now that you have seen it what do you think the value is?”

It is OK for the Business Owner to say “this is now valued a 7 (ie lower value) than we original thought.” Sometimes this is because the business situation changed. Or it might be that after having seen the capability it’s just not as useful as it was. Whatever the case, we need this data to figure out how we can improve. Is there something we could have done to improve our communication? Is there something we could have done to understand the change in business environment? The key thing to understand is that there is no blame assignment here; the Team did not somehow fail. It is just data and we might be able to learn something from it. Metrics like the Predictability will fluctuate based changes in the Team’s ability to deliver and the changing business environment. We use this data in the Inspect and Adapt to help drive improvements.

Sometimes we find that Business Owners say they do not have the time to attend a System Demo or that they need to wait on something else before providing the value. My feeling is that the RTE should set up a working agreement with the Business Owner where in the event that the Business Owner is unable to provide a value, the RTE will simply assign the value that was originally given. Like most working agreements of this time, the other side of the agreement will be that the RTE will work hard to show the value to the Business Owner so they can provide feedback. In general this is a bit of an anti-pattern (we want Business Owners involved as much as possible), but especially as you are establishing Trains in a new organization, it something that you will often see.

Want to Know More?

How Do We Write Good PI Objectives?

What is the Purpose of Sprint or Iteration Goals?: While originally written from the perspective of Scrum Teams creating Sprint Goals, PI Objectives can be seen as Uber-Sprint Goals and so can provide additional background such as why we do PI Objectives at all and how these are different to features we are working as part of a PI Planning process.

https://www.scaledagileframework.com/pi-objectives : Original write-up from SAFe

http://www.yakyma.com/2013/02/the-17-truths-about-psi-objectives.html : Excellent write-up by Alex Yakyma discussing concepts and thinking behind PI Objectives

I am indebted to Bernie Clarke who originally helped me think through the process.