Measuring Value – Scrum Day Europe 2019

‘Value Delivery with Scrum’ was the theme of the 8th edition of Scrum Day Europe 2019. To me this was a particularly interesting topic, since my current job title is Product Owner. In other words, someone who is supposed to optimize for value. In this post I will share my highlights from the conference.

What is Value?

So, what is value? Value happens to be a word that has a different meaning for everyone. Stanislava Potupchik led a workshop where we had to interpret Dixit cards to describe our concept of value. We discussed commonalities and differences within small groups. It is interesting to see Scrum Masters thinking in terms of processes, team collaboration, etc. Product Owners tend to think in customers and products. I also realized that I put different emphasis on aspects of value, depending on what context I’m in. A couple of months ago I would have emphasized the vision or the dream. Nowadays, as I have developed a vision for my product, I’m more thinking in trade-offs and choices to optimize value continuously, while moving towards the product vision. It is good to be aware of the differences in mental models between you and someone else when you try to communicate your ideas.

So, now that we can arrive at a common understanding of what we mean by the word value, how do we know we are delivering value with our product or service? When we want to track progress of something, we need to measure it. Therefore the question is: How do we measure value delivery? Patricia Kong gave a keynote on Evidence Based Management. Many organizations fall in the trap of measuring the activities they undertake or the output that is being produced. For example, they focus on improving velocity of Scrum teams. Or they might track how fast they deliver new features. But these kind of metrics say nothing about the value that has been created for the end user or for the business itself.

Evidence Based Management

Evidence Based Management recognizes four so-called Key Value Areas. Each Key Value Area (KVA) tells you something about the delivered value and the ability to deliver value. The four KVAs are:

Current Value describes the value that the product delivers to customers and our own organization today. Are your customers happy? Are your employees happy? Are your investors or other stakeholdes happy?

Unrealized Value gives an indication of the potential future value that the product could deliver for the organization. Is there any improvement possible? It such an improvement worth the risk and effort?

Time to Market looks at how quickly the organization can deliver new capabilities, services or products. How fast is new value delivered to the customer? How long does it take to learn from new experiments and information? And how fast can we adapt?

Ability to Innovate is an indicator of the ability of the organization to deliver new capabilities that better meet customer needs. Is there something that prevents the organization to deliver new value? Are there impediments to customers or users to benefit from the new innovations?

Evidence Based Management – Scrum.org

All of the four Key Value Areas are important and influence each other. Reflecting on what we measure in these four areas, we can base decisions on how to move forward with our product or our company. If there is a lot of Unrealized Value and a high time to market, we know we have to improve our processes. When there is high current value (high market share, no real competitors, satisfied customers), we do not need to invest so much in that product.

There are many measures that can be used to measure these KVAs. But the usefulness of measures varies depending on your context. These measures can be leading (fairly rapidly changing) or lagging (taking a long time to change). When you need to wait long for your metric to reflect the change you made, you cannot adapt. Fortunately, for many lagging measures you can find a leading proxy measure. For example, ‘customer satisfaction’ can be proxied by ‘usage’ or ‘number of abandoned transactions’.

Your mindset is reflected in your measures

Many organizations measure progress. They look whether the team velocity has improved and whether product development is becoming more predictable. However, measuring on progress will only misalign the business and the development organization. The number of story points delivered is not an indicator of increased value to the customer and the business. Agility is about effectiveness, not about efficiency. In a complex world, where we know that the first attempt is almost always wrong, we have to keep a clear focus on the goal. A good goal will inform a team what they need to measure.

In order to come up with useful measures, we need goals about the outcome of the product. Another mistake many organizations make is that they measure output. For example, we can measure our conversion funnel on a webshop, which indicates how many people have been able to buy a product. But this does not say necessarily anything about whether the customer is happy with the outcome.

In order to say anything about the desired outcomes, we need to look at the strategy. Roman Pichler showed a strategy map where we go from business strategy via portfolio strategy to product vision. Which can be expressed via a business model and KPIs. This strategy on the various levels will inform your product roadmap and product backlog. It is mandatory for your team to understand the goals and strategy in order to be fully agile. When the goals are not known, how can the team find a way to reach them?

Strategy vs. Product
Strategy vs. Product

Strategic insight is also useful to know when we can stop improving on the product. Looking at the four Key Value Areas and comparing them with the product vision and business strategy will give you insight in the next steps for your product. What you measure says something about whether you are output or outcome oriented. It makes clear whether you have some decent strategy in place or not.

Conclusion

My key takeaways of Scrum Day Europe 2019 are the following:

  • Evaluate your strategy and product vision. Make sure they are all about the outcome for the customer and for your organization.
  • Communicate your vision to the organization and your team. Be aware that different people have different mental models about generic words, like value.
  • Look at the four Key Value Areas and find leading measures that give useful information about the value you are delivering.
  • Keep inspecting and adapting.