Blog: 3 Common Agile Metrics Mistakes

3 Common “Agile Metrics” Mistakes

Your organization likely tracks a number of metrics. Have you ever stopped to think about why you track what you track? As your organization moved from traditional “waterfall” projects to Agile, the metrics you collect probably changed, but did the why behind them change?

One of my mantras is, “what you do matters; why you do it matters more.” Your organization is doing certain practices. Why? You are tracking certain metrics. Why? The intent behind traditional project thinking is vastly different from Agile. Yet I find the why behind the metrics that many track to be relatively unchanged.

In my book, Pursuing Timeless Agility: the Path to Lasting Agile Transformation, I spend a good bit of time contrasting what Agile really is with what many think it is.  As I also explain in my blog, The Unintentional Intentional Distortion of Agile, there are reasons why Agile has become distorted, misunderstood, and misapplied. The 3 common “Agile metrics” mistakes stem from that misunderstanding.

Mistake #1 – Calling them “Agile Metrics”

The bottom line is that the why behind most so-called “Agile metrics” does not align with the intent of Agile, so they are therefore not “Agile” metrics.

Secondly, it is a pet peeve of mine, where the word “agile” is placed in front of everything, such as Agile team, Agile project, or Agile metrics. Agile is often contrasted with Waterfall, yet no one ever said “Waterfall metrics,” “Waterfall project,” or “Waterfall team.” When the word “agile” is put before everything, it indicates to me that Agile is misunderstood. When Agile is misunderstood, it is misapplied, and mis-measured.

Whatever you value and measure success by, you will adopt metrics that you think measure that objective. Those metrics are always squeezed out of the practices you employ, regardless of what those practices are.

Teams start doing Scrum, for example, but still want to track traditional measures of success, so we end up with “velocity” and “burn down” as “Agile metrics.” These are called “Agile metrics” because they are used with so-called “Agile methodologies.”

What these metrics place value on is no different than before Agile. How that data is collected is different when teams follow something like Scrum, but what the organization values in this instance has not changed, so what it wants to measure success by has not changed. These values, however, do not align with Agile. Therefore, they are NOT “Agile metrics.”

Mistake #2 – Output Focus

I explain in my book how many organizations believe the measure of success is found in tracking velocity, budget vs. actual cost, planned vs. actual stories per iteration, and estimation accuracy. In other words, faster delivery, on-time delivery, and more predictability. These are the same output-focused objectives organizations valued before Agile and what many hold to after deciding to “do” Agile.

Agile is values, principles, and concepts – more about why we do what we do than specifically what we do and how. What Agile values is people, frequent inspection and adaption (empiricism), flexibility, risk reduction, improved quality, and continuous improvement. No where in these values will you find doing more in less time, being “on-time,” or predicting long-term plans. Rather, Agile is more focused on outcomes – better products, not more; the next right thing, not the next thing in the plan. Agile is outcome based. Timeless Agility puts additional focus on your why. Why are you doing what you are doing?

Teams move to 2-4 week sprints. Why? For many, they do it because Scrum says so. For others, it is to deliver something faster and more often. Ok…why? Why do you want to deliver working software faster and more often? For some, it is about doing more in less time, or at least the appearance of it. For others, it is about being able to track progress against the plan via shorter cycles. In these instances, the focus is outputs – not Agile. Conversely, if your why is so that you can learn sooner in order to adjust sooner so that you can get to a better product sooner, then you are thinking more Agile. How you measure success will be vastly different because you will then be thinking about outcomes and value, not outputs and plans.

Measuring value delivered, measuring the pace of learning (supported by frequent delivery and empiricism), measuring the flow of work, and measuring mindset transformation are better “Agile metrics,” but even then, they are merely success measures.

Anything that measures your teams’ progress toward the mindset transformation required to intuitively work in this manner could perhaps be called an “Agile metric,” and I talk about that in my book, but what most are calling Agile metrics does not fit. If outputs are still what you measure success by, then Agile may not be fit for your organization. Learn more in my blog about How to Know When You Should Not Pursue Agile.

Mistake #3 – Pushing Metrics Where They Do Not Belong

Metrics can be dangerous when assessed independently and without context, which is what happens when numbers and charts are sent to management. More so, when the wrong things are measured, the wrong story is told.

Many of the things Agile-minded teams measure around product and team outcomes is meant to help the team improve, not to be a management metric. Yet, far too often these measures are pushed upward to management, in reports, with thresholds to meet. That’s not the intended point, and it creates unhelpful behaviors.

For example, from a “productivity” perspective, one might think the measure of success for a team is the number of stories, or number of story points, it completes in a sprint – its velocity. Furthermore, such a thought process may suggest that a successful team will be able to predict nearly perfectly how many stories it will complete in a sprint, so it measures variance.

When managed by it, a team may increase its estimates to produce more perceived velocity. When penalized for doing more than the targeted story points in a sprint, a team may defer work. When penalized for not finishing what it committed to, the team may skimp on quality or work overtime to get the work done. Using these metrics for management tells the wrong story and causes unhelpful behaviors. None of these behaviors or outcomes are supported by Agile values and principles.

There are valid success measures, and valid measures to share with management. The Measuring Success chapter in my book breaks down some of my favorite measures and explains the story they can tell, in context with other measures.

As with all metrics, they only tell a story in context with other data. The best way for #management to understand how teams are doing is to talk with them, not look at metrics reports in isolation. #Agile #projectmanagement… Click To Tweet

Moving to Agile requires redefining how you measure success in software product development. Otherwise, you are not really Agile.

There are metrics that should stay at the team level, and there are new measures management needs to learn to use. It is quite possible that Agile is not for your organization. I help you figure that out in my book by exposing the errors of “common wisdom,” revealing what Agile is really about, and then helping you assess where your organization fits in all that. If Agile, as it was intended is what you really want, and you are currently off track, the back half of the book walks you through next steps for getting on the path to lasting Agile transformation.

In summary, try to stop using the word “agile” in front of everything, except maybe “coach.” Assess your current metrics and measures of success and ask why you track them. Does your why align with Agile? If you are still output focused, reach out to me to explore how you can shift to a more outcome-focused approach – perhaps start with purchasing a copy of my book, Pursuing Timeless Agility: the Path to Lasting Agile Transformation.

About The Author

Leave a Comment