Enough Already, There is No “Value” in Earned Value Management | Vitality Chicago Inc. (2024)

My colleague Bill Baxter informed me that PMI had just released the Earned Value Management (EVM) guide in December 2019 and that I should take a look. My first thought was, “Geez I hope they aren’t going to force EVM on to agile projects”. Which of course they did.

Those of you born after 1990 have probably never heard of EVM. Unless you are working in the government sector where it may be a requirement.

Earned Value is an old-school way of viewing project costs, schedule and scope together to determine your performance. EVM focuses on delivery against those “triple constraints” which unfortunately tends to be the primary objective of traditional project managers who strive to deliver OTOBOS. While it may be required for some government projects, EVM should never be used on Agile initiatives. Yet that is exactly what PMI is trying to do.

My complaints about Earned Value Management include:

1. EVM calculations are based on detailed and accurate upfront planning that doesn’t exist in agile project.
2. EVM ignores the delivery of business benefits.
3. EVM misuses the term “value”
4. In reality, no value is delivered until the customer receives the working solution.

Let’s take a closer look, starting with a little bit of background.

Some Background on Earned Value Management

New EVM Standard Published in December 2019

This most recent standard for Earned Value Management was published in December 2019. Up till now, I don’t think many people were trying to force-fit EVM on Agile Initiatives.

The most previous version of the PMI Earned Value Management Standard was the second edition, published in 2011. That 2011 version doesn’t mention agile projects at all.

True to the PMI tradition, the 2019 EVM standard is a bloated 182 pages of detailed explanation for a topic that could be explained in 18 pages. Thankfully, the two sections of the document that describe how EVM can be applied to Agile and Hybrid environments is a relatively sparse 9 pages; it probably could have been one simple statement: “Don’t Do It!”

Calculating Earned Value Management

I don’t profess to be an expert on Earned Value Management. I learned just enough about EVM back in the 1990s to teach it in various IT Project Management courses. I never worked in an environment where I was required to use it.

In short, EVM is a way to track whether or not you are spending and completing work at the speed that you planned it. Nothing else. It does this by comparing your planned cost and schedule per item to the actual. Rather than just provide a % complete spend, it can reveal problems by showing how much of the planned “work” you spent got done.

Calculating EVM is straightforward. EVM looks at the relationship between 3 project constraints: time, cost and scope. We use the plan and actual for these triple constraints to calculate Planned Value, Earned Value and Actual Cost:

• Planned Value (PV) – This is the cost of the work you plan to get done at a specific point in time. Though it is labeled as value, it is truly the cost of the work.
• Earned Value (EV) – Earned value represents the cost of the work that you actually completed at a point in time. value of the work delivered.
• Actual Cost (AC) – Actual cost is the cost in dollars that you actually spent to complete the work at a specific point in time.

EVM is Based on Detailed Planning Upfront

One of the biggest problems with EVM is that it is all based on having detailed plans upfront. And not having too much change which doesn’t fit with agile initiatives.

Mike Griffiths has been straddling the agile and PMI schools of thought for over a decade now. I recommend his PMI-ACP prep book to everyone who is pursuing that certification. Here is what Mike wrote in 2007 about applying EVM to agile projects:

There are fundamental problems using EVA on agile projects relating to baseline plan quality. Also there are better alternatives available for agile projects.

Earned Value analysis and reporting measures conformance and performance to a baselined plan. So, given that on agile projects we know that our initial plans are likely to change, why track progress against a weak plan?

— Mike Griffiths, Using Earned Value on Agile Projects

You can also see the problem in the statements that PMI makes in the Agile Practice Guide. You may recall that PMI jointly produced the Agile Practice Guide with the Agile Alliance back in 2017. I went back to look at my previous post on the Agile Practice Guide and saw that I had criticized the inclusion of Earned Value in my review. I called it a “weird fascination”.

The statement below from the Practice Guide reflects a key challenge with applying EVM to agile:

Traditional EVM metrics like schedule performance index (SPI) and cost performance index (CPI) can be easily translated into agile terms. For example, if the team planned to complete 30 story points in an iteration, but only completed 25 then the SPI is 25/30 or 0.83 (the team is working at only 83% of the rate planned). Likewise, CPI is the earned value (completed features value) to date divided by the actual costs to date or, as shown in Figure 5-6, $2.2M / $2.8M = 0.79. This means a result of only 79 cents on the dollar compared to plan (but of course this assumes that the prediction is still correct.)

–Project Management Institute, Agile Practice Guide (2017)

The problem is that our calculations are from a baseline that is a best guess and is expected to change. We then calculate performance indexes against those guestimates and come up with conclusions like the following:

• “the team is working at only 83% of the planned rate“
• “a result of only 79 cents on the dollar compared to plan“

First, it sounds like the team is not working as hard as they should. And second, it is a measure from someone outside the team about the best guesses (story points) that the team could complete in a sprint. Never mind that a focus on measures like this will have the undesirable impact of the team gaming the measure so that they always hit (or exceed) their estimates in the future.

The problem here is that EVM is based on a detailed and accurate “plan” at the start of the initiative. And Agile avoids that detailed upfront planning because it usually represents waste and throwaway work.

In an agile project, the backlog and release plan may be a best guess based on high-level estimates of the work using imprecise measures like story points. So your actual results are highly likely to differ from the baseline you created. Remember the Agile Value statement, Responding to Change vs. Following a Plan?

If your results fall short of the plan, how can you possibly know if it is because the team did not perform well or that your original estimates were wrong? This comparison is back to reliance on plan-driven methods and assumes or is predicated on the ability to accurately forecast the future. Focusing on the variance is navel-gazing at best.

This effort to compare plan to actual serves no purpose except to encourage gaming and other dysfunctions, and to create a lot of work for the person calculating EVM. In fact, PMs who apply EVM to agile initiatives will find themselves chasing their tails trying to determine why their actuals are not matching the plan (they almost never do!). If you love to hunker down at your desk, endlessly manipulating MS Project and exporting it to MS Excel for hours on end, this is a great tool to help you achieve that.

One other problem is that the PMI proposal for EVM would take agile projects back in time to the “% complete” mentality and style of progress reporting. Which is exactly the wrong direction. According to the standard:

The key assumption in agile environments is that the ratio of story points completed to the total story points planned for a release is a good measure of the actual percent complete”.

— Project Management Institute, The Standard for Earned Value Management (2019)

Aha, we are back to using % complete to report progress. Awesome. What happened to value-driven delivery?

EVM Ignores the Delivery of Business Benefits

Section 3.5 of the 2019 EVM Standard (Applying EVM in an Agile/Hybrid Environment) starts off with a statement about how both plan-driven and agile approaches need to be aligned to achieve business objectives and we need to measure progress.

“All project teams should ensure that work is aligned to key business objectives. It is important that management at all levels can see that the overall progress is measured, risks are managed, and the wider view across the project is maintained”.

— Project Management Institute, The Standard for Earned Value Management (2019)

And that is exactly the rub – EVM doesn’t do anything to ensure the work is aligned to key business objectives, nor does it provide visibility to the progress toward those objectives. EVM’s magical properties only extend to tracking the relationships between planned and actual cost, schedule and scope. If your business objectives are to achieve some sort of business benefit or value, you won’t get there using Earned Value.

I previously ranted about how PMI focuses too much on the triple constraints in Time to Retire the OTOBOS and how they ignore business value in Why Scope Creep is Complete Bullsh*t so I won’t repeat it here. The point is, PMI focuses only on ticking off all the tasks in the WBS, and completely ignores the achievement of business benefits. It assumes the achievement of the plan is the goal, not the delivery of business benefit.

Earned Value Management Misuses the word “Value”

When PMI tries to apply EVM to agile initiatives, it mis-uses basic terms and sows confusion. Let’s look at the description of each of these terms from the PMI EVM Standard (see section 4.4.3.2 Calculating PV, EV, and AC within Agile).

The comparison of EV with PV lies at the core of EVM. PV is the value of the work planned for a certain date. It is the entire budget for work to be completed at the planned date. In agile terminology: It is the sum of the estimated feature sizes for all the features up until the planned date.

— Project Management Institute, The Standard for Earned Value Management (2019)

Notice that the PV or planned value, is stated as the value of the work, but then in the next sentence, it becomes clear that it is the budget for that work or the sum of the feature sizes. What PMI is saying is that PV is the cost or what you planned to spend, and that is equal to your estimate in story points. To that I say, cost and story points are not “value”.

Let’s look at Earned Value (EV):

For agile projects or control accounts, EV represents the value of work completed (i.e., total number of story points for all completed user stories times number of completed iterations) at the same date as used for PV. EV is not synonymous with actual cost, nor does the term refer to business value. EV refers to technical performance (work) earned or completed against the baseline or work planned. In agile terminology, it is the value of the completed user stories up until the calculation/status date.

— Project Management Institute, The Standard for Earned Value Management (2019)

In this section, it states the EV is the value of work completed or the number of story points completed. Then the kicker, “EV is not synonymous with actual cost, nor does the term refer to business value“. It goes on to point out that EV is a comparison of actual work compared to a baseline. So the earned value is the spent money and it doesn’t relate to business value.

And then we have another misleading statement “In agile terminology, it is the value of the completed user stories up until the calculation/status date.” No, it is not thevalue, it is either the cost or the number of story points.

Finally, let’s look at Actual Cost:

Actual cost is what the name implies: the cost in dollars to complete a set of user stories by the team per iteration and is usually derived from work hours recorded by the organization’s time reporting/tracking system.

— Project Management Institute, The Standard for Earned Value Management (2019)

Finally, a cost correctly identified as a cost. In reality, PV, EV, and AC all represent costs. None of them represent value.

Unfortunately, the PMI Standard ignores this and keeps on using the term value when it means cost. How about this statement highlighted below where “story points are mapped into financial value based on a value-per-point conversion rate of $3,500 per point (i.e. the burn rate).”

Agile EVM works by comparing the current release plan (taking into consideration changes to the scope in the product backlog) against actual work completed as captured in the release burnup chart. For example, progress can be shown on a burnup chart as depicted in Figure 4-7, where story points are mapped into financial value based on a value-per-point conversion rate of $3,500 per point (i.e., the burn rate).

Oh my, more misusing story points. The PMI Standard probably also contains a conversion chart of story points to hours as well.

In case it is not clear from the above how EVM works, let us take a very simple example.

A Simple EVM Example

Let’s assume that our initiative is to buy Christmas presents for my wife. I have a budget of $200 and I am going to get her 4 presents, one per week for the weeks leading up to Christmas. Each present will cost $50.

1. Sweater ($50)
2. Yoga Mat ($50)
3. Gift Card ($50)
4. Framed photo ($50)

While the first three are items I can buy off Amazon, the fourth is a combination of a nice picture frame, a printed photo, and my effort to come up with some kind words to add as a statement on the photo.

In terms of schedule, I am going to buy one present per week in December. However, though the deliverables may all be bought each week, they won’t be delivered to the customer until Christmas. That is when the customer (my wife) is going to open them up.

Here is what the EVM charts look like at the end of each week:

Week 1: I successfully purchased the sweater for $65. Planned Value was 50, Earned value was $50 and actual cost was $65.

Week 2: I got busy and did not do any shopping. Cumulative Planned Value was 100, Earned value was $50 and actual cost was $65.

Week 3: Whew, I carved out some time and was able to get out and purchase the yoga mat for $25 and the gift card for $50. Cumulative Planned Value was 150, Earned value was $150 and actual cost was $140.

Week 4: In week 4, I purchased the photo frame, and then I spent some time editing the photo to add a cool caption. Cumulative Planned Value was $200, Earned value was $200 and actual cost was $190.

The EVM Charts Don’t Tell the Story

This simple example illustrates two key points:

1. The “value” perceived by the customer is not the same as the cost
2. The customer doesn’t get any “value” until the solution is delivered

Let’s explain.

The Perceived Value is NOT the same as the Cost

The cost of the four example presents was $190 in total. However, that is not the Value of those presents to the customer no matter what the PV and EV say.

When my wife opens all the presents on Christmas, the “value” she places on the first 3 is almost nothing. The first one (sweater) actually had to be returned since it was the wrong size (my bad). The other two are just meh. But the 4th gift, the framed photo of her and I, this gift makes her cry, and cry. She hugs me. She takes the photo and puts it on a prominent place on her dresser and she looks at it every day.

The EVM calculations are about costs and waste, and they have absolutely nothing to do with value. Value is measured by the customer, and it is detached from the cost. The least costly of the 3 gifts above was the most prized.

There is No Value Until Delivered to the Customer

True value is not delivered until the customer is able to use the solution. In the simple example, the presents are purchased each week but they are not opened until Christmas Day. So there is no value in the customer’s eyes until Christmas. Getting those first presents bought and wrapped incurred cost but there is no value until the day the customer gets it into their hands.

In Lean thinking, all interim deliverables represent work in progress and are therefore waste. So any gifts bought that are not delivered to the customer represent waste.

At the end of the third week, EVM says you have earned value of $150 but the reality is all that you have created is waste at that point. And since we don’t get feedback until the end of the project, we also create risk.

What is the Point? Stop Applying EVM to Agile!

By trying to force-fit traditional approaches like EVM to agile initiatives, PMI continues to miss the boat. While PMI is trying to make some strides in the agile space (See: PMI Goes all in with Disciplined Agile and Flex), this EVM effort just shows that they don’t fully appreciate agile ways of work.

I know that I have probably have readers who love EVM. Don’t send me your hate mail to explain that it is required for your government agile program and how valuable you think it is and all the important project status charts you can get, yada yada yada. That may all be true, and good for you if it is helpful.

Like Scope Creep, the focus on EVM in agile environments is misplaced. Stop.

PMI should start focusing on value delivery, a term they use in the PMI-ACP certification. Not “Earned Value”, but the delivery of true customer value.

For those of you looking to track your agile projects, check out my post on creating reality based forecasts for agile projects. You can use this to create realistic release forecasts and track progress with burndown charts.