In a recent twitter exchange, a formula was presented that goes like this, from the Original Post
Rewriting this into algebra in LaTeX gives us
Let's see if we canunpack the equation:
- The total uncertaintyis the ratio of the estimated past, plus the estimated future dividedby the actual past + actual future.
- There can certainlybe anestimate of the past that is used, for example in our domain, thisestimate of the past would be used to compute the variance of the estimating process, once theactualscome in. We'd compare theactuals to theestimate to determine the fidelity of the estimating process. This would be the Estimate at Completion (EAC) and Estimate to Complete (ETC) recorded at a specific time in the past.
- But that past EAC and ETC are updated when the actuals come in (monthly).
- So it's not clear what it means whenActual Future is used in an equation ofTotal Uncertainty.
It's Also Not clear:
- Before the project is complete, we can know theactual past but how do we know theactual future?
- So it can certainly be that theestimate of the pastis not equal to theactual past -and it rarely is, otherwise, thatestimate, wouldn't be anestimate, it'd bethe actuals.
Let's see if we can unpack the last part of the OP:
- When the project starts there are only future uncertainties since no work has been done, only estimates made about those futureuncertainties.
- When the project is done, those uncertainties are gone, since all we have is actuals and there are no uncertaintiesabout those.
- As well, Actual Future Uncertainty is an oxymoron. The value under measurement is either anactual or anuncertain value, but it can't be both.
- Since we can't know theactual value of the future uncertainty since it'suncertain, not sure what the value of the number in the denominatoris.
Let's Try Another Approach
All project (or product developments) have planned work. This can be a traditional planning approach or an Agile Planning approach. In the traditional approach, work is planned out usually inwork packages of scope according to some sequence needed to produce the desiredvalue for the cost and duration. The level of detail is usually domain specific. A schedule of the work is built. This schedule can represent all the detailsor it can be arolling wave of details, with theplanning horizon defining how much detailin the schedule at any one time.
This is one of the fallacies agilest like to use, the rolling wave process, the incrementalspiral commit process, and otherincrementalanditerative processes are baked into our Federal Acquisitionprocess and used in our enterprise IT processes.
There is no such thing as Waterfall, other than on badly managed projects, that willfully ignore current best practices for project management that have been around for decades.
So let's look at some ways to assess theuncertainty
Here's a notional 35-day project, with one approach ofjust do all the work over the 35 days and a second approach ofbreak down the work into its Work Packages and Tasks. In both cases, the assignuncertainty for the Aleatory risk (irreducible) uncertainty of the work effort as minus 5% and + 15%.
Both these values are for theTotal Uncertainty at the beginning of the project, whose work looks like this...
Now when we hear -oh we don't use Gantt Charts for our work, fine, whatever you use, Kanban board, Scrum Product Back Log, sticky notes on the wall, strings, it doesn't matter. Somehow you're sequencing the work to be done, assessing the progress of that work, measuring compliance with what the customer ordered. It doesn't matter. The uncertainty in the effort, duration, cost, and technical performance resulting from that work is still there.
If this is not the case, stop reading now, you're notmanaging project work in the presence of uncertainty.You're doing something else, but it's not the management of other people's money to produce value.
The two simulation runs below are for thesimple 35-day task on the left and themore complex 35-day collection of Work Packages on the right.
For thesimple(do all work) approach, there is a 50% chance that we'll finish on or before Jan 29, 2018. For themore complex set of Work Packages, there is a 39% chance that we'll finish on or before Jan 29, 2018. This difference have to do with how the Monte Carlo Simulation (Risky Project) treats thenetwork of work. This is a seperate topic.
As the project progresses and we get status for each of the task -simple andmore complex- let's see how the risk of NOT finishing on time changes. The status date for the project is Jan 4th, 2018 and let's assume the work complete for both thesimple model and themore complexmodel is the same. On Jan 4th, we're 57% complete. This measure ofPhysical Percent Complete is a Critical Success Factor for all project work. This means:
- We have units of measure meaningful to the decision makers.
- These measures can be Measures of Effectiveness, Measures of Performance, Key Performance Parameters, and or Technical Performance Measures.
- But that have to produce tangible evidence of progress to plan. This is calledQuantifiable Backup Data where we work.
In other words, we NEVER measure progress to plan (what ever type of plan you have) without tangible evidentiary materials to confirm that progress representsPhysical Percent Complete. No handwaving, noratios that hide the individual measures, no personal opinion. Just data from apredefined plan for the data.What Percent Complete SHOULD we be on this date? What Percent Complete are we? If we're less - in the aggregate, then we're late, likely over budget as well, since more money will be spent to show up late or need to get back on schedule.
So here's the project status on Jan 4, 2018, with the Percent Complete bars on dark blue. Technically (and this is a very technical issue) the bar shown here is the percent duration complete, not the other 4 measures of complete. But that requires a more complex set up and profiling of the labor spreads across the work and calculations to be set up. This is anotional example and on any REAL program we work, the MSFT Project Percent Complete is never used, since it can hide thePhysical Percent Complete. But for the typical programs we work (> $100M) our approach is much too complex for the typical sofware development process.So now that we've moved along in our project and made progress - but not actually the progress as planned, since we're showing late on a few tasks, we now have a newprobability of completing on or before the planned completion date.
We have a 47% chance of completingon or before Jan 30, 2018, for theSimple Linear Work and a 35% chance of completing on or before Jan 30, 2018, for theMore Complex Work. So our probability of completingas needed is going down.
This is a critical understanding.
- We want to know theprobability of arriving on or before the need date, at or below the needed cost.
- To sayuncertainty is not being reduced is of little use unless we know how that uncertainty is impacting theprobability of success.
- So when the performance of the project doesn't match the planned performance - all those measures - AND the reducing uncertainty, we need to do something beyond just pointing out that uncertainty didn't reduce. That something is to take corrective or preventative actions to keep the performance measuresinside the planned bounds.
The Punch Line
Just stating what the uncertainty is in a project - that approach taken by critics of theCone of Uncertainty is useful, but not actionable. We need to answer the question
Knowing something about the uncertainty in the future, our past performance, the currentPhysical Percent Complete, the remaining work, and any changes in the uncertainty of that work (I didn't make any changes to the work that wasbaselined at -5%/+10% aleatory uncertainty for each task), but I could have updated that uncertainty for the future). Know all these things -WHAT IS THE PROBABILITY OF COMPLETE ON OR BEFORE THE NEED DATE, and AT OR BELOW THE PLANNED COST. (I didn't cost load the work either, but that's an easy task).(Video) Project Management - Estimating
Knowing theTotal Uncertainty is interesting but not very useful. The cost and schedule are driven by this uncertainty, but that uncertainty had better be reducing as the project progresses - and reducing at some planned rate - otherwise, you're late, over budget, and the product is not likely to be working.
So the critics of the Cone of Uncertainty, are criticizing the wrong problem. They claim:
- They have data that shows the uncertainty does not reduce.
- OK, fine - Why is the uncertainty not reducing as needed to maintain the probability of completing on or before the need date and at or below the needed cost?
- One article states the reasons for the data not showing reducing uncertainty, but doesn't appear that any corrective or preventative actions were taken. So the result was the uncertainty didn't reduce.
- A follow-up letter (IEEE Sofwtare, 2006)to that article in the same magazine states the same.
- As well it states, usingrelative uncertainty hides thedriversof the uncertainty, since the Cone of Uncertainty was meant and it always considers theabsolute uncertainty. Also stated in the Letter to the Editor ofIEEE Software.
- Since theCone represents thebest case uncertainty, it's always possible to be worse - that is to NOT control the aleatory and epistemic risks to the project and have data that goes outside theCone. This is not an issue with theCone it's an issue with the management of the project.
The Cone of Uncertainty Does Not Reduce Itself. The CoU is defined as the desired reduction of uncertainty at specific phases of the projects needed to informed the decision makers of the Probability of Project Success.
So when someone has data that doesn't have its uncertainty reduced in accordance with some plan, and doesn't take corrective or preventative actions to reduce that uncertainty, then they're going to be disappointed in the results for the cost and schedule performance of the project. They can rationalize that he customersloved the product. But that doesn't remove the fact you showed up late and over budget.
Project Management in the presence of uncertainty is a closed loop control system. Cost, Schedule, Risk, and production of Value are a few of the dependent variables of this closed loop control system. When those variables haveexcursions outside the planned boudnries cporrective or preventative actions must be taken it get them back in the boundaries.Keep the Program GREEN is a favorite syaing where we work.
Here's two useful resources for applying the cone of uncertainty that Google will find for you
- "Improving Software Development Tracking and Estimation Inside the Cone of Uncertainty," Pongtip Aroonvatanaporn, Thanida Hongsongkiat, and Barry Boehm.
- "Reducing Estimation Uncertainty with Continuous Assessment:Tracking the “Cone of Uncertainty,”Pongtip Aroonvatanaporn, Chatchai Sinthop, Barry Boehm
Read these and learn how the CoU is to be properly used to increase the probability of project success