Better Timelines: Using Monte Carlo Analyses for Digital Projects
"When will it be done?" As a digital project manager, I hear this all the time. From a client’s perspective, it’s a perfectly reasonable question. Stakeholders and managers need certainty about when a website or app will launch in order to coordinate other organizational initiatives such as marketing campaigns, product launches, and so on.
From a project management perspective, the problem is we don’t have a crystal ball. At the outset of a project, or even halfway through completion, we can’t foresee all the roadblocks we’ll encounter, whether they are new requirements, unanticipated technical challenges, internal resource changes, or other risks. So when we’re asked for a date, we often draw up a timeline blown-out with a risk buffer. While this is a time-honored project management strategy, at bottom it just isn’t the most accurate representation of what we foresee. As project managers familiar with our team’s velocity and the project’s risks, we anticipate the likely scenario, not the worst-case scenario. But because we know our feet will be held to the fire, we often give management the most conservative date we can.
A Better Way
For one of our PM consulting clients, things had been moving slowly with her internal team, so she asked me for help compiling a full timeline for her management team. Instead of giving her a standard-fare schedule complete with an arbitrary risk buffer, I created a Monte Carlo Analysis to give her a probabilistic assessment of our projection. That may sound fancier than it is–a Monte Carlo Analysis is just a way to calculate outcomes using ranges (150 to 175 story points, for example) instead of using singular numbers. The estimate culminated in a timeline that looked something like this:
In this particular case, I plugged in a range of values for three key variables: how many story points were left in the project (140 to 170), the number of story points per sprint (15 to 20), and potential variation in sprint length (4 to 5 weeks, to account for holidays and other external factors).
After handing it over, I was able to explain that if her management team needed to plan against project completion with 95% confidence, she should tell them to count on 14 months. But if her management team was able to tolerate more risk, say 50% confidence, she could give them a date of 11 months.
By giving her a range of outcomes, the conversation with her management will shift in two key ways. First, management is reminded of the uncertainty involved in software development. By speaking probabilistically, we reinforce the notion there are always “unknown unknowns.” Second, it will cultivate a more productive, collaborative conversation around project resources and scope. Instead of hearing a deadline, writing it down, and holding us to it, giving a range of outcomes helps managers ask better questions and ultimately plan smarter or make changes.
How to Create a Monte Carlo Analysis
Here at Aten, we value open source software, which means we avoid some of the more traditional enterprise project management tools which provide Monte Carlo Analyses. In what follows, I’ll show you how I created the analysis above using the online, open source app Guesstimate. Guesstimate is an open source risk solver–think of it as a kind of specialized google sheet. To get started, I used their scratchpad, which you can access for free.
After clearing out the sample entries on the scratchpad, I entered a few cells representing a basic formula for predicting how long the project would take to complete, along with some sample numbers. The formula I used was Number of Sprints = Total Story Points * Story Points per Sprint. Simple, right? Here’s what that looks like:
To add a little icing to the cake, I decided to calculate the total number of weeks from start to finish, accounting for varying sprint lengths. This variation helps accommodate for lost weeks during the holidays and any other unforeseen scheduling snafus. Here’s my beefed-up spreadsheet:
Next (this is where the magic happens), we can change these singular values to be ranges. To do this, for example, click into the Total Story Points cell and change the value from 100 to [150,170]. The formula will now be calculated using a range. Finally, I added one more cell to convert the number of weeks into the number of months. Here’s what the result looks like:
After doing that, I clicked the “Expand” button shown in the above image to get a zoomed-in view of the Monte Carlo graph:
After a little photoshop magic to remove the “X” in the screenshot I took, I ended up with the chart you saw at the beginning of the post.
Putting it All Together
Being a great project manager isn’t about pretending you can predict the future with certainty. It’s about communicating honestly and directly with stakeholders to give them the most realistic assessment of work. A Monte Carlo analysis makes the uncertainty and risk inherent in a project digestible. While it can be overkill for smaller projects, for larger, more complex work it can help light the way to a project’s success.