That was gratuitous!Once again, those are ignoring all random variables except for my collected unpredictable schedule data.
I like that 20-Day Project result: 3–5 weeks is an easy to remember range for a 4-week-expected-length project.
Fake data for a more consistent scheduleJust for fun, let’s try constructing some fake data for a developer with less variance in their schedule than me.
To do it, I removed 70 of the more extreme data points from my 222 data points.
I used no mathematical strategy, I simply tried to greatly reduce the thick left tail on the histogram, and make it look more like a bell curve shape.
Here is a histogram of the fake data.
The distribution of all of the days in my fake dataset.
The 152 days were categorized into bins (bars) based on the number of minutes “worked on main project” that day.
Hopefully that looks reasonable enough.
It much more resembles a bell curve than my real data, which almost looked like a uniform distribution (die roll).
And now, let’s do 1 million simulations of a 10-day project again, using the fake data that is more consistent.
Here is the result:The distribution of 1 million simulated runs of a 10-day project.
Uses fake schedule data instead of my real schedule data.
The runs simulate the effect of a fairly consistent schedule.
The project length is still has fairly high variance, but less than my real data.
Here are some observations this time:41.
7% of runs are ±1 or more days away from the expected (instead of my 60.
1%).
95% confidence interval: 8.
0 to 12.
9 days (mine was 7.
0 to 14.
5 days).
So, sure enough, there’s less variance, but more than I would have expected before starting this experiment.
Keep in mind when looking at the graphs, that it’s nearly impossible to have fewer than 2 bars.
That’s because, a “10-Day Project” as I’ve defined it, is expected to take 10 full days on average.
That’s precisely on the border of Day 10 and Day 11.
So even if there is only a tiny bit of variance, then half the runs would finish on Day 10, and the other half would finish on Day 11, probably.
So really, if you think about the graphs long enough, you start to prefer to think about the 95% confidence intervals, perhaps.
Adding back in other random variablesRemember, project length is affected by many random variables that cause variance, but all the above simulations in this entire article only consider the “unpredictable schedule” random variable.
But just for fun, we can take a guess at how much the other random variables affect project length, and then run the simulation again and include that randomness!Let’s identify some other random variables and invent how much they add/subtract to project length:Unexpected difficulties: Sometimes projects take longer or shorter because of unexpected technical difficulties or unexpected easy parts.
Let’s say, 33% of projects take 20% longer, and 33% of projects take 20% shorter.
This is certainly conservative, we’ve all had projects where we realize they’re twice as hard as we thought…but whatever.
Unclear or changing requirements: Sometimes projects take longer or shorter because of requirements that are required but not revealed until during the project.
Let’s say, 33% of projects take 20% longer, and 33% of projects take 20% shorter.
Also probably conservative?Energy and morale: Sometimes a project is particularly tedious, or you just have other stuff going on in your life.
Or sometimes you’re super into it.
Let’s say, 33% of projects take 10% longer, and 33% of projects take 10% shorter.
Also probably conservative — personally, when the open office is quiet, and I’m in the zone, and I don’t have ten things I’m trying to remember before the end of the day, I work blazingly fast…Estimation skill: Whenever we guess we’re on a 2000 minute project, we don’t know that.
We guessed, and we don’t have perfect estimation skill.
We might actually be on a project that will take on average 1700 minutes, or 2300 minutes, or whatever.
So for 33% of projects, let’s add 20% more work, and for 33% of projects, let’s subtract 20% of the work.
No idea if this is conservative or not.
So now, for each run, we’ll still start with 2000 minutes (10 full days), but then we will compute the above random variable additions & subtractions and modify that 2000 minutes.
Then we will actually simulate the day 1/2/3… of the project by sampling my schedule data, just like before.
Here it is:The distribution of 1 million simulated runs of a 10-day project.
Uses my real schedule data, but also randomly modifies the project length using make-believe formulas to represent unexpected difficulties, changing requirements, energy/morale, and estimation error.
78.
0% of runs are ±1 day or more away from the expected (instead of original 60.
1%).
95% confidence interval: 3.
9 to 17.
5 days (original was 7.
0 to 14.
5 days).
Many software developers recommend that if you are asked to choose a date and commit to it, on a project, then: you should take your estimate of the expected average length, and double it, and that’s your commitment date.
Looking at the above graph, that seems like a reasonably good call.
Although, I think most developers tend to underestimate the expected average…so those developers should probably triple it.
Appendix: Rules for data gatheringHere are some important rules I used for recording data and choosing to include it in this experiment:If I took an unexpected day off, then that still got recorded as a “0 minutes worked on my main project this day”, and still got included in this dataset.
So, sick days, for example.
Out of the 25 “zero minute” days in my dataset, there were probably only about 10 of these (the others were days where I just got interrupted too much, presumably).
If I took a planned day off, like if I knew I was taking that day off whenever I last gave an estimate/commit date for the project, that day was not recorded at all and not included in this dataset.
There were small periods of time where I was not assigned a “main project”, I was just in-between projects, doing whatever.
These days were not included in this dataset (I did record them somewhere in case I want them some time though).
This doesn’t happen often.
There was one large project that was clearly an outlier from other main projects I’ve worked on, and so that time period was not included in this dataset.
It was a project I was passionate about and had a strict business-driven deadline.
If I had included it, my schedule’s variance would have been even more extreme and would have proved my point further, but it would have felt like cheating.
Strictly speaking, to do a statistical study like this, each data point should be independent of the others.
My data is probably not actually entirely independent — surely there are interruptions that recurred over multiple subsequent days.
But, I think it’s close enough, you can take a look at a scatter plot, to give it an eyeball test for independence:Every day included in my dataset, in the actual order they occurred.
This is supposed to show that each data point doesn’t affect the adjacent data points to the left and right, much, or similar patterns.
Pretty darn random.
I suspect independence is rarely perfectly achieved anyway for most statistical studies about real life.
.