Why are projects late? And how to fix that.

I used to work with this very interesting fellow -- I'll call him "Roger" ('cause that's his name) -- who had some great ideas about project management. After some investigation and some brilliant insights, he shrank software development schedules without cutting quality or features, and without spending more money or getting people to work longer hours. He did this both in a large company (HP) and in small startups in the ’90s. For some years, I nagged him to write a book about it, but never succeeded in persuading him to do so.

Fast forward to 2007. I'm talking with Lissa, who I think also knew Roger at HP, and she mentions one of these ideas. "Brilliant!" I say. "Did you learn that from Roger, or did you come up with it yourself?"

Neither -- she got it from Goldratt's Critical Chain. Whoa, was this the book Roger was supposed to write? I ordered it from amazon.com and found out: He doesn't have everything Roger came up with, but he's got a lot. I think every manager and every director in Engineering ought to read this book (and probably more by this guy), but let me outline why projects are late and some things you can do about it. Page numbers below are taken from Goldratt's book, in the above edition.

1. They shouldn't be late!
   
   • People put safety factors in their estimates
     (NOTE: given the typical distribution for completion times, P(success) of 80% implies the estimate is 3x the median, i.e., padded 200% -- pp. 43-46)
     
   • Managers take the padded estimates and add their own padding
   Hence we have safety and more safety, but projects are still late!
   
   
2. In a sequence of steps... (pp. 121-122)
   
   • delays from one step are passed on in full to the next one;
     
   • worse, the longest delay from all steps is passed on to the next (dependent) step;
     
   • advances ("We finished early!") in one step generally do not result in an early start on the next step
   Roger reminds me that Dan used to call this the "idiot shift right" -- slide the schedule to the right and the past is forgotten.
   
   
3. People don't start on a task until they think they have to (the so-called "student syndrome", p. 124); since estimates are all padded, they don't even start when they say they will.
   

   Two other factors cause later starts in the general case: one is the desire to postpone investment (present value of money, pp. 68-69) and another is the desire to limit the number of irons currently in the fire -- in other words, management focus is a scarce resource (p. 70)

4. In some situations, critical resources are forced to multitask based on conflicting priorities; this can easily double lead-time on all paths (p. 126)
   
   
5. The key step in fixing this is to make "safety" explicit and visible. Goldratt calls it the "buffer" (pp. 153-155). Roger called it "contingency" or "contingency account". Any of these phrases can work, but the idea is that when your step takes more time than you thought, that eats into the buffer/contingency.
   
   
   • You track the current "balance" in the contingency account, or the amount left in the "buffer", and that will tell you how well things are going. (p. 163). Do not track due dates, or that will trigger the student syndrome!
     

     Also -- don't track percentage of resources/time used versus projected; that makes it look like you're on time even when you aren't (p. 73), and will tend to optimize the wrong management behavior (pp. 73-74).

   • Goldratt has the "project buffer" only on the critical path; he creates "feeding buffers" at the places where noncritical paths join the critical path (p. 158), "resource buffers" (which I'm too dense to understand, p. 160), and the eponymous "critical chain" (pp. 213-219), etc. According to Goldratt's paradigm, it is very important to keep track of which path/chain really is critical so that you can subordinate (pp. 159-160) noncritical paths to the critical path and thus reduce multi-tasking (which is evil, p. 126)
     
     
   • In contrast, Roger made a PERT chart and then locked it away somewhere. Instead of trying to figure out when the critical path is late, he wants to know when any task takes longer than estimated. This is far less complex than Goldratt's schema, and also takes into account a reality that Goldratt ignores: that dependencies aren't really "hard."
     

     Roger used to say, "If you really can't make any progress on anything because you're waiting on someone else, take some time off. And no need to report it to HR -- it's on me." Because if you think a little, you almost always can find something to do that will move the project ahead. And no one has ever taken him up on his offer.

     Roger's secret weapon on this front was: "Well, is your documentation complete?" That might have something to do with why nobody took him up on his offer.

     I tend to think Roger has it right, i.e., that managers tend to take the "critical path" a little too seriously.

     The other thing he did was to create a "right-to-left" PERT chart, the "integration plan", which mathematically "should be identical" to the usual (left-to-right) one. But there's some effect of doing it in the "reverse" direction that makes the critical path stand out.

   Whether Goldratt's scheme (with a lot of focus on the critical path, various buffers, critical chain, etc.) or Roger's (one "contingency account", treat the critical path with benign neglect) is ultimately correct, I think the first step is to Just Do It™ -- take a step away from "pad everybody's schedule and track milestones", toward a tracking scheme based on buffer(s) (or contingency account(s)).
   

I obviously have not done justice to Goldratt's book (I've only read this one), and certainly not to Roger's management ideas, but this is a start, anyway.

Update:
Roger read the above and emailed me back the other day, to clarify and remind me of some things I'd forgotten. The green text is my summary of his comments.

— collin 2007-04-10