Project Estimation By André Vondran - Software Quality Assurance Analyst

Estimates

If I had to think of one subject that was not taught in school, and only covered in industry certifications in the most naïve way – it would be estimation. It seems so simple. You figure out what needs to be done, figure out how to do it, break the how down into tasks, and add them up. This is pure functional decomposition, right? Easy as pie. Sadly, in the real world, not so much. Do any of these sound familiar?

Forced Due Dates. In this case, nobody really does estimates. Or perhaps the boss does something quick, sloppy, and “aggressive.” In any event, the boss needs the project done in two weeks. Period.

'Someone else' does the estimates. Estimation is a skill, and it’s a skill that few junior staffers have. So having a senior staffer (or manager) do the estimates for a junior person might be helpful. The problem is that is separates the estimate from the person responsible. In other words, making someone else’s estimate is like spending someone else’s gift money – it is just detached enough from reality that you won’t be careful enough.

"Four weeks? Really? I could do it in one …" This is a manipulation technique used at the big status meeting, and I hate it. In some cases, the bully really can do it in one week, and it’s a great win for the company to switch. More often, all it takes is an offer to switch to shut the bully down. Junior people tend to buckle, promise the impossible, 'fail', and then look bad.

Bring me a rock estimates. "Hmm … three months? Can you shave a little time off that? … two months? We were looking for something a little more aggressive … One Month? OK, that sounds good. Remember, that is YOUR NUMBER, and we’ll hold you to that estimate!"

Slippery-Slope Requirements. Perhaps, by some miracle, you are allowed to estimate the project with your own date. Halfway through, the customer realizes that the requirements won’t meet his needs. This leaves you with two options – (A) Add the features and slip the date, making the project “late” or (B) Deny the features and deliver a product that doesn’t fit it’s purpose.


Why Estimates Are Important
In many cases, the "success" or "failure" of the project depends on weather or not we hit the date. In most of the scenarios above, that’s not a judgment of the technical contributors – it’s an evaluation of the person who made the original SWAG compared to reality.
In my next post in the series, I will discuss what we can do about it.

Estimates - Interlude

The previous post was on estimates for technologist in general. Software Testers, things are a little more ... challenging. For example -

What quality will be the code be when it is delivered to test?
Most of the time, we don't know. Vast differences in quality could make testing take a longer - or shorter - length of time.

What is the management expectation for the software when it is released?
Most of the time, these expectations aren't quantified - if they are articulated at all.

Once identified, how quickly will the developers fix the bugs?
Unless you've had the same team for several projects, and the projects are similar, you probably won't know this. If the answer is "not that fast", then it's probably not a testing phase - it's a fixing phase - and it's not bound by the testers.

For every one hundred bugs the developers fix, how many new bugs will be injected?
I've seen teams where this number is fifty, and I've seen team where this number is much less - but I can't think of a team where this number is zero. Obviously, if this number is bigger, testing will take longer.

I could go on - the point is that we've got a whole lot of variables that are beyond the control of the testers. In fact, it's probably impossible for the testers to even know what those variables are. So how can you provide an estimate?

Offhand, I know of only one career field that has to deal with a similar problem. That is -- your friendly 401(k) investment advisor. I always thought that "advisor" was a bit of a misnomer, because providing advice is the one thing that guy doesn't do. Instead, he carefully explains that he doesn't know where social security will be in fifty years, and that he doesn't know the value of your house, how it will appreciate, and weather you will pay it off early or take home equity loans. He doesn't know what inflation will be, or how you investments will perform.

So he gives you a piece of paper with a big, complex formula, and tells you to figure it out for yourself. After all, he will say "It's your money."

Testing is an investment of time by a business owner. It is senior management's money, yet we couldn't get away with this, now could we?

Don't worry. I have a few ideas and solutions for you.

Estimates - Laws

I'm a big fan of "Laws" of software development - Rules that express complex ideas in very few words. Moore's Law, for example, that computing power doubles (roughly) every eighteen months.
Or Jerry Weinberg's advice to not attribute to malice what can be explained by incompetence.

Or Paul Jorgensen's rule of large numbers in software development: Big Numbers times Big Numbers equals REALLY big numbers.

Now me, I'm working on a theory for software estimates, but here's another interlude, Heusser's Rule, which explains a lot of the ... odd behavior you'll see on software projects:

Most people really like to be able to sleep at night

This explains why people would rather be certain and wrong than uncertain and right. It explains why people hold on to bad ideas that should have been discredited long ago. It explains why people can force themselves that they will meet a deadline, despite a mountain of evidence to the contrary.

And it explains why they shoot the messenger. It's not that you're wrong; no, you are very much right. But by calling a spade a spade, you've robbed the entire room of the ability to sleep at night.

Estimates - II

As usual, some of my commenter’s have written my article for me ...
Seriously, Heidy points out that is it always possible to give an estimate, but that all estimates are wrong (if they were right, they would be commitments). Gustavo points out that we get lousy requirements for our estimates. For example, when asked: "When will it be done?"
You probably have a vague and fluffy definition of "IT" and a vague or unrealistic definition of "done."
For example, let me re-translate: "When will it be done?"
Could be: "How long will it take you to do everything I've written down here, with scope creep that I deem reasonable, and without a single defect or flaw - given that anything over a month is too long?"

Of course, I promised some actual answers. So here goes ...

The "GAP" between what the software actually IS and how it was analyzed is a serious pain point in software development. That gap widens with time - so my first suggestion is to never run a project more than about a month. (Yes, require/dev/test/prod in thirty days.) Schedule any "big" project as a series of small ones.

A worst, you might be two weeks late. That's not the end of the world.

If you have to run longer than a month, then periodically bring your code up to production quality. Develop features end-to-end, in thin slices, and add them to the system.

That's not really an estimating hint - it's a project organization hint. Estimates of less than a month are drastically easier than more. Moreover, they are small enough that an employee can feel a real personal commitment to the date, and work a little overtime to make it. (On a ten-month project, once you realize you're late, overtime will just kill you, not bring the project back on track.) So you organize the project so you never have to estimate more than a month at a time.

Plus, you can prioritize the customer's feature requests, so you build the most important thing first.

So what if that is not an option?

Ok, next idea. Do two types of estimates - realistic and pessimistic. Make the realistic estimate your goal date, and make the pessimistic a commitment back to the business. Within the team, talk about the goal; with management, talk about the commitment.

A third - Estimates can be done in two different ways. The first is an analysis task where break down the features into chunks and add up the chunks, or you take the spec and do some math to come up with an approximate number of test cases. The second way to do estimates is to actually start doing the work - at a very high level. Go ahead and _DO_ the design, or start to decompose the test cases, or write "Tracer bullet" objects to do the highest levels of disk, screen, and database interaction. Leave comments or "TODO" markers at where the next step will be. When you've finished that, come back to the TODO markers, estimate them, and add them up. Finally, add a little extra for risk - the bigger the risk, the bigger the buffer.

The thing is, the further you go into the project, the more you'll know. I have one colleague who simply avoids making any commitments or estimates at all. He'll say things like "if you say so" or "that's certainly our goal" but never make a promise. I'm not advocating that - but certainly the further you get into the project, the more accurate your dates will be.

(In Rapid Development, Steve McConnell actually suggests that you communicate estimates in a range that gets smaller over time, with the larger number first. "Six to four months" sounds strange - but if you say the smaller number first, people tend to forget the larger one.)

Looking back, most of this is not how you should do the estimates - but how to communicate them so they are not abused or mis-understood. This can be very challenging; as my old friend Eric likes to say "Sooner or later your three paragraphs of very specific if-thens and assumptions are going to turn into a PowerPoint bullet with a date on it. Tread lightly."

If that's the case, then you have a few options. One is to hit any arbitrary date by structuring the project so that you can go to production every month. Another is to make no commitments and blame someone else. A third is to way is to continually and consistently communicate risk-adjusted dates back to your customers.

My final thought is to express the cost of every change. It's usually not the project that kills us - it's the feature changes at the end. And that's fine - it's good - it makes us provide software that has better fitness for use. When that happens, though, either make the cost explicit (by adding the feature and insisting on moving the date) or pay for it with your nights and weekends.

The choice is yours ....

Estimates - III (Serious this time)

The code is big stinking mess, all over the floor. The team is dependent on a vendor who hasn't delivered a good build to us yet. We have no idea when anything is going to be ready. My best bet is three months out, but nobody really knows. Senior Management is Antsy. What will the project manager say?
Ok, so I wasn't there, but I know what happens: He walks into the big office and says "THE PROJECT WILL BE DONE IN FIVE MONTHS."

Where did that number come from? I have no idea. There was probably a gannt chart somewhere that added up to five months, but nobody really knew, and nobody technical trusted the vendor.

All I do know is that senior management went home that night - and they slept well, because they had a date. A firm commitment. None of this "gee, we don't really know, how long does it take to find your lost keys?" business. A firm date.

It was bogus, but, whatever. They could sleep. ... Time passes ...

Four and a half months later, and it becomes obvious that the team won't deliver in two weeks. I've been pulled off to work on another, different death march (*), but I take a general, cynical tester's interest in the project.

Again, the word comes down from on high "WE'VE HAD ANOTHER SLIP. THE PROJECT WILL SHIP IN FOUR MONTHS ..."

Of course, four months goes by ...
Eventually, the project was cancelled: Over a year late.
Now, I criticize a lot of things about that project. 99% of the time, I think the project management was irresponsible, incompetent, and (possibly) unethical.

Then again, 99% of the time, senior management was able to sleep at night, because, dang nab it, they had a date. None of this "uncertainty" business.
Which reminds me of an old quote by DeMarco/Lister - all too often, people would rather be certain and wrong than uncertain. After all, it's better to be a team player than be negative.

So, the basic technique is this: If you have no idea when you'll be done, make up a date so far in the future that you've got a good shot. When it becomes obvious that you cannot hit that date, well ... make up a new one.
Or, in other words: When uncertainty is politically impossible, do what gets you rewards.
Like I said before, 99% of the time, I think this is an incompetent and bogus way to "lead" projects. But that 1% of the time, I think: Man, if you have no idea when it's going to be done, making up a date real far away and slipping if you have to is a decent approach. (It can't be too far out - otherwise you'll get push back.)
All of that presupposes that you actually can ship at all. The problem in the story is that we couldn't, and anyone with decent systems thinking skills could see that, advice management, and would, well ...
... "find other projects."

(*) - Actually, to be honest, that one was a relief; it was a death march, and I worked massive overtime, and it was painful, AND IT SHIPPED.

André Vondran SQA Analyst
www.aqasol.com.ar