The first thing we can do is to break our requirements down into very small sections; or stories as we call them. This on its own makes things much easier to understand and estimate. We might then find that we understand many parts of the problem and there are only a few small items left that we don’t fully comprehend.
The second thing we can do is a thing we call ‘spiking,’ which means driving a metaphorical spike right the way through the heart of the requirement until we have enough knowledge to estimate it with certainty. We can experiment, if you like.
But it’s not enough just to allow the developers to go off and experiment on an ad-hoc basis whenever they decide they don’t know enough, or haven’t the confidence, to produce a reliable estimate. Spikes also need to be estimated.
Each spike should have a set time limit, at the end of which the developer should be able to produce the estimate for the original requirement. Without this discipline, there would be no way of determining the end of your planning period.
It’s very common now to engage in two-part contracts with software developers; the first part being a requirements gathering and analysis stage culminating in a formal build proposal. This is the ideal time for spiking to take place too, so that when you deliver your proposal you are as certain as you can be of the figures you are proposing.
Of course, producing the estimates is only half of the story. Production needs to be continuously monitored, the second part of the contract, so we know whether we are going to reach our deadline. We can measure the ratio of real-time in man-days against the estimated work that actually gets completed. If we do this on a team basis, we usually find that individual inconsistencies cancel themselves out.
For example, we may find that each week we complete stories whose estimates add up to 30 days. Regardless of how many real man-days it took to deliver on those estimates, if the team doesn't change and we have 300 days of estimates for the whole project, we can estimate that the whole project will take around ten weeks, not counting holidays, sickness, etc. As long as the ratio of real to estimated man-days remains consistent, we have a good idea of when we will deliver.
If the ratio starts increasing (i.e., the delivery date is receding into the future), we need to take action and that usually means a customer decision to either: change the date (unlikely), add resources (unlikely too) or (preferably) alter the scope (leave out some lower-priority functionality).
So, in summary, there is no need to guess when you’re estimating, you can reduce uncertainty and risk by breaking your requirements down into more manageable chunks and experimenting to remove any remaining uncertainties. You should also monitor the gap between your estimates and reality as you go, taking corrective action where necessary (that is, in good time). ®