I am having a difficult time with management asking for estimates on programming tasks that are using third-party controls that I have no prior experience with.
I definitely understand why they would want the estimates, but I feel as though any estimate I give will either be a) too short and make me look bad or b) too long and make me look bad.
What estimate or reply could I give management to get them off of my back so that I can continue doing my work!
The best answer you can give is to ask for time to knock up a quick prototype to allow you to give a more accurate estimate. Without some experience with a tool or a problem, any estimate you give is essentially meaningless.
As an aside, there is very rarely a problem with giving too long an estimate. Unanticipated problems occur, priorities change, and requirements are "updated". Even if you don't use all the time you asked for, you will have more testing time, or can release "early".
I've always been far too optimistic in my estimates, and it can put a lot of stress into your life, especially when you are a young programmer without the experience and self-confidence to tell bosses uncomfortable truths.
I'll let you in on a secret. Even if you were an expert with that technology, your estimate is likely to be highly inaccurate. It is the nature of the beast when doing something that is an inherently R&D task. Unfortunately management often tries to apply a manufacturing model and demand accurate estimates. To illustrate my point, consider the difficulty in accurately estimating the following two efforts:
A) Manufacture another 11K umbrellas that are the exact same as the 2K you churned out last month. B) Design a new kind of umbrella and build the first one.
Software development is B, but they are asking for an estimate assuming A.
The best you can do is break the task down into the smallest pieces possible (no more than 1/2 day each) and then triple the final number you come up with.(Spolsky Method)
Alternately, Steve McConnell has a whole book (arguably several) on this aspect of software engineering. http://www.amazon.com/Software-Estimation-Demystifying-Practices-Microsoft/dp/0735605351
Steve McConnell (and others) talks about about the cone of uncertainty. Basically you provide an estimate that looks something like this:
The work will take between 3 and 9 weeks with 4 weeks being the most likely.
As the project progresses you can refine your estimate. As you do more of the work and understand the effort required better you can refine your estimate to be more accurate.
It's worked for me, but it can require some effort to get other stake holders in the project to understand the process.
You might want to consider giving both an estimate and a confidence level, i.e., it's 50/50 that it will take 3-6 months or 6-9 months or 75% chance to be done in 9 months and 90% that you will be done in a year.
Another thing you might want to consider is using the "wisdom of crowds" approach. Go around and ask 25-50 people how long they think it will take and average their estimates. Mike Cohn's Planning Poker is, I think, very similar to this, though difficult to plan with just one developer.
Break down your estimate into:
Offer to adjust the estimate and certain milestones along the way. Any unknown unknowns will become known unknowns, the known unknowns should become known knowns as you gain experience, and the estimate of you known knowns can be adjusted based on progress to date. You can do an initial estimate, then re-estimate when you about 25% done, then again at 50%, then again at 85%. At each milestone your estimate should start converging on the actual time the tasks will take.
I use a definite labeling system for my estimates... class A, class B, and class C.
The class C estimate is the first they get. It is openly stated as plus or minus 50% due to unknowns. If they want me to continue to give them a class B, then I need money to research.
The class B is plus or minus 25%. Sometimes this is good enough and they give me the money to build. If not, lesser money and more research.
The class A is plus or minus 10%, the final and go or no go. If it is a go and I stray too far from the estimate I confess often and early.
I think that if you remove the phrase "that are utilizing 3rd party controls that I have no prior experience with", you might have a better description of your larger problem.
If "Agile" has taught us anything, it's that, if management expects you, on an ongoing basis, to estimate projects that way, and you will "look bad" if you say it can't be provided because you don't have enough information, you're on the highway to FAIL.
The biggest problem is going to be the issues that you have no control over, and which you haven't even identified yet. How often have you looked back and said to yourself "Well, I hit my estimate right on the button - on the third try, after I figured out that ... and that I needed version ... and that the dba would be on vacation for a week and that the Project Manager would need me for ... for a week and that my wife was pregnant and ...".
I'd try real hard to say, "I can identify the critical risk factors and come up with a checklist of deliverables to test them in xx days. At that point I'll give you another incremental estimate."
And it would be real nice if you could suggest that they should "Please insist that I never try to give you a credible estimate of that type in the future. Fire me if I try."
(Overstated, but only slightly.)
Don't even try to estimate. There isn't any way your estimate will be correct. It is after all just an estimate.
I would instead recommend that you split the feature into small pieces (no more than, say 1-2 days) and try to deliver these pieces as working, complete, testable and valuable code to the customer/manager. That way he'll see for himself your progress on a day-to-day basis. This also means that he can in fact decide to stop development once satisfied and consider it complete, even though it may not have reached all the goals.
Have a look at the agile practices "Release Planning" and "Iteration Planning" for more in-depth information about this approach.
Keep in mind if you ask for a larger time estimate but make it under time, it looks way better than under estimating and having to ask for an extension.
I would try to mock a prototype so that you have a better idea of the time it will take. Be honest with your management so they can budget for unexpected delays in the learning curve.
EDIT: You might also see if you can get a more "iterative" deadline. In "Pragmatic Thinking and Learning", Andy Hunt makes a good point that people are project experts closer to the end of the project and least knowledgeable at the very beginning. It doesn't make much sense to do all of your design and time estimation at the very beginning when everyone is the least knowledgeable about the project. If you "iterate" the deadlines and solve the problem in chunks, you will have more success.
A lot of accurate estimation is self-knowledge. If you've written a lot of code, if you've had to learn a lot of APIs, you start to get a feel for questions like:
Throughout that, you should get a sense of such things as:
Based on all of these things, you will be able to develop a sense of how long something will take and be able to state your assumptions ("Assuming the API is sane...") even in the face of woefully incomplete information.
Estimate how long you need in order to learn enough to make a better estimate, for example, "I don't know: I've never work with this before. It will probably take me insert your estimate here in order to work out what you need to learn about it which I'd have to know before I can give you a good estimate for using this to finish your project."
You just guess an outside number and get to evaluating immediately, let them know that future information may effect your estimates, but that you will keep them up to date.
As you evaluate, keep them informed--either through a document published on the web or weekly updates, but always include an updated "estimated end date", and reasons (if any) for extensions.
Most managers should understand that--by asking for end dates, they are really saying "give us some idea how we can plan our schedule" and "don't just take forever".
If you end up extending more than once or twice, re-evaluate your entire schedule based on your new knowledge that you suck at estimating.
When programming I have always taken what I really thought it would take me and multiply that by 3 to provide an estimate. If I think I can do a job in 1 week, I tell the client it will take 3--if I think it will really take me 3 weeks I tell the client 9 weeks.
By doing this I set myself-up to "under promise, over deliver" -- if you can successfully do this your life will be much better and your clients will be extremely happy.
In your case you will certainly want to get at least SOME understanding of what you are diving into before providing an estimate. Maybe you even need to provide an estimate on how long it is going to take to provide an estimate. Multiply by 3 keeps the clients happy.
Break it down into things that you do have some experience with. The act of chopping it up will give you a better idea about what you know and what you don't know.
Once the pieces are small enough that they can be seen as single defined tasks, a few of them will be entirely un-estimable. For those, either prototype first, or just leave yourself some reasonable amount of time, depending on the size of the piece. If you're finding that you've got un-estimable pieces bigger than a 2-4 weeks of work, go back to chopping them up first.
Eventually you'll get down to some very weird technological solutions (that you think should work, but don't know for sure), and a whole lot of work to be done to back that stuff up, once it works. There'll be a few bits of missing design, for which it's best to pick some well-known library or a very simple algorithm for the initial version.
If you can't break down the tasks, then you should hire someone with enough experience who can (since your lack of experience is going to haunt you in other ways too). If you can't hire someone, then you should just bargain for a randomly long period (6 months to 2 years), and head straight into a messy prototype (until you've managed to give yourself enough experience to know what's right and what's wrong). But, if you end up flailing at it, it is important to not kid yourself and think that you're doing it the right "way". Prototypes were meant to be thrown away. Hopefully once the prototype countdown is completed, you're ready to build it for real.
Paul.
I'd add to what RB said, when I find myself in this situation I estimate how long it would take with tools I'm familiar with, and then double that estimate to build in some learning curve.
The important part is communicating to management that the estimate is a guess, if they press for a more accurate estimate or if they try to - dear God - sell you a time limit (surely it'll only take you 2 days to build the Starship Enterprise, you're good you are) STICK TO YOUR GUNS, don't compromise your estimate, or the fact that it's unreliable.
If management override you and timebox the task e.g. "Well, it has to be done in 2 days", again let them know that's their estimate, which is exactly as reliable as your own.
Get all this down in writing.
I deal with estimation quite a bit in my job and it's a real challenge. One of my biggest challenges is estimating how long it will take a different developer to accomplish a task with no knowledge of how skilled that developer will be.
While you might see some initial success with the "under promise, over deliver" method, you will find that over time you will be outbid by other people who follow the "over promise, under deliver" school of thought. Lack of accuracy will come back to bite you either way. Accuracy is very much tied to experience and limiting the number of unknowns with the technology.
One thing I would suggest is to get some idea of what kind of budget your estimate will be working against. If you have a small budget, don't go crazy with unfamiliar technology and stick with what you know. If your budget is a little more flexible, then you can afford to experiment a little bit.
Also recognize that there will be some tasks where all you can provide is a Wild Ass Guess (WAG). For these you should set a minimum time to your estimate and make it clear you don't know what the max is. Often times this kind of estimate is enough reason for certain features/req's to be cut out by management.
That's a very common situation: the necessity to deal with the unknown. A useful way to tackle this is realize that besides the actual programming tasks, you have some learning to do - and management should be aware of that.
When you are in a situation like this, the project suddenly becomes an R&D project and a longer time than normal shall not make you look bad, since you are learning and producing programs. I don't know how fast you are learning or what resources you have to deal with any problems you may find (Stack Overflow is one of the options you have).
I would say that you should estimate as usual and then multiply either by 1.5 (if you are a fast learner and have access to resources to solve your questions) or by 2.5 if you are an average learner and are relying only on yourself.
I hope this helps!
All the above responses have covered pretty much everything regarding coming up with the estimate itself.
One thing I would emphasize is to keep track of your estimate (a small Excel spreadsheet a la Joel, or even a Notepad document if it's very simple), and at the end of every day update this to the most accurate figures you can now provide. Even if you don't need to pass this back to your bosses, keeping this up to date gives you a better idea about how things are progressing, and more importantly you'll get a good feel for why your estimate changes as the work progresses.
Doing this will make you better at estimating in future, both for this specific technology and others that you haven't used before, simply because it requires you at some level to notice when your expectations change at regular intervals, and work out why that happened.
Estimating how long something's going to take is part of your job. So long as it's understood to be an estimate rather than a deadline you should have nothing to worry about. There is no one better placed to provide an estimate than the person who is going to write the code. If you can't provide a good estimate then you need to make management aware of the risk attached to your bad estimate so they can reconsider whether it is worth taking the risk on programming against these unknown 3rd party controls.
Can you give a range? 40-60 hours, something like that?
The smaller the tasks are the more difficult it is, if you can group them you'll have a little more "slop" as the errors may balance at the end of the project.
Look at any area you do have experience with and use if as a guide. "Last time a needed to create a feature that changed the database it took me X". Good luck.
Try your hardest to split the task up in managable pieces. With some luck there are specific tasks that are related to the 3rd party component involved and others that are less coupled (and therefore easier to estimate). Give management the split up estimates and point out where the most uncertain estimates live.
I fully agree with whoever suggested to play around and prototyping some. Set a fixed timebox for the prototyping activity. ("I need two days first to make this part of my estimate better.")
