I've gone through times when management would treat estimates as deadlines, and were deaf to any sort of reason about why it could be otherwise, like the usual thing of them changing the specification repeatedly.

So when those times have occurred I've (we've more accurately) adopted what I refer to the "deer in the headlights" response to just about anything non-trivial. "Hoo boy, that could be doozy. I think someone on the team needs to take an hour or so and figure out what this is really going to take." Then you'll get asked to "ballpark it" because that's what managers do, and they get a number that makes them rise up in their chair, and yes, that is the number they remember. And then you do your hour of due diligence, and try your best not to actually give any other number than the ballpark at any time, and then you get it done "ahead of time" and look good.

Now, I've had good managers who totally didn't need this strategy, and I loved 'em to death. But for the other numbnuts who can't be bothered to learn their career skills, they get the whites of my eyes.

Also, just made meetings a lot more fun.

The article is about modernization projects, which have soft deadlines because ostensibly the legacy software is still running while you're developing the replacement. There's always budget pressure, and promises may have been made, and users may be hoping for new features and eliminated frustrations... but if the replacement is a day late, it really wouldn't matter much.

Conversely, if you're trying to launch a space probe and the planets are no longer in the right positions for the required gravity assist, your spacecraft will not get where it needs to go. Or if you're a little $100M/yr toolmaker, and Ford asks you for a die for the 2026 F150 production line, to be delivered by March, and the contract states you owe a $20,000 per MINUTE penalty if you're late...you don't wait until February to say something surprising happened and it's not going to be ready. You don't sign on that dotted line unless you know for certain that you can do it.

Ford or NASA won't bat an eye when you tell them that a quote is going to cost $XX,XXX. They won't be surprised when they give you an ECO and you say that it's going to take 3 weeks and $8,000 to deliver a part that everyone knows you can probably make by hand in 30 minutes, they know that you're hedging against those deadlines, and pricing in the acceptance phase and inspection phase and contingency plans and everything else that makes their deadline-heavy industry function.

But if you tell someone at OP's modernization group that due to incomplete information you think that the 30-minute task to change the text of that button will take "no more than 3 weeks and $8,000" they'll laugh you out the door. Optimistic estimates get rewarded, pessimistic estimates get discouraged, accurate estimates are irrelevant, and in the end you're constantly behind schedule and no one's really surprised.

In 1505 Michelangelo gave an estimate of five years to finish the tomb for Pope Julius II.

Due to small side projects like the painting of the Sistine Chapel ceiling it took around 40.

Failure to meet the deadline informed by the estimate meant that the scale of the project was massively reduced because: Pope Julius II had died prior to completion, there were changes requested by the customer (both Julius and his heirs), supply chain issues, contract renegotiations, labor disputes, shortages of qualified workers, and money running out due to the long duration of the project.

So, since 1505 at least?

The funny thing is that the pope isn't even interred there.

I learned something important early in my career: the first number you put out will be remembered.

Unfortunately it’s often true. People keep saying: "but didn’t you initially say X?"

"Sure I did, but I have new knowledge" won't always work.

A nasty side-effect is that people who are aware of this shy away from giving you numbers.

"Can you imagine if the insurance company started arguing with the repair shop, asking them—no—telling them that they would only pay the $18,000 and not the additional $20,000 because that was the original estimate? Does that sound ridiculous to you? It does to me, too. Thank heavens, reality does not operate like this."

That happens all the time with insurance. I'm surprised at the confident tone in "reality does not operate like this". Not just car/home insurance either...health insurance also. They do often negotiate to a reasonable place, but not always.

One trick, if you can get away with it, is to ensure that you are always estimating for a fixed scope exclusive of unknown unknowns.

You should not provide an estimate for "feature X implemented", but rather for "feature X engine". If you discover additional work to be done, then you need to add "existing code refactor", "feature X+Y integration", etc. as discovered milestones.

Unfortunately, you need that nomenclature and understanding to go up the chain for this to work. If someone turns your "feature X engine" milestone into "feature X complete" with the same estimate, you are screwed.

------

There is a related problem that I've seen in my career: leadership thinks that deadlines are "motivating".

These are the same people that want to heat their home to a temperature of 72F, but set the thermostat to 80F "so it will do it faster".

I was once in a leadership meeting, where the other participants forgot that I, lowly engineer, was invited to this meeting. Someone asked if we should accept that deadline X was very unlikely to be met, and substitute a more realistic deadline. To which the senior PM responded that "we never move deadlines! Engineering will just take any time given to them!"

Engineering, in that case, gave the time back when I left that team.

After too many iterations of providing "some wild-ass guess" estimates and them turning into hard deadlines, I now try to champion a No Estimates[0][1] approach with stakeholders.

There is often understandable resistance to this at first. To address concerns, I find it helpful to share with stakeholders that a reasonably accurate estimate, one which could be legitimately used in planning concerns, is really only possible in one of two situations:

  A) the outstanding work is a carbon-copy of a previous
     effort, such as the second time provisioning a data
     center for the same system.

  B) the remaining new functional work is determined
     by the team to be in the last quartile and is
     well-defined, including remaining risks to
     successful completion.

Micro-estimates are the enabler of micro-management. A healthy team identifies the highest priority tasks to perform and does so in descending order, where priority is defined as risk to project success.

0 - https://www.youtube.com/watch?v=MhbT7EvYN0c

1 - https://www.goodreads.com/book/show/30650836-noestimates

This is a fun formula that caught my eye a while ago in HN, it looks flashy and very cool. Of course, just like others do their estimations, this one is just a made up formula and without any formal validity, apart from supposedly personal experience:

https://news.ycombinator.com/item?id=37965582

  My estimate math:
  
  R = t × [1.1^ln(n+p) + 1.3^X]
  
  R - time it really takes.
  
  t - shortest possible time it would take without need to communicate.
  
  n - number of people working and involved during the process, both customers and developing organization.
  
  p - longest communication distance network involved in the project (typically from the lowest level developer to the end user)
  
  X - number of new tools, libraries, techniques, used in the process.
  
  Example. Project involving one developing writing code. Project would take 2 weeks (t=2), but it has 5 people (n=5) involved total, only 1 new tool (X=1) and longest communication distance is 4.
  
  2×(1.1^ln(5+4) + 1.3^1) = 4.5 weeks.

The best estimation system I've seen is to actually BET on the completion date, closest gets a free lunch paid for by other members of the group.

Those dates were mostly informed guesses of what would actually happen or go wrong. Importantly this was between friends. Needless to say, they turned out very accurate.

Not only that, in the name of efficiency where I work estimates are actively being pushed down and no spillovers are allowed. I've been in crunch like mode for over a year with no recuperation at all. I kept on hoping it will get better but it seems it's getting worse. On top of it we're all rotated to different areas of the product all the time with no recourse. Though I'm still running along I feel absolutely spent mentally...

Of course businesses want to be able to de-risk by having highly accurate predictions of the future. Too bad those don't exist in any domain of business planning.

More often, the focus on estimating comes from management layers where incentives are not structured to reward anyone for accurate estimates, merely to punish them for missed deadlines.

Time to finished is only one dimension of estimation. With any unit of engineering work there may be code debt added or removed, complexity increased or decreased, morale increased or decreased, etc.

Focusing only on time, especially in a punitive way surely negatively impacts the others.

My estimation technique is to completely ignore the nature of the task, and instead just try to figure out the highest number the person asking will accept.

I often see takes on this topic from the engineering side. "It's hard!". "Managers just don't understand".

It feels like as a community, it would be useful to get more articles seeing things from the other side and exploring functional approaches beyond provide-a-worst-case-scenario-estimate.

There's a reason this dynamic is so pervasive. In order for everyone in an organization to do their job well, people do often need a realistic set of estimates. Can sales promise the prospect this integration? Can marketing plan a launch for the new feature? Can the CPO make a reasonable bet on a new line of work?

In my experience, the nuance here is more about handling the mis-estimates. How do we discuss the risks up front? How much work should we put into contingency planning? How do we handle the weeks / months before a deadline when it is clear that we need to limit scope?

Estimates are tricky because different manager roles and different personalities bias toward totally different/incompatible concepts of what an estimate actually means. The author's article is conflating realistic and pessimistic estimates:

- Realistic e.g. tech managers and people who favors agile/lean/XP/etc.

- Optimistic e.g. sales managers and people who want to promote.

- Pessimistic e.g. risk managers and people who need firm deadlines.

- Equilabristic e.g. project managers and people doing critical chain

The abbreviation is ROPE, and it turns out to work really well in practice to cover all four bases. My notes are below. Constructive criticism welcome.

https://github.com/SixArm/project-management-rope-estimate

In my direct personal experience, for me it was 1969 during my first major gig. I remember one project, called the March 1 system, that slowly came into being sometime that October, if my memory serves me correctly. There was tension, of course. This was exacerbated by having to work nights to get access to the system to continue development.

I eventually learned, when asked for a "quick" estimate, I would give something drastically longer than I knew would be accepted. I said "But I can give you a better estimate if you give me a few days to do a better plan." This always got me the extra time to provide an estimate.

My technique is to give an estimate with error bars. Something like 6 weeks plus or minus 2. That then leads into discussion as to what is unknown/undefined leading to the uncertainty. Sometimes the error bars are larger than the estimate because I know there will be endless feature creep and UI revisions for something like "analytics". And sometimes the number could be negative like 6 weeks plus or minus 8. That is when the functionality already exists (eg the data is already there - you can already load it into Excel and a pivot table) and could be sufficient.

I would say that the trend against actual agile and towards waterfall (PRDs are totally en vogue) suggests that deadlines are how most company management looks at this. Definitely not suggesting it is right, but "really aggressive 'estimates' (that are accurate beyond human ability) you can plan on", aka deadlines, are expected of any product/engineering org today.

Tell me a story I want to believe, even if it isn't true. Then you can make it the teams' fault because they said they would and didn't.

When I give an estimate, I always reiterate that it's just an estimate that is based on limited and incomplete information, and that the real number can be more or less. If they don't like it, they're free to put someone else on the project.

> When did estimates turn into deadlines?

In my personal experience: the first time I gave what could be construed as an official estimate.

In my experience, there are two different reasons for why I get asked for estimates.

There's the cases where it's used to roughly schedule work, or to prioritize features. My boss wants to know roughly how much is on our plates, so he can plan for known upcoming work.

Then there's the cases where it's more of a XY situation, where the boss is asking for estimates because in reality they've got a customer on the hook but they won't sign unless we can implement some functionality before go-live, or something along those lines. Typically that'll be a hard deadline, as customer will either have to switch to us or pay another year of licensing, and the boss wants to know if I can deliver.

I try to suss out if it's the latter, and if I'm unsure I will simply ask why they want the estimate.

If that's the case and it'll be a struggle to make the deadline, I'll try to help figure out if we can perhaps solve the core issue some other way. Perhaps a temporary solution that the client can live with for a week or two extra while we finish the proper solution, or perhaps we just simplify our proposed solution, enabling us to leverage existing infrastructure, and that turns out to be good enough for the customer.

It's really simple. To succeed you need to look good in person and on paper. Under estimate and it can blow up in your face. Over estimate and you start to look incompetent. Fail to walk the tight rope and you'll soon be laid off. Succeed and you'll survive long enough to get laid off anyway when the next recession hits.

I normally only commit to "We will give an update in X (minutes/hours) to make sure we understand the problem first. Then start giving estimated timelines in ranges with specific call outs for updates and possible changes to the timeline.

I've found that most management just want to be involved in the process and have definite times set for updates and can handle timeline changes as long as information is coming at regular intervals.

That's cool that they went to Sokcho!! Not many people go to Sokcho but imo it's the best city in Korea, the vibes there are on point, nobody really speaks English and it's pretty dead for the most part, but I really love Sokcho for the vibes, it's maybe the place i've felt the most at peace in my life. If you ever get the chance, I recommend Sokcho. :)

Working in smaller steps is how you should build software. Constantly get feedback and re-evaluate what you're working on with other members of the team. Instead of giving an estimate, use t-shirt size.

With constant feedback, the whole team is participating in the emergent complexity, instead of being passive and just annoying you with "is it done yet"?

Even worse -- giving a thorough estimate, having the other party decline and reply with a different, significantly shorter estimate, and then turning their new estimate into your new deadline. Woof!

Nobody is going to fix the problem without fixing the culture, which isn't easy to do.

The issue isn't around tasks that are predictable in nature and therefore easy to estimate with a small margin of error, it's around complexity in software, unforeseen things, bugs, etc, which can compound for larger long term projects.

If engineers give estimates close to what it would be if everything goes right, then they risk overpromising and underdelivering if something goes wrong (hofstader's law). They might've just wanted to do the right thing by saving the company money and time, but in the end they footgunned themselves.

Or engineers intentionally over-estimate in order to manage the complexity, but then you end up with a lot of padding and parkinson's law. Because as soon as the engineer starts underpromising and overdelivering consistently, management will pressure them to lower their estimates because they have a track record of doing that, so instead they're incentivized to pad and then fill up the entire time they estimated even if it took less time.

Sprints were probably invented in order to deal with some of these issues, so that people just work with a bunch of smaller tickets that are much easier to estimate, with the more complex long term estimates going to management, which are incentivized to get it right because they're shareholders. That often leads to micromanagement and burnout, and it doesn't fix the padding/overestimation issue either, it might even amplify it in a lot of cases.

People here mention giving ranges or probability distributions, and have also equally mentioned that they don't work because management wants a single number, or management just assumes the best-case or middle-case of the range as the actual estimate, and then they still get in trouble for giving ranges and it didn't solve anything. It also doesn't solve the problem of unanticipated setbacks, the whole you don't know what you don't know thing, which can only really be solved culturally in some way.

While there are certainly bad managers that want to squeeze their workers, a lot of the time management is probably also pressured to give estimates and that's why they want and need that accuracy, because they're pressured by investors and clients that want to know how much time and money something will cost.

Overall the entire problem is a system cultural issue around managing complexity.

I learned how to manage client expectations from Scotty in Star Trek: https://youtu.be/L3jXhmr_o9A

This is especially problematic in early business when the team is small and manager act with same policy/process as when at BigCo.

In this case don't estimate the time to build a poorly defined $something - invert the problem to estimate the value of $somethig.

It's amazing how many of those managers asking for estimates push back when they have to put one out. With all the same reasoning that engineers have when estimating.

A good manager should start with the Value first and allocate time-budget that makes that Value payoff.

When managers refused to accept that we just can't predict the future of the creative work that is software design and implementation.

And that's because their entire existence is based upon money, not results. I've only ever had one good manager, and that was because he knew what he didn't know and accepted that we do and are trying our best.

Multiply your original estimate by 3, works most of the time

I definitely relate to the basic issue of estimates with my one-person consulting projects. The bit they didn't mention is: who pays for the cost of the inspections and analysis? In software, it can take a long time to analyse the requirements and the solution, in order to come up with an estimate (or fixed quote), and I find it awkward trying to get paid for that time.

When incompetent management who scam their sorry asses to the top for a lick of the brass ring before getting their asses kicked to the curb became the norm.

This is a problem people and we're not impressed.

when an external / vendor team picks a project...

estimates = contract amount = project budget ...

it is hard for a project team to negotiate later

often the estimates need to be competitive or bottom most for you to win the contract

no one acknowledges the unknown and risks at the beginning of the project

writing down more than 4-5 risks along with the bid amount is taken as a sign of a team that will not be easy to work with and fight over every bullet point whether something is in scope or a change request

and often the one who estimates and wins the project may not be on the actual project team with delivery accountability

The problem of estimates as they exist in a "Agile" process - they force decisions to be made when the least amount of empirical data is available. Then once work starts and information starts flowing in, you can't change your estimate. The scientific method is explicitly banned! This is often by design; data-driven decisions are incompatible with management-vibe-driven decisions.

At the very least, you need to do a bit of legwork to gather data prior to giving an estimate. Call it design, call it architecture, call it research, call it proof-of-concept, I don't care. Just stop insisting that decisions be made in a vacuum of data. Real results from running code trumps everything.

To be clear, you can produce software without using the scientific method. You can build anything without a data-driven process. But you get what you pay for. The head-in-the-sand approach ignores valuable information and yields poor quality as a result - it doesn't fit the definition of engineering.

Sadly, estimates are a negotiation. Whoever provides a number first generally loses because of "The Wince".

"What's your estimate?" "I'm not sure." "Just ballpark it."

"Well, when would you want it by?"

This is the trap that new managers will fall into every time. If they give you an answer? Bingo. You give them "The Wince":

You suck air between your teeth, and with a frown say, "Oh, that's completely unrealistic. Where in the world did you get that number from??" Then you provide a number that's many multiples higher, or offer a reduced amount of work: "Oh, gosh. We'd only be able to get X feature done in that amount of time, and only if we got lucky and cut feature Y from the other project."

Regardless, whatever you do, don't be the first to provide a number. It takes a little bit to get the feel for it, like poker or buying a car.

Time is money, even in a big corporation. Treat it as such: It's a zero sum game.

> Can you imagine if the insurance company started arguing with the repair shop, asking them—no—telling them that they would only pay the $18,000 and not the additional $20,000 because that was the original estimate?

Well yeah, because there's not an inherent power imbalance like there is in employment.

Part of this imbalance results in the ability for managers to employ Taylorization upon their directs. The majority of the time Taylorization hinders workers but management loves it because they can have more control in outcomes. What ends up happening though, is that an shadow work plan ends up getting established that management has less control over unless they want to drive out top talent by employing technocratic solutions to social problems.

Estimates turned into deadlines around the time oneone came up with the concept of an estimate.

Sometime around 1956 at a guess.

A lot of ink has been spilled on this topic.

The solution is simple: get better at estimating.

Software engineers act as if they're the only ones in the world who are asked to estimate and then held accountable.

It's a skill issue.

Estimates have always been deadlines. I've noticed it since at least 1999. Either your estimate goes up a certain number of management levels and then they don't want to lose face so you are stuck with it. Or a salesperson has seen it and promised it to customer and nobody wants to be the one that loses the deal so you are stuck with it.

>that hurt will not go away anytime soon

Feel the Bern.

> Can you imagine if the insurance company started arguing with the repair shop, asking them—no—telling them that they would only pay the $18,000 and not the additional $20,000 because that was the original estimate? Does that sound ridiculous to you? It does to me, too.

No, actually, this sounds quite realistic and in many cases even reasonable. Medical insurance does this literally all the time. Insurance companies of many different kinds in general have to do this to prevent fraud and keep costs (and by extension premiums) low.

> There is no fixed path when modernizing a complex legacy system. There is no rulebook to follow.

Of course not. But as an engineer tasked with this kind of project your estimate should reflect some kind of plan for accomplishing the project, where the plan has more concrete and actionable steps that make estimation easier. And if you are good at your job, your estimate should account for known-unknowns (eg this part is contingent on something partially out of our control and may be delayed by X months) and give enough slack to handle inevitable unknown-unknowns without excessive padding. And uncertainty regarding any particular risks or variance in how long somethign takes should be explicitly noted and communicated.

My $0.02:

The unfixable estimate vs deadline problem ultimately boils down to money. A particular project might be worth it if it ties up 10 people for 3 months, but not worth it if it ties up 10 people for 12 months. And relatedly, businesses oftentimes find themselves needing something finished by a certain time to close a sale/keep a customer happy/catch up to a competitor, and the stakeholders on the other end (customers, investors, partners) need some kind of date for their own planning.

Good estimation is really about identifying the probability distribution of the completion date, rather than a single "expected" completion date, and identifying/communicating the related risks. For a big project this itself probably will take a few days to months to complete and communicate. If you do this properly, there should never be any ambiguity as to what is an estimate and what is a deadline. If you don't do this, you are basically assuming that your manager (and their managers and so on) can read your mind and have the exact same understanding as you do regarding how firm the estimate is + what the risks and possibly "actual completion times" are. You're also denying them the opportunity to help you derisk or accelerate the project by eg adding more people to work on it, reach out to some external stakeholders, or communicating the end date to eg investors/customers in a way that reflects risks. You also need to update the people that care about the completion semi regularly to tell them about any new risks or known unknowns for the same reasons.

If you are wildly off with your estimate even after spending the time to breakdown the project with a more actionable plan, identify risks, and come up with ranges/a distribution of outcomes - let's say you blow past your p99 time to completion - you are probably just bad at your job (or trying to operate above your skill level) and it would probably be very reasonable for your manager to hold you to a deadline with the threat of firing you/canceling the project. Yes, the sunk cost fallacy dictates that even projects that run over should probably be completed in many cases (though since overall budget/personnel are usually constrained I think the opportunity cost of doing other things becomes pretty important). But if people can't trust you to estimate properly they probably can't trust you to lead a project or to have estimated the remaining time to finish the project.

When marches were replaced with sprints.

David Parnas wrote in A Rational Design Process (1985) [1] that accurate estimates are essentially unattainable, but we should try anyway. The point is, I think, that the ceremony of a structured process facilitates communication better than no ceremony.

[1] https://users.ece.utexas.edu/~perry/education/SE-Intro/fakei...

These are all just tricks to reduce cost and make sure you get things done. You have competing constraints:

# You need to align the rest of your team on things: maybe you have marketing tied to real world events, you're hiring and training to a sales cycle. If you do it wrong you waste money

# You need to fight scope creep which happens with unbounded timelines

# You need to fight a minimal product that's not viable because it isn't done enough for your market

and more. The ideal is for the decision maker to constantly have full information on progress and make instantaneous minute adjustments to keep things going at full steam. But there's no way to have full information and there's no way to make instantaneous minute adjustments if you're not a one-man shop. So everything else comes from the organizational effort of trying to work with that.

[dead]

[dead]