It is a rookie mistake to think that programming is splitting a big task into smaller tasks that are splitted into even smaller tasks, and that once the process is done, you can use mindless drones to complete these microtasks.
It assumes the perfect knowledge, the fixed work volume.
In practice, you don't know how much work might be required (a small task may blow up). It is inevitable that sometimes you have to drop features to ship in time. Programming is an iterative probabilistic process.
It is also a rookie mistake to think that just because some tasks might blow up, you can't accurately estimate a large class of them to within a reasonable confidence window (say, within one day 99% of the time), and likewise identify those tasks with a large chance of blowing up. Breaking down tasks into smaller tasks seems to help build this skill faster in some people.
If you don't want to be a mindless drone, you should also see the need to practice skills beyond just cranking out code.
It may also be a mistake to assume that it's always possible to break down a system into logical parts and simplify that system until it fits your complexity budget prior to development without knowing the upper bound of your problem's complexity.
Some things still have to be discovered before they can be measured.
Your job as an engineer, broadly, is to bring order to the disordered. You only rarely have enough time to fully understand the problem before trying to solve it.
The real, fundamental mistake being danced around here is the failure to recognize the inherent tension between product and engineering.
Stop fighting, start harmonizing. Understand how you fit into the bigger picture, and “deadlines” vs. “complexity” will start to make more sense.
It’s obvious that you aren’t going to make money off of nothing, so you have to build something, and it’s obvious you can’t build something out of thin air, so figure out how to build small, specific things that people will pay for.
While I agree that my job is broadly to bring order to the disordered, I would prefer if the disordered accepted that my proposed way of doing things will eventually achieve order without doubting me every morning, as it is after all my job to find the optimal algorithm to order things since I have myself optimized myself to find just that.
I don't want to fight, I'm simply unable to process your disorder in real-time.
Ah, you've already fallen into the trap; it's not about the "optimal", it's about the "functional".
Very, very few people are paid to find the "optimal" anything. The trust you're looking for can be found once you recognize what it is you're actually being asked for (again, rarely 'optimal', just 'functional').
We just got started and while I still can't find any obvious deficiencies with your thinking, I find you're investing far too much energy in telling me how I should work and not allocating nearly enough energy into describing what your problem is.
A standup is not "doubting you" and five minutes once a day is not "real time". It's rather the opposite; everyone is trusting you to raise issues when relevant, and making space to do that, rather than letting anyone interrupt anyone else anytime they think of something.
It can go wrong if you have shitty teammates, a detached PO, a selfish team lead, etc. So will everything else.
If it's really 5 minutes a day and if you're really satisfied with every monkey nodding once, then sure, this ritual can be accomplished with a moderately-sized team. But please provide data that shows that this has ever been achieved organization-wide anywhere with more than 10 employees. Seriously. I need to know.
Otherwise stop polling devs once a day when they already said the earliest you'll get anything is 2-3 days. They literally will quit over this in the long run and it's the simplest thing you can do to stop losing devs over communication issues between your team members.
They cost enough per head and you want to literally start their day with a reminder that they've unwittingly joined a cult to pay the rent?
Our 7 person standup takes about 30 seconds if nobody is blocked or has any questions. Most days it takes longer because most days at least 2-3 developers want to say something.
If your team lead (or god forbid somehow a PO is present) is lecturing or questioning individuals to report about specifics during standup, I'm sorry you have a shitty boss.
What's the point of having a standup for devs while at the same time assigning a manager to study and organize their JIRA entries? Wouldn't it be better to simply route all notifications through the manager, 7 to 1 (or 6 to 1 if you can find a dev who can reliably manage the communication network between your devs) and then just route the information into JIRA or upstream towards the bosses?
I've never had a case where it made sense to wait until the next day to bring something up to my organization and I've never been to a planned meeting where someone didn't get abused by management for only bringing up the issue at that time.
But it doesn’t need to be possible all the time, just often enough to set bounds acceptable to the rest of the organization. And how are you going to know unless you’ve already done it a hundred times, and/or try?
The problem with acceptable bounds is when they're applied uniformly to all your code monkeys without adjusting for experience and mental state at any point in time and we can't sample those values in real-time.
Performance metrics are often too noisy to be useful so proper bounds are too difficult to set without the "experience" of seeing the team work over a very long period of time.
Predicting the future of a high entropy system is predicted to always be hard if you're aiming for 99% accuracy. Solving for hard problems AND solving for this particularly hard problem on a daily basis is less energy efficient than if you stop trying to predict the future at every standup and trust that your average best guess while ignoring the future is good enough to keep you going until the end of this task.
TL;DR: Spend less energy sampling the efficiency of human creativity and spend more (but still not too much) on removing barriers that limit creativity.
What pop nonsense. An unpredictable software engineering team is itself a barrier limiting the creativity of the product team (and vice versa). If you don't want to be treated a crank to turn, you also need to bring some self-reflection, compromise, and willingness to communicate to the table.
Picking the 99% or top decile or whatever is not padding, it's the actual job of estimating. Anyone who talks about "padding" rather than uncertainty doesn't understand estimating yet.
A product team without an engineering process can reliably do nothing (but not vice versa). If you don't want to be treated like an optimization that only makes sense in a large money-printing machine, you need to stop optimizing for money-printing.
Communication isn't just about gracefully accepting expected input and feeling good about getting it. It's also about reliably figuring out the logic behind unexpected input to gracefully bridge the gap.
So hacking means creatively staring at unpredictable systems until they make sense. You can timebox how long you can afford to stare at it and then incrementally review whether your staring method should be improved or whether it makes more sense to do something else after the timebox but you shouldn't poll the state of the world every day.
TL;DR: Stop opening the oven door every 5 minutes, you're losing heat every time. Muffins don't like that.
It does not matter that you can estimate 99% of [simple] tasks. The total time is dominated by complex (big uncertainty) tasks that you can't predict/estimate reliably (e.g., think lognormal random distribution).
Programmers are good at automating predictable boring tasks. If you are competent, there always be unpredictable elements in your work.
Software estimation is similar to the coastline paradox if you don't know how small your measuring stick should be in advance. The smaller the stick the longer the coastline might be (fractal nature). An analog of taking a big software task, splitting into several smaller tasks and using it as an estimate would be like drawing a square on a map and rely on it as a good estimation for the coastline length that you can use for any scale (it is wrong if the coastline is a fractal (if the software task is complex)) https://youtube.com/watch?v=I_rw-AJqpCM
It assumes the perfect knowledge, the fixed work volume.
In practice, you don't know how much work might be required (a small task may blow up). It is inevitable that sometimes you have to drop features to ship in time. Programming is an iterative probabilistic process.