I want to agree with this post, but I don't. This person is in a position of power over their product.
Instead, I have found that decisions work by "path dependency". If something works, even if it is a poor implementation, the path to get there is holy ground and the amount of effort to change a 10 minute decision that was made flippintly by two business people chatting in the urinals on a Wednesday is like trying to siege the walls of Jehrico.
There's no solving this problem because it is fundamentally human in nature. The egos are invested in the existing thing -- so improving the proof-of-concept-that-is-now-hastily-shoved-into-production problem requires that you unravel every person's thoughts and pride on the current thing, regardless of the seriousness of the problem. And so the problem cannot get solved and change cannot happen unless something catastrophic occurs.
Thus, the line-worker software engineer, who has no real organizational power, has to hope that the proverbial printer falls down a flight of stairs "accidentally" so that they can swap out the black ink cartridge to a new one that doesn't make weird bleed lines on the edge of every page. That way, when the new printer comes in, and someone complains that the ink bleed is gone and how they liked that, that you can feign ignorance.
> it is fundamentally human in nature. The egos are invested in the existing thing
That is one thing that can happen, because it is part of human nature. But it is not the whole of human nature. if you have a culture of experimentation and two-way doors, people learn to invest not in one particular experiment, but the process of experimentation. They invest their egos not in one particular outcome, but in the team and their long-term success.
I get that the dominant culture in American business is managerialism [1], where we all have to pretend that people currently holding power are brilliant, while they have pissing contests over their place in the corporate dominance hierarchy. But that's not the only way things can possibly work. I've lived different approaches, as has the author of the piece.
Sociocracy (from the Quakers) provides an interesting framework: consent-based decision-making instead of consensus or authority-based hierarchies.
To call hierarchical authority "human nature" is to ignore not only theoretical viability, but also demonstrated present-day viability as well as modern anthropological and archaeological understandings of realities explored in our pasts.
Sadly most of us have bosses or subject others to it, and we bind ourselves to these structures by legal contract and risk being identified as insubordinate if we try to do better.
Sociocratisch Centrum co-founder Reijmer has summarized the difference as follows: "By consensus, I must convince you that I'm right; by consent, you ask whether you can live with the decision".
This is so true. It still shocks me and I don’t know why. The most frustrating version for me is when the original author of a solution didn’t care all that much and would admit it was a hack but they’re no longer around - and their successor thinks that what they’ve inherited is some sort of sacred object built on infinite and unknowable wisdom. So this obvious design flaw must actually be a carefully designed feature that were simply not capable of understanding. Maddening.
It's especially difficult, too, because the stagnancy is often (hopefully) based, for better or worse, in humility, which is broadly a good trait to hav. Especially around Chesterton's Fence and product decisions, moving carefully and respectfully for prior art is a generally safe and mature way to work.
But dangit, sometimes you need to trust that you have a better solution and just execute! But it's not that easy.
First step to solving that problem is letting people dive deep into codebases, so that they can understand the situation. Which requires having that time available to them, rather than always being on busy work. I think people sometimes lump this in with maintenance, but it is something different and worth putting into your estimates. Add time to grok the existing solution first, not just time to build on top of it.
Devil's advocate: if nothing bad happens resulting from the path chosen with that 10-minute-decision solution, is it fair to call it poor from the business perspective? Or even if something "mildly bad" happens, if the flip side was spending 3 weeks doing research until finally arriving at a 24% better approach, which of those costs is higher?
If the team/org is too high-ego to actually treat 2-way decisions as 2-way decisions after they get made, I think that's a bit of a different problem. The org has to invest in the "so we can change it" part of the plan.
I mean, if it was a good decision, it was a good decision. But there are a lot of ways for a decision to be bad without legibly causing a legible problem—and the "business" might be happy with that—but that doesn't make it a good idea!
A common pattern I've seen is a team or organization getting in the habit of collecting little papercuts where no individual problem is bad enough to register but, over time, the codebase becomes slow and painful to work on. What's especially dangerous is that the state of the system informs people's expectations: what's easy, what's hard, what's possible at all. You get to the point where changes that should be easy are seen as inherently difficult and people start making strategic decisions that implicitly compensate for this, masking the accumulated problems even further.
It's a 20 year old codebase, so it's gotten so bad that we spend 2 weeks planning something that could take me 1 week to write in the current codebase or 1 day to write in a sane codebase, on a product that I could probably re-write from scratch in a month.
In my experience, a quick decision is often a poor decision. Most of the time it’s suboptimal but occasional it’s outright damaging. Most places I’ve worked at, claim that decisions are reversible, but they aren’t. Unless a process or a piece of software is actively and visibly misbehaving, nobody refactors it.
There’s gotta be a balance between waterfall and fake agile, but I haven’t found it yet.
I am a big advocate of refactoring bad code, but what you say is SO important. Badly written code that is non-critical or rarely used or simply where all the known issues are known and easily worked around, is not a good candidate for refactoring. What a lot of refactoring advocates fail to mention (or sometimes realize) is that the act of refactoring has hidden costs. There is an inherent risk taken on that when refactoring something you are going to break something. If you are lucky, it breaks “hard” and it is obvious. If you are unlucky, it soft-breaks and only shows up occasionally or years later on edge cases. This all must be taken into consideration when deciding whether to refactor a bit of code. Deciding to refactor should not be a casual decision.
I’m working with a distributed monolith right now. The system is misbehaving as a whole, but it’s difficult to point to one individual piece and say — this is the culprit.
A number of reversible decisions over the course of the years resulted in a terrible architecture.
Something is broken but nobody knows exactly what it is and there’s no organizational will to get it fixed.
While I agree with the sentiment, I can see 2 counter-points:
- "Nothing bad happens" might be the accumulation of risk until something _very_ bad happens.
- Something bad might actually be already happening, only silently. For example, teams suffering unnecessary difficulty whenever they need to work on some part of the codebase, but it's considered "normal" because that's life now.
The version that I’ve always experienced in practice is “we spent six months fixing bugs and it’s still broken, because we did not read the first page of the docs. No we do not want to read the docs now. Look I found a stack overflow post where some unknown, unqualified fool who is also incapable of reading the docs says to do it this way”.
Weeks of bug fixing can save you literally hours of planning.
> if something "mildly bad" happens, if the flip side was spending 3 weeks doing research until finally arriving at a 24% better approach, which of those costs is higher?
The flip side is often to ask the people doing the work for input, there's a good chance they know what's best. There's often a group that can make decisions and implement the decisions, and another group that can only make decisions, and so naturally we divide the work so that those who cannot implement the decisions make the decisions and those who can do both don't ever get to make decisions.
I agree completely. Path dependency is such a huge component of how the world is built and many people are oblivious to its effects.
Even the freeest "two-way door" decision becomes one-way over time because every day is a step farther away from that door, and another step you have to backtrack in order to go back through it.
The concept of line-worker software engineers seems like the problem in this parable. People need the ability to have some small amount of override and the ability to make decisions without risking their jobs, even if it's somewhat boxed to their experience level. Leaving everything up to management only is a way to ensure that little gets done and no one's happy doing it.
It took me awhile to go from thinking tech companies rely on their machines and algos, and the best way to change / better them is to present a good technical solutions, to the idea that organizations are made of people, and tech can only be a good argument after one establishes that your advice is welcome on an emotional level.
None of us like to admit we were wrong, the more power / responsibility we hold the harder it gets. A lot of people feel insecure deep down because their position in the org was gained roughly by accident and they can’t reliably reproduce their success in another place, so their ego / reputation becomes an incredibly touch subject.
We as devs have an incredible luxury of being able to pack up our stuff and deliver a working solution in another team / company / country / continent reliably. A lot of business people either don’t have that, or believe they don’t, same thing really.
So the way to approach it becomes understanding how vulnerable people in power feel, helping them out and building alliances that way, and when they start trusting you, then you can bring to bare even outlandish proposals and be supported.
Sadly we are all still primates, building the most elaborate tree ever, and one has to take that into account.
The espoused amazon groupthink seems to be from "earlier" amazon which, to your point, was practically greenfield for everything.
IMO only now is Amazon starting to run into the long term consequences of massive adhoc inhouse tech tools, probably only worsened by defensive coding techniques to combat stack rank evals.
I'd like to say that the most modern example of two way door decisions is hiring someone, because you can always just fire them immediately as the stack rank sacrificial lamb. See? Easily undone.
> There's no solving this problem because it is fundamentally human in nature. The egos are invested in the existing thing -- so improving the proof-of-concept-that-is-now-hastily-shoved-into-production problem requires that you unravel every person's thoughts and pride on the current thing, regardless of the seriousness of the problem. And so the problem cannot get solved and change cannot happen unless something catastrophic occurs.
Yes. I see this with internal home grown tools. They were home grown by early and influential employees and you ain’t getting rid of them. It is worse when they kind of work and capture a lot of organisational practice such that a rewrite or off shelf solution can now be “good enough is enemy of perfection” type thing. But the problem with home grown tools is someone has to support them. They suck up a lot of time.
There's no solving this problem because it is fundamentally human in nature.
the solution is unity. you are right that with a someone in a position in power making decisions it can be very difficult to change that, but when the whole team makes a decision then the whole team can also change that decision.
what is important here is to develop that mindset that everyones input to a decision is valuable. the challenge is that this requires trust that needs to be developed
"Action produces information". If you "do something, so we can change it" be sure to do it strongly in one direction or the other, so that you set the direction of exploration, too.
> be sure to do it strongly in one direction or the other, so that you set the direction of exploration, too.
This reminds me of a game design technique Blizzard learned to use back in the day when balancing their RTS games. Sometimes they'd make a small change – say increasing damage by 4% – and playtest the result. It seemed, maybe better? So they'd ship it. Some time later, they'd realize they had way undershot, and the ideal increase would have been 25%. They sometimes found themselves buffing or nerfing the same thing over and over, trying to get it right.
The approach that worked better for them was to err on the side of first overcorrecting – say, try increasing damage by 40%. This way, in playtesting they could clearly see the effects of having gone too far, then back off the change as appropriate.
One of my big rules has always been, "double it, or cut it in half". Don't waste your time adjusting something by 5 percent, then another 5 percent, then another.. just double it, and see if it even had the effect you thought it was going to have at all.
This is what musicians do - put in nonsensical filler lyrics until they work out the rest of the song. Sometimes the throwaway lyrics are even kept. The Beatles did this to great effect in the Get Back documentary. It was so cool to see how the now legendary songs took shape. You got to start somewhere. Everything is iterative.
Video games are like that too, just put in default models, textures, placeholder music, and iterate until it's done. Mind you, some placeholders have copyright protection, IIRC a game shipped recently with copyrighted music that was used as a placeholder.
I'll admit I only skimmed this article, but I think that I have noticed this sentiment as a boon during software development: Sometimes I will find myself in a position where I need some level of sign-off or approval from the team about something I'm trying to do. If I ask for opinions without showing anything concrete it's often crickets. However, if I make a pull request (or even a couple pull requests show casing different options) then it's much easier to find out people's thoughts. It may be more work for me, but it's better than the paralysis feeling of not being able to move forward, and it also helps me really understand some of the options at hand.
I'm sorry if my comment isn't regarded as constructive... but you skimmed an 'article' consisting of three paragraphs, that takes maybe two minutes to read, but bothered to leave a comment on it?
When I read "I'll admit I only skimmed this article" I though maybe due to blocked JavaScripts the article didn't load for me and I only read the preface of the actual article or something...
As a solid manager once described it, "It's a lot easier to argue over an implementation prototype than to argue over a design doc."
This was not an argument in favor of not writing design docs, but instead an argument in favor of moving towards prototype while waiting for people to weigh in on a design instead of doing nothing with that downtime.
A lot of times its a more productive debate over the tangible 'something' thats out there too (even if its wrong), rather than the theoretic or a design!
Or you're like me, and the beginning of a project is crippling (mentally), and you analyze every "what if" leading to a mental use of 20% of your actual production time. Then deadlines kick in, and I work late and put extra time outside of office hours in. The software actually ends up being better because I've examined as many situations as I can, but it can't be good for my long-term health. I feel like I'm not the only one to suffer this curse. Yay anxiety, and booooo anxiety!
A terribly common error is having a debate over how something should be designed, and then never resolving the debate. Brian Valentine, the lead developer on Windows 2000, was famous for his motto “Decisions in 10 minutes or less, or the next one is free.”
Windows 95 and 7 may be exceptions. I recall both being refreshing and met with enthusiastism and praise. Early versions of Microsoft Office as well were (at least at one time) considered a step up.
Yeah, I'm a big believer in finding 2-way decisions and starting fast, but even for 1-way decisions there's a pretty pervasive problem at a bunch of places of meetings that are arguments instead of decisions. Ego stroking for engineers instead of getting something done.
The majority of arguments I’ve seen at work turn out to be people taking past each other because they are using different definitions for the same words or they are starting with different assumptions. Eventually someone says, “When you say X, do you mean Y?” And “Are you assuming that Z?”. Suddenly the argument goes away or is greatly simplifies. But it takes so much to get to that point. I don’t know why humans are so poor at communicating.
Communication is NP-hard and the difficulty scales exponentially with each added layer of complexity.
On top of all that, people in specialized roles tend to invent jargon, then use it as an excuse to treat others as inferior or outsiders instead of putting their ideas to the test by explaining them to the rest of the world. People who do that tend to be interested in maintaining an upper-hand status-wise rather than actually getting shit done.
Communication is only NP-hard if you have zero prior context.
In most cases, you already have 99.9% of the common language, including an acceptable common metalanguage embedded into it. Coming up with mutually understandable definitions is a bit of work, but it's far from unsurmountable.
Note how many legal papers and even scientific papers start with definitios. Other technical communication should, too.
One of the best things I ever did was to come up with a straight-forward definition of UI components in our CMS, so our designers and front-end developers could speak to each other and be on the same level. When new features were developed, and new components introduced, we would all agree upon what made up that component and document it.
Love that essay, but the number of specs I've written for programs or hardware systems that not a single soul has bothered to read is so high it makes me steam at the ears just to consider tabulating the number.
That's the person who launched Vista and fled to Amazon?
He's also the person who once approved some request for money and complained that it was a waste of his time to ask for so little.
I agreed, but, corporate policy.
I spent a minute at Amazon so I'm familiar with the one/two way door paradigm. It was one of the big takeaways I got from my time there, especially the aspect of trying to make more one way doors into two way doors. Often you don’t even consider this path because you’re so hung up on thinking through all the consequences of the one way door that you don’t just… stand back and check the frame, to extend a metaphor.
> “we support what you’re doing, let’s talk, and keep talking”
I was talking with a first-time co-founder recently, and had to break the news to them that many people in startup space will always say they're interested, and never say no.
I've guessed it's to keep their options open, and to avoid/delay negative associations with the speaker that would come from saying no. (I suppose some might also do it for selfless reasons, of not wanting to discourage someone.)
> and when I put them in the slide deck or team page that’s how I learn who is really with me
I like that general idea, for cutting through some of the noise. I'll have to think about how to adapt it to my style.
I like this. I call it "making something solid enough to argue about." Once you get out of the realm of design docs and create something people can poke at the next steps become clear. Just do something to move the ball up the field.
> The director blinked. “Well, do something, so we can change it!”
Related, but maybe distinct...
I don't know how common it is among choreographers, but a methodological tool of at least one modern dance choreographer, working on a new piece, is to have a session with the dancers, and have them individually improvise a particular part, under some direction. Then she'll do things like, "Oo, everyone try what Jane is doing."
The problem is that you can do that when is safely and reversible, the half of the advise that is missing is: fail fast[1] and safely[2] so you can improve[3] over the feedback[4] you get from a source that has the best interest in mind for your success[5].
1 is the easiest.
2 should keep you out of "failing fast" with irreversible decisions or ones that have consequences of causing permanent damage or giving up your freedom or power.
3 are you coacheable enough to embrace the required level of mindset change? will you develop mastery for the next iteration at the required speed?
4 collecting quality feedback is not always trivial or cheap.
5 this is the hardest to get, are you surrounded/listening to true friends (celebrators of your success) or fake ones (envy in disguise poisonig misleading advice so they don't feel outshined by your success)?
> Should we pay billions to buy a company, then systematically obliterate its accumulated reputation and userbase in an attempt to inflate our overgrown yet still incredibly fragile egos?
Author explicitly said he was talking about Amazon, but I don't think he was talking about Amazon.
That part was referencing Twitter, not Amazon. See the following (44bn number is a giveaway):
> Maybe you put some conditions on your $44 billion acquisition offer, and – if the deal goes through – maybe you refrain from repeatedly permanently sabotaging it, like some kind of ridiculous gibbon. You know, that kind of thing.
I do agree, most of the time I find people arguing about theoretical issues - once we get something built it is so much easier to improve it because it is so much easier to point and describe what needs to be changed or removed. Discussion on something that is actually done is much easier than the discussion about the theoretical issues.
For an opposing view, go read the showrunner rant.[1][2]
The rant against "I'll know it when I see it."
Well... it is true that not everyone believes that knowing what
they want, and reaching out to those who need to know it in order
to perform, is a necessity for success in the world of
television... and this is the part where they come out from their
slimy, shit-stained hole and excuse their lack of vision (or
their unwillingness to impart that vision) with a defense I
consider to be the most cowardly and thieving seven words in the
showrunner's lexicon:
"I'll know it when I see it."
If you ever find yourself saying that, kindly consider the
possibility that - and I mean this, from the heart - your
impostor syndrome is most likely real and you are, in fact, a
shrill, shrieking fraud.
Here's what "I'll know it when I see it" means to me and to
everyone who hears it from a showrunner: "I have no original
ideas of my own but am perfectly willing to let everyone else
spin their wheels and exhaust themselves emotionally and
creatively so that I can eventually cherry-pick the best of their
genius and claim it for my own."
Instead, I have found that decisions work by "path dependency". If something works, even if it is a poor implementation, the path to get there is holy ground and the amount of effort to change a 10 minute decision that was made flippintly by two business people chatting in the urinals on a Wednesday is like trying to siege the walls of Jehrico.
There's no solving this problem because it is fundamentally human in nature. The egos are invested in the existing thing -- so improving the proof-of-concept-that-is-now-hastily-shoved-into-production problem requires that you unravel every person's thoughts and pride on the current thing, regardless of the seriousness of the problem. And so the problem cannot get solved and change cannot happen unless something catastrophic occurs.
Thus, the line-worker software engineer, who has no real organizational power, has to hope that the proverbial printer falls down a flight of stairs "accidentally" so that they can swap out the black ink cartridge to a new one that doesn't make weird bleed lines on the edge of every page. That way, when the new printer comes in, and someone complains that the ink bleed is gone and how they liked that, that you can feign ignorance.