There's huge amounts of anti-process arguments in the threads here that is frankly immature.
Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity. These side quests also tend to not contribute to business aligned goals.
It'd be great if we could all just do what we want, but in my experience, process is crucial when aligning great people's efforts and energy with the business paying them.
I'm not saying scrum is the way, but calls for "just let engineers decide and get on with what they like" sounds like 20-somethings with little experience of when and how often that goes horribly wrong.
Suitable, proper process keeps chaos at bay and ensures everyone's expectations are aligned. That keeps business running and jobs and growth happening.
Funny thing is, 15 years ago, I'd have been the first to agree that processes are "infantile handholding" (to quote one poster here).
15 years later and much wiser, I'd never hire anyone with that attitude.
I'm also 15 years in and have worked at all different kinds of companies with a few different kinds of processes, and I don't really relate to this.
To your point: I don't think any senior engineer argues that it is tenable to have absolutely no process. There needs to be a way for business stakeholders to communicate with engineers and vice versa and for everyone to plan and contribute to prioritization together.
But many senior engineers debate the time scale on which and the audience with which that coordination and planning and prioritization should be happening. The threads here where I've seen the most pushback are where people have been advocating for processes that are designed to give very broad audiences very nearly real time insight into what engineers are doing with their time. And I think that's pretty bad and I think lots of experienced people agree with that, not just whiny youngsters.
What I support is small teams where intra-team visibility is not really an issue, and cross-team coordination on longer time scales (bi-weekly, monthly, quarterly, half-ly, I've seen all of these work fine, with different trade-offs).
I don't disagree with anything you've said. My core point was that process, of whatever form, is right and necessary.
I've seen things go wrong due to no process and I've seen things go wrong due to inappropriate process, and any process applied poorly.
The process isn't the evil, it is poor leadership, regardless of the process that kills projects.
Different projects in different industries, with different software IC contributor types calls for different leadership ... and I've used Scrum and others successfully as tools in that leadership.
Blaming tools (like scrum) or any tool blaming I feel is immature and scapegoating.
Granted my original post was a touch inflammatory, partly out of frustration.
I made my living repairing under performing teams and good processes top to bottom are the number one thing I've found that stabilises things and turns it around.
The opposite of scrum isn't "just let engineers decide and get on with what they like".
In good tech companies, product managers + design + eng collaborate on how to meet a business need with a delightful product. The leaders of those orgs keep everyone aligned around a big goal and high-level milestones, but distribute as much of the lower-level decision-making and planning as possible.
That's mostly orthogonal to the processes for how a project is actually run (scrum vs informal vs plan-build-ship) and how the process style is decided (centrally controlled vs suggested vs team-by-team).
> Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity. These side quests also tend to not contribute to business aligned goals.
The way FAANG fixes that is to make "impact" the first metric of their engineer evaluation process. Basically, you're quite autonomous to pursue your ideas, but need to be able to justify that you contributed the project in meaningful ways.
>The way FAANG fixes that is to make "impact" the first metric of their engineer evaluation process. Basically, you're quite autonomous to pursue your ideas, but need to be able to justify that you contributed the project in meaningful ways.
Isn't the joke that Google as a result makes and kills tons of half baked products without producing any actual business value overall? The product lives just long enough to get people's promotions approved.
There is no viable heuristic because everyone has an incentive to game the system. Even managers. "When a measure becomes a target, it ceases to be a good measure"
The closest one is short term improvements tied to revenue and measured using ab testing. But even those get gamed.
That's interesting when there's wiggle room for that.
I've led projects where there isn't (in defence). Specifications are prescribed exactly and in detail, deadlines are fixed and the ICs job is to implement and that's all.
It was a sad state of affairs, but one caused by software being a small cog in a massive machine that isn't software driven.
Outside FAANG and software centric businesses, the landscape of responsibility and the availability of autonomy can look very different IMHO
Less like craftsmen making fine furniture and more like an assembly line for IKEA furniture. Processes like Scrum form that assembly line which ... like it or not as an IC ... is appropriate in some businesses and industries.
> Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity. These side quests also tend to not contribute to business aligned goals.
At my last job, there was a strong bottom-up engineering culture where ICs were empowered to work on what they thought would deliver the most business impact. This was actually quite effective because there were performance incentives to ensure individuals and the organization had aligned goals.
> Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity. These side quests also tend to not contribute to business aligned goals.
Is this actually true? I know it’s a common fear of managers who are in over their heads in one way or another, but isn’t it a bit of an infantilising view? I would never want to work at a company where this was how the leadership saw engineering (and I’m a product manager, not an engineer).
I think it is a fair view that a manager is in over their heads if they can't stop this happening. Sadly ultimate autonomy of many ICs without proper leadership results in this and good processes prevent that.
It is the sign of a team being lost, due to bad leadership, and good management prevents a team being lost.
A good way to prevent a team being lost is, in my experience, processes that keep everyone integrated and moving in the right direction ... scrum being one approach (not always the right one).
At the end of the day leadership is needed and a framework that lets everyone know where they stand without it all turning into design-by-committee nightmare.
I tend to think you’d have more success having no manager at all vs the bad manager who thinks his team is going to go off and start making a video game or something more fun instead of working on the business if they don’t crack the whip. It’s a kind of paranoid fantasy coming from insecurity, it’s not actual reality.
I think it's a fantasy to expect that all teams are capable of self direction like that.
I've had teams that were, who embodied the idea of building teams smarter than ones self.
I've had team where given more freedom, they just went in chaotic directions. They just intrinsically were a composition of people who couldn't care less and wanted to work on whatever they found interesting. Usually because they didn't have any stake in the business, interest in what it wanted or respect for its goals. And yeah, I didn't blame them, but I still had to move them in a unified direction writing boring software for a boring industry with no interesting challenges.
There’s so many normal people out there who are basically motivated to do a good job, even in boring industries.. why would anyone go out of their way to build out a team of uninterested people who don’t want to be there? Seems like we’ve just found another path back to bad management.
I agree, thinking back to one project I led though, finding engineers who would have actual interest in the domain we operated in was tough. It was dry, tedious code in a legacy language (for sound business reasons!)
It's sometimes tough. I managed to find compromise by working with ICs to make it interesting by engineering our way out of the boring through seriously cool automation.
I can see how, in hindsight, that was a lucky move and I don't blame other managers in similar situations for hustling teams along with more process. It's the right call.
I would disagree - I think you are reading something into “let engineers decide” that goes with the many micromanaging structures like scrum, which is to start with a close to complete lack of trust in ICs.
In a well managed organisation ICs don’t simply go off and do what they want. In a new organisation with a lack of any experienced engineers a failure in management could easily lead to that as management has failed to do its job and the ICs are not sufficiently experienced or informed to be able to compensate for such a failure.
More experienced engineers can generally identify poor or incompetent management and can compensate (to an extent). But by and large “all the engineers went off and did random things” is absolutely a failure in management, not the ICs.
If there are just a few ICs off in the weeds then that implies the ICs are the “problem”, and that may simply mean having some more hands on management - helping them understand prioritization, etc not micromanaging them - or possibly having them get some help from a more experienced IC, so that they can learn how another IC does their job. At no point have I said anything about peer programming, which should not ever just be a “this is how all dev shall happen” nonsense.
Things like peer programming is a fairly occasional tool in my experience, something that has happened when I’ve been working on a particularly irksome problem with another person on a whiteboard, and it seems like the code might be gnarly, but that latter is very very rare; vs the simply working out the solution.
I agree with what you're saying. What this might discount though is where an organisation can't afford high enough calibre of engineers to make these things possible. That's not always the case.
I've been (and am) a software engineer but also operate in higher level management of dev teams.
In orgs that have an technical product or are software businesses, what you say is true. That culture doesn't exist in all sectors that require software engineers.
I believe that for most organisations strong processes from PRs and code review all the way through time-boxed goals to high level stakeholder meetings its important they're structural and focus on accountability and goal achieving expectation.
I also strongly believe the right ICs should be one of the driving stakeholders of business objectives when hired well.
Yes it was reductive, but not frightened. I think building the right team is the antidote to over management.
Sadly I've seen too often teams are not built with this in mind, and stringent process can work to keep efforts in line with requirements. It almost allows broader hiring.
I'm not saying there's a right or wrong, just that arguments dismissing lots of process here don't work for me for their reasons.
I'd argue yes, if management allows the situation to persist.
Bottom line is if the eng team don't meet goals the manager is responsible.
For me, good process, not micromanagement allows the eng team to perform to expectation and know the boundaries they operate in. Scrum is a tool in that toolbox.
My interpretation is that if this is an issue for any meaningful proportion of ICs, it implies management is the source of the problem - if an IC does not have a good understanding of targets and priorities, then management has failed to adequately communicate.
If it’s only a small proportion of ICs, then you can look at the ICs being responsible. If it is a new/inexperienced IC I might still lay blame on management for failing to provide adequate support while an IC learns how to do their job. School and/or OSS contributions are good* at teach the mechanics of a job (how do you write code, how do you design a circuit, how do you do design, etc), but atrocious at teaching how to work in an actual work environment where you have to work with other people on your team, other teams, other people on other teams, etc, and how you do things like prioritizing goals.
Individual projects at school obviously do nothing for team interaction, group projects are IME atrocious at reflecting a real environment on far too many levels to express without writing a book (and so learning how to write readable long form prose :) )
100%. A lot of problems at organisations come managers having far too much job security. This gets really acute the larger an organisation gets, where the goals of managers can become completely divorced from that of the company. For example a managers goals might be to increase their budget and team size in order to justify higher pay and promotions due to experience “delivering” “larger” projects. Whereas more team members (especially those easy to hire project managers or engineering padding like architects) end up decreasing productivity. A really good example in the article was Skype, which went Scrum with dozens of staff (if not hundreds) and was beaten by a team of three engineers with no “formal” process at WhatsApp.
Scrum needs to seen for what it is: a way for management consultants to bill lots of hours and a way for organisations to manage projects where engineers have no agency. In a consultancy, for example, where scrum is almost always used, the real money comes from all of the management consulting fees and design iterations and etc etc, and a build is only done when all of that has been signed off and paid for by the client. Then the engineering team is told to go build it. This is VERY similar to the waterfall methodology that Scrum was supposed to replace. And seen in that context, it’s no wonder that successful tech companies have zero time for it.
>Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity. These side quests also tend to not contribute to business aligned goals.
I mean e.g. Facebook does real tech stuff, like TAO.
Sounds the goal of Scrum is herding to write your CRUD app.
Yeah that's fair but there are a hell of a lot of CRUD apps that need writing by middling software engineers who'd rather do some fun refactoring than write yet another boring API endpoint.
I've managed enough of those teams and yes, boredom is a real problem for both motivation and staff retention.
If I let people on those teams do what they wished, the app would've been rewritten 10 times by now in different cool new frameworks and the business wouldn't benefit because the core app was so dull.
But there's a lot of boring software that needs writing that makes money.
It is true that those teams suffered churn of people looking for less boring work ... at the end of the day the right processes (like scrum) and other things that appear like heavy management are just risk mitigation for the fact engineers don't want to work on the core problem
I've spent many years working out how to deal with these situations without degrading engineers with heavy handed management.
This might come across a but harsh but I don't know how else to put it without resorting to my native language but I feel kind of sad for you. You sound like someone's who's lost hope in (at least part of) humanity.
Your anecdote suggests this is a truth about all engineers and while I cannot disprove that I might as well counter with my personal anecdote.
In my experience, engineers tend to be very sensitive to the environment they work in. When unsatisfied with the workplace environment, they might go off on side quests but trying to "fix" that with tightly controlled process is fighting symptoms, not causes and will generally result in an even more unsatisfied engineer who appears more valuable because they've learned how to hide the unproductiveness.
I've seen this happen and it definitely didn't add any value for the business.
The real solution is to try and create an environment where people have trust in each other and are committed to a common goal. I've been lucky enough to experience these kind of jobs where it's just professionals committed to a product they believe in. We're all adults so if someone disappears on a side quest that's hurting the team we talk about it. Maybe someone just got distracted and needs someone to point it out, a simple mistake which is ok every once in a while. Maybe someone is having a bad time and needs to take a vacation, understandable and also ok since we're humans first, engineers second.
But most importantly: if someone is regularly hurting the team's goals and isn't willing to improve the situation or acknowledge it then it's simply not a good engineer/colleague/asset to the company. No amount of process is going to change that and even if it does you're probably losing productivity from the rest of the team because they were productive in the old approach.
A flat tire doesn't mean cars are a bad mode of transportation and switching to a horse is not going to get you to your destination any faster. Why not replace the defective tire instead?
If the situation I described above sounds unrealistic to you maybe the companies you've worked for need to review their hiring process. If you don't trust the engineers you've hired with the process you'd be doing yourself a disfavor by trusting them to do the job they were hired to do.
"Your anecdote suggests this is a truth about all engineers"
I don't disagree with what you're saying. The scope of what I was saying is more limited to certain projects than it sounds.
I have hope. If I didn't, I'd quit. Unfortunately I make a silly amount of money dealing with these problems in a certain way, and it is a way that (for some projects) involves prescribed process; and it works.
> Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity
Sounds like you have infants. Perhaps hand-holding is appropriate here.
Most engineers when "empowered" and left to their own devices without much accountability go off on their own side quests that often add no value to the business, but scratch that engineer's curiosity. These side quests also tend to not contribute to business aligned goals.
It'd be great if we could all just do what we want, but in my experience, process is crucial when aligning great people's efforts and energy with the business paying them.
I'm not saying scrum is the way, but calls for "just let engineers decide and get on with what they like" sounds like 20-somethings with little experience of when and how often that goes horribly wrong.
Suitable, proper process keeps chaos at bay and ensures everyone's expectations are aligned. That keeps business running and jobs and growth happening.
Funny thing is, 15 years ago, I'd have been the first to agree that processes are "infantile handholding" (to quote one poster here).
15 years later and much wiser, I'd never hire anyone with that attitude.