In my experience that comes mostly down to politics, in the worst cases including outright abuse of your team's performance. Not always but often enough. Selling yourself well enough and advance by changing gigs is also working nicely, just jump before any issues catch up with you in your old gig. And then there is the luck part, to always be at the rigzjt place at the right time.

The rare few higher managers that achieved their positions also through domain knowledge and competence were a delight to work for and with. The others not so much.

I wonder how a well meaning manager can avoid falling into this sort of situation though. I know that that many managers are well meaning and do want their team to succeed but have the unsavory job of getting their teams to do shit on time, by force if necessary since the business depends on it.

It’s possible that there is no good answer I guess. Managers might just be the bogeyman bringing bad news but it’s the business (and execs) that ultimately make these choices.

Well meaning or otherwise, it's impossible to avoid falling into this sort of situation at some point or another. What makes or breaks a manager is how well they navigate it.

Pushing your team to do shit on time by force may work, but it's a risky and generally unsustainable strategy. And the immediate repercussions of doing so are largely opaque higher than you as the immediate manager (and potentially your boss). Strategic failure or missed deadlines can generally get the visibility you need to ensure you get the consideration or resources going forward to prevent it from becoming a norm. But that also has political costs and considerations, so you have to have a firm grasp of your position in the org, your positioning and messaging on the issue, and how well you capitalize on the follow up before the window for change/resources closes.

Business executives may not understand what you or your team do, but their decision making process is largely similar to a systems engineering problem. When joining the company as a manager, you have to rapidly understand the general structure of the legacy system that is your new company, where your team fits into that architecture, the integration points and dependencies your team and reporting structure has with the wider company (which is as much personal and political as it is technical), and approach the maintenance, oversight, and positioning of your team as appropriate.

It's a very active process, and without the benefit of source code, documentation, or logging you may have when getting up to speed on a technical system. A passive manager just acts as a message bus passing through demands from the business until the service that is their team becomes overloaded and things only get better if the upstream demand decreases. An active manager attempts to influence the overall system in such a way where those spikes are rare enough to be manageable, trigger the appropriate alerts (from an organization perspective), such that the upstream demands and downstream capacity to service those demands are in alignment.