That sounds good in theory but most leaders are so removed from engineering that it would take them a week ramp up to produce even the most basic tiny change/feature to push to production. A VP should not be spending one week of his or her time doing that. They should rely on engineers to identify and fix whatever is broken at that level. That's why we have staff+ engineers.
But that's also pretty divorced from the topic of what makes good interview questions. There's no way that a VP who spent a week to push out a color change to a button in prod would have any meaningful insight into how to change the coding interviews. That should also be left up to the engineers themselves to decide.
They absolutely should be spending their time doing that. They are in the position to say "I have to do X, Y, and Z to push 2 lines of code??" and actually get it fixed. That week could save the company years of developer hours lost to overhead.
Relying on VPs to do every front-line job in order to identify and fix problems would indicate that something is fundamentally broken at the company. That should never be the primary method by which a company identifies and fixes such problems.
As a former engineering manager, if someone on my team walked me through why getting PRs out to prod was an insane nightmare, I'd take note, work with them to gather evidence, and present it to my director and try to escalate it to the point where we could take action to improve it. If the VP is any good at their job, they'll listen and work with us to fix it.
If they aren't good at their job or it is status quo, they will brush it off. Sometimes it takes a new perspective to say what the fuck is wrong with this?
As an example, I joined a company with ~8k employees recently. They over communicate on email. I get 50+ emails a day. I filter heavily. My inbox is still unusable due to the volume of automated junk. I raised this issue in Slack and the majority of responses were just "well, that's how it is".
I am sure the development process has very similar deficiencies that I am blind to because I participate in it everyday.
> If they aren't good at their job or it is status quo, they will brush it off.
In your example, you rely on them to be even better: you assume that they'll be a competent engineer and be able to understand the complexities of day-to-day software development by making a toy PR when many of them haven't done it for years for decades. That's a much stronger assumption than the one that I'm making: that a good VP will listen to their subordinates.
That's about the time I'd expect for a new hire to ramp up on the codebase and submit a PR behind a feature flag and probably experiment that makes it to production. It takes a day to even set up the environment and be able to start looking at and running the code locally. Four days to read through a brand new codebase, identify the changes that are necessary, write a small tech spec (optional based on how small the change is), submit the PRs, get them reviewed and approved, and then merged sounds reasonable to someone who's never worked in the codebase before.
> That should also be left up to the engineers themselves to decide.
I agree with the rest, but I don't agree with this part. Engineers should have a lot of input into the hiring process, but fundamentally management is accountable for business performance and one of the biggest drivers of success is getting the right people in the door rather than just more people like the ones you already have (which is what happens almost always if you don't deliberately shape the hiring process).
But that's also pretty divorced from the topic of what makes good interview questions. There's no way that a VP who spent a week to push out a color change to a button in prod would have any meaningful insight into how to change the coding interviews. That should also be left up to the engineers themselves to decide.