What about a physical killswitch goes against Apple's design? The iPhone has always had a ringer/vibrate switch on its side since it was introduced. Obviously it doesn't do anything hardware-wise, but it's still a switch that disables some functionality.
That has a common use case of not wanting your phone ringing in a meeting.
People expect their computers not to listen to them when not actively using the mic. Your switch is the program that records you, a second is redundant and confusing.
Almost all of Apple's security features are invisible and by default and don't get in the way. They're striking the balance between user experience and security.