The big missing piece is roles. No service uses a root access key directly. Instead, there's a webserver role with access to a relevant secrets group but no access to data warehouse secrets, for example.
Access keys can be provisioned and downloaded straight onto the box from the service. Sure, a compromise is bad, but only exposes the secrets that would be available on the pwned box regardless.
OK, so "root" wasn't really the right term. I get an X credential so I can be an X, and nobody needs to worry that I also got enough to be any part of a Y. Thanks.
This sort of model also fits nicely with the AWS ecosystem. EC2 instances (virtual machines) can be given an IAM Role when they boot-up. An IAM Role is essentially an automatically generated access key which is unique to that EC2 instance, and has pre-determined permissions.
So in other words – a unique key is generated every time a virtual machine is created. It's fully automated, never shared between instances, and never needs to be handled manually. That key will give the virtual machine permission to access other AWS services, in this case the AWS Secrets Manager.
So as long as you're using EC2 instances, you won't need to worry about securely passing a 'master password' to your VMs in order for them to access secrets.
Access keys can be provisioned and downloaded straight onto the box from the service. Sure, a compromise is bad, but only exposes the secrets that would be available on the pwned box regardless.