Agreed. This is one reason I limit the number of devices that have my email credentials. Using 2FA everywhere is sadly not practical yet, so there are a nontrivial number of accounts that are, as you point out, effectively owned by anyone who can access my email.
I actually occasionally fantasize about implementing a mechanism that I could use from my desktop (where my password manager is) to send passwords as needed (e.g. one at a time) to my devices (I really like not worrying about syncing whole vaults). Encrypt the password using an epehemeral key (gets deleted after 60 seconds, for example) on the transfer service and a local key derived from a random six digit number. Display the number, send a url to the device, and anyone hitting that URL has 60 seconds to enter the six digit code and it decrypts the password and drops it on the device clipboard. This is about 1000 times better (and over-engineered, naturally) than my current practice of "paste it in a slack message to myself."
I actually occasionally fantasize about implementing a mechanism that I could use from my desktop (where my password manager is) to send passwords as needed (e.g. one at a time) to my devices (I really like not worrying about syncing whole vaults). Encrypt the password using an epehemeral key (gets deleted after 60 seconds, for example) on the transfer service and a local key derived from a random six digit number. Display the number, send a url to the device, and anyone hitting that URL has 60 seconds to enter the six digit code and it decrypts the password and drops it on the device clipboard. This is about 1000 times better (and over-engineered, naturally) than my current practice of "paste it in a slack message to myself."