I think it is if the the extension does it’s thing without any user interaction?

At least I remember reading that that was why the Bitwarden extension is so safe. It doesn’t do anything until I press a button.

Indeed - in the past, some browser extensions would auto fill into iframes and similar, using the origin identity of the page container, even when the field was invisible. That's obviously an issue, but sticking to manual actions (partly) helps there.

The downside of not using a password manager is that users enter (or paste) their passwords without any robust domain validation. In phishing scenarios, a missing auto fill prompt is likely to be enough to encourage a pause and think.

Tavis Ormandy is one of the leading security experts in the world. Here’s a blog post that highlights a number of the risks related to password manager extensions: https://lock.cmpxchg8b.com/passmgrs.html