Indeed - in the past, some browser extensions would auto fill into iframes and similar, using the origin identity of the page container, even when the field was invisible. That's obviously an issue, but sticking to manual actions (partly) helps there.

The downside of not using a password manager is that users enter (or paste) their passwords without any robust domain validation. In phishing scenarios, a missing auto fill prompt is likely to be enough to encourage a pause and think.