I'm a light weight emacs user (mostly for org mode). I'm forever grateful to the dev I worked with who introduced and encouraged me to learn the vi bindings.

I'm probably missing something in your message, but there is support in the shell for vi bindings (https://opensource.com/article/17/3/fun-vi-mode-your-shell)