A thousand times yes. I'm using Vim keybindings in most of my editors, and every time it is something somebody implemented anew which is kinda like vim, but not really like vim. It would be great to take the complete set of keybindings > actions out of vim, abstract it away, and put it into a library.
I looked into the vim code a bit, and it seems that this is a lot of work.
GNU Readline already provides a lot of VI's (so almost VIM) functionality as a library. Maybe that could be used as a starting point? It probably still is a lot of work though.