Unix was effectively free for a long time, ported to pretty much everything, and with source-based portability a lot of software could be adapted between places.
VMS could be way more solid than whatever was cobbled in Unix (outside expensive niche implementations) but over time Unix would grow and grow while VMS was limited in users and platforms. [1]
Digital selling off some of the software packages they used to provide with VMS only made for quicker demise.
[1] This is also oft-forgotten reason why Oracle got so widespread in enterprises - you could get Oracle with most features on most platforms, even if some were more blessed than others. It might run slower than packages optimized for given platform (think DB2 on S/390, Rdb on VMS) but it would be mostly the same database whether you connected to development server on PC, some department's VAX Cluster, or your recent acquisition's IBM Mainframe.
VMS could be way more solid than whatever was cobbled in Unix (outside expensive niche implementations) but over time Unix would grow and grow while VMS was limited in users and platforms. [1]
Digital selling off some of the software packages they used to provide with VMS only made for quicker demise.
[1] This is also oft-forgotten reason why Oracle got so widespread in enterprises - you could get Oracle with most features on most platforms, even if some were more blessed than others. It might run slower than packages optimized for given platform (think DB2 on S/390, Rdb on VMS) but it would be mostly the same database whether you connected to development server on PC, some department's VAX Cluster, or your recent acquisition's IBM Mainframe.