Anyone else remember we had widely available and completely understandable options to rent everything from baseline web-hosting all the way up to private rack services for actual years before AWS came along and apparently all of that was forgotten, Warhammer 40K style?
I've rented a VPS from a vendor for going on 20 years now (Holy fuck I'm old) and I've never once been surprised at the bill.
You know that book "JavaScript: The Good Parts"? There needs to be a similar one "AWS: just the good parts". It would probably talk about EC2 (VPS), S3 (cheap bulk storage), SES (emails), and that's about it. When folks get into the elastic-super-beanstalk-container-swarm-v0.3 products, that's when they really kill themselves on the bills.
That said, yes, just using a VPS vendor is the easy way to stick to the good parts.
You still can. My company rents space from Cogent communications, and they are very good at customer service, although the offering of "rent a cage and get a pipe" is a lot more DIY than AWS.
I've rented a VPS from a vendor for going on 20 years now (Holy fuck I'm old) and I've never once been surprised at the bill.