As others have mentioned, Linode is great value. Pair it with Dokku and you've got an even cheaper / more customizable (but not scalable) Heroku.
However, be weary of Linode's "backup" service if you intend to store a lot of files (as opposed to lots of data / large files). It's file-based (not block-based) backup, and I can confirm it does fail. In our case, we weren't storing tiny files either, they were images uploaded to our infrastructure. Granted, don't do that(!), use S3.
Also, regarding AWS/S3, there's a plethora of ways to get into AWS's Activate program (https://aws.amazon.com/activate/) which includes a decent chunk of free credits.
However, be weary of Linode's "backup" service if you intend to store a lot of files (as opposed to lots of data / large files). It's file-based (not block-based) backup, and I can confirm it does fail. In our case, we weren't storing tiny files either, they were images uploaded to our infrastructure. Granted, don't do that(!), use S3.
Also, regarding AWS/S3, there's a plethora of ways to get into AWS's Activate program (https://aws.amazon.com/activate/) which includes a decent chunk of free credits.