I'm not sure how useful this really is. It just handles the application layer, leaving you on your own for dealing with the database. Now, Amazon has some services to help you out (RDS, SimpleDB), but SimpleDB is kind of crap, and RDS is expensive.
More importantly, that loses some of the simplicity, meaning that you still have to think about infrastructure to a degree.
Heroku is git push to deploy, and if you have a standard Rails app, that's it.
Whereas to deploy a standard Rails app, after you package up your app as a .war with JRuby (and deal with possible gem incompatibilities), you have to spin up an RDS instance too. And change the config in your Rails app so that it knows how to talk to the RDS instance. And if your RDS instance crashes, they backed it up for you, but you still have to manually restore from the backup.
And once you're already thinking about infrastructure to that degree, you've lost a lot of the benefit of a PaaS.
RDS pricing appears competitive to Heroku's dedicated databases. It's hard to compare them directly since RDS separates instance type, on-demand and reserved instances, provisioned storage, replication (crossing availability zone boundaries), and number of requests. Heroku on the other hand provides a fixed amount of storage (and one that's twice the max one can provision for RDS), a mix of instance types, and a connection cap limit. It's not clear if Heroku provides multiple availability zone instances similar to RDS.
Ignoring provisioned storage and request costs one can have a small on-demand RDS instance for ~$80/month. Heroku's instance (RAM/CPU) equivalent (Ronin) is $200/month.
As one of the two people working on something similar to Heroku or GAE (we're called ep.io), Beanstalk is simultaneously worrying and quite interesting - while it is an interesting player into the market, it's not _that_ hands-off, from what I can see, especially when it comes to databases, or things like Redis, which it's nice to have managed for you as well.
It's possible we'll see people springing up on EC2 offering those as separate services, however.
The other interesting part is that, since they don't do shared hosting but make you use a whole EC2 instance, there's potential for it to cost considerably more for smaller sites, but only once you get past that free plan of theirs (then again, the disk I/O on the Micro instances is truly awful)
More importantly, that loses some of the simplicity, meaning that you still have to think about infrastructure to a degree.
Heroku is git push to deploy, and if you have a standard Rails app, that's it.
Whereas to deploy a standard Rails app, after you package up your app as a .war with JRuby (and deal with possible gem incompatibilities), you have to spin up an RDS instance too. And change the config in your Rails app so that it knows how to talk to the RDS instance. And if your RDS instance crashes, they backed it up for you, but you still have to manually restore from the backup.
And once you're already thinking about infrastructure to that degree, you've lost a lot of the benefit of a PaaS.