I’ve come to the conclusion that Heroku must be using adaptive processes to sandbag us. For example, to save on their resources, they must be creating static caches of the pages which are being requested from these cron pings. They could avoid loading my whole app back into memory with a trick like this. Please correct me if I’m wrong, but I see no other way for these pings to be successful and my app to still have bad initial load times when it is not visited for a while.
You are incorrect. We idle single-dyno apps that have not recently received requests. We do no caching, unless you are running an old bamboo app and are therefore going through varnish. See https://devcenter.heroku.com/articles/dynos#dyno-idling
Regardless of his slight misunderstanding of the technical side of things, he is correct when he says that you can be paying over $100 a month for a service and still be subject to dyno idling, which does seem pretty ridiculous, especially since Heroku takes a 30% cut of all add-on revenue (https://addons.heroku.com/provider/resources/HerokuAddonsLic...)
He is paying $100 for a bunch of add-ons, which have no idling. He is not paying anything for his dyno. The only current way to avoid this behavior is to pay for a dyno.
If you don't like this behavior, that's something to complain about. I'm just stating facts to help explain why his assumptions are incorrect.
It's interesting. I created a simple app (using the free dyno), use the free new relic monitoring and that keeps the app fresh (no idling). And I have no traffic coming to the site. All i'm paying for is SSL. I would think he would have no idling with multiple services pinging it for uptime.
I don't understand how my app could be set to idle when I have pings hitting it from three separate sources. Also, the pingdom ping is looking for a specific word on one of my pages. It has to find this word to be considered successful.
How often are the pings? Do the pings wait for startup or timeout? A lot of filtering methods on the traffic data could ignore isolated pings as well. They don't have to be using on/off. They could be using a higher threshold. E.g. one ping every 10 minutes, just startup and handle it then shutdown. 10 page loads every 10 minutes? Startup and stay up for 10 minutes. It's weird to even call page loads pings rather than the actual traditional ping as well, but whatever.
The pingdom ping is every minute (http check type with a string match optional setting). The heroku scheduler is running once every ten minutes. I'm not sure how often new relic runs.
Edit: The obvious solution is to buy another dyno. This isn’t the issue, however. I wrote this post to show the unfortunate grey area in their pricing structure. It’s possible for low-traffic customers to pay a lot and still be subject to the poor performance that comes with a dyno being set to idle.
The greater underlying problem is there are some customers who only need one dyno (especially those of us who are running unicorn. My dyno can handle three times the number of requests) and we would rather spend a little extra on an upgraded database or some other addons.
Well, that's that it seems. Could it be that he's running a single dyno app (with a bunch of add-ons) and that's what explains the behavior he's seeing?
Imagine how many free apps are created that are never used again. Should Heroku really devote resources to these apps? I think it's perfectly reasonable to idle them after a period of inactivity. If they didn't, they would need more resources and ultimately the customers would be paying for that. They aren't a charity, they are a business.
It's pretty simple, if you don't want your app to idle out after 1 hour of inactivity, buy a dyno for $35 p/month.
From the documentation they "...will have that web dyno idled out after one hour of inactivity."
The issue is that there is no hour of inactivity because there is web monitoring/querying supposedly going on automatically. So there should be no idling.
They are trying to "cheat" the system by setting up automated pings to keep it alive. Do you really think that's in the spirit of the offering from Heroku? I don't know why it isn't working but I hope Heroku just "handles" that scenario.
Like I said, if Heroku did provide the resources for all these free apps (and there must be so many that aren't used) then that would be reflected in their pricing. I pay for Heroku and don't want to be paying more to compensate resources for their free apps.
I don't think it's in the spirit of the offering, but that's up to individual interpretation. What is very clear is that it doesn't seem to be behaving as documented. That's a problem.
Also, bgentry stated "We idle single-dyno apps that have not recently received requests. We do no caching." This also seems to contradict what is happening and adds to the confusion.
Why not just clarify what exactly is going on? It's completely reasonable to be handling this kind of situation however they are doing it, but it's not reasonable (IMHO) to repeat and support misleading/incorrect documentation.
Well, it's documented in the sense that the free web dyno is documented to spin down sometimes; if you pay $35/month for the extra web dyno, it ought not to.
This seems like a reasonable pricing model to me. And it seems reasonable to me that they don't document _exactly_ in what circumstances the free dyno might spin down on idle. Free is free.
If on the other hand even with the paid extra web dyno you were still getting ~10 second spin up times after idle, that would be something I'd get annoyed at too.
If you're willing to pay over $100/month on services you may or may not need just for the learning experience -- perhaps scale some of those back, and instead pay $35/month for the web dyno you do actually need to avoid the idle spin up time.
I do think it's true that heroku can end up being a lot more expensive than you initially expect for what you need, the extras can add up quick. This certainly can be frustrating. Even when you really are paying only for what you need, things can get expensive for a non-tiny app. But if you want to minimize your costs, you definitely want to avoid paying for add-ons you don't need.
Exactly. It's not a problem that Heroku has implemented such a system, but it is a problem that it's not disclosed. The details do not have to be spelled out, but the system should be documented.
I don't need another dyno. I'm already running way more than this app requires. (with addons, etc..) I think the service would be more fair if the time threshold for being set to idle was governed by the amount you are paying.
You are paying for 0 web dynos. They happen to be giving you one for free, but the free one comes with limitations.
Yes, you are paying for a bunch of other services; but, well, those services aren't the actual web dyno that processes the actual request.
Perhaps they could phrase their offerings better, but my impression was that the 1 free dyno with idling is intended for development and testing, not for production use. Once you actually need to use it in production, you need to pay for a dyno.
Yes, that was my impression also, especially since there's no limit on the number of free dynos you can have (for different projects). I was actually surprised to learn that when I looked into setting up a separate staging server for my current project -- I'd thought the free dyno was one to a customer. Nope. The staging dyno is free; I just have to wait a few seconds for it to spin up if I haven't been working with it for a while.
So now I have 3 dynos (2 production, one staging) for $35/month, I can scale up or down at will, deployment is dirt simple, and I don't have to do the sysadmin crap myself.
It's an extreme bargain, IMO.
Now if the project takes off to the point where I need dozens of dynos all the time, that will be different, but ideally in that case I could hire a sysadmin. :-)
I think it's much fairer that you can choose what you pay for. If you don't want it to idle, pay $35 and get a dyno. I wouldn't want to have to pay $100+ (or some other amount) before it stops idling.
I feel your pain, as a fellow bootstrapper, but a single paid dyno, which will keep your app unidled, is gonna cost you $35 a month. I did run into a bug recently where one of our dynos was being idled even after we paid up, but it has since been resolved.
I'd say just pay them another $35 a month. Considering the miracles Heroku regularly performs as an ops team, it's worth it.
Fine-tuning an image like that only matters when your site is getting a ton of traffic. I am only employing these other services for the learning experience.
1. Reducing a half-megabyte image is hardly "fine tuning", especially on a basic information page.
2. It matters to every user that loads the page, regardless of how much traffic you're getting.
3. As a side note, this info page is nearly 2MB on wire. Turn on gzip (like 700KB+ of JS/CSS and stuff) and figure out what else is taking up so much transfer.
You should look into setting up a small vps. It would be far cheaper, you can run any services you want, and you'd control uptime of processes etc. it's really not hard, and it's quite possible to scale to millions of users a year on a small vps with some caching. Right now you're paying for lots of stuff you're never going to use, and underpaying for the stuff you need ( actual web server processes.) Heroku is not for you, and they're not going to change their policy on free dynos for you because you're trying to use the free trial of this for a real website, which is not going to work or allow you to learn about scaling a website.
This could be something as simple as them ignoring pings from pingdom when deciding to spin up workers, have you tried using curl to simulate hits? Heroku would probably be happy with you moving to linode, it suits what you want to do a lot more.
Re the image, if you're serious about learning about site optimisation, and dealing with load, yes it does matter, far more than your db until you reach much larger scale.
Re: Image: I agree with you about the image. What I really should have said is a one-man team who is trying to launch something can't always optimize every little thing. There is always one person like ericabiz above who goes through every hn post and criticizes the author over some petty image optimization issue on their landing page.
Wait, now its just for learning? I thought you were expecting a lot of traffic (hence all the addons you stuffed in). You need to scale those back and purchase a dyno. Problem will be solved and you'll pay less. Optimize only when you need to.
for $108 / month you could have a very nice setup on any number of VPSs, RS, ec2, etc... you could have at least a 2GB setup either split between 2 hosts or all on one depending on how you plan on scaling... but really IMO if you plan on running anything for any matter of time beyond a (hey checkout my neat idea app) get off heroku and figure out how to setup your own systems, nginx/haproxy, etc... sure it's nice to focus only on the app side with heroku so learn stuff there but eventually unless you're getting funded - you'll need to know how to manage your own servers... just my thoughts.. and i do put lots of little projects on heroku but they're mostly none critical services... or ideas
Why do you need to know how to setup and manage your own services. If it's not your expertise, spend your time where you can add the most value. I think Heroku is a perfectly fine option until you reach the point where you could hire a full time dev ops.
It's a pretty cheap way to have servers set up with 24/7 monitoring and technical support.
It's not very cheap for what this user wants to do - he wants web processes up all the time, so he'd have to pay heroku the price for that. He is paying for things which are not required (db cache etc) and trying to skimp on the thing which is (dynos). Heroku is not cheap, but it's a great service if you can afford it and want the service they are offering.
Setting up a vps and automating deployments is not very hard, it would be useful experience if he plans working on startups, and it would also be massively cheaper and could host tens of sites like his. If he wants to pay heroku to deal with all of that, he should accept that it will be costly, and reasses his priorities and direct resources where they are needed. heroku gives you the tools to do that - ditch crane db for example and other caching add ons, and buy dynos rather than trying to use the free dyno for a proper website.
I'm running a Hetzner server for 10yrs now. Debian, now Ubuntu. I've installed Redis, Nginx and some other stuff with apt-get and have an basic iptables setup.
Could you detail what I would not need to do when I'd chose Heroku? What "expertise" you are speaking of, apt-get?
I'm confused these days about cloud and how this would save me "full time dev ops". Could you elaborate?
One night, working for a CMS design agency, and running a launch for a new Rails app, I tried installing nodejs onto our dedicated server so Rails could compile the new coffeescript stuff we were running with.
Of course Node then tried to update NGINX or something (I'm a little fuzzy on the details, but this server had been around for a while) and then NGINX complained about something being misconfigured and wouldn't let us restart it, and so then we were stuck with 100+ rails apps sitting on this server, and we couldn't change the configuration for any of the apps, or launch this new app, and it's 11 PM, and I just want to drink a beer and go to bed, and the client's yelling, and
— so, I'm perfectly OK with not having 10 years of server management experience. Heroku just works. Almost 100% of the time. And every app's isolated.
I got into this business to ship products for customers, not figure out how to make dependencies of 3rd-party libraries on 4-year-old-server-builds play nice. If that's what you've gotta do, that's what you've gotta do, but I'm not really inclined to pick up 10 years of experience managing servers unless I have to.
You wouldn't need to rely on ten years of experience running your own server. Since you already have that you're in good shape - that doesn't mean your solution will work for everyone else though.
That's the point, I've been running servers for 10 years with iptables and apt-get knowledge. There is no "ten years of experience running your own server."
My hypothesis is that everything interesting that happens to servers is not covered by Heroku.
If that were the case, Heroku wouldn't be able to charge for what they provide, and everyone with more than just a trivial app would run their own server.
I think you underestimate what it takes to run your own infrastructure. I'm not saying it's black-magic hard, but it does require a not-insignificant skill set for all but the simplest of setups.
I might underestimate what it takes to run a server, and what Heroku brings to the table, I'll grant you that.
Taking another of my hats, the CTO of a million-user business hat, I still don't know what TCO wise the cloud/Heroku brings to the table. I know about the fast scale up and scale down benefits, no 12 months contract, OPEX vs. CAPEX etc. but have not found any TCO cost calculation based on real tasks people need to do on rented servers vs. cloud.
I do agree that heroku, while not cheap, is a value for what you get.
But I wonder if that comparison "I think Heroku is a perfectly fine option until you reach the point where you could hire a full time dev ops" is actually one anyone makes in reality. A full time ops person is going to cost way more than $100 a month, quite a bit more than several thousand a month in fact. Are there really people who would go with heroku until they're paying $5 or $6k a month or more, and somewhere around there (that's a lot of heroku), say "Gee, _now_ we're paying enough (barely) to hire a fulltime person to manage the servers ourselves, let's switch everything to that."
I'd be curious what dsower's heroku logs look like. Would you mind posting the portion of this where it happened?
The logs should show 'idling' when idling and also show the 'pings' from pingdom and/or new relic. Looking through that it'd give us all something concrete to discuss - because quite possibly this is just a bug with Heroku's dyno idling that could have simply been submitted as a support ticket.
Reposting my comment from the website here (unreadable there):
I contacted support about this a while ago (was in a similar situation). Their response was "Use more than one web dyno". I didn't need more than one, but that was the only way they offered to allow my app to not "go to sleep".
This is kind of an annoying problem if you have a lot of small sites for whatever reason. I think their policy is reasonable and they are upfront about it, so I certainly don't begrudge them, but it is still annoying.
What is the best way to host a bunch of low-traffic sites for less than $35 (I believe that is what it costs for the second dyno, required to keep your app from being spun down) each per month? The usual answer is "PHP", but I sometimes wonder if there'd be a market (and way to make money) for Heroku-like ease of deployment/management, billed based on resources (much like a VPS) rather than per application.
VPS and Dedicated Servers certainly work and are usually going to be more economical, but then you are going to have to deal with all of the setup/maintenance tasks that Heroku abstracts away from you.
I'd venture that there are far more people using Heroku for its easy deployment and management than its ability to scale quickly and easily.
>I'd venture that there are far more people using Heroku for its easy deployment and management than its ability to scale quickly and easily.
That's well and good but I wouldn't be so quick to pass off on dedicated. A simple 50-100 line fabric script that takes 30 minutes to write can manage your custom deployment for you. And guess what, your next project will be even easier to deploy because of that same script.
If the only reason to use Heroku or other of its kind is easy deployment - maybe we just need better deployment apps that can work seamlessly across servers.
> If the only reason to use Heroku or other of its kind is easy deployment - maybe we just need better deployment apps that can work seamlessly across servers.
Heroku certainly thought so. That's why they got a few dozen brilliant programmers into the same building and made it happen.
Of course, with a ~$5M/year burn rate, it's probably also why they charge money for their services.
(making up numbers here, but assume 30 developers at $150K => ~$4.5M so I would guess I'm laughably low)
How is OVH for actually serving websites? I use a 8G dedicated server for a seedbox and PLEX. The servers in France were really bad for me in the US. But I just switched to the datacenter in Canada and it is working well.
But I am still weary about using them for anything serious. I thought they were just for bittorrent.
Using them just for bit-torrent is actually against the TOS/AUP unless it is completely legal. For their French/UK datacenters they don't care about torrenting, for their Canadian datacenter they have been coming down harsh on those people I know that were using them as seedboxes for copyright material.
As for actual hosting? They are fast, have plenty of banwidth, and I am able to saturate my home cable link (Comcast in Colorado) using my service located in their Canadian datacenter.
Ping times aren't that fantastic, but for most of what I serve up it isn't an issue that the connection took 50 ms to set up, rather than the 35 ms from a US based host (Although a lot of that comes down to the extremely poor routing choices that Level 3 seems to be making on my way to Canada.)
What I have noticed is that my European visitors are happier, from across the ocean to OVH's BHS datacenter is about 90 msec, whereas from there to my server at Softlayer (Dallas, TX) that easily goes up to 150 msec.
A few of my sites that are heavier load, or require more static assets that can easily be cached have Cloudflare thrown in front of the static asset part. So the web app/dynamic content is served from the server itself, and the static content is on a secondary domain that is set to Cloudflare.
I love the way the comments have steered into technology nonsense, while the issue has nothing to do with the way in which Heroku screws him over, just the mere fact that they ARE screwing him. Commenters who have suggested 'just pay $35/month for not having 10 second lag' are idiots.
Heroku is a great and easy service, but I find the add-on 'no lag for $35/month' which is what the dyno basically is to the poster, unacceptable. But if most commenters are willing to pay into the hype, go ahead.
How is giving him a free dyno (albeit one that spins down when it's idle) "screwing him over"? Be specific.
The cheapest Linode is $19.95/month, and that's just a bare VM. You have to do all the sysadmin stuff yourself. For $15/month more, Heroku takes care of all that for you. If your time is worth anything at all, maintaining your Linode is going to burn up more than $15/month.
Now, granted, for some use cases the Linode makes more sense (I have one of those, too, and have been happy with it and the company overall), but I don't think either company is screwing anybody over. Both are offering good value on their products, but those products are different. Linode gives maximum flexibility, but at the cost of full responsibility. Heroku gives maximum convenience, but at the cost of less flexibility.
I call it being screwed over when you pay over $100/month to have your site 'spin down'.
I grant you that Heroku is a great service. Having a reasonably big site you'd need a system admin who costs $100/hour, Heroku saves on that too. I am not arguing against the technical convenience that it brings you. But I believe the policy of spin downs should depend more on if and how much you are paying, rather than if you are paying for a specific element of their service, in this case a dyno. It does not cost $15/month more to achieve what the poster wants, but $35/month.
He's not paying "$100/month" for dynos. If he wants to avoid spin downs, he has to pay for a dyno. That's not hidden.
" But I believe the policy of spin downs should depend more on if and how much you are paying, rather than if you are paying for a specific element of their service, in this case a dyno."
You're seriously arguing that if you buy several items from a company, they're then obligated to give you a completely different item for free?
It is easy to play devil's advocate on something like this. Technically yes, you are right, it is not hidden. What I am saying is that I find it bad form that they spin your stuff down when you are paying so much for a SERVICE.
The purpose of the dyno is not to prevent spin downs, it is a provision for background processes, think processing orders, updating information from external APIs every hour, etc.
Your comparison is obviously bogus as we are dealing with a service, not products. It is more like paying for a spotify premium account and still having to listen to ads. He is being treated like a free user, while he is paying for premium service.
I have some feedback on your site (http://www.mycelial.com) that is unrelated to your post. I had no idea what your site was until I went to the About page. I think the contents of the About page should be put on the homepage.
We use Pingdom to keep a couple low-traffic sites alive without issue. I think we could provide you with more help if you posted a condensed sample of your Heroku logs.
Check your browser plugins, especially adblock. The background on the comments should be dark grey #ccc. You're probably blocking or rewriting something from Disqus.
To clarify: I understand that getting another dyno would solve this problem. I love heroku but I think that they need to rethink the idle process. Some users only need 1 dyno and a bunch of addons. If we are paying decent money then the idle threshold should be increased, in my opinion.
It's just semantics. They are giving away the first dyno. Would you prefer they took that away when you purchased one for real? 2 for the price of 1 seems pretty nice.
That and fundamentally (as you noted about being optimistic) you're paying $100+ a month for an app that gets zero traffic. That seems to be the core issue.
A cron job where a request is sent at regular intervals to a dyno from Phantom should solve your issue. It should be justified if you are buying addons. For free sites with no revenue to Heroku, it's reasonable since they have many sites that only get visited monthly, if that.
Seems to me the real problem is you spend 50 for a bigger DB instead of 35 for a dyno. If you got no traffic why do you need so much db cache especially in addition to memcache. You can easily fix that with pgbackups and import into a basic db.
Your post says that you want to hear from Heroku, but it doesn't mention anything about you trying to contact them directly. Did you try to contact them before making a somewhat accusatory blog post without much proof? What did they say?
Write 'intelligent' automated tests (you're a HN'er so you have these already right ;) and schedule them to run CI on your production site. That may prevent any adaptive processes from caching.
You are incorrect. We idle single-dyno apps that have not recently received requests. We do no caching, unless you are running an old bamboo app and are therefore going through varnish. See https://devcenter.heroku.com/articles/dynos#dyno-idling