While I continue to dig in to the specifics of the billing and support issues described, I can confirm this bit from the blog post:
if you stored 2 million images and delivered 1 million images, your total cost for that month for the Images product should be ~$210, not $400+
I've reached out to the author to get some additional information which will help me investigate this further. Also happy to chat with anyone else with questions or issues (zaid at cloudflare)
It seems like the author already went above and beyond to try to get this resolved through the proper channels. Why did it take an HN front page post to get it actually looked into? It does not inspire confidence in your service.
The thing is that Cloudflare has no proper channels. Anyone that has tried Cloudfare's support (even as a paying customer) knows that it's almost impossible to get a sensible answer to anything.
That's the way tech companies work today. You need to have connections or spend social capital to reach people who can solve issues. Or you need to bang your head in support channels for weeks hoping that someone will escalate your issue.
Sometimes there's workaround, if you pay big bucks, you might get "personal" manager who can actually connect you with necessary people. But that service is not available for every company out there. And if you don't pay big bucks, you don't have a chance.
This scheme probably makes sense. At certain scale you just can't talk to everyone. When you have 20 developers and 20 million clients, one person can have only so much time. And most support issues are stupid anyway.
I recall several similar situations with Cloudflare: a user has billing/service problems and the only way to get some support is to end on HN frontpage or reddit ...
Skimming through the blog post, it sounds like in the end they were not overcharged, but the timing and calculation of prorated charges for upgrades, vs. the credits for what what already paid, was a little weird and not obvious.
What contact info did you use? My partner is the author of this blog post, and neither of us has been contacted by Cloudflare that we know of.
My original ticket number was 3029706 but it seems to have disappeared after the support platform migration.
I clicked assuming this was analyzing how hilariously expensive Images is compared to most of their offerings, but it's about how hilariously bad Images' payment structure is. Nice!
> It’s likely that we will replace Cloudflare Images in the long run. Looking at storage costs alone, S3 would be 4 times less expensive for our use-case.
Cloudflare R2 would likely be a good choice too, if you're willing to bet that the (inevitable?) shakedown for money once you find the limits of "unlimited egress with no fees" won't be too bad.
I'd recommend just going on-premise for storage. You can build a full system with more than enough RAID-1 storage which will have 100% ROI in a few months compared to what they're supposed to pay for Cloudflare Images. Just use that as a storage backend for your webservers in the cloud.
If Cloudflare really is serious about this for R2, that's going to be difficult for them to communicate in a trustworthy way. Their other products (in particular their flagship DDOS mitigation service) definitely seem to operate on a "free, but if your site is large enough you'll get an email from salespeople threatening to disconnect you unless you pay $$$" basis, using the boilerplate "don't overburden our servers" clause in their ToS.
Speaking as a bystander, a more-charitable option might be: "DDoS protection is free for targets which fit a generic risk-profile and are targeted randomly or only for small-scale spite."
P.S.: In contrast, imagine a website dedicated to journalistic content about how {dictator} is a crook and a parasite that has made {country} weak and a joke to the rest of the world. If there's a regular nation-state level DDoS from nodes in {country}, Cloudflare might say: "Hey, uh, your situation is not normal anymore, and it's not exactly your fault be we are a business that needs to recoup costs..."
Thats basically for everything. Do you think home insurance will pay if you burn down house every year. Or car insurance if you trash it few times a year. I see DDOS protection as kind of insurance for event happening once in a while, if it happens all the time one definitely needs something more than a free service.
That reminds me of an NPR piece [0] discussing how a worrying number of people seem to be getting wrong/illegal tax-advice from "influencers" on TikTok etc. I'm sure a similar thing with insurance fraud either has happened or will happen eventually.
You've posted these links without much context, but yes, I do think Cloudflare does a pretty good job of backing up the specific claim that they'll absorb a large, random DDoS attack for you. (Although it doesn't seem like the attack in your last link could even have been attributed to any specific customer in the first place.)
Where things start to get shaky is if your site uses a lot of bandwidth for legitimate traffic, or otherwise uses the service in an unusual way. Personally I see it like an old shared hosting plan that will probably let you use some burst capacity if you get Slashdotted, but operates under a vague shared understanding that the service is only for "normal websites". (Which includes a lot of policies and content guidelines that only become problems if you show up on someone's "sort by usage descending" dashboard.)
I think publishing a specific amount of bandwidth that customers are allowed to consume would go a long way to putting R2 in the former category. Maybe that number is your maximum object size multiplied by your GET request limit, maybe it's your current total network capacity, maybe it's eight octillion zottabits per second.
Is there a reason you decided to make it "free" in your marketing instead of "free up to some (absurdly) high volume, and then cheap"? The former seems suspicion-inducing (consumers are familiar with many things that are "free*", such as "unlimited" mobile wireless plans), and would make for worse PR if/when you have a corporate customer that pushes the limits of what's profitable for you and you have to throttle/charge/cut them. The latter provides a clearly-visible data limit for "free" that's still high enough that everyone except the tiniest fraction of your customers will never even get close. If anything, I'd expect that "100 PB free a month" would be awe-inspiring and good PR.
While I honestly and wholeheartedly respect what you are doing with Cloudflare, and also yourself as a person and an engineer, and having been using Cloudflare and advocating for Cloudflare myself since 2011., I must correct you that given the recent scandals with "mandatory optional upgrades" and "waiving the abuse away - if you pay" (see the relatively recent post on robindev substack), nothing Cloudflare promises can be considered free and without shakedown. The reputation of the company has finally been tarnished. I'm truly sorry to have seen it happen.
could something like steam or a linux distro use it to distribute software? seems crazy not to.
I think what people are saying is we know there is probably a silent line somewhere, and we have no way of knowing when we cross it. It feels like it has an implied "most of the people reading this can treat it as unlimited in practice"
Egress was pretty much always unmetered until AWS came along. And it still is with most providers other then the usual suspects (AWS,azure, Google cloud)
Right, Cloudflare is operating in good faith, and while the deal could change (because business is business and sometimes the offering has to change), one should ensure they have other options available for egress from origin. That's not a Cloudflare issue, that's a vendor risk management issue. You should always have alternatives planned if a vendor's specific offering at a price point is a material component of your business costs.
There might not be precise measurement system which bills you by the byte, but many provides monitor (or periodically sample) bandwidth utilization, so and using too much data will catch someone's attention.
That’s wonky. If their normal servers include 20TB, why would they consider 250TB on their ‘unlimited’ servers too much. Makes you wonder who they think opts for unlimited servers. If 20TB was enough they clearly wouldn’t.
The docs say it's possible to resize images from an external source like S3 using Cloudflare Images. It seems like it's not straightforward, but would be possible with some configuration.
iDrive pricing seems too good to be true, their focus is clearly consumers (which they target with shady "90% off for your first year" gimmicks), and I can't find any write-ups by developers using it for real products. Can you point me to anything which supports that they're a real option for more than consumer/developer hobbyist use?
its s3 compatible so there should be no hurdle in using it
the price is their strong point but you shouldn't use S3 or any of the sort for any critical stuff in the first place.
reply to comment below: if you require S3 support more than once a year then you definitely should not be using idrive
but for the rest of us who don't need that idrive is a great alternative!
To summarize, I think this issue comes down to the fact CF didn't design well around the scenario that someone may upgrade their image storage capacity multiple times in a billing cycle, which seems to be the case for the author. If you only upgrade zero or once each cycle, it's not too bad.
I understand they want it to be prepaid. But I wonder why they don't just do the calculation upfront and charge you the difference (times the remaining days in the billing cycle) every time you upgrade.
I've written about the issues with CF Images before[0], and a lot of these issues still remain. A while back they joined their two image resizing products together though, and now CF Images is basically a hosted version of the previously named "Image Resizing", but weirdly (though for some use-cases great) priced, and you can bring your own storage if you so like.
In my opinion, CF Images is best if you use it with a Worker in front and R2 as storage, as you have (most) all the convenience of R2, the CDN, and CF Images, but without the unusual pricing.
Perhaps it's time I did a write-up on how to do this - it works very well for the purposes of our SaaS[1] at least :)
I refer back to your exact blog post every 6 months or so to see how (or if) its various issues have been smoothed out, and whether I should recommend it in a new build. Thanks much for keeping your post up-to-date!
didn't know you could place CF Images + Worker in front of R2, would love to know how you did that + what the benefits are. I've been serving images from Workers + R2 and it's great. I don't deal w/ lots of images though, and don't need resizing (but it'd be nice!)
I would be interested to know pageviews/bandwidth because this feels like it's still at the scale of it could run on a single box (rather than Heroku or a whole kubernetes cluster as seems to be the current case). For reference, we served a genealogy image/audio/video site from two servers behind an LB for ~$60/mo and handled 100K pageviews a day with nary a bother.
I would second this. Based on the initial numbers he provided (1 million images served a month, image size between 10kb and 1mb) I would expect to be able to trivially host this on pretty much anything in any major metro.
Even at his worst case (all images are 1mb) - they're only serving ~1tb per month in image data (admittedly, he mentioned these were stale numbers, so it'll have increased since then).
But I serve 1tb/month (actually much more than this) on old gaming machines sitting in my basement with just a standard comcast business line, just hosting media for my extended family.
I pay 200/m for the business line, and then another 15 in electric costs. And I need the internet either way.
Nope. I'm paying for a business line. I'm using what's allocated to that business line (actually FAR under what's allocated most times).
Every now and then I get a rep calling trying to upsell me, and that's about it.
At the end of 2 years they wanted to renew me. So my strong suspicion is they don't care in the slightest.
I'll add - I've also done this on a consumer Google Fiber line (which is more restrictive in the sense that contractually I can't host paid services). They also don't blink an eye when I push out 10mbps (or 500mpbs) over extended periods.
I can recommend Bunny CDN (https://bunny.net). You'll pay $20/month for storing 2TB, then a fixed $9.95/month to use their image optimization service with unlimited requests. And it might even perform better.
Yep, this is what I use as well. Works well! (In my case, I have it just running as a CDN in front of Wasabi, which actually stores the images - likely even cheaper if most of the images aren't viewed)
You can also just pay to use their image optimization service with another store (like R2). We do this for millions of images, it's much cheaper than CF's image service.
I actually have a similar story about Cloudflare, image hosting, billing and support, but the amounts involved were nowhere near as high.
I was evaluating image hosts a while ago. CF Images seemed promising but I was afraid that my use-case would grow to become cost-prohibitive, and I really didn't want to deal with a migration off it later. So I settled on rolling-my-own with R2 and ffmpeg to pre-generate a few renditions [1].
In short, R2 billing is weird. I actually haven't updated my bucket in months and yet the storage amount fluctuates a bit month-to-month. (Is R2 storage accounting probabilistic?!?) I also got hit by a billing line item in May for "R2 late usage" from Feb to March which I could not find any information about.
Their support only recently got back to me, with a very unhelpful / irrelevant answer of course.
The amounts involved are only ~$1 so I don't care too much - I was mostly curious - and I don't blame them for maybe not wanting to expend too much effort looking into irregularities on such a tiny account.
No regrets though. Using CF Images would still be 5x more expensive month-to-month even at my (tiny) scale. The $5-per-100k flat-rate is probably the thing that allows them to make any money on image hosting.
[1] Word to the wise - there are a lot of subtleties around building production-scale image hosting that often times it may be much easier to simply pay the premium for a managed service. For example, what if you want to only generate renditions just-in-time to save on storage? What about updating your rendition list on the fly, eg adding a new one, or even removing older ones from storage? When / how often should you sweep storage of unused renditions?
I looked at Cloudflare for DNS and domain registration. It's so hypercomplicated that I can easily understand that people may not understand how billing really works, and that some of those people work inside CF. I'm not trying to be insulting, it's just that from my limited experience, man, they're complicated.
My CF billing seems to be "day of the month" which might explain the differing prices based on the month; R2 storage is supposedly "GB-month" but I wouldn't be surprised if it's actually GB-hour on the backend. Does it follow that scale (e.g. June with 30 days should be ~96.8% the cost of May's 31 days)?
Interesting ... I checked and the difference is closer to 5% ($0.99 May vs $1.04 June), and June's invoice was actually higher than May's. They are billing that amount because the storage amount itself is fluctuating (alternating 66 and 69 GB) with zero writes / deletes to the bucket since around March, and staying well within the free tier for reads. Will probably have to leave it be as one of the mysteries of life :)
We actually switched FROM cloudflare TO cloudinary. CF seemed cheaper, but we found the service, the dashboard, and the service to be wildly erratic. Price went all over the place, we'd get double billed, the dashboard would stop working for a week.
Maybe Cloudinary is more expensive (though not by a lot for our use case) but I can say that nobody has had to log in or think about it for months. The thing just works.
Most of the CF add-on services seem to be hobby-quality. We used to use their video stream/encoding service too but have since switched to another more robust platform.
I've never used imgix but I find Cloudinary (and specifically the Media Optimizer product) to be extremely reliable, reasonably priced, and as close to zero-maintenance as a thing can be.
Cloudflare billing is pretty crazy. We had an annual contract with them for Cloudflare Stream, their video transcoding / streaming / hosting service.
We had negotiated a contract with a fixed capacity and overage charges for when we went over them. They estimated our usage for six months out and twelve months out, then spread the fixed cost out across two six month periods, billing monthly. So in effect, we would be paying for capacity six months in advance that we didn’t need, progressively getting closer to accurate billing.
The real problem came about ten months into the contract. We went over the fixed capacity. Still, no problem, right? That’s what the overage charges are for. Nope! They just switched our service off, causing us downtime.
They fixed it by manually applying a random amount of credit. Which we weren’t billed for. After that, they were really keen to negotiate a new fixed contract, even though they weren’t billing us for the overage charges.
We couldn’t figure it out at first because they were behaving very strangely about a bunch of things. Eventually we realised that there was one thing that explained all their weird behaviour: they just hadn’t implemented overage handling at all. They had no way of stopping our service from being automatically disabled when we went over our limit; the only way they could fix it was to manually apply credit; and the only way they had to bill us for the overages was to negotiate a new contract.
I’m still not certain if that was the case, but it was the only explanation we could come up with, and they were less than forthcoming about it all. The whole service was an absolute shitshow in many other ways as well; it was beta quality at best and they had no interest in fixing the issues. So can I believe they just didn’t implement overages? Yup.
Founder of Mux here. Not to pile on, but happy to chat (steve at mux). We've had detailed usage-based billing from day one, and even have APIs for reporting on your usage.
> Please accept our apologies for the delay in responding to you. We are experiencing an unprecedented demand for our service, which is causing delays for our customers.
This is such weasel wording. It's frustrating because it's most likely true and the people writing it are probably not lying and honestly swamped.
But considering that growth is the main purpose of most companies like this... by definition, they will at nearly all times be dealing with unprecedented demand!
I feel many cloud billing models are set up to initially account for the abusers of the service, and then account for the legitimate users. (in other words everyone’s assume to be a scammer or abuser, until you prove otherwise).
It’s an interesting concept, a corollary would be walking into a physical store and being treated as a thief, until you check out and make payment. (in fact the opposite is true generally as they check your receipt on exit, or you walk through the magnetometers on exit). I’m not saying it’s right or wrong, just making an observation.
These days I get the feeling of being treated as a thief whenever I walk into a Walmart or some Home Depots. Gotta go through a pedestrian gate set up well inside the store (turnstile basically) and lots of cameras in your face specifically to say “Hey you thief, we’re watching you with our AI”.
I've not tried Amazon walkout technology but it sounds like a good application of technology. if the system knows you took something, just charge the user for it rather than making a fuss about it.
> Though it seemed completely automated, Just Walk Out relied on more than 1,000 people in India watching and labeling videos to ensure accurate checkouts. The cashiers were simply moved off-site, and they watched you as you shopped.
The way Wal-Mart, Target and others work though isn't that you ever get to explain things.
Something scans wrong? Employee punches something in and you don't get wrung up for an item? Some AI camera system decides you've done some sort of switcheroo?
They will have the police come to arrest you next time you're at the store and have long since forgotten the details of your last purchase. Now your chance to explain involves lawyer fees - way more expensive than most credit card fees.
If you notice the mistake and try to explain to an employee before it's chargable as theft, and think you get it cleared up: there's still a good chance you still get charged and also the employee is fired and/or charged too. Major retailers assume their employees are just as much of a thief as they think you are.
Between the cameras, the guards everywhere, the anti-thief systems on products, the alarm portals and the ticket check, the whole experience is super aggressive.
Many stores have visible anti-theft defenses which inconvenience honest people, such as placing items in locked cabinets where you have to call a seller if you want to look at them.
This is more common in the higher-crime areas, but even in the low-crime places, this seems to be very common a lot with small and expensive things (jewelry, high-end electronics).
I received an invoice for cloudflare domains with incorrect billing period when I renewed it early. It calculated the billing period from the date invoice was created, instead of the the actual year in the future I was renewing it for.
It was not a huge issue, but I reported it via support and they acknowledge the issue. This was in 2023.
The number of large companies that practice this "you're too small for us to care about your complicated question, even if it's billing related" system is too large.
This is like the 5th or 6th time now, that I hear about Cloudflare's horrible billing and support. I was in the middle of setting up a new account the last time I read about another Cloudflare horror story. Changed my mind and setup a Bunny.net account instead. I don't have time to be jumping between services and chasing down bills that are wrong.
If someone makes a startup solely to use LLMs to decipher billing statements, they will probably make a lot of money. Or, that will get bought out before they put sunlight on the tactics of innumerable companies.
There are a bunch of companies doing this in a non AI fashion for AWS billing so there's got to be one/some out there doing it with AI, but at the point where you have an SQL database of your bill and are just constructing the right queries to ask it, using ChatGPT to help you construct the SQL is already there.
We recently looked at an image services for our platform. Nothing crazy a few hundred millions requests per day.
We (well I) shortlisted;
- Fastly
- Cloudflare
- Imgix
- Cloudinary
Imgix (for us) was the clear winner. We had only initially planned to use in a small performance-sensitive part of the platform. That quickly escalated (in a good way) to doing all our of public images on a custom plan. Imgix were great. We have no complaints.
Going by an estimate of 1tb total in imgs (1m x 1mb) it would cost $15 a month to chuck em on r2 and request charges are .36 per million so negligible for your usecase
- Opportunistically serve more efficient image formats when browsers support them without converting/storing
- Cropping and resizing without needing to build your own compute to do it (no creating and storing thumbnails)
- Watermarking on the fly (e.g., on cropped images)
You could absolutely build it all yourself, but tying it all together with your CDN and handling user uploads safely and doing auth (so users can't, for instance, just remove a watermark) isn't very easy, and it's nice to have a managed service that just does the thing.
The article notes that the image-centric EphemeraSearch uses image services like Cloudflare Images and Cloudinary to do more than just simple file serving, which the author and his partner would otherwise have to build. But the conclusion notes that Cloudflare Images is ultimately not worth the cost for their use case, and that "it’s likely that we will replace Cloudflare Images in the long run".
Honestly, the main point of the story here isn't the weird billing processes.
Its, yet again, the utter refusal by CF to provide even basic support. Even for paying users. Even for users who bought the so-called "pro" plan, mainly so that they can access the support that comes with it.
I myself had a metering/billing issue that:
1. I first posted on the forum. Instead of responding there, I was told by staff to raise a ticket.
2. No response to the ticket. Autoclosed after ~1 month.
3. I immediately reopen it. No response, autoclosed after ~1 month.
4. Repeat, for 5 months.
5. Then, I wrote an angry rant in the ticket, and finally the support response arrives: "the customer is misinterpreting the reported numbers, they mean X, not Y". Fair, enough, not obvious from the UI, but could you not have told me this without so much ... friction?
I love what CF has built (I even own a small amount of stock!). Its not perfect, but its far more of a serveless/cloud model than any other player. Makes so many things so much easier.
But, _PLEASE_, CF: improve tooling, and commit to a basic level of support.
Cloudflare support being terrible and non-helpful? Say it aint so /s
Seriously though, it feels like support exists as a hurdle to get to engineering. I have never had a serious issue that was resolved by support. I miss when customer engineers or customer support actually was treated like a value add part of the business.
The author discovers the wild world of financial engineering and "float".
Instead of settling usage changes for it's image service with a single charge, Cloudflare settles it with two charges in such a way that the customer "floats" Cloudflare money for some portion of their billing cycle.
This is the bullshit that you start to do when you become publicly traded. It adds zero value for the customer.
It's a little worse than that. If you need to upgrade at least once per billing cycle, the system keeps charging you more and more (since an upgrade will always have a higher total cost than the previous upgrade) each cycle while accruing credits that will never start paying down unless/until you have a billing cycle without an upgrade. That's not a great design for most systems, but especially bad for a scalable image service intended to facilitate the flexibility of growth.
To both of those companies credit they have never stuck me with a surprise bill after providing services for the amount of money they said they wanted to charge.
They have plenty of dark patterns, mind you, but post-paid usage-costed cloud services have so much more potential to mess you up.
What? I’m no Cloudflare advocate (i don’t trust them for MITM reasons) but their pricing is extremely gracious and one of the lowest cost for quality that exist today. Their Images service just seems poorly planned.
The hard part is the replication and up time, because you can serve a ton of images from a VPS for pennies on the dollar.
But even then, images are quite simple to deal with compared to big files like videos because you don't have to chunk them. So just copying them several time and putting a load balancing in front of it will get you 99% there for half of the price, dev time included.
Founder of Gumlet.com here (we serve more than 2 billion images per day).
Here are the reasons image processing on VPS is hard.
- Reliability and uptime is a huge concern. If service goes down, images can show broken
- When you build it yourself, you pay for image processing, bandwidth AND developer time. image processing at multiple terabytes and half a million image processed is not trivial ( inputs can be crazy and output needs to be modified as per the requesting platform)
- Images form a big part of Google web vitals so slow image processing can hurt LCP score.
- Newer formats like AVIF and JXL are increasingly complex. They need way more processing power than plain old JPEG.
I was thinking the same thing when I saw the title. But, my next thought was: what a reasonable profit margin before it's considered "over charging"... 20%? 50%? 100%? 200%?
Is that even the right way to look at it? I mean, perhaps some company is providing enough value and enjoyment of their product that a 500% profit margin is gleefully paid by their customers.
While I continue to dig in to the specifics of the billing and support issues described, I can confirm this bit from the blog post:
if you stored 2 million images and delivered 1 million images, your total cost for that month for the Images product should be ~$210, not $400+
I've reached out to the author to get some additional information which will help me investigate this further. Also happy to chat with anyone else with questions or issues (zaid at cloudflare)