Hacker News new | past | comments | ask | show | jobs | submit login
The Yahoo Firehose: Get access to our public Updates activity stream (yahoo.net)
46 points by nreece on April 13, 2010 | hide | past | favorite | 6 comments



Ha! When I tweeted the following a few days ago I had no idea this announcement was coming.

http://twitter.com/random_walker/status/11917456790

"within a few years there will be a real-time stream that aggregates essentially all activity anywhere on the web"

While Twitter, Wordpress, etc. have firehoses, this announcement is different -- aggregation across multiple services. I think the concept is very powerful and is something I've been looking into over the past few weeks.

I'm curious how many people know about Spinn3r (spinn3r.com) -- they claim to provide a real-time hose of all updates in the blogosphere. They quoted me $6k/mo. Unfortunately, Spinn3r's main product was deadpooled last year, so I'm not sure what their future is as a company. I think they should have made the real-time stream their main product. In a way they were too early.

Another startup called Superfeedr takes a different approach. They charge on a per-feed (or per-entry) basis, so their up-front costs aren't as huge. They currently cater to customers who might only want a few hundred or a few thousand specific feeds, but as they scale up they should be able to offer a bulk stream of essentially the entire blogosphere.

Then there's Google Buzz, which is also about aggregating different services. They don't currently offer a firehose but apparently you can build one yourself using PubSubHubbub, although developers report that it isn't working so great: http://groups.google.com/group/google-buzz-api/browse_thread...

But the long-term trend is clear. We're going to have an "uberhose" at some point! I have a few applications in mind of things that such a pipe would enable that are fundamentally not possible today, but I'm curious what ideas other people have.


Nice market segmentation :) As for Superfeedr, we already have a pretty big chunk of the blogsphere and as long as you know what you want (feeds), we can provide that to you. Other services that you mention suffer from spam, because they have a "extensive" approach. In our case, we'd rather let our users decide what they want, rather than filter for them.


Just wanted to add:

We at trendiction.com also provide a raw and filtered feed of different data (currently blog posts, blog comments, news posts, news comments, and message board posts).

But we restrict ourself to mainly german content right now. (there is already a lot to cover there)

Thibaut


First off, why would Yahoo!'s firehose have "YouTube favoriting" in it?

If they continue to aggregate 3rd party data and put it all in an easily accessible firehose, I could see this being really useful.

I haven't used them a lot, but Yahoo!'s developer stuff seems pretty handy. Would be nice to see them reinvent themselves or really push this division much like Amazon AWS has.


How much will this cost?

It's a very cool service that only a company the size of Yahoo! (and Google, Bing, etc) can provide due to the infrastructure needed for crawling and serving. It would be interesting if this was the first of many Yahoo! moves in which they shift from a pure search + portal to data providers. Their APIs already do that, but this is much bigger.


Yahoo suffers a lot of stigma. Maybe because most of us passed a fase where all serious programers had switched to google. I know ppl thar dislikes yui only based on preconcepts about yahoo.

how comfortable are you on depending on yahoo services and why?




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: