I'm bookmarking this to read later, but how much more sophisticated does the algorithm have to be than:
1) Create a list of 500 - 1000 active, relatively popular Twitter users: this would eliminate most celebrities who only tweet casually or delegate it to their PR people...presumably, by the time they tweet something, it's already huge.
2) Segregate the sample group of Twitter users into cliques
3) When any topic spreads between multiple cliques at an accelerated rate, that topic will likely trend
In addition, have a list of mega-popular celebrities and assume that most of what they tweet has a high probability of being a trending topic.
Twitter has some kind of formula for removing constantly-popular topics (or else Justin Bieber would forever be on the list)...if there's an easy way to include that, then it seems like predicting trending would be straightforward?
I agree with you --- this seems like a perfectly good way to tackle the problem of trend prediction directly. What we had in mind was something that would be more generally applicable to any kind of time series data, and we figured it would be interesting to test it on Twitter trends.
1) Create a list of 500 - 1000 active, relatively popular Twitter users: this would eliminate most celebrities who only tweet casually or delegate it to their PR people...presumably, by the time they tweet something, it's already huge.
2) Segregate the sample group of Twitter users into cliques
3) When any topic spreads between multiple cliques at an accelerated rate, that topic will likely trend
In addition, have a list of mega-popular celebrities and assume that most of what they tweet has a high probability of being a trending topic.
Twitter has some kind of formula for removing constantly-popular topics (or else Justin Bieber would forever be on the list)...if there's an easy way to include that, then it seems like predicting trending would be straightforward?