That seems to be true with the "blitz" of stories like the Airbnb ones, but at a more general level, a lot of duplicates seem to be occurring because there are lots of different subgroups now using HN at different times of the day.
For example, these are the same story but did well with similar karma and similar discussions, a week apart from each other:
I certainly visit HN far too much but I notice "repeat" stories like these frequently.
MetaFilter (which is certainly pretty resistant to change) ended up implementing an automatic search feature at the posting stage to highlight potential conflicts like these. It seems to work well (it's not enforced - it just makes users think first). Edit: Realized I should add I don't necessarily think this is a good idea for HN. Personally I'd prefer to see the per-page # of items go up a bit. A 50 item front page would better cope with the higher levels of FP items.
When the articles appear simultaneously it's obviously a pain to have discussion fragmented, but if they're appearing distinctly, is it such a bad thing if people are discovering something new?
I am fine with repeats, but I think a mechanism that detects "related" articles and providing links to previous/later discussions would be useful. I certainly would like to hear more of what people say if a topic or a specific article interests me.
I am not necessarily advocating that this should be a built-in feature; on the contrary, I think it would be more elegant as external feature and I think in that form it would be more useful, since this sounds like one of those problems where the general solution is easier than a particular solution
I think that's quite subjective. It may not be a bad thing! I just think it indicates fragmentation of the readership in a way that didn't used to occur. Perhaps this is a natural part of a community growing beyond a certain size.
That said, there was once a time when any new front page link was something I was sure not to have seen before. Now? Unless it's up to date news, I've seen perhaps 1 in 5 links on Twitter, Reddit, or even my own newsletters already. People are getting 3 week old JRuby release notes on the front page nowadays fer chrissakes..
For example, these are the same story but did well with similar karma and similar discussions, a week apart from each other:
http://news.ycombinator.com/item?id=2778219 http://news.ycombinator.com/item?id=2801263
I certainly visit HN far too much but I notice "repeat" stories like these frequently.
MetaFilter (which is certainly pretty resistant to change) ended up implementing an automatic search feature at the posting stage to highlight potential conflicts like these. It seems to work well (it's not enforced - it just makes users think first). Edit: Realized I should add I don't necessarily think this is a good idea for HN. Personally I'd prefer to see the per-page # of items go up a bit. A 50 item front page would better cope with the higher levels of FP items.