I feel like it would only be right for this kind of project to not "stoop" to background-removal, but instead do something more ridiclous by throwing even more ML at the problem:

1. classify the image

2. use word-net to figure out what "container" word is most closely associated with the image's classification

3. search the container word, and pull out its dominant colors

4. return the colors that are in the image but not in its classificatory container. Unless that set is empty (e.g. a polar bear in snow), in which case return the colors you'd have given without all these extra steps ;)

I agree. Otherwise it would try to strip the background from an ocean or sky image.

I (and probably a few others) searched for "Hacker News" and it comes up with a really nice colour that is probably a cross between the top bar and the background, but not a colour I'd associate with the site in general.