Hacker News new | past | comments | ask | show | jobs | submit login

Which, unless I missed something, says nothing about you keep your Elasticsearch index in sync with your source-of-truth database.



Depending on your problem domain, you either do periodic dumps of your data into ElasticSearch (oftentimes done nightly, when server load is low), or you do a dual-write layer in your application.

Search is one area where user expectations around freshness are somewhat relaxed. If your search results don't include up-to-the-minute results, in most domains users will forgive you. It's better than having poor relevance.

If up-to-the-minute results are critical, then update your DB abstraction layer to write into ElasticSearch as well as PostGres. It's a pain, but manageable. Consistency, again, is not as critical for search as for a source-of-truth store, because a missing or stale record is the same as a poor ranking algorithm that didn't retrieve it in the first place. And if your application detects a bad search result, they can always go back to the source-of-truth and re-index it.


"oftentimes done nightly, when server load is low" what does this mean? Unless you are running a highly localized website, websites don't have a nighttime. We see pretty much constant usage 24/7 (though a bit higher when it is daytime in Asia). This probably would have made more sense thousands of years ago when the world was flat and the sky barge would pull the sun over the horizon into the underworld to travel back to the other side during the night, but nowadays? Not so much.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: