Does anyone use OpenSearch for its knn and vector capabilities? Is it any good? It’s always hard to know with systems like this whether it works at scale until your team is fighting fires.
I don't know about OpenSearch implementation, but recently I implemented from scratch Vector Sets for Redis using the HNSW as a data structure, and there are many other stores that use the same data structure. When HNSWs are well implemented, you can stay assured they scale very well compared to the task at hand, but you can expect insertion speed only on the order of a few thousands per second, if you are hitting a single HNSW. Reads are much faster, in Redis I get 80k/s easily (but it uses multiple cores).
So if you want to build a very, very large index using HSNWs, you have to understand if you normally have many writes that accumulate evenly, or if your index is a mostly read-only thing that is rebuilt from time to time. Mass-insertion the first time is going to be very slow. You can parallelize it if you build N parallel HNSWs, since the searches can be composed as the union of the results (sorted by cosine similarity). But often the bottleneck is the embedding model itself.
What is really not super scalable is the size of HNSWs. They use of memory is big (Redis by default uses 8 bit quantization for this reason), and on disk they require seeks. If you have large vectors, like 1024 components, quantization is a must.
Irrespective of opensearch, if the dimension of your vector embedding is reasonably large you'll probably want an approximate nearest neighbours approach like HNSW rather than knn itself
For whatever an endorsement from a random stranger is worth, we've been using opensearch for a vectordb for hybrid search across text and multimodal embeddings as well as traditional metadata and it's been great but we're not "full production" yet so I can't really speak to scale, but it's opensearch so I expect the scale to be fine most probably.
I use it all the time. If it’s “good” depends more on your model for embeddings, but you do need to know a bit to tune the index. Whatever algo you choose, read the paper.
If you’re using lucene HNSW, it will scale but will eat lots and lots of Heap RAM. If you’re using FAISS or nmslib plugins keep an eye out for JNI RAM consumption as well as its outside the heap.
Overall, I’d say that it is a challenge to easily scale ANN past 100M vectors unless it’s given significant attention from the team.
It works with some caveats. I've seen it handle searches with millions of documents no problem, but the KNN search requires to load the entirety of the embedding graph in memory. So watch your RAM consumption.
The quality of your results will depend mostly on the quality of your embeddings
