There's an API that abstracts vector search only. vectorize.search() and that part is not unique to LLMs but it does require selection of an embedding model. Some people have called embedding models LLMs.

vectorize.rag() requires selection of a chat completion model. Thats more specific to LLM than vector search IMO.