This section of the docs may be confusing. What you described will actually almost work. See: https://postgresml.org/docs/open-source/korvus/guides/rag#ra...

It does work by running the LLM on the database server but you can configure the LLM to run on the GPU

if you are using your database extensively how do you scale up your GPU resources for korvus?

You can use Korvus for search and feed the results to an external model

Great question! Making calls to external services is not something we plan to support. The point of Korvus is to write SQL queries that take advantage of the pgml and pgvector extensions. Making calls to external services is something that could be done by users after retrieval.

Unfortunately the pgml extension does not work on AWS RDS so there is not.

You can totally use an existing postgres database. Just make sure to install the pgvector and pgml postgres extensions and it will work!

We would love help developing a Ruby SDK! We programmatically generate our Python, JavaScript, and C bindings from our Rust library. Check out the rust-bridge folder for more info on how we do that.

I think there are a couple things worth noting here:

Semantic search performs well at capturing documents keyword search misses. As noted in the article, when searching for exact keywords, keyword will outperform semantic search. It is when users do not know the exact phrase they are looking for that keyword search shines.

Semantic search should only be a part of your search system, not your entire search system. We find that combining keyword search + semantic search and then using a reranker gives the best results. It is best if the reranker is fine tuned on your search history, but general crossencoders perform surprisingly well.

Hey thanks for bringing this up. I'm not able to reproduce the error you are getting. I just ran `cargo install lsp-ai -F llama_cpp -F metal` on my mac and received no errors and a clean install. Can give me the full output of your command either here or in a github issue?

LSP-AI is meant to work with plugins. It can provide auto complete features without them, but to have the kind of experience Copilot provides with VS Code, you will need editor specific plugins.

LSP-AI makes writing these plugins easier. Check out "The Case for LSP-AI" https://github.com/SilasMarvin/lsp-ai?tab=readme-ov-file#the... for more info on why I think that is true