It's going to depend on what you're running on, but phi2 is pretty fast so you c...

		CuriouslyC on March 6, 2024 \| parent \| context \| favorite \| on: Pg_vectorize: Vector search and RAG on Postgres It's going to depend on what you're running on, but phi2 is pretty fast so you can reasonably expect to be hitting ~50 tokens a second. Given that, if you are ingesting a 100k token document you can expect it to take 30-40 minutes if done serially, and you can of course spread stuff in parallel.

thanks for the info--good to know we aren't the only ones contending with speed for large documents lol