+1 on this one, I've been pleasantly surprised by this for a small (<3GB) local project

does duckdb scale well over large datasets for vector search ?

What order of magnitude would you define as „large“ in this case?

like over 1tb.

Some people are using DuckDB for large datasets, https://duckdb.org/docs/stable/guides/performance/working_wi... , but you'd probably do some testing under the specific conditions of your rig to figure out if it is a good match or not.

its clear many DuckDB sql queries can handle terabytes of data, but the question here was about vector search..

Pandas - https://pandas.pydata.org/docs/

I found "Writing Without Bullshit" helpful. https://withoutbullshit.com/book

Apache arrow

Wrote 3 technical books [1] with asciidoctor, love it! Simple format, easy to include code, builds PDF, ePUB, mobi ...

[1] https://gumroad.com/tebeka

Inclusions are absolutely stellar, I know some people dismiss Asciidoctor because it feels neither here not there kinda lang. It amazes me how pliable the tool actually is (for Rubyist off course), there are some things there that would make org-mode aficionados and LaTeX die-harts envy.

Can you give some examples of what sets it apart?

I feels and works more like a real programming language while still having uncluttered syntax, you have variables, cases, etc. You can make your own plugins relatively easy. It's not in your way when you are writing.

Wrote too (small) books so far - Go Brain Teasers (https://gum.co/iIQT) - Python Brain Teasers (https://gum.co/iIQT)

Working on the 3rd ...

No serizalization is safe

- https://docs.microsoft.com/en-us/security-updates/securitybu... - https://en.wikipedia.org/wiki/Billion_laughs_attack - https://en.wikipedia.org/wiki/Zip_bomb - ...

None of those are serialization schemes. XML can be used for serialization, but if you look at the whole ecosystem it is a Turing-complete complexity monster, so of course it isn't safe.

The Knowledge Project https://fs.blog/the-knowledge-project/

I heard good things about http://vim.spf13.com/

Peter Norvig's Spell Checker http://norvig.com/spell-correct.html

A lot of other code he writes as well.

I really love this blog post by Norvig. Last Christmas I was looking for a trivial project to try Clojure and had a lot of fun working through this. Highly recommended - https://github.com/prakhar1989/clj-spellchecker

Particularly "Solving Every Sudoku Puzzle": http://www.norvig.com/sudoku.html

Also one of my favorites, and also like another reply to this comment I ported it[1] to a language I was learning, Golang.

1. https://github.com/montanaflynn/toy-spelling-corrector