I recently ran an experiment with Node and was able to take writes from around 750 / second to around 24K / second just by coordinating them using Node IPC. That is, I had the main thread own the sole write connection and all other threads sent their writes operations to it, thread-local connections were read-only.
It's pretty cool how far you can push SQLite and it just keeps humming right along.
My go-to method of inter-process communication in Node without using any third party modules is to just use process.send[1] to a send a message to the main thread, and then have it forward messages to the workers in the cluster, which you can listen for using the message event [2].
Using cluster and worker threads. The code is here, but completely undocumented, as I’ve been side tracked by a few other things[0]. I’m currently looking into porting that project to Bun, so it’s possibly dead in its current form.
I was planning on turning that into a library, but Bun nerd sniped me.
You need the “hey” tool to run the benchmarks[1] the way that I was running them.
I recently ran an experiment with Node and was able to take writes from around 750 / second to around 24K / second just by coordinating them using Node IPC. That is, I had the main thread own the sole write connection and all other threads sent their writes operations to it, thread-local connections were read-only.
It's pretty cool how far you can push SQLite and it just keeps humming right along.