I wrote the CatBench vector search playground toy app exactly for this reason! [1] ("cat-similarity search for recommendation engines and cat-fraud detection"). I built it both for learning & fun, but also it's useful for demoing vector search functionality, plugged in to regular RDBMS application schemas in business context. I used cats & dogs as it's something everyone understands, instead of diving deep into some narrow industry vertical specific use case.
Pretty cool. At some point I wrote my own routines with pure DuckDB SQL, so I wouldn't have to import UDFs, it would all be in just SQL files. DuckDB already has a basic `bar()` function for bar charts [1] and also you could use YouPlot [2] if you prefer just piping the output to some other command instead of using a library or extension. I created my own bar charts though, as I wanted to use the "partial" unicode block characters for more fine-grained bar charts.
I created my own sparklines too with LISTAGG and unicode characters for rendering latency histograms, but the bigger problem there was that some latency buckets didn't have any data falling into them. That missing row of data still needs to be rendered as an empty bucket. So while the pure SQL approach worked, it was a convoluted query where I used an outer join with a list of synthetically generated latency buckets to align the rest of the buckets right.
That was a prototype anyway, later on I switched to returning the filled bucket numbers (latency:count) with LISTAGG as CSV that I later parsed and rendered in the app layer [3]
Nice flashback to the '90s... Back in 1999, I worked with a 3 TB sized Oracle DB that was supposedly the biggest Oracle database in Europe on Windows! 32-bit Windows NT 4.0 running on a machine with 16 GB of RAM, so we had to use Oracle/Windows very large memory hacks to "swap" physical pages in to the 32-bit address space as needed...
Edit: Yes, we had blue screens every couple of weeks
The MIT Athena computer labs while I was there were 80% SPARC and 20% SGI. Despite whatever porting frameworks Microsoft had to glom on, IE for Solaris was still better than the pig Netscape was at that point.
The late 90's and early 2000's were a dark time for browsers. IE was basically the only game in town. Netscape was approaching unusable. I remember it having massive memory leaks and segfaulting every half hour or so on Solaris. Firefox wasn't out yet. Chrome was 5+ years away.
The latest version uses modern eBPF/libbpf/Rust-based blazesym and in basic mode does not inject any tracepoints or probes onto the critical path of other threads in the systems (such passive sampling is achieved using sleepable eBPF programs, task iterators, etc).
Google's (Postgres-compatible) AlloyDB Omni also has similar functionality now - the main DB action, persistence, etc still has to land on persistent block storage, but additional data can be cached for reading on local NVMe disks.
Oracle's Exadata is a whole another beast (I co-authored a book about it back in 2010 and it has improved even further since then). It's a hybrid, shared storage distributed system - not consensus-based replication (although they support RAFT for global data systems now too), but a distributed, coherent buffer cache (global cache) based database system. As it's shared storage, you can write copies of blocks, WAL to multiple separate storage cells (NVMe or even just remote RAM) via direct RDMA operations, without OS kernel or system calls involved.
For analytic queries, yep Oracle can push down filtering, column projection, many aggregations and join filters (bloom filters) for early filtering into the storage cells. The bloom filters are used for early filtering of the next table in the join, based on the output of the previous query plan nodes so far.
On Linux this is not true, on an IO heavy system - with lots of synchronous I/Os done concurrently by many threads - your load average may be well over the number of CPUs, without having a CPU shortage. Say, you have 16 CPUs, load avg is 20, but only 10 threads out of 20 are in Runnable (R) mode on average, and the other 10 are in Uninterruptible sleep (D) mode. You don't have a CPU shortage in this case.
Note that synchronous I/O completion checks for previously submitted asynchronous I/Os (both with libaio and io_uring) do not contribute to system load as they sleep in the interruptible sleep (S) mode.
That's why I tend to break down the system load (demand) by the sleep type, system call and wchan/kernel stack location when possible. I've written about the techniques and one extreme scenario ("system load in thousands, little CPU usage") here:
FWIW, I've used VMWare Fusion on Mac OS for years (and their Workstation product on Linux/Windows since 1999). It has worked well for me so far. VirtualBox used to be not great on MacOS with multiple CPUs and lots of interrupts - I noticed problems when moving from a Mac laptop with a spinning HDD to NVMe SSDs. The slow inter-processor interrupts (IPIs) and some related contention caused concurrent IOs to get very slow with fast SSDs with VirtualBox (back when still running on an Intel Mac with VMWare kernel module). In fact, I could get more IOPS out of the VM with fio when giving just 1-2 vCPUs to the VM, vs 4-8 vCPUs.
I haven't tested VirtuaBox on latest Mac versions as I'm happy with VMWare Fusion (also on ARM M1/M2 machines). The product is still good, regardless of Broadcom's business shenanigans.
Edit: If Parallels works for you, but only the SSH terminal sessions get stuck, maybe the "shared network" connection drops your connections for some reason. Before diving too deep into troubleshooting this across layers, you could try out the usual suspects workaround, setting SSH TCP keepalive. I have this for some connections in my ~/.ssh/config file:
I found this article interesting (in fact, posted it earlier, but it didn't get traction then). I think some context is needed: When you operate at eBPF/kernel level you don't get easy direct access to the higher level goodies, like various container metadata (other than perhaps the cgroup id/name). So with eBPF you extract various numbers and IDs and then use userspace code+services to retrieve the meaningful (human-readable) context and strings using these IDs.
A plain Linux example would be that eBPF will only give you user/group IDs (uid/gid), not usernames, so you need to use post-processing in userspace code to convert these IDs into something meaningful.
"3I/ATLAS formed in another star system and was somehow ejected into interstellar space, which is the space between the stars. For millions or even billions of years, it has drifted until it recently arrived at our solar system."
[1]: https://tanelpoder.com/posts/catbench-vector-search-query-th...