1. You have given up on any sort of consistency.
2. The database implementation will never be able to optimize a lookup workload which is pretty close to pessimal. Not caring about 99% is only viable if can actually avoid the other 1%.
what is a document? How is an ORANUM or a bignum not a document?
One motivation for creating documents is that modeling document contents as relations requires the creation of a bunch of primary keys which no natural definition. A simple document might be an ordered collection of paragraphs, [p23, p57, ...]
Modifying such things is difficult. In fact, the most effective way of structuring modification seems to be OTs based on document offsets. What Google docs does.
Been doing this for nearly 40 years. No sign of doom. What I think you're discovering is just that there is a large amount of low-aspiration work out there. That is as true of FANG as anywhere else.
IMO specializing in any particular bit of trendy tech. is a recipe for disaster. Trends come and go. First-principles CS and being able to communicate it continues to be extremely valuable and satisfying.
PathQuery was started by Warren Harris. At Metaweb, Warren did a rewrite of MQL in OCaml which we never shipped because of the Google acquisition. He spent much of the first year at Google working on a prototype of PathQuery in Haskell. The prototype was rewritten in C++ and used in the first KG-serving infrastructure sometime around 2012/2013.
I am a huge fan of Frank McSherry's work and don't necessarily agree with the premise that DD is somehow failing. However,...
Batch data processing is very well understood, cheap and getting cheaper every year. So, if you can afford to boil the ocean every night, DD is a tough sell.
The addressable market, customers with problems which can only be solved with DD (instantaneous exactly correct answers) is probably small right now.
>Deep reading has in large part informed our development as humans
As a statistic, reading is only significant in the past 200 years or so. Given that there was a similar kerfuffle a couple thousand years ago when we shifted from memorizing to writing things down, I doubt that this is too significant.
Literacy is also shifting. Growing up with memes and Instagram and Tiktok has produced a generation that is vastly more literate in design, photography and film.
>I can't wait until I have enough money to be financially independent and never have to do this nonsense again.
There are a lot of people operating on that plan. Mostly it doesn't work out for them.
Play this forward ten years with a family to support and a mortgage which committed you to personal inflation and a track record of ...? You can absolutely fool people for a week or a year, but a decade of non-experience will stick out like a sore thumb. Experience compounds, and the one you describe is compounding down. Do this for ten years and you will be much less accomplished than you could be.
> Play this forward ten years with a family to support and a mortgage which committed you to personal inflation and a track record of ...?
Being an excellent employee?
I understand your point, but I actually have been doing this for about 10 years now, and I consistently have improved my "career trajectory" by switching jobs when I start to feel that my coworkers are overachieving and throwing me under the bus. I look at it like a forcing function to switch jobs and get a raise.
I already had a decade of pre-work experience just from being a lifelong hacker, so it's not like I'm relying on my job to learn new technology. I don't have a fear of falling behind. If anything, I've noticed a huge decrease in SWE competence over my time in industry.
Some of my most "successful" friends at FAANGs are bad developers that just devote more time to playing the system than I am willing to do. I find the social con-man stuff to be really draining and I hate doing it. I also hate interviewing, which limited my career more than I'd care to admit.
Also, I've opted to not support a family or carry a mortgage. I'm doing something similar to FIRE. I'm hoping to be FI with a total of 15 years as a tech employee.
"But do you really hate working so much that you would sacrifice having children!?". Yes. Very yes.
It's also likely that tech workers are "overpaid" right now, as we're compartmentalizing more work, lowering the skills required to join tech companies, and remote work is now the norm. It's a race to the bottom, and I want out as soon as possible.
Concurrency is hard because we invented a bunch of theoretical machinery (semaphores, mutexes, condition variables,...) and embedded it in languages such that most of the programs one can express are bad: racey or deadlock prone. Compare with modern sequential languages which have near-perfect denotational semantics: anything which compiles means something.
If you want practical guidance, my best is to avoid all of the concurrency constructs that are taught in schools. Use log-structured single-writer multiple reader, wait-free, processes and shared memory.
A large company in actual financial distress will sell off a division.