thank you, i appreciate it! yes, the software is api first, and much of it’s utility comes from working in other environments like google docs, ios shortcuts, etc..
in essence, the core of the project is a vector database that works for end users who want no fuss quick capture, semantic and fts search, the ability to create new relationships with marginalia.
im not trying to replace tools like obsidian or notion, i think ycb works better with them doing what they do best! i also plan to make the stack self hostable in the near future :)
I recently changed my thought process on the responsibility of readers and writers. I think it is the responsibility of the writer to say something interesting, and then publish said interesting thing somewhere where it can be reached by readers. I think it’s an element of the modern freneticism of the Internet that we expect instant eyeballs.
Old writing is constantly reinvigorated by new readers. In fact, that is the only way that it can continue to live. It is the responsibility of the reader to decide whether a piece of writing is important or not in their context.
As such, I’ve been blogging quite a bit. Because posting on a blog allows for the critical mass of future readers. Anything interesting that I happen to say can be found through a medium that reaches billions of people.
Today, or in the future.
HTTP is basically the printing press. And my website is my media company.
Just because billions of people aren’t consistently reading what’s written, doesn’t mean that the writer has lost their responsibility to write.
A method that has worked well for me: divorced databases.
The first database is a plaintext database that stores rows: id, data, and metadata and the second database is a vector database that stores id, embedding. whenever a new row is added the first database makes a POST request to the second database. The second database embeds the data and returns the id of its row. The first database uses that ID to store the plain text.
When searching, the second database is optimized for cosine sim with an HNSW index. It returns the IDs to the first database, which fetch the plaintext to return to the user.
The advantages of this are that the plaintext data can be A/B tested across multiple embedding models without affecting the source, and each database can be provisioned for a specific task. Also lowers hosting costs and security because there only needs to be one central vector database and small provisioned plaintext databases.
Post-co author here. This is actually something that we are considering implementing in future versions of pgai Vectorizer. You point the vectorizer at database A but tell it to create and store embeddings in database B. You can always do joins across the two databases with postgres FDWs and it would solve issues of load management if those are concerns. Neat idea and one on our radar!
From my last comment in August 2024 [1], I have made progress!
The project I'm developing is called Your Commonbase, a self organizing scrapbook built around Personal Library Science.
The big updates are:
1. From Fleeting Note to Connected Note with The Entry/Comment Model - An entry has its own marginalia (comments), which are also embedded as first class data. These comments allow your search model to improve over time, and create surprising clusters. See this video to see an example of the Entry/Comment model in a d3 graph [2]. All the connections are created automatically! Entries go from "fleeting entries" inbox (not linked or commented on) to "main entries" which spread and connect ideas from across the space.
2. Using yCb as a creator - I created a Google Docs extension and have been using it to create Zettelkasten's for my blogs. On average each blog references 12 or more books. This is a real use case of a PKM system outside doing it for the love of the game [3].
There's so much more including mobile upload, [[links]], audio upload, and more, but you can explore the Notion page in [2] to see the features I've added.
An active reading habit (of books) is one of the most reliable ways to achieve wisdom. Why?
- Books require a "Goldilocks" amount of sustained, slow effort given to one topic. Not too costly to pursue that it is a bodily harm risk, but not trivial either in terms of time and attention investment. This slows down the frenetic nature of a mind that latches onto heuristics.
- By being exposed to different well-formed ideas (especially if you disagree) widens perspective in ways lived experience can't. A reader is uniquely exposed to ideas from other times in human history, from people who are long gone.
- A reader develops a practical trust in themselves, particularly in that they know they take auto didacticism seriously, and won't shy away from intellectual challenges, which the world presents constantly. As such, they make better judgements.
People often think that being booksmart is having your head filled with useless facts, but this is almost opposite from reality.
> People often think that being booksmart is having your head filled with useless facts, but this is almost opposite from reality.
Indeed. Reading a lot of books does not mean things are retained by mind for a faithful recall. In fact, a lot of detail is lost during the course of reading and afterward. The things that stick are what relate to our experiences, beliefs or aspirations.
My personal hunch is that the more you read, the better your model of reality is. If you have an interest to a particular time/topic and you've read almost all you can find about it you have a better understanding of "something" from a different point of views. Like read all about a particular event in WW2 from all the sides and on different levels (military, economy, social, political). A term coined by Mortimer Adler is "Syntopical Reading", but by deeply exploring a simple concept you can build your knowledge on how the world works.
Several benefits from deeply knowing a limited subject:
- when you listen people who are "well known" talking stupid shit about a topic you understand that the info you're getting is not always that high and you need to check multiple sources (if you need to make a decision)
- you understand complexity better and "survival" bias
- you can even bet for fun on a prediction market when you see a stupid narrative getting puffed up (I've made around 1k after the CrowdStrike bet that it wasn't a "hack")
But the biggest win is to put more knowledge on your "tree" you've grown from a simple subject to related areas. This mental skill is applicable in almost all complex situations and this might be related to "the wisdom" you're talking about.
The downside is the more you know, the more cynical you get with your sources :)
I think reading is necessary but insufficient for wisdom.
Reading lets you acquire the experiences that you personally cannot live out in your own life. It does provide leverage in that way.
However syntopical reading isn’t sufficient to produce good work. Mortimer Adler himself tried doing it (I believe the Syntopicon was a product of syntopical reading) but those works never did well and were soon forgotten.
Books encode canonical knowledge and this works in established domains but in a domains without much existing knowledge, taking action in perturbing the world actually produces more useful knowledge.
This is why in these spaces it’s important to test ideas in reality and learn from them, rather than only constructing mental models. (We should strive to do both instead of the just latter)
Action often produces more information than reading alone. Having ideas is often the easy part — executing on them is the hard part. If Mortimer Adler actually had some gauge on the market before pursuing the very ambitious multi year effort of producing the Synopticon, he might have produced something noteworthy (which he did with the much smaller experiment “How to read a book.”
To me, wisdom isn’t about thinking rightly (which is what rationalists hold) but about being able to make good decisions with good judgment.
Reading is very important, but hardly sufficient. One problem with over reliance on books is that your world view will be molded by your reading list. You correctly pointed out the importance of reading material you disagree with, but there are often many contradictory ways to disagree with any specific world view.
Books must be mixed with a close relationship to people and the world. The world itself can be seen as the "ultimate book", thus the importance of the so-called "school of life".
I would never follow someone whose wisdom was built solely or mostly by reading books.
No education is complete, or even half complete, without a thorough and direct exposure to different cultures, countries and their people.
I think this makes sense, because, in general, wisdom requires gathering experience and reflecting on it (well in advance of any questions). Reading is an activity that promotes that.
Very fun game concept! I got stuck on level two. The difficulty is a bit too much, and it seems when i’m halfway through my pattern input a random haptic feedback happens from my phone, which throws me off/signals i’ve failed? iPhone 15 Pro Max/iOS 18
Yes! The way the app works is if you mess up then it has an error haptic that tells you messed up and plays the sequence again, and if you get it right then there is a clapping like haptic.
There is a help section if you click the ? at the top right of the screen when you first open the app (to see it again just close the app from the background and the re-open it, and you will see it again) It has a full tutorial explaining everything well
Thank you so much for downloading the app too! I have seen other comments about it being too hard too and thats something I will try to fix in an update! :)
So it gives the error haptic as soon as you mess up? I get into this weird loop where I mess up on the second tik, so it gives the error haptic and restarts the pattern while I'm still tapping the last three taps, so then my last few taps of the 1st try are used as the first taps of the retry, which is again immediately wrong, and I'm in a death spiral. :-)
Oh allow me to clarify. When you make a mistake, it plays an error haptic, and then repeats the sequence. As soon as you feel the error, there is no use in continuing to tap on the screen, because the entire sequence must be correct in order to move on. Therefore as soon as you feel the error haptic, you can stop tapping, and then focus on the sequence being played again to hopefully see why your timing was off.
I hope this helps! Once again visiting the help screen (?) at the top right when you launch the app after closing it from the background will give a in depth tutorial. Happy playing!!
Developer Hegemony [1] will help your thinking on how to price software in a world that traditionally rewards business people, as well as being pretty emotionally revealing about the trade we’re involved in.
If your blog provider supports it, adding a “Open a Random Post” button on your blog makes the experience much more fulfilling in the long term, as you (and others) can revisit different posts from different eras. Websites don’t have physical form that readers can navigate, so we can take advantage of that by adding serendipity manually.
For my own personal, non-technical blog that I have kept going since 2006, I added an "on this day" feature that shows posts for today's date (or closest matching) for past years. Collapsed version shows posts from 1, 3, 5 and 10 years ago; expanded version shows all 18 years. It's like a little time machine that gives me little gifts of past posts.
To be honest, the OP has it right. I write this blog for the same reason I did decades ago: it's fun for its own reason. I used to have hundreds of hits back then. Now I think the only reader is me and my one friend with an RSS reader. But one day if discovered by an LLM, maybe it'll scoop me in and I'll be one bit of a machine intelligence :D
The technique I find most useful is to implement a “linked list” strategy where a chunk has multiple pointers to the entry it is referenced by. This task is done manually, but the diversity of the ways you can reference a particular node go up dramatically.
Another way to look at it, comments. Imagine every comment under this post is a pointer back to the original post. Some will be close in distance, and others will be farther, due to the perception of the authors of the comments themselves. But if you assign each comment a “parent_id”, your access to the post multiplies.
You can see an example of this technique here [1]. I don’t attempt to mind read what the end user will query for, I simply let them tell me, and then index that as a pointer. There are only a finite number of options to represent a given object. But some representations are very, very, very far from the semantic meaning of the core object.
I also wrote a script that searches the web and works pretty well (using the vercel ai sdk)[1]
[0] - https://brave.com/search/api/
[1] - https://gist.github.com/bramses/41e90b27d156590154bcefd4119f...