Yeah, it's a balance. I love being able to help, and I am generally in favor of asking questions early, but not ones of the form "hey so I ran this code and it errored. Help?"
"... did you read the stack trace? Did you look at the code referenced by the stack trace?"
This is where I've learned responding with "Sure! What have you tried so far?" is relevant.
This is an interesting point, because they are kinda infinite spaces but they also impose a structure on the space, which I would assert is a part of why they are so successful, in contrast to the "infinite structureless blank paper" that the OP is talking about.
The trick is getting the amount of structure just right, not too much to be too restrictive and not so little that users are lost in the way the person you're replying to describes.
Infinite canvases require panning and zooming with the mouse and tracking motion visually. A lot of users probably don't like the cognitive load of that compared to other types of apps.
Excel handles this better because navigating around a large sheet feels more like "snapping" as there's no UI motion as your view zooms in and out. Plus with shortcuts like CTRL + rArrow, which immediately snaps your selected cell to the rightmost end of the current range of cells, the infinite canvas feels downright zippy. Excel sheets also have tabs that signal to the users about how they can split up their data instead of filling up Sheet1 with every scenario. Infinite canvases make you create a new file.
This is fair -- the newest token can attend perfectly to the oldest token, within the context window.
but also, on a broader scale, if a transformer model is presented with a long input that does not fit in its context (e.g.: you are building a chatbot, and you have a very long chat history), it must "compress" or "forget" some of that information (e.g.: repeatedly summarizing historical messages, dropping them and appending the summary at the beginning of the input).
Mamba/RWKV/other "recurrent" architectures, can theoretically operate on unbounded input lengths; they "forget" information from earlier tokens over time, but is that not comparable to what a transformer must do with input lengths greater than their context window?
I love this! Simple and solid execution. I've been wanting to build something similar for some time now, might fork and play around with it. Thank you for open sourcing it!
I've started using Obsidian with a new note for each day and separating "blocks" with a Markdown horizontal rule (`---`) to achieve something similar, but this is much cleaner.
The strength of such an approach is making capture extremely easy -- new block, start writing, no thinking about where this goes and how to fit it into pre-existing structure. I find that if I'm trying to do that, then by the time I find where my idea goes, I've lost the idea.
The downside, of course, is finding things again. The ability to tag or title a block and search by tag or title would be great. More ambitiously, it would be cool to experiment with incorporating LLMs and embeddings to automatically tag, summarize, categorize, cluster etc. your blocks.
There's a lot of different directions one could take this, but I'll echo the sentiment of others to refrain from adding too many features and losing the original appeal of simplicity. :)
Also: How do you handle performance when the buffer gets very large?
If you put any restrictions on usage or what can be done with it (like selling), then it's absolutely not open source.
open source doesn't mean source code is there. open source has a specific definition. There is a list of acceptable open source licenses, as defined by OSI. similarly there is a list of acceptable free software licenses, as defined by FSF. Broadly, the two lists are the same. Commons Clause is definitely not open source.
There exists a niche of commercial software developers who are actively attempting to water down the commonly accepted meaning of "open source" for their own gain, and I suspect they are voting you down. :(
I didn't put too much consideration into picking a license. If someone has compelling arguments for why I should license it differently, I would absolutely consider it.
Performance is mostly handled by CodeMirror (https://codemirror.net/), the underlying editor that Heynote is built upon. It seems to handle quite large buffers well. Where I have seen some minor performance issues is when working with very large blocks in certain language modes.
I use Obsidian for my programming notes, troubleshooting logs, thinking on “paper”, writing and checking assumptions. It’s very powerful and quite performant. AMA.
Have you looked into quantization? At 8-bit quantization, a 7B model requires ~7GB of RAM (plus a bit of overhead); at 4-bit, it would require around 3.5GB and fit entirely into the RAM you have. Quality of generation does degrade a bit the smaller you quantize, but not as much as you may think.
It seems like there's opportunity then in helping make regulations more legible/understandable, highlighting local differences, assisting with permitting and compliance, etc.? Seems like an area where LLMs could help out
You've probably already seen https://riffle.systems/essays/prelude/ - one of my favorite recent research projects in this space, it is very much in line with what you're describing. Sad that it didn't seem to go anywhere after a couple introductory posts.
Not affiliated, just wanted to pop in and say this team seems to really have caught lightning in a bottle. Their userbase is growing rapidly, they've onboarded notable public figures and Twitter power users, and it seems poised to really take over the mantle from Twitter. And the underlying protocol is really fascinating.
The team is active on the app and they are all good people who work hard and think deeply about this stuff, I'd highly recommend applying! (I'd apply myself but I'm a Python/data guy, which doesn't line up w/ advertised roles right now)
"... did you read the stack trace? Did you look at the code referenced by the stack trace?"
This is where I've learned responding with "Sure! What have you tried so far?" is relevant.