At the risk of sabotaging stuff I've worked on, it's _astoundingly easy_ to glue together Y.js, prosemirror, and a little bit of express.js to get a distributed Notion clone with conflict free resolution.
Like I have something that I got working in 3 days with multiple pages, so links between things, and cursor syncing (most of this work is provided by wonderful people with full-featured libraries!). There's some trickiness with how Y.js actually works (in particular map ID conflict resolution meaning you don't want a dictionary of page names to pages but UUIDs and then some sort of manual revision checking....). But you can get that and probably with a bit more work get ~instant search and the like. All with free offline capabilities.
Of course then I stared at this and was like "I like notion because it looks nice" (I personally don't care about the AI stuff or the database stuff), so just went and paid for notion for my family usage.
fyi, from my experience it’s still quite a lot of work to build on top of Prosemirror and TipTap directly to do so.
So I’ve started working on https://www.blocknotejs.org to make this even easier for developers; its a batteries-included library with Notion style UI elements out of the box.
Definitely bullish on this ecosystem including Yjs and local first dev!
YousefED, love Blocknote, but I ended up not using it for a new project, just because it's too prescriptive. Which I totally get is a positive for many. I.e they want the UI and controls built in and off the shelf. I would love to just use the Core API (block etc) and get some useful hooks, so I can extend it in my own ways
Thanks for the feedback! It was designed from the philosophy that existing frameworks like PM or even TipTap are too low-level - but I can understand you need more ways to customize UI elements. It's certainly possible, but I think we need to document it better
I built on the same stack and have a similar opinion. It’s only easy if you’re depending on the webrtc implementation. If you’re coordinating changes across your own server, you gotta put in some work.
I did not rely on the webrtc implementation. Really you gotta do a lot of "echo server"-style shraing of messages and some sync messages from time to time but Yjs/CRDT stuff works pretty smoothly on this front.
Nice, yeah from what little I've done with it, I've learned that the difficulty can vary greatly depending on your requirements.
For me, I had a pre-existing note-taking app so there were tons of documents stored in html format. So I had this challenge where I had to read them from html, convert them to yjs, then sync the changes back to the DB in html format. It was kinda tough.
With Yjs, how do you persist data in a "normal" database?
And yeah, it may he easy for text editing (as pretty much all work is already done) but what if you have some other features where you want to get "live syncing"? It starts out easy but can become harder imo.
The core idea is that you are syncing a data structure, for example a map, so you can get a lot done with that (yjs’s presence stuff can be helpful for other live features).
In the end I have just been stuffing the data structure directly into postgres “as-is”. I think there are some tricks you can use to shrink some historical data but I think it’s pretty straightforward if you get to a size where the data size matters
The landing page they linked suggests it’s possible, but makes it sound like it might be more work to do so:
> BlockNote is built for React and exposes easy-to-use components. However, it is possible to use BlockNote with other frameworks or Vanilla JS (advanced users).
blocknote looks very nice! I am unsure why I didn't take this as an option earlier, as I think I looked at it. I will check it out.
FWIW I picked up prosemirror because I have experience working with CodeMirror 6, and I trust the general model to be right, if complicated to use. But honestly I might not need the level of complications provided by PM.
Would love to see a demo and/or the code if you were open to open-sourcing it! I'm sure there's someone out there who'd be willing to contribute to the styling.
I've found reliable persistence on the backend irritating with yjs. Seems like the official path is to fork their example library and edit it. (The example is insufficient because, for example, it will silently eat data if the onchange webhook fails).
yrs initially looks tempting but it's unsound at it's core. (The plan is to port the API directly from JS, use unsafe to silence the borrow checker, then gradually fix specific instances of undefined behavior if they cause actual real world issues.[1] I don't this this is an approach that can work. That's a shame because a lot of impressive work has gone into yrs.)
aesthetics are very nice, and the slash menu mechanism + syncing make it nice when working with people. If Google docs looked nice this convo would be different but Google docs doesn’t look nice!
Aesthetic aside, I really admire Notion’s extensibility and rich API ecosystem — it’s given rise to website builders like https://super.so and https://potion.so, something that I haven’t seen with Google Docs/One Note
First of all, apologies for the typo lol, apparently I can't spell "WYSIWYG" :')
Some context: I've always been a huge fan of Notion's editor, and since Vercel launched the AI SDK yesterday (https://sdk.vercel.ai/docs), I decided to take this as an opportunity to recreate Notion's block editor + incorporate some of Notion's latest AI features with `npm i ai`
Here's basically how it works:
1. Tiptap Editor setup with custom slash command extension
2. `useCompletion` hook at the parent editor + the slash command menu – using `id` to share states
3. `/api/generate` route to stream responses from OpenAI
Novel also handles image uploads via drag & drop / copy & paste. Images are stored as base64 strings in localStorage currently, but I'm planning to implement uploads to Vercel Blob in the future.
If anyone wants to try this in their own env, I published a docker image on dockerhub. `docker run -p 3000:3000 keeb/novel`.
To generate the image, I simply cloned the provided github URL and used the nextjs Dockerfile[1] and removed all of the cruft. The Dockerfile is available here[2].
Tangent: On the note of web based text field for "normal users", what's the best UX you can get while also trying to be as no-JS-friendly as possible? Are there CSS-only tricks to make editing in an input field nice?
I'm making a low-JS site, but it also has a lot of input fields, user content generation, etc. In "modern CSS" i feel like Input boxes should be better than they are. Is JS the only way?
I would like to add: If the GP goes for any JS based solution, they should make sure to serve the JS themselves, and not outsource it to 3 CDNs. For us not liking JS too much people, the next best thing to no JS is little JS server by the website's server itself. Best with most of the website already rendering without JS, so that we see things when hitting the website and may consider it worth allowing its first party JS. Being able to see the website and get what it will use JS for goes a long way building trust.
I do plan on self-hosting all (the very select few i use) JS and CSS. It's part of the binary for dead simple self-hosting. I'm going to attempt to be very privacy friendly, as much as possible, which will be an interesting adventure because it's ActivityPub related.
Regardless this is all just an exercise for me because, frankly, i hate most websites lol. The ones that look good also tend to be slow. The ones that are fast look like they're from 1990. There has to be a middleground, right?
I love that feeling of visiting some (very few) sites and they load so fast you have to double-take. I want to be that site. I want accessibility to be forefront. etc
We'll see how far i can get. I'm just a humble backend dev hah. But the site being 95% plain HTML is a good start, at the very least.
Here’s a comparison for you because I just did the same.
Old site.
- Built with Gatsby, hosted on Netlify.
- Lots of client-side JS, but page loads are instantaneous because everything is prefetched and a page load is a client-side JS event not a server request.
content-editable is a crap shoot without some JS layer on top of it normalized the output. Otherwise you end up with the same visual output being able to defined as 50 different inputs.
The other thing is content editable allows all kinds of garbage to be pasted into it included any kind of inline style.
Now I maintain a fork of Htmlawed which can do a pretty decent job of filtering down arbitrary HTML into a safe and restricted subset but with no JS there can be a huge disconnect between what the user sees in their editor and what appears as the sanitized output.
As a result I’m currently using SlateJS as a rich text editor foundation. I’ve used others in the past but this is the nicest so far.
Rich text editing is probably not possible or not worth it.
However, most people who disable JS are probably fine with not using rich text and definitely shouldn’t expect it. Just add a CSS stylesheet which turns <textarea> into something half-decent, put a note that markdown is supported, and make a plain text editor.
> Just add a CSS stylesheet which turns <textarea> into something half-decent, put a note that markdown is supported, and make a plain text editor.
Any examples that you can think of which make a plain text area half-decent? So many sites go with heavy-JS options that i'm not sure what is possible for a good (half decent) visual editing experience in just CSS
It's bit harder to do correctly manually just because of cross-platform browser quirks, you'll probably at least want normalize.css or CSS Reset. Even MDN docs has a half-decent textarea in 20 lines of CSS: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/te... (it looks ugly with monospace but change the font and I think it looks great)
As much as possible, the specifics matter i think, so i'll weigh what script(s) offer in terms of UX compared to load time/bandwidth/etc. In general i'm just trying to stay super lightweight, only giving in when necessary.
Also, focusing on CSS first means - hopefully - that the UX is "as good as it gets" for JS-less individuals. If i add a small JS framework on-top of that it's likely fine.
edit: I'm sort of experimenting with how good i can make modern websites. Both on UX, accessibility, low bandwidth (folks on satellite connections appreciate this), etc. I'd like the site to "look like any modern site" (minus obtrusive crap), but suffer as few flaws as possible.
> Just add a CSS stylesheet which turns <textarea> into something half-decent, put a note that markdown is supported, and make a plain text editor.
If you're not going with prosemirror or similar off-the-shelf component, then this is the only way to not annoy your users. Augment it with a side-b-y-side RO view of the rendered markdown.
I mean, even the linked page is a confusing UI: you enter `**` for a bold sentence, and then when you arrow-left to the beginning of the sentence you cannot delete just the `*` you typed in.[1]
[1] I feel a better way to do this would be to not allow markdown to actually be typed in, so that the user never sees it. This sidesteps the confusion when trying to remove just the markdown. Let the user use shortcut keys and/or buttons on selections, because to remove a formatting rule they simply repeat what they did to create that rule.
This. I wish websites had this as a fallback mode, when I do not allow their 5 third party script sources. I also wish this was done in products like Jira (basically anything out of Atlassian), which have very bad WYSIWYG kind of editors.
Not quite sure what you mean by normal. But if you want a nice input field - as in it allows you to input text and does what you expect with plain text, something like water.css maybe. If you want rich text, the Web was designed to offload this to js. Markdown and other similar systems were created to provide an alternative to this, but there's a barrier to entry because you want more than text. Text plus markup requires interpretation and either a ux that adds tooling or the user to do some learning.
I just mean .. average users. Where a text box has some very limited features that may be thought of as the baseline, the standard, etc. Not even sure what qualifies as that, but i imagine a lot of users expect to be able to highlight text and make it bold, italics, etc.
I love markdown and want to support it as the primary interface.. but i don't want to solely force it. I'd love a really, really thin layer of JS which just gives in-line highlighting for markdown, for example.
I might try to make it honestly, if i can't find it. But that's a long ways down the road.
No real issues i think, but take this HN input box i'm typing in now. It's .. okay, like.. it's about as barebones as you can get. CSS has seen a ton of advancements in just about every area (it feels like atleast. As a backend dev), but yet this HN box still feels them same as it did in 1995 heh.
I see so many huge JS beasts with WYSIWYG features that i'm unsure what advancements, if any, have been made for CSS to provide something nice. Something that a user would feel it's more than nothing done at all.
The HN box feels like nothing at all. Just a plain HTML input text area.
Pure HTML document editor. Open source, PWA, based on TinyMCE, with CSS and markup editing, JS macros, saves to local drive, plus OpenAI integration if you have a key.
- I was about to tell you that there really should be a demo, but just now I realized the "World Wide Web Phase 2 Begins Here" button links to the demo. You really should change that text to "Live Demo" or something more clear.
- The demo itself takes a long time to load and seemed to refresh multiple times
- The GitHub should also have a link to the demo, and an image would be nice as well
- The landing page is very ugly. Any generic template would do better and inspire more confidence.
- Why not highlight some of those features you mentioned on the landing page?
Since we're shilling projects, this is the Notion clone I've been working on: https://mindgarden.app/. It's not ready yet though, hopefully will be in a few weeks.
Are there any examples of something similar that has the ability to seamlessly interact with tables as well?
Notion seems to be the only player in the space that effectively uses its tables as linkable databases, most similar products seem to either lack table functionality or they're just simple formatted text.
So much of my daily work revolves around databases and Excel, I would love the ability to have multiple views of database tables embedded in a text editor, varied by context, with full backend control over the database itself. Bonus points if I could easily have scripts run on them in the background, almost like a programmatic kind of Excel.
Have you heard of Obsidian? It's a note-taking app build on locally stored markdown files with bidirectional linking and a great ecosystem of third party plugins. One of the most popular plugins is https://github.com/blacksmithgu/obsidian-dataview which lets you treat your notes as databases and query them to form tables. The creator has been working on its successor, Datacore https://github.com/blacksmithgu/datacore for a while - Datacore might come close to what you're looking for, its goals include WYSIWYG views and live editing inside tables.
I have used Obsidian before, and the lack of any excel-like functionality (Dataview just wasn't enough) was what drove me to Notion. I'll keep an eye on Datacore though, it does seem like it might be able to achieve enough of what I'm looking to do.
Logseq mostly does this, I think. Rather than starting with a table and adding objects to it, you start with the objects in your database — pages and blocks — and then form tables by querying your database for those objects based on their attributes (tags, contents, etc).
> What I miss (but not enough to change) is the ability to have a plain Markdown page, and not only rely on bullets.
This. So much this. I prefer Logseq's UI over Obsidian's but the lack of support for plain (non-outline) text files made me switch to Obsidian in the end.
Not the answer anyone was expecting, but SharePoint Online. Wiki pages, every list you create is a SQL table with a robust API, and the power platform is one of the best no/low code solutions available. I create full react applications around this and host the SPA in a SharePoint document library.
... I'm not sure if this is the greatest idea I've ever heard, or if I want to immediately run away screaming.
I've always thought SharePoint was terribly slow, but that was just from experience interacting with the web interface itself. Does the API perform well? Or is there that same inherent latency to put up with?
And do you have any general tips/ideas/learnings to share about using SharePoint in this way? I definitely want to give this a try.
The first step might actually be a non-starter for most people. Notion is free, SharePoint is usually offered as a capability within a business or school's Office 365 subscription. I believe you can either subscribe yourself for $5.00 / month using this link: https://www.microsoft.com/en-us/microsoft-365/sharepoint/com..., or maybe you're lucky and already have an account with your school / work. If neither of these work for you, you can also sign up for the Microsoft Developer Program, which is actually pretty cool; here is a guide for setting up a SharePoint site through the developer program: https://www.sharepointdiary.com/2021/05/how-to-create-sharep...
I should take a moment to clarify that I'm not affiliated with Microsoft or any of the links I'm posting here in any way. I make a living building web sites and as mentioned above, many of them leverage SharePoint. I find it a pretty easy way to build intranet solutions for my customers.
Assuming you've got your SharePoint site created within SharePoint Online, creating a "List" is pretty simple. Here's an article from Microsoft explaining how to do so:
https://support.microsoft.com/en-us/office/create-a-list-0d3.... Think of a list as a database table. From there you can create various types of columns including "look-up" columns, which are basically joining to another table on the same site with a foreign key.
I think from there you could dive into the Power Platform, but there's so much out there, I think this would be a good start.
There is a project called “AnyType” that is real and i use it but it’s “open source promised soon”. The gist of it is a Person knowledge management system except the items are in a linked graph (and the relationships are typed).
It actually has an editor that looks eerily like the linked one so I wonder if they’re related. It lets you use slash-commands that’s much more expressive than everything except maybe notion. But again, it lets you operate against a database-like set of objects.
I don’t think you can point it at real databases though.
It has far more data power than Coda, with tables that can be used as databases, a strong formula language, and loads of other nifty features. Like Notion, it's a closed-source SaaS.
> Novel is not just your average WYSIWYG editor; it takes things to a whole new level with its AI-powered autocompletions. With the seamless integration of Tiptap, OpenAI, and the Vercel AI SDK, it can fuck up your prose in ways you never thought possible. Imagine effortlessly composing sentences that flow with elegance and precision, as if written by a seasoned wordsmith. Novel's AI-powered autocompletions analyze your writing style, context, and even the tone you want to convey if you're a complete idiot. Novel's autocompletions are designed to enhance your writing experience by seamlessly integrating AI technology. Whether you're a professional writer or a beginner, Novel's intuitive interface and powerful features make it easy to compose engaging and polished content. With the combined bullshittery of AI and your own incompetence, you can now create literary disasters that will leave readers scratching their heads in confusion. But hey, at least it's all in the name of innovation, right?
This project is not open source but instead uses the “Business Source License”. I only mention this in case open source is something that is important to you.
I hadn't heard of the “Business Source License”, but reading it makes it seem pretty scary. Basically any usage of the software that could be considered production is prohibited, which probably could even mean personal usage in a non-development fashion.
I’ve basically completely stopped using notion after they crammed “AI” in there with absolutely no option to turn it off without a business tier plan (even more $$$ than the personal). If I want to use chat gpt I’m gonna open that in a completely separate window and interact with it there, I cannot stand LLMs being unceremoniously shoved into seemingly every pre-existing tool I use.
For a text editor like this a base LLM would be more appropriate, one that does just autocomplete, without being instruction tuned. Unfortunately these base models are either unavailable (GPT-4), outdated (GPT-3/davinci) or otherwise restricted (GPT-3.5 aka code-davinci-002 is only available via Microsoft Azure and more expensive than the instruction tuned GPT-3.5-turbo model).
divinci-003 is outdated? I actually switched back to it after trying gpt4 for a while because I thought it was too assistant-y. I have a switch to use gpt4 when I feel like I need it to be smarter but I rarely reach for it.
There is no model called "davinci-003". I was talking about "davinci", aka GPT-3 175B. You are probably talking about "text-davinci-003", which is an instruction tuned version of the base GPT-3.5 ("code-davinci-002").
This is a super cool project! And I love the super-simple Vercel deployment.
I wasn't able to get the AI completions to work with my OpenAI API key; not sure why.
I don't completely understand how I'm supposed to use this as a stand-alone deployment if I can't create more than one page. I think the answer is just that the project is young, but I'm not certain.
Anyone has an alternative where I can use simple markdown files (in a folder or google drive) as the primary datasource?
I am using obsidian on the Mac, but on the go I have no real solution. (I want to avoid using the obsidian mobile app because it forces you to use Obsidian Sync)
You need to use Obsidian Sync if you have an iPhone. IIRC it’s because the third-party storage APIs are limited on iOS in such a way that makes it non-feasible for Obsidian. Something about bulk editing/access.
"I am an AI writing assistant that continues existing text based on context from prior text. Give more weight/priority to the later characters than the beginning ones."
I think the nature/feel of the quick response is likely due to the fact the completion call is streaming "stream: true" [1]. Then, they are parsing it back to text as it streams the response back.
I'd have been more surprised if there wasn't streaming. All the LLMs I have ever used, stream tokens in real-time. I'm talking about the time spent per token.
Tiptap, of which Novel.sh is built on top of, has cross framework support I believe (React, Vue)! Novel is fully open-source so feel free to check out the code and plug and play into your framework of choice: http://novel.sh/github
WYSYWIG does indeed mean "What You See Is What You Get."
A WYSIWYG editor is an editor that shows the text with all formatting, exactly how it will look once it's "printed". You know, like Word or Google Docs. But unlike when you edit markdown or latex in a plain text editor. For example, writing comments on this site isn't WYSIWYG.
‘<<>>’ indicates where I typed ‘++’ and the resultant text is unedited.
It seems as though the AI response became “stuck” on a theme, and my renewed prompts were insufficient to cause it to change / reconsider context. . .
_____
Novel is a Notion-style WYSYWIG editor with AI-powered autocompletions.
what backend is used for the autocompletions in Novel?
The auto completions in Novel are provided by <<>> The auto completions in Novel are provided by a sophisticated backend powered by state-of-the-art AI algorithms.
These start-of-the-art AI algorithms rely on underlying technology from <<>> leading natural language processing (NLP) frameworks such as OpenAI's GPT-3 and BERT. These frameworks have been trained on vast amounts of text data to have a deep understanding of language patterns and context. By leveraging the power of thesed on massive amounts of text data and are capable of generating high-quality and contextually relevant suggestions for autocompletions. The backend of Novel leverages the power of these advanced NLP frameworks to enhance the user experience and provide accurate and helpful suggestions
Novel is funded by <<>> d on vast amounts of text data to understand and generate human-like text. The backend infrastructure of Novel leverages the power of these pretrained models to provide accurate and contextually relevant autocompletions. The sophisticated algorithms in the backend analyze the input text
How is Nobel funded <<>> d on vast amounts of text data and have achieved impressive results in various NLP tasks. The combination of GPT-3 and BERT allows Novel to generate highly accurate and contextually relevant autocompletions for users. The sophisticated backend continuously learns
what does novel cos <<>> d on vast amounts of text data to generate high-quality autocompletions. The sophisticated backend in Novel leverages the power of GPT-3 and BERT, combining their strengths to deliver accurate and contextually relevant suggestions. With GPT-
It appears to be <<>> d on vast amounts of text data to generate highly accurate and contextually relevant autocompletions. The backend infrastructure of Novel leverages the power of these NLP frameworks to process and analyze user input, allowing it to generate intelligent suggestions in real-time
What if I want to write about <<>> d on vast amounts of text data, allowing them to generate accurate and contextually relevant suggestions. The backend of Novel utilizes a combination of these NLP frameworks to enhance the autocompletion capabilities. With GPT-3, the system can generate coherent
start a new topic <<>> d on vast amounts of text data and have the ability to generate coherent and contextually relevant suggestions. The combination of GPT-3 and BERT allows Novel to offer a highly intelligent and accurate autocompletion feature to its users. The backend architecture
_____
This is pretty slick. Unobtrusive prompt and then you get a pretty intrusive drop-down. Pretty it up. It has a wonderful response and flow and I would be interested in using this in future projects.
Like I have something that I got working in 3 days with multiple pages, so links between things, and cursor syncing (most of this work is provided by wonderful people with full-featured libraries!). There's some trickiness with how Y.js actually works (in particular map ID conflict resolution meaning you don't want a dictionary of page names to pages but UUIDs and then some sort of manual revision checking....). But you can get that and probably with a bit more work get ~instant search and the like. All with free offline capabilities.
Of course then I stared at this and was like "I like notion because it looks nice" (I personally don't care about the AI stuff or the database stuff), so just went and paid for notion for my family usage.