Unless I’m missing something, Etherpad is just for text editing. Not arbitrary application data like we’re working on. And each etherpad document lives on a centralised server. At least, that was true when I worked with the original etherpad devs in 2011. I can’t find a roadmap for 2023.
Replica will work with native apps first. Data will be local to your devices. Servers are necessary for peer discovery, sync and delivery. But you can still make changes offline and sync later. Or ideally, sync directly between your devices over a local network.
Think about it like GitHub. You can use their website, but the data itself (in git) is cloned onto your local computer - and you can replicate it to other devices however you like. The model for replica will be similar: We’ll run an official website to use for replication, backup and discovery. But your data is, and always will be your own.
I don't see anything in your current page that can't be layered into the e-mail model, and my prediction is that this will never get serious traction for that reason, because anything people want from it will just be integrated into e-mail if it starts to get attention.
Consider that SMTP was "shoehorned into" the email model, and so was HTML, and attachments. My first e-mail exchange involved UUCP rather than SMTP. For many years my e-mail clients did not understand this newfangled HTML thing. E-mail is extremely malleable because e-mail is extremely simple.
Add a "To: " header to a text file and you have a basic e-mail. Copy the file to another machine and you have a mail transport (that's roughly how UUCP worked - by implementing basic file transfer between hosts, and layering distribution of that mail on top with a program that'd read a file of concatenated e-mail messages and distribute them to mailboxes).
E-mail is at its core a simple key-value header format, a message body and an address. Everything else is auxiliary. Transport layers can coexist: UUCP coexisted with SMTP for many years. If you invent a new one you just need to decide if you care about having a gateway to let "outsiders" contact you. Message formats can evolve: As long as you provide the basic MIME headers, you're in full control of the format of the message past the first set of headers. You choose how much interop you want by whether or not you use multipart/alternative to provide version readable by "old style" clients.
These features are why e-mail in its current form is still around.
What you gain* from this is interop and an opportunity to have your system evangelise itself as you use it.
To address your page more specifically:
>+ choose the sites where you participate
>+ select which members of a site can message you
If you enforce DMARC (see below), you can do this in a mail server simply with an option to whitelist domains or users or both.
>+ always know from which site a message originated
This is supported in e-mail today via DMARC. The reason you still receive e-mail for which it's not the case is lack of universal deployment. You can solve this issue simply by configuring a mail server to reject all e-mail that fails DMARC.
> + can block anyone with whom you’ve made contact
> + may leave a site and never see traffic from it again
If you enforce DMARC, you can certainly block sites at the server level or client level or both, or just block individuals.
The upside of doing this with DMARC is that people with it correctly set up will only notice anything different if they try to send you unsolicited messages and get an error message back.
> + message formatting & layout via Markdown (aka CommonMark)
Solved with client support - just needs the appropriate MIME type set and a ideally a multipart/alternative to deal with any clients that fail to fall back on plain text rendering.
> + forms/surveys whose results are collected into tables
Needs server side support if you want something standardised, but works fine with links to the web today with appropriate client support and HTML, and if you want to provide an upgradeable path it's easily done with client support and a MIME type + requiring multipart alternative w/ a link to a web version for clients without support.
So to summarise how I would have done this if I wanted to meet the goals stated on your page:
* Pick a mail server that supports JMAP for the client.
* Configure it to reject all mail that fails DMARC with a suitable message explaining why. Now you've created pressure to add DMARC for those who don't.
* Add a JMAP profile for the client to tell the server if it wants whitelisting or blacklisting, and which users and domains it wants to whitelist and blacklist. I'll tell you right away that most users will reject a whitelisting option, so consider greylisting or quarantine as options.
* Optionally define a MIME type to carry subscription/follow/unfollow requests to remote sites, and a DNS record to hold public keys to sign requests.
* Define your desired message format and pick a MIME type.
* Specify the rendering of a plain text version of the message formats for delivery via multipart/alternative, which includes a footer that specifies that this is a possibly imprecise rendering and links to an info page.
Using JMAP means people would be able to connect with any JMAP capable client and "just" lose out on functionality not supported by that client. Using a mail server means most well configured mail servers can already talk to you if whitelisted/not blacklisted. Using multipart/alternative that way means clients can "upgrade" gradually and possibly after users see a link to an info page, and the footer in the plain text parts (or other rendered alternative) would help evangelise the system.
(Author here.) Thanks for your interest & support!
Yes, the default concept for an Internet messaging service is "everyone can reach anyone". This isn't helpful for online services and businesses who need to reach their members/customers directly & securely, and with a rich content model (e.g. survey/form in message, and structured result in reply).
(Author here.) Development has been paused for ~1y as I've been at work creating a vehicle to take this forward.
The protocol specifics (e.g. identifiers & types) aren't polished, because I focused on features in working code. Thanks for the feedback!
You can't have alias collisions; it checks whether the string you propose is acceptable & available :)
A sender's alias is (or will be) checked against their recorded aliases.
The reason why lack of E2EE in email is a serious problem is that email traverses at least two "federated" services, which are typically third parties with whom you have no intrinsic need for a relationship. A TMTP message passes through a single service which you explicitly joined.
I don't see how your feedback justifies "poorly thought out"; it's merely an unpolished preview.
>I don't see how your feedback justifies "poorly thought out"; it's merely an unpolished preview.
I don't think they meant this with sincerity. It's unpolished, which is accurate, but there's promise there, and I think their frustration was the last 20%, which is often overlooked- the cohesion of marketing babble and featureset.
> The protocol specifics (e.g. identifiers & types) aren't polished, because I focused on features in working code. Thanks for the feedback!
Got it. IMHO it's better to start with a clean protocol spec before implementing features. At least for the very basic stuff. Of course for an MVP I wouldn't start with a spec but then again this seems past the MVP stage seeing as it started 2017 and wants to be a protocol and not just product.
> You can't have alias collisions; it checks whether the string you propose is acceptable & available :)
You can't use a volatile cache to check for collisions though. Unless I'm missing something and the "cache" is prefilled with all existing aliases somewhere and not volatile but then it wouldn't be a cache? Even if it did, it's asking for trouble.
> A sender's alias is (or will be) checked against their recorded aliases.
What about the following: 1. user A adds alias Foo 2. Server restarts 3. user B adds alias Foo 3. random other user sends message to alias Foo.
My point is a cache should just be used to speed up certain operations but has to be handled with care (cache invalidation etc). Better to skip this as it's just premature optimization and can just cause trouble.
In fact all this work to juggle JSON in flat files will cause more headache than it's worth. Just use something like SQLite and you'll save yourself a lot of time and issues. It'll make the implementation much easier especially to adapt. But it'll pay dividends for a long time (migrations etc). You'll run into filesystem limitations quite quickly anyways with these flat files.
> The reason why lack of E2EE in email is a serious problem is that email traverses at least two "federated" services, which are typically third parties with whom you have no intrinsic need for a relationship. A TMTP message passes through a single service which you explicitly joined.
But what prevents a "gmnm" here? Companies don't want to host heir own email servers and they might not want to run their own mnm servers. Nothing here guarantees to the user that he's directly communicating with the other party. And all data is also stored unencrypted.
As GekkePrutser mentioned in a sibling comment, it's 2022 and any new messaging protocol without E2EE will be dead on arrival. I'm maybe exagerating but it'll have a tough time. Especially if it justifies itself partially because email doesn't have E2EE.
> I don't see how your feedback justifies "poorly thought out"; it's merely an unpolished preview.
Maybe so. I just saw it mentioning problems with mature protocols like SMTP and Matrix and I agree with some of that criticism but the solutions are not in TMTP as far as I can see. What about synchronization between multiple client devices? What about permissions in groups? What about identity verifications? What about mobile clients which don't want to open dozens persistent connections? There is a long list of topics to cover when it comes to messaging protocols.
To create a new protocol isn't an easy task. A protocol should be precise and robust. To create a product? Much easier.
BTW no one forces people to federate when using Matrix. Or one could use something like the Signal protocol. Sure they are more complex than TMTP but they have to be because they solve a ton of non-trivial issues.
PS: Seems like the demo doesn't actually use the protocol at all? It looks like it just loads all static data from https://mnmnotmail.org/w/data-demo.js and basically is a static page.
Wow, have to ask whether you're interested enough to contribute? :)
> You can't use a volatile cache ...
> all this work to juggle JSON in flat files
I really don't think I've misused the cache! I didn't use SQLite because ordinary admins need to be able to inspect and occasionally fix the database. That decision can be revisited down the line.
> Companies don't want to host heir own email servers
TMTP is more like HTTP than SMTP. There must be many ways to host a service, just like a website. Any admin must be able to bring up a TMTP server. (It's nothing like getting a new MTA host to work with the SMTP network!)
I discuss the E2EE issue on the mnm website FAQ. Only certain kinds of TMTP sites will want it to be the default. If a consumer tech brand offers an online TMTP client, it would present a proprietary API to the browser, as Gmail does; TMTP can't protect those messages.
> synchronization between multiple client devices?
> permissions in groups?
> identity verifications?
> mobile clients which don't want to open dozens persistent connections?
> the demo doesn't actually use the protocol?
Client sync is implemented already \o/ - that's the replicas feature.
It already has simple, user-defined groups. Admin-defined groups is on the roadmap; need user input to chart a course here.
I've implemented OpenID Connect auth for new account registration. It could also be required per login, if there's demand.
JMAP offers a model for how to do non-persistent connections on mobile devices.
The demo is just the browser-based UI to the client, reconfigured to use canned data. (The client is a localhost web app.)
And, hey, can't a protocol can be a work in progress? I'm only willing to spec what I am ready to implement!
> Wow, have to ask whether you're interested enough to contribute? :)
Sorry but I don't have a lot of spare time. I think you have some ambition with this project and it'll be a good amount of work.
> I really don't think I've misused the cache!
Hm what about the scenario I outlined? I might be missing something but I don't see how a cache can be used to check if something exists in persistent data.
> I didn't use SQLite because ordinary admins need to be able to inspect and occasionally fix the database.
I see your point though I think any sysadmin nowerdays needs to be able to do basic tasks for databases and especially something like SQLite.
> TMTP is more like HTTP than SMTP. There must be many ways to host a service, just like a website. Any admin must be able to bring up a TMTP server. (It's nothing like getting a new MTA host to work with the SMTP network!)
Maybe in terms of simplicity of setting up a server yea but other than that this protocol and HTTP have not much in common.
> And, hey, can't a protocol can be a work in progress? I'm only willing to spec what I am ready to implement!
Of course. Well, you're trying to build something so more power to you I guess. Just trying to give some constructive criticism even if it initially might have been a bit harsh. Good luck with it anyways!
Wrong place to share. The harmless people here will nitpick you to death as the adult version of sucking-up to teacher & showing how smart they are. The dangerous ones will silently steal the better parts of your idea & run with them, but with better funding & connections.
(Author here.) TMTP clients maintain concurrent (possibly intermittent) connections to any number of TMTP services, and each service gets its own inbox. So you don't need a separate app instance per service.
Inboxes for a set of related services could conceivably be merged, tho I haven't done any work on that yet.
