[exe.dev co-founder here] You are right, you cannot! It was quite a bit of work. We have a blog post in the works that should come out in a couple of weeks with all the details.
I was just sufficiently nerd sniped by this, so let me know if I’m close:
Based on what the commenter below found about sshpiper I believe that you use the ssh identity + the ip from the slot to resolve the vm target. sshpiper knows how to route the ssh identity + slot ip to the correct VM. I suspect you have a custom sshpiper plugin to do that routing.
You use the slot record indirection so you can change the ip of a slot without having to update everyone’s A records across the customer base. It also makes it easy to shuffle around vm-slot mappings within a customer. I haven’t tested, but I’m guessing this dns server is internal (coredns?), and the ips too.
I did something similar (ip + identity routing) for a project a few weeks ago. Yours is a lot more elegant with the dns indirection.
I’m no ssh expert, but in theory you should be able to ssh -J exe.dev myvm.exe.xyz for a one-liner? Or maybe you don't even need it, if that DNS server within the ssh exe.dev is the same as the public DNS. Pardon for not testing it yet!
[exe.dev co-founder here] Thank you! Not to give too many secrets away, but my hope is to follow a business model I have been part of before, and make it as cheap as possible for individuals so they encourage their employers to buy it for work. So I would very much love to get cheaper.
The two constraints are that, one, when small underlying resources are expensive (we hope to fix that soon by not being small!), and two, we do not want to make the resource allocation so small that the VM feels unpleasant to use. So there is a floor on how small we make them.
That said, I very very much want to drop prices. We started with conservative numbers.
[exe.dev co-founder here] Hi there, I am not sure exactly where you are, but your VM is ubuntu derived and definitely starts with apt and bash. Perhaps try `ssh yourvm.exe.xyz`?
While at tailscale you built sketch.dev only to actually build this product ? Love it. Ultimate yak shave.
Kind of how like Antithesis was the product inside foundationdb.
[exe.dev co-founder here] It is planned! The reason we have not got to it yet is it needs to be very different than IPv4 support. We have spent a lot of time on machinery to allow `ssh yourmachine.exe.xyz` work without having to allocate you an IPv4 address. The mechanisms for IPv6 can and should be different, but they will also interact with how assigning public static IPv4 addresses will work in the future.
We do not want to end up in the state AWS is in, where any production work requires navigating the differences between how AWS manage v4 and v6. And that means rolling out v6 is going to be a lot of work for us. It will get done.
[exe.dev co-founder] Hi! There is a mobile site. It is not super visible right now but you can use it to create VMs (and even build something on them with our agent if you like). If you ran into a particular bug I would love to get it in the issue tracker so we can fix it.
Sorry if I missed this in the docs, but how robust is the persistence? ie is it the disk that comes with a standard AWS VM? or is it a share backed by e.g. Ceph with multiple redundant copies?
Details coming in the next few weeks. The contents are regularly replicated to a disk cluster, though we have some more experimentation to do before we commit to exactly how frequently. This space has a lot of trade-offs, we believe we have found a new and interesting one.
I have not used E2B (though I really like their web site), though it looks like there are quite a few differences. Our disks are persistent (without manual snapshotting), we have a TLS proxy by default with built-in auth and link sharing.
It also looks like they have many features we do not have (yet).
I believe the target use is also quite different. You can use exe.dev VMs for running your agent. But you can also use it for hosting your site. E.g. blog.exe.dev is an exe.dev VM.
Thanks for the response. In the "How exe works" page, it's mentioned that exe runs on bare metal with Kata containers, how is it different from firecracker? Were there any advantages?
The mention of Kata is out of date, we are fixing that, thanks! Our underlying VMM is very similar to firecracker (same upstream source). We believe our advantages are in how we run it. Several blog posts are in the works about technical details!
No I apologize for the confusion (exe.dev person here). What is different about this service is you get dedicated resources that you share between your VMs. The initial allocation is conservative, we want to give people more (or drop the price).
The goal is to reduce the marginal cost of creating a VM to zero. Instead of installing a container manager or using Unix users, just make another VM.
(I will get a better version of this table online tonight.)
You guys really need to work on simplifying your communication on your website. I was also very confused about how the 8GB - whether it is per VM, shared etc.
>Instead of installing a container manager or using Unix users, just make another VM.
What is the advantage of this? Unless you need something exotic like different kernel configurations per instance, what's the problem with using containers on the same instance?
BTW, a Hetzner dedicated server with 2 CPUs/8GB RAM that would let me run my own hypervisor is about $14 USD/month. For anyone who's a big enough power user to care about the distinction of running distributed workflows on VMs versus containers, I'm not sure that an extra $5/month is worth your "hypervisor as a service." But then again, HN commenters infamously poopooed Dropbox [0], so what do I know? :-)
Containers aren’t enough for me. I like to do things like create TUN devices, run docker compose, etc. I believe the VM is a fundamentally better abstraction.
Consider this: sometimes when you are using a VPS, you start a new project and say to yourself, "I should put this on a new VPS." Not all the time, but it does happen. And when it does, we are faced with the problem that starting a new project immediately costs us $X/month. I would like a new project to initially cost nothing.
Is that possible and useful with exe.dev? The docs say:
On the networking side, we don't give your VM its own public IP. Instead, we terminate HTTPS/TLS requests, and proxy them securely to your VM's web servers. For SSH, we handle ssh vmname.exe.xyz.
> run docker compose
You can run multiple compose stacks in a single VPS.
> you start a new project and say to yourself, "I should put this on a new VPS."
Tun devices are possible and useful on exe.dev today, because it means, for example, you can install the full copy of Tailscale.
But to your point: if a single VPS meets all your needs and you do not feel constrained by the marginal cost of another VPS then the exe pricing model is not going to bring you much value. Perhaps the automatic TLS termination and auth proxy with link sharing is useful. But if not, then it sounds like you are well served by existing products.
exedev@scarlet-canyon:~$ rsync --version
rsync version 3.2.7 protocol version 31
Copyright (C) 1996-2022 by Andrew Tridgell, Wayne Davison, and others.
Our base image is most of an Ubuntu server. We trim out, for example, systemd features that are designed for working with system hardware, and then add a lot of standard software, as our block device cloning is a lot faster and more efficient than apt. So you will find vim, git, go, curl, sqlite3, etc all installed. If you think something obvious is missing please let us know!
Would love it if Tailscale came pre-installed! Or even better: some simple way to make it so every VM I start up is automatically/easily part of my Tailnet.
p.s. thanks for making Tailscale. And I'm loving exe.dev so far!
[exe.dev cofounder here] That's a good idea! I will add it to a list I have for exeuntu. Automatic login would be really nice, let me see if I can figure out how to do that. Thanks for trying exe.dev!
Apologies for the vagueness of the home page, we were not expecting to be here today. There is a little more info in our first blog post https://blog.exe.dev/meet-exe.dev and docs, but far needs to be written.
(We have also built some interesting tech behind this that we are excited to write up, I have a doc two pages long of blog posts we want to write.)
The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.
Please contact the website owners to inform them of this problem.
I just tried this out in Firefox on macOS and there are no issues, so this might have something to do with our LetsEncrypt wildcard cert and the CA roots installed on Android. Could you tell me what version of Android you are using?
An error occurred during a connection to blog.exe.dev. SSL received a record that exceeded the maximum permissible length.
Error code: SSL_ERROR_RX_RECORD_TOO_LONG
The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.
Please contact the website owners to inform them of this problem.
Hello, I am behind this company. My co-founder Josh Bleecher Snyder has also been hanging around the internet for a while. There are several of us hacking away. It is very early days, we have a lot of work to do to earn your trust but it is my intention to do so.
Both are also early Go engineers and developers who hacked on the Go stdlib for years. Most people in the Go community know them. Great people, and the idea speaks for it. I wish them best of luck.
