An equivalent, but simpler, solution would be to use a network-based KVM, like PiKVM. You connect a USB connector to the PiKVM so it can simulate a keyboard (and mouse), an HDMI connector so it can show you what’s on the server screen, and you also connect a special cable to the server motherboard power and reset pins, so the PiKVM can “press” those buttons remotely as well.
Mandos works with initramfs images created by both initramfs-tools and Dracut, and is present in Debian since 2011, so no need to use a third-party package.
The kind of tool that I'm waiting for would allow me to log into a machine remotely, then reboot it once into a provided image (which could then run Linux in a ramdisk). I.e., take over the entire machine until it is restarted again. Does something like this exist?
I want something like "kexec" but taking over the entire hardware at the lowest possible level.
Despite the “tang/clevis” system being too clever by half (and even more complex), it is essentially very similar. Mandos, though, is much simpler; you can actually understand it, since it uses standard components and protocols like DNS-SD¹, OpenPGP and TLS. (Mandos is documented in man pages, in contrast to page you linked, which is both huge, and – for me at least – keeps reloading itself and replacing the entire page with a 500 server error.)
Mandos is also for Debian (and all Debian-based systems), and not Red Hat (although there is nothing preventing a port, since Mandos now also supports initramfs images created by Dracut).
Finally, Mandos was initially created in in 2007, many years before tang/clevis, and literally by a person in a dorm room, not by whatever academics/scientists who seem to have dreamed up the cryptography/protocol tangle that is tang/clevis. Mandos has changed some since then, from initially using broadcast UDP and X.509 certificates, to using DNS-SD and raw public keys², but has otherwise remained very similar to its initial design.
I think FOSDEM had a talk a few years ago about that solution and their tang server. It is very similar in concept. They use McCallum-Relyea exchange, and we use PGP encryption, but the basics are the same in that you need information stored at the server and information stored in the unencrypted initramfs in order to be combined and turned into the encryption key. If my understanding of the McCallum-Relyea exchange is correct, they also combine the key creation and perfect forward security of the transaction into a single protocol, while we use TLS and client keys for the perfect forward security aspect and PGP for data at rest.
With tang you need to verify the hash of the tang server in order to prevent a MITM threat from extracting the server side part during the key creation process, while with Mandos you configure it manually with configuration files.
Tang uses REST and JSON. Mandos sends a single protocol number and then sends the data. Both operate primarily through local LAN, through Mandos client also support as an option to use global reachable ip addresses.
Outside of those design differences, there are some packaging aspects. Tang was designed for Redhat/Dracut, while Mandos was designed for Debian and initramfs-tools (Today Mandos also support Dracut, but it is not packaged for Redhat). Redhat packaging has been requested multiple times for Mandos, but neither of us two developer are Redhat users.
To add some historical context, I recall a Debconf BoF by the developers of initramfs-tools about if they were going to continue developing initramfs-tools or giving up and port everything to Dracut, and the silence was fairly deafening. People did not want to give up on initramfs-tools, but everyone recognized the massive duplication that those two project are. Similarly, the reason why Redhat had chosen initially to develop Dracut rather than just port initramfs-tools is also fairly big mystery, and is generally considered a Not Invented Here syndrome. Today there are however some distinct difference in design between the two systems.
clevis and tang do currently work seamlessly on Debian and Ubuntu using initramfs-tools. So while the initramfs-tools/dracut discussion is valid, it seems mostly orthogonal to this topic.
I was unaware that they no longer depended on Dracut and now support initramfs-tools, which also seem to be the earliest clevis version that got packaged in Debian. That makes the initramfs-tools/dracut distinction a historical aspect of the project.
Why is this needed at all? As the decrypted key is in memory before the reboot, can’t it just be written to a know location in memory and have kexec be instructed to read it early on?
> As the decrypted key is in memory before the reboot, can’t it just be written to a know location in memory and have kexec be instructed to read it early on?
I set up what you are suggesting (sort of anyway[1]) on a personal VPS to reboot after updates, that require one. I just generate an initrd in tmpfs that contains a keyfile[2] and kexec with that ephemeral initrd; The newest kernel can be found by looking at what the /boot/vmlinuz symlink points to. Been running this for years. It is 100% reliable, and simple. And, for the purposes of this box, secure enough.
For remote unlocks from initial power on, Debian has had that since forever using keyscripts and dropbear in the initrd.
[1] You could pull the key from memory, and use that to unlock the disk from within the generated initrd, but it would be more work than just setting up a keyfile in advance. It was my first thought as well.
[2] Easiest way was to use a mount namespace to use a diff crypttab file that points to the keyfile, since cannot specify crypttab location when creating the initrd. E.g.,
Oh for sure something is needed for a full start from zero. But the much more common case for a computer with backup power is regular restarts after applying patches that require a reboot. Would be much more pleasant for that to work out of the box with no manual interaction at all.
Good FAQ, clearly stating the weak point of physical access. For a server that threatmodel can work, for a fleet of edge/iot devices in unsecured locations without permanent uptime there is no real solution to be expected without custom silicon logic (like in smartcards) on the soc.
That only works with RAID 1. If the server uses RAID 5 or RAID 6, this won’t work.
> extract what you need
Well, yes. This is addressed in the FAQ.
> or change the image.
> Then you turn off the server, and just start a vm with the captured init and capture the key.
Well, as explained in the FAQ, an attacker will have to do so quickly, before the Mandos server decides that the Mandos client has been offline for too long, and disables that client. The default value is five minutes, but is configurable per client.
Ok, that is assuming /boot is ON the raid which I wouldn't want to rate for probability
But even if it is, you could just pull one after the other and wait for the resilver before pulling the next one (you will hear if it resilvers automatically)
This doesn't work with secure boot and UKIs, since the entire "pre-rootfs switch" is signed in a single binary. If your threat model is what you have that is the least you should have.
Can't I just extract the key from uefi/tmp in this case?
Not that it's easy, but with the right tools you can so it offline with all the time in the world
The whole architecure of Mandos is very plugin-based, it will therefore likely not be hard to add. But I am not sure what you are asking for? The Mandos server will, by default, unlock all clients, without asking. There is support for not unlocking immediately and instead wait for external approval for clients explicitly so configured, but what is the scenario? Is the client the one supplying the passkeys/webauthn? Or are you providing that manually on a web page somewhere? The latter is possible; the web page server process would then, when a passkey/webauthn has been verified, send a D-Bus “Approve” message to the Mandos server process, which would then send the client its password.
No mention whatsoever of the $TERM environment variable. sigh
Outputting raw escape codes and hope that they work is not how you do it. This is not how any reasonable library (or bash, or git, etc.) does it. These programs and libraries start with the $TERM environment variable to find out what terminal the user is using, and then use something like termcap(5) or terminfo(5) to look up what capabilities that specific terminal has, and what actual escape codes to output to get that effect. In reality, though, most regular programs delegate all this to a library, like ncurses.
(And also make sure not to output any terminal codes at all if the standard output is not a terminal, as in isatty(3) or tty(1).)
This way, you can check for and use modern cool stuff like sixels, but not use them if some user is using something older like XTerm or the Windows Console.
Aside from the 256-color section, I believe all of the examples given in the article are basic sequences which are supported in ~every terminal. Are you aware of any notable terminals where that is not the case?
Maybe I've been lucky, but I've written plenty software that blindly shoots CSI's at the console and still haven't hit any snags. Especially convenient when the channel is unidirectional; curl ocv.me :)
Shelll-mode in emacs uses $TERM=dumb and doesn’t support escapes code by default. Same with compile-mode. Also there’s some tools that persist using escapes code even when piped to another program (like less).
I have read that people still use 9term. Its terminfo entry doesn't show any sign of ANSI escape code support.
If I remember correctly, ANSI-like terminal emulators in RGB mode (aka direct mode) only support 16-color and red/green/blue escape codes. Not the 256-color palette used in this article.
ANSI-compatible terminal emulators widely disagree on whether RGB values should be separated by semicolons or colons.
I have read articles (possibly on hackaday) this decade by people working on retro hardware projects, expressing frustration with command line programs that spew ANSI garbage to their non-ANSI terminals. At least one author resorted to the screen program, because it can translate some ANSI-isms to the correct codes for the active terminal.
I had genuine TeleVideo 912 glass terminals hooked up until not terribly long ago.
In any case, I don't view this as a matter of whether unusual terminals are "notable". We have an abstraction that plays nicely with them (and allows new terminal protocols to be developed). It's a POSIX standard. It's easy to use. I suggest using it.
Hence my question -- my belief was/is that all of the escape-sequences mentioned in the article will work on 99.99% of all terminals, with the exception of 256-color. Assuming this is true, IMO the remaining 0.01% does not justify introducing another dependency (ncurses -> tput).
Everything is either xterm or something else that's emulating most if not all of its features. Whatever isn't is probably terrible and broken and not worth supporting unless you're getting paid fat stacks to do it.
> No mention whatsoever of the $TERM environment variable. sigh
I have mixed feelings about this... Sometimes I feel young and reckless and want to just output whatever I need without checking $TERM. In practice, all modern terminal emulators are essentially xterm-256color compatible. If something breaks, the worse you get is some funny characters. Is that such a big deal? Much better than propagate the silly termcap/terminfo complexity.
reply