There are two issues here. One is the ability to obtain, given sufficient local resources, a file descriptor to /dev/urandom. I think defining away this problem is fine: running a program in an environment without a valid /dev is simply user error. In such an environment, operations that might require entropy should simply fail.
The second issue is resource limits: low level components of LibreSSL cannot cope with entropy-generating functions failing. On OpenBSD, these functions cannot fail, but on Linux, they can. That's not a problem with Linux, but with LibreSSL's architecture. It's LibreSSL's responsibility to ensure that it allocates. Every call to LibreSSL's internal RNG is preceded by some kind of resource-allocating call that can fail. It's in this call that LibreSSL should obtain the resources needed to do its work. That it doesn't is simply a bug in LibreSSL, not a deficiency in Linux.
It becomes the application's problem. LibreSSL on its own can't really go out and obtain resources whenever it wants, nor can it ensure the application won't take these away. The application using LibreSSL would have to make LibreSSL do these things, at the right time and right place. You're pushing responsibility to users and applications and while at it admitting that it is impossible for a library to abstract it all away and provide a guaranteed safe API that cannot ever fail. It definitely sounds very Linuxy.
LibreSSL can pre-allocate resources it needs in places where applications expect it to fail, then go on to use those resources in places where applications do not expect it to fail. Complaining about having to manage a file descriptor is strange, since LibreSSL already manages another resource, heap memory. Nobody expects LibreSSL to be able to do everything it promises if the system runs out of memory, so why should people expect LibreSSL to do everything it promises if the systems is misconfigured or out of file descriptors? Fundamentally, applications need to cope with resource constraints, and it's the job of libraries to tell applications when resource constraints have been exceeded. raise(SIGKILL) is an exceptionally poor way of informing an application that its resource demand has outstripped supply.
> Nobody expects LibreSSL to be able to do everything it promises if the system runs out of memory, so why should people expect LibreSSL to do everything it promises if the systems is misconfigured or out of file descriptors?
The issue is not that anyone expects it to do everything it promises if the system is misconfigured, but that if it should fail, it should take care to try to avoid failing in ways that could open massive security holes.
This is the issue here: The developers believe that as the existence of systems with unsafe core files is well established, their options are limited, as there is a risk of exposing enough state to less privileged users with a core dump to leave the system vulnerable. Someone building for a system they know has properly secured core files, can disable the homegrown entropy code, and the code will fail hard if /dev/urandom and sysctl() are both unavailable, and the problem goes away.
But what do you suggest they do for the case where they do not know whether failing will expose sensitive data? They've chosen the option they see as the lesser of two evils: Do as best they can - only as a fallback, mind you - and include a large comment documenting the issues.
If they had full freedom to design their own API this would not be an issue. They could e.g. have put in place a callback that should return entropy or fail in an application defined safe way, or many other options. But as long as part of the point is to be able to support the OpenSSL API, their hands are fairly tied.
If LibreSSL was the origin of the API, I might agree with you. But LibreSSL is trying to largely conform to an API they have inherited. If that isn't feasible to do safely on Linux, then while it may be arguable whether or not it is a deficiency in Linux, it is a problem for Linux users.
For my part, I believe strongly that it is a deficiency if we have to go through all these kinds of hoops in order to safely obtain entropy, when the solution is so simple on the kernel end: Ensure we retain a syscall.
Exactly what part of LibreSSL's API prohibits the maintenance of an internal file descriptor and early checks that this file descriptor can be filled with a handle to /dev/urandom?
Resource management is not a "hoop". It is a fact of life.
Nothing. But nothing also prevents the client from intentionally or accidentally closing that file descriptor. As I've pointed out elsewhere, looping over all file descriptors and closing them on fork() is a common pattern for applications where you want to ensure you don't have resource leaks (whether for capacity or security).
Which is fine if you have a safe way of returning errors in all code paths, but as the comments points out, they believe they don't. Maybe they're wrong, but they seem to have spent some time thinking about it. They've also provided an easy define to change the behaviour to failing hard for people building it on systems where their caveats against failing hard does not apply (e.g. systems with secure core files)
If they can't fail early in a safe manner without potentially creating a security leak (as they potentially would if an unprivileged user could induce an unsafe core dump), and can't even be guaranteed that they're able to safely log the error (there's no guarantee they'd be able to write it anywhere), it's hard to see alternatives but to try to do something that is "good enough" as the alternative could be much worse.
> looping over all file descriptors and closing them on fork() is a common pattern for applications where you want to ensure you don't have resource leaks (whether for capacity or security).
Any application that does that is broken. Please stop trying to bring up this behavior as something a library needs to support. It isn't. If application try to close all file descriptors, then go on to do real work, plenty of things other than LibreSSL will break.
Would you go around calling munmap on random addresses and expect your application to keep working? Would you write a library that tried to guard against this behavior?
> if you have a safe way of returning errors in all code paths, but as the comments points out, they believe they don't.
That's an internal LibreSSL problem. There's nothing stopping the LibreSSL team from implementing the correct plumbing for telling callers errors about errors. AFAICT, there is no sequence of valid OpenSSL API calls such that the library needs entropy, but at no point in this sequence of calls can indicate failure.
The problems you highlight are not things libraries should try to work around. They're systemic issues. Libraries calling raise(SIGKILL) because their authors don't believe systems have sufficiently secured their core file generation is absurd and only makes the problem worse because it makes overall system operation less predictable. (Imagine a poor system administrator trying to figure out why his programs occasionally commit suicide with no log message or core file.)
These are not problems that require system-level fixes. They're problems that require changes from LibreSSL.
> Any application that does that is broken. Please stop trying to bring up this behavior as something a library needs to support. It isn't. If application try to close all file descriptors, then go on to do real work, plenty of things other than LibreSSL will break.
It doesn't matter that it is broken. It matters whether or not it is done and how to deal with it when it happens.
> If application try to close all file descriptors, then go on to do real work, plenty of things other than LibreSSL will break.
Plenty of things that applications that do this will have to have successfully dealt with. Effectively failing hard now will be a change of behaviour that makes LibreSSL incompatible with the OpenSSL API it is trying to implement, and possibly causing security problems in the process.
Either they do this properly, or they need to break compatibility with OpenSSL sufficiently that people don't accidentally start exposing sensitive data because they mistakenly thought LibreSSL was a drop in replacement (which it won't be if it does things like calls abort() in this case).
> Would you go around calling munmap on random addresses and expect your application to keep working? Would you write a library that tried to guard against this behavior?
Strawman. munmap() on random addresses is not something I have seen. Looping over all file descriptors and closing is something I have seen in lots of code. Code that works fine unless someone introduces a breaking change like suddenly holding onto a file descriptor the library previously didn't.
And when the risk is exposing sensitive data to a potential attacker, I would most certainly weigh the risks of failing hard vs. attempting a fallback very carefully.
> There's nothing stopping the LibreSSL team from implementing the correct plumbing for telling callers errors about errors.
There is: It would mean LibreSSL does not work as a drop-in replacement for OpenSSL. That may very well be a decision they have to make sooner or later, and may very well be the right thing to do, but there are big tradeoffs. They've chosen this avenue for now, with a large comment in the source making it clear that this is in part a statement about their belief that the best solution would be for Linux to keep a safe API to obtain entropy.
Note that this is even code that will never be executed when running on a current mainline kernel. It will break on systems where people have been overzealous about disabling sysctl(), or on systems moving to some future mainline kernel which we don't know when will be released.
> The problems you highlight are not things libraries should try to work around. They're systemic issues. Libraries calling raise(SIGKILL) because their authors don't believe systems have sufficiently secured their core file generation is absurd
It's not absurd when we know for a fact that this often happens and is a common source of security breeches.
For systems where this is not an issue, disabling the fallback is a define away for your friendly distro packager.
> and only makes the problem worse because it makes overall system operation less predictable. (Imagine a poor system administrator trying to figure out why his programs occasionally commit suicide with no log message or core file.)
I'd rather be the system administrator trying to figure this out, than the system administrator that doesn't know that various data found in my core files have been leaked to an attacker.
It will also only happen if: /dev/urandom is inaccessible and you're running on a kernel without sysctl() and you've chosen this alternative over the built in fallback entropy source.
> These are not problems that require system-level fixes. They're problems that require changes from LibreSSL.
They're problems that may not be possible to fix for LibreSSL without failing to meet one of it's main near-term goals of being a drop-in replacement for OpenSSL.
This is also likely to not only be a problem for LibreSSL - to me it raises the question of how many applications blindly assumes /dev/urandom will always be available and readable. It is a system-level problem when every application that wants entropy needs to carefully consider how to do it to avoid creating new security holes, when the solution simply is to retain a capability that is currently there (the sysctl() avenue) or implementing getentropy().
We're not likely to agree on this, ever. We're going in circles now, and just reiterating largely the same argument from different angles.
I won't comment any more on this, other than saying that for me, it's a matter of a basic principle: Assume everything will fail, and think about how to be the most secure possible in this scenario. To me, that makes the decisions the LibreSSL developers the seemingly only sane choice in a bad situation assuming the constraint of sticking to the OpenSSL API. Long term I think they ought to clean up the API too, but short term I think we'd get far more benefit out of them making it possible to safely replace OpenSSL first. And that may require sub-optimal choices to deal with the worst case scenarios, but then so be it.
> Plenty of things that applications that do this will have to have successfully dealt with.
If I have a choice of accommodating broken applications that close all file descriptors (and you have still not named one) and having a system in which libraries can retain internal kernel handles, I'll take the latter. LibreSSL already breaks compatibility with OpenSSL in areas like FIPS compliance. Compatibility with broken applications is another "feature" that would be best to remove.
> Strawman. munmap() on random addresses is not something I have seen. Looping over all file descriptors and closing is something I have seen in lots of code. Code that works fine unless someone introduces a breaking change like suddenly holding onto a file descriptor the library previously didn't.
There is no fundamental difference between a library-internal resource that happens to be a memory mapping and one that happens to be a file descriptor. Are you claiming that no libraries in the future should be able to use internal file descriptors because there are a few broken applications out there that like to go on close(2)-sprees?
If that level of compatibility is important to you, do what Microsoft did and implement appcompat shims. Did you know the Windows heap code has modes designed specifically for applications that corrupt the heap?
If you're not prepared to go down that road, please recognize that broken behavior is not guaranteed to work forever. There was a time when use-after-free was a very common pattern: on a single-thread system, why not use a piece of memory after free but before the next malloc? That pattern had to be beaten out of software in order to allow progress to be made. As it was then with memory, now it is again with indiscriminate file descriptor closing.
> There is: [proper error plumbing] would mean LibreSSL does not work as a drop-in replacement for OpenSSL
This claim is simply untrue. The OpenSSL API has a rich error-reporting interface. No compatibility break is required for reporting entropy-gathering failures. The only needed changes are inside LibreSSL, and its developers appear to be refusing to make these changes.
> for me, it's a matter of a basic principle
Another basic principle is that you can't get very far if you assume everything can fail. You have to have a certain axiomatic base. The ability to have private resources in a library is a perfectly reasonable basic assumption.
That doesn't count. It's closing file descriptors before exec (and in one case, before doing some tightly-scoped work). It has nothing to do with closing all file descriptors, then expecting to use arbitrary third-party libraries.
Yes, letting child processes inherit stray file descriptors is dangerous. We're not talking about calling all file descriptors immediately before exec. We're talking about applications that say, "Ok now, I'm going to close all file descriptors and go back to what I was doing". You can't expect libraries to work when you're freed their internal resources from under them.
The second issue is resource limits: low level components of LibreSSL cannot cope with entropy-generating functions failing. On OpenBSD, these functions cannot fail, but on Linux, they can. That's not a problem with Linux, but with LibreSSL's architecture. It's LibreSSL's responsibility to ensure that it allocates. Every call to LibreSSL's internal RNG is preceded by some kind of resource-allocating call that can fail. It's in this call that LibreSSL should obtain the resources needed to do its work. That it doesn't is simply a bug in LibreSSL, not a deficiency in Linux.