The very thing that makes pipes useful is what also makes them slow. I don't think there is much we can do to fix that without breaking POSIX compatibility entirely.
Personally I think there's much worse ugliness in POSIX than pipes. For example, I've just spent the last couple of days debugging a number of bugs in a shell's job control code (`fg`, `bg`, `jobs`, etc).
But despite its warts, I'm still grateful we have something like POSIX to build against.
They work as expected on Redhat and Debian. "POSIX" leaves open a lot of possibility for less-well-tested systems. They could be writing shellscripts on Minix or HelenOS.
I'm talking about shell implementation not shell usage.
To implement job control, there are several signals you need to be aware of:
- SIGSTSP (what the TTY sends if it receives ^Z)
- SIGSTOP (what a shell sends to a process to suspend it)
- SIGCONT (what a shell sends to a process to resume it)
- SIGCHLD (what the shell needs to listen for to see there is a change in state for a child process -- this is also sometimes referred to as SIGCLD)
- SIGTIN (received if a process read from stdin)
- SIGTOU (received if a process cannot write to stdout nor set its modes)
Some of these signals are received by the shell, some are by the process. Some are sent from the shell and others from the kernel.
SIGCHLD isn't just raised for when a child process goes into suspend, it can be raised for a few different changes of state. So if you receive SIGCHLD you then need to inspect your children (of course you don't know what child has triggered SIGCHLD because signals don't contain metadata) to see if any of them have changed their state in any way. Which is "fun"....
And all of this only works if you manage to fork your children with special flags to set their PGID (not PID, another meta ID which represents what process group they belong to), and send magic syscalls to keep passing ownership of the TTY (if you don't tell the kernel which process owns the TTY, ie is in the foreground, then either your child process and/or your shell will crash due to permission issues).
None of this is 100% portable (see footnote [1]) and all of this also depends on well behaving applications not catching signals themselves and doing something funky with them.
The bug I've got is that Helix editor is one of those applications doing something non-standard with SIGTSTP and assuming anything that breaks as a result is a parent process which doesn't support job control. Except my shell does support job control and still crashes as a result of Helix's non-standard implementation.
In fairness to Helix, my shell does also implement job control in a non-standard way because I wanted to add some wrappers around signals and TTYs to make the terminal experience a little more comfortable than it is with POSIX-compliant shells like Bash. But because job control (and signals and TTYs in general) are so archaic, the result is that there are always going to be edge case bugs with applications (like Helix) that have implemented things a little differently themselves too.
So they're definitely not easy to use and can break in unexpected ways if even just one application doesn't implement things in expected ways.
[1] By the way, this is all ignoring subtle problems that different implementations of PTYs (eg terminal emulators, terminal multiplexors, etc) and different POSIX kernels can introduce too. And those can be a nightmare to track down and debug!
Personally I think there's much worse ugliness in POSIX than pipes. For example, I've just spent the last couple of days debugging a number of bugs in a shell's job control code (`fg`, `bg`, `jobs`, etc).
But despite its warts, I'm still grateful we have something like POSIX to build against.