The IPC is in SystemV flavor but I think these days we normally should adopt Posix IPC instead especially for Linux/BSD, am I missing something?
"On Linux and FreeBSD there is big advantage of posix queues, as handler given by mq_open are basically file descriptor which can be polled/epolled/selected/kqueued"
"all POSIX IPC is thread-safe, while most SysV IPC is NOT"
If you use pandoc, have you considered epub as a target format? I am getting good results going from latex to epub using pandoc. Not sure what your sources and toolchain are like, though.
Disagree. ePub (or any flowable medium) sucks when code blocks are involved. eBooks are great for novels and long form prose. Not so much for technical books.
Why? Because inevitably the code is going to be either wrapped, too tiny or overflowing. Furthermore, syntax highlighting is a complete utter mess in ebooks.
Reviews don’t lie. People don’t like it. And it’s obvious where ebooks shine and where they don’t.
Personally: I want proper typesetting and constancy in the layout. But with eReaders, ebooks work pretty well and PDFs don’t. With iPads, PDFs are better IMO. But for technical books PDFs set by a professional typesetter like Springer is fantastic. Great typography and layout.
I think the problem with PDFs on ereaders is that the PDF layout is typically designed for a much larger physical page. I designed the PDF version of Dernocua (http://canonical.org/~kragen/dernocua) for comfortable reading on small screens, even including cellphones; if you're willing to try it, I'm curious to hear what your experience is with perusing it on your ereader.
This is simply brilliant. I've never seen anyone produce PDFs for small size (or large fonts). I just tried it on iPad mini and it is fantastic. I don't have my eReader handy but I can only imagine it would work well. I think you nailed the problem with PDFs and eReaders - its the choppy panning/zooming that's fatiguing to be able to read fonts at a legible size. This solves it.
I hope this idea catches on. ePub/Mobi files are no good IMO.
Oh, thanks! I'm glad you liked it. I'm interested to hear how it works on the eReader too, since the iPad is a lot more capable of rapid scrolling, which makes it a significantly more forgiving environment for this kind of thing.
It didn't work as well on Kindle, see [1]. The version of Kindle I have is from 2012 and it doesn't have a 'fit-to-width' feature. So, the PDF would need to be of correct aspect ratio.
Thanks! Yeah, that looks pretty hard to read. I see it has 150%, 200%, 300%, and "actual size" options; do any of those make it usable? How about landscape mode?
On a laptop you should probably use the HTML tarball; less of the formatting is broken that way. Unlike Dercuano there isn't SVG or anything in this version that gets lost in the PDF, but there are tables and "ASCII" Unicode art.
I'm pretty sure you can. But most e-readers have screens in the 6-7 inch range. You can't fit very wide lines in a readable font size, so the only way code samples work out in that format is if they are written to be narrow - say, no more than 40-50 characters wide.
So if the code you're writing is amenable to those constraints, it would probably work quite well, but AbstractWidgetLocalizerManagerFactoryBeans are right out.
Beej, so nice to see you on this site, such a fan of your work. Used your guide a decade ago to implement socket select() on a system-critical app. Mad props for doing some awesome work!
I just got through my Graduate Intro to OS course at Georgia Tech last year, and your networking guide is still handed from student to student every single semester to survive the first project. I didn't think I'd get the chance, but since you're hanging out here, thank you very much for making it!
In Dercuano, Derctuo, and Dernocua, I was able to get what I thought was pretty reasonable formatting for 80-column code even on a cellphone screen in a PDF with the lmtlc font, which, despite the abbreviation, is called Latin Modern Mono Condensed Light; it's an extended version of Computer Modern Mono Condensed Light with some Unicode coverage. I wrote a little about this at https://news.ycombinator.com/item?id=21946327, and Dernocua is at http://canonical.org/~kragen/dernocua among other places if you want to see how it worked out.
I'm reasonably pleased with the Dercuano formatting, though some people hate it; it doesn't work very well on a laptop screen, but on a cellphone screen I think it's close to optimal, except where it's just broken. It's probably passable on a small ereader but would be painful on a large one, except maybe in landscape facing-pages mode. (I hacked together the HTML renderer to generate the PDF in a few days, and I didn't implement, for example, table layout; and when it has to fall back to a glyph that isn't in lmtlc, it screws up the column widths.)
I think I've said this before, but I really appreciate Beej's Socket Guide! Thank you for writing it!
fork() is rarely ideal, unless followed by exec() to launch another processes. That copy-on-write thing often wastes too much physical memory. I think threads are usually better, the shared memory allows to marshal large volumes of data without making copies.
I like poll() much better than signals. Signals are too limiting, and too arcane. It's much easier to use poll() and dispatch things manually based on the signaled handles, at least there's a guarantee your code won't be interrupted by another event.
Pretty much all modern kernel things are compatible with poll. There's eventfd() which acts as an event or semaphore, message queues (but not the ones from the article, better ones, created with mq_open not msgget), Unix domain sockets for cross-process messaging. There's more, poll() can track termination of processes (see pidfd_open), page flip events in a GPU with DRM, consumed or available video/audio samples with V4L2 or ALSA.
I partly agree, but a lot of this stuff is very context-dependent.
Signals are interrupts, which are faster than an event loop but very bug-prone, especially in C (few C library functions are safe in interrupts), but shared-memory threads are too, for almost exactly the same reasons. If you want to share memory between processes, it's easy enough to use mmap().
Both threads and signals are somewhat tamer in Python, but lose some of their advantages on the way.
Interrupts are really needed for three reasons: device driver latency and buffering (multithreaded CPUs probably being a superior alternative), killing infinite loops, and userspace virtual memory. Unix signals are useless for the first and arguably so bad at the other two that Unix would have been better off without them.
Generally running multiple processes of the same program with different exec() calls uses more memory than forking it at startup, not less. It's true that threads use less still, and event loops less than that, when you don't have a multicore scaling problem.
If you're looking at writing your own event loop, select() is more convenient than poll(), and io_uring is worth checking out. kqueue, iocp, and epoll are faster than poll(). Also though maybe just try libev, libevent, or libuv instead of writing your own.
NEVER use select(). It's like gets() - an API that should never be called in new code. poll is the standardized alternative (epoll/kqueue are more performant with large numbers of fds, but are os-dependent).
The biggest problem with select() is that it will corrupt your stack if you use it with an fd with a value >= 1024. Since you can't easily control fd values, you must assume that calling select() is UB unless explicitly proven otherwise. There are also performance problems due to having to set up the sets before each call, but these are minor in comparison.
You can easily control fd values, and attackers can't control them, so even if you're using fd_set, it's not like gets(). You can use select() safely by dynamically allocating your bitvectors, assuming you're using a sensible kernel instead of one that only cares about standards-compliance.
Or you could just not have thousands of open files in a single process, which is the way every single Unix program worked for the first 25 years of Unix. I have 269 processes running on this machine right now and none of them is using a file descriptor greater than 255. Even Firefox is only using up to fd 75. The worst offender is gnome-terminal, using fds up to 226, and its children. Having thousands of simultaneously open files is a useful way to structure a few specialized programs like chat servers and load balancers, but it's not something most programs need to worry about.
Of course in a library the situation is different, but you probably shouldn't be calling poll() or select() in a library unless it's something like libevent, libev, or libuv, because you can't use two such libraries in the same program unless you're willing to suffer multithreading.
> If you want to share memory between processes, it's easy enough to use mmap()
It's only easy when all of the following is true: (1) sharing one time only (2) Sharing exactly one continuous buffer, as opposed to anything more complicated with pointers inside (3) Sharing between processes one-to-one. When any of these things is false, mmap becomes hard. For instance, despite C++ std::vector is a continuous buffer, when sharing a mutable one that grow/reallocate thing gonna cause issues.
> select() is more convenient than poll()
Strongly disagree. These macros and the bitmap is pure BS. For some handle types like sockets, for hardware reasons (multicore CPUs) Linux no longer guarantees sequential file descriptors. Even the manual page says "WARNING: select() can monitor only file descriptors numbers that are less than FD_SETSIZE (1024) - an unreasonably low limit for many modern applications - and this limitation will not change. All modern applications should instead use poll(2) or epoll(7)"
poll() is awesome in comparison, the API with that array of simple structures is perfect for the job. At least in terms of usability, AFAIK epoll does better performance-wise.
> io_uring is worth checking out
Right, but the use case is different, it's only for I/O.
> kqueue, iocp
Not available on Linux?
> epoll
Yeah, but I think to truly benefit from this one, need multithreading. Multithreaded async I/O is hard, I think people only doing that when they absolutely have to for performance reasons.
> just try libev, libevent, or libuv instead of writing your own
My use cases are probably different. I rarely using poll() for network sockets, it's usually other things: process and threads synchronization, timers with timerfd(), hardware devices (GPU, sound card, hardware video codecs).
You can have the same file mmap()ed at the same address in two separate processes, so putting pointers in it is a valid thing to do. In most languages doing dynamic allocation in shared memory is a world of pain, but maybe the STL allocator approach makes it a reasonable thing to do in C++? Sharing a complex linked data structure between different threads is a somewhat tricky thing to do in any case, though in some cases it may be the only reasonable solution. In many cases something like FlatBuffers may be a better solution.
I'm not sure what you mean by "sharing one time only" or "sharing between processes one-to-one"; can you elaborate?
It isn't really true that select() can only monitor file descriptor numbers less than FD_SETSIZE; you just have to allocate the bitmaps differently instead of using the fd_set type. Libevent does this, for example.
io_uring isn't really only for I/O; you can use it for madvise(), poll(), and timers as well. Normally I'd be comfortable with saying that's "I/O" but in this context poll() and timers are the things you're implicitly opposing to "I/O".
Yes, kqueue and iocp are not available on Linux. But io_uring and epoll aren't available on other operating systems, so I thought it was worthwhile mentioning the corresponding facilities there.
Why do you say you need multithreading to truly benefit from epoll? If you have an I/O-bound process that has a lot of file descriptors open, running in an event loop, epoll is a faster alternative to select() (or, as you say, poll()), isn't it?
This thing about nonsequential socket file descriptors is news to me; I hadn't heard anything about it, though I admit I haven't been paying attention. Where can I find out more?
> I'm not sure what you mean by "sharing one time only" or "sharing between processes one-to-one"
By one time only, I meant that you only want to pass the data once, as opposed to sharing the data and being able to update it on one side and see updates on the other one.
It’s not impossible, but practically hard to implement growable/shrinkable shared memory areas, the communicating processes need to unmap and remap the virtual memory regions.
It’s similar with more than 2 processes sharing the memory. Technically doable but hard in practice, for instance it’s easy to leak these memfd handles.
None of these issues are present for threads sharing the address space.
> you just have to allocate the bitmaps differently instead of using the fd_set type
When really unlucky, with these bitmaps one gonna need to waste kilobytes of memory to represent a set with only a couple of handles. The RAM itself is usually cheap at that scale, the problem is the performance overhead of searching for the set bits.
> epoll is a faster alternative to select() (or, as you say, poll()), isn't it?
Right, but for IO bound things, epoll() + multithreading is even faster. Modern network cards are aware of multi-core CPUs on the other side of the PCIe bus. Faster cards like 10 Gbit/second have multiple transmit/receive queues designed to interact with different CPU cores.
OTOH, when things aren’t that I/O bound, the single dispatcher thread is adequate, and the API overhead of poll() is reasonable too, given the usability win.
> Where can I find out more?
Yeah, you’re right about that, accept() never does that, according to specs. However, note the F_DUPFD parameter of fcntl: duplicate the file descriptor using the lowest-numbered available file descriptor greater than or equal to arg. This alone means that for libraries or long-lived projects, relying on file handles to be small numbers is fundamentally unreliable.
Typically when many processes share memory it is with a fixed-size buffer for things like a database page cache or the Apache scoreboard. Or sometimes one process creates a data file and then one or many processes mmap it read-only (your "one-time" case, I think). It's true that sharing dynamically growing and shrinking data structures becomes complicated, but multithreading doesn't make it less complicated, it just moves the complication somewhere else.
Probably you shouldn't use select() or poll() in a library unless it's something like libevent. Libraries that want to own the event loop are impossible to use together unless you're willing to resort to multithreading.
Yes, it's true that you can use F_DUPFD (or dup2()) to request a high-numbered file descriptor. As I see it, that's not really a matter of luck. The only reason I can think of to do that is for some sort of art project or side-channel communication where your file descriptor numbers spell out a message or something.
> For some handle types like sockets, for hardware reasons (multicore CPUs) Linux no longer guarantees sequential file descriptors.
huh? this would be a severe POSIX violation, which says that for socket and accept, "The file descriptor shall be allocated as described in File Descriptor Allocation.": "All functions that open one or more file descriptors shall, unless specified otherwise, atomically allocate the lowest numbered available (that is, not already open in the calling process) file descriptor at the time of each allocation."
Signals have some wierd properties; a process-directed signal (i.e. the usual one you’d send with kill) chooses an arbitrary thread and invokes the signal handler. You can get in to some weird situations with signals and threads. If you’re using signals as an ipc mechanism, a useful tip is to just block the signal and register a signalfd on poll/epoll to regain control flow control.
The classic solution would be to fork a special signal catching thread and play with the signal masks to ensure the signal goes there. This is what glib does if I recall correctly.
... Which of course seems very silly today. Signalfd is a much nicer API to use for new programs!
Dbus is a good place to start. Raw sockets and shared memory are still necessary to get good enough performance in some cases. Dbus can be very slow! If something needs to happen several times per second, or big amounts of binary data should be transferred, dbus can become a bottleneck in embedded applications.
I think dbus works great for service discovery and for low-frequency low-bandwidth communication. For other cases, people just exchange a file descriptor over dbus and proceed to talk directly to each other.
I personally dislike Beej's writing style. I tried to like his Network Programming guide but I found that most of what it conveys can be learned more quickly by simply reading the relevant POSIX/Linux manpages.
I don't know C very well, but what little I know is from Beej.
I don't know network programming very well, but what little I know was learned from Beej.
It's actually been quite rare in my professional programming career that I've had to use any of these things that I learned from Beej but... when I needed them.... I knew where to look and, knowing even just that, it's made me look like a wizard. Honestly, I can't see a 'donate' link here, but I really feel that I should.
Beej's guides are amazing with a memorable and entertaining writing style. I remember reading his network guide almost two decades ago when I was a kid, had just heard of this thing called a socket, but the only connotation I had for one at the time was a wall socket. His guide helped me understand networking but more importantly it made it a fun experience for a programmer just starting out.
> Before we go on, why would I even begin to bother pointing out that a pound sign is called an octothorpe? The answer is simple: I think the word octothorpe is so excellently funny, I have to gratuitously spread its name around whenever I get the opportunity. Octothorpe. Octothorpe, octothorpe, octothorpe.
I can't find it right now but somewhere in the guide in a different chapter, he randomly inserts "Octothorpe"... good comedy. I think this kind of writing is exceedingly rare and immensely enjoyable.
Been at it again with the amazing content. Love everything this guy does. He has taught me so much about network programming and C in a very accessible way.
I remember in college in my networking class beej's guide was recommended by the professor and its stuck with me for 7 years. I've written a lot of socket based programs over the years and I still revisit it from time to time. Its the perfect guide. I also just point people to it whenever they ask about socket programming and basic networking because the guide explains it so much better than I ever could.
Same here, found his guide when doing socket programming assignments for a networking course. It certainly was more interesting than the lecture material...
It's very entertaining to read though which holds my attention well, and is well suited to people with existing programming experience. I've tried to read K&R but I keep coming back to Beej's.
I really want to like Beej, and I get that he wants to make things easy for people to understand, but his guides are full of unnecessary shortcuts which are actually wrong.
For example, he doesn't seem to understand the difference between arrays and pointers -- at [1] he says:
You can’t [get the length of an array] ish. C doesn’t record this information [footnote 50]. You have to manage it separately in another variable.
[footnote 50] says "Since arrays are just pointers to the first element of the array under the hood, there’s no additional information recording the length"
And then later he says:
There is a trick to get the number of elements in an array in the scope in which an array is declared. But, generally speaking, this won’t work the way you want if you pass the array to a function [footnote 51]
[footnote 51] says "Because when you pass an array to a function, you’re actually just passing a pointer to the first element of that array, not the “entire” array."
Footnote 51 is correct, and contradicts footnote 50.
There is a simple, and correct, way to explain this:
Arrays and pointers are distinct types in C but arrays are converted to a pointer to their first element, except when it is the operand of the sizeof operator, the _Alignof operator, or the unary & operator, or is a string literal used to initialize an array.
Yes, it's more wordy, but it's what's actually going on. Arrays aren't pointers, but he insists on claiming they are throughout. Footnote 51 shouldn't be a footnote, it should be prominent in the text.
P.S. Not going to go into it here, but his "unix" network programming guides used to be full of linux-specific things (without pointing this out), but I think it's improved over the last 10 or so years.
