Mind. Officially. Blown. Sometimes the coincidental timeliness of things popping up on Hacker News is just weird. u ROCK deepseagirl!
I've recently wanted to start building Linux kernels and play with device drivers again. Wanted to build and then execute on top of QEMU. Downloaded, configured, and built Linux kernel 6.6 following recipes from arch and a few other places. Doing a reasonable .config was a painful process (picking what to include etc.) The kernel build took several hours. The resulting image refused to boot under Virtual Machine Manager, to say nothing of using gdb etc. to interactively debug the kernel.
So, I figured I was looking at a solid weekend of googling, stack-overflowing, swearing, reading kernel Documentation files etc. and finally getting everything set up and working happily.
NOT! The easylkb script did the entire thing completely automatically, culminating in a login prompt on a running virtual machine!! The Linux kernel .config was one of the standard ones I believe, but I hadn't yet found it. The complete kernel build took on the order of several minutes instead of several hours.
There were a couple of minor issues I had with easylkb, which I will share as comments on github.
Another useful tip is to grab an existing config, eg. the one for your current system found under /boot, rename it .config and then do 'make olddefconfig' (followed by building the kernel in the usual way).
Distro kernels are normally quite "full fat" so this may tend to build a lot, but it's a good way to get a kernel which closely matches what you're typically running, both in terms of device coverage and compilation options / hardening.
I personally wen't this route last year and eventually got things working (networking etc. OK with only the kernel). My company then took a direction in which each compute node has its custom kernel, so that we can control the initial stages of boot better (initrd is dynamically downloaded from intranet, given that the network drivers are baked-in) to manage heterogeneous nodes with different "roles" (GPUs, add-on devices, etc.).
Something I have been surprised about is how much functionality exists in the Linux kernel and how much more reliable the modules are compared to many user-space tools (e.g. race-conditions).
I just want to learn. Unfortunately those days with simple source commentary long gone. Hence whilst we have this good tutorial about setup … but how you start to learn more about.
Considering and size and complexity of the Linux kernel, I don't think any amount of source code comments would help you much, unless you already have a strong understanding of the underlying subsystem you want to hack on.
"Understanding the Linux kernel", and "Linux device drivers" are pretty good, pragmatic books IMHO to give you a solid understanding of most useful subsystem.
As for this kernel building script, I guess it could be handy, but unless you're hacking very frequently I never found the actual build of the kernel to be the more cumbersome part.
> unless you're hacking very frequently I never found the actual build of the kernel to be the more cumbersome part
Doesn't kernel use incremental builds? All well-designed C projects that I know of compile .c files into .o files and link them, so a single file modification only requires a single recompilation/relinking.
It does use incremental builds so long as you don't reset everything.
I'm not sure that a lot of people who don't work with make realize that it's smart enough to only rebuild what has changed, and that it's not just a command runner!
A bit out of date, but not to the point of being useless. I still give them to juniors and interns that want to learn more about kernel work. There's also a much more recent (2022) book called Linux Device Driver Development published by packt. In general, these sorts of books are mainly useful for "what not to do", as general architecture stuff is usually well documented by source code examples and in-kernel docs.
I'm toying with building a little OS using the Linux kernel and an all-Go userspace, and one of the goals is to make the entire system as easy to cross-compile (both host OS and host/guest CPU arch) as possible. Linux being non-trivial to compile (let alone cross-compile) has been so far quite a nuissance, so I'll definitely be having a closer look at this.
The idea there sounds a lot like https://gokrazy.org/, which builds a minimal go userland, wrapping one or more user provided go applications, and bundles in a linux kernel.
Targets mostly at single board computers, and I think it downloads pre-built kernels (and bootloaters if needed), rather than trying to build them directly, since getting a working cross compilation toolchain set up and plumbed into the kernel compilation process is still a pain.
I've personally only used yocto/open-embedded for that which does nicely handle building the cross-compilation toolchain, kernel image, and modules. But it is kinda overkill for that task, being designed to build a whole userland too.
Yep, both gokrazy and u-root serve as inspirations/references (there's a lot of tricky stuff like configuring the network or the framebuffer). Using a kernel from Debian while figuring out my options - glad there seem to be so many.
The ultimate goal is to make it very straightforward to change any part of the system source and try it out in action, so the only things you need installed to get started are Go and Qemu. Further down the road I'd love to extend these principles to hacking the kernel - I'd probably like a simpler kernel, maybe OpenBSD or NetBSD, but I'm less familiar with those.
It uses musl and builds using a musl cross compiler, working x86 and arm64 builds. You could easily build a toolchain for your desired arch and make a simple distro, adding in any user space tools.
cross compilation equally non-trivial.
set CROSS_COMPILE= and ARCH= and making the appropriate changes to .config (either set ARCH and `make defconfig`, or modify the relevant parts in `make menuconfig`). then build as usual
This caught my attention because I'm obsessing about storage performance (again) and have included a kernel build in my benchmarks. I found and followed the instructions at https://wiki.debian.org/BuildADebianKernelPackage which were trivially simple.
As an aside, I was astonished that the build took less than 100s. My recollection is that this took about an hour on on my previous system (I7-4770K, 32GB RAM and SATA SSD formatted with ZFS.) New system is Ryzen 7 7700X, 32GB RAM and NVME SSD. I expected a performance boost, but not that much. I'm still wondering what I did wrong/differently.
It indeed is a journey for the most patient of us. Or, at least, was about 8 years ago, when I needed to build gcc for the Cortex M3 target. I guess not much has changed since then, and one still has a great opportunity to learn what mpfr, gmp and the rest of scary words mean, before either giving up or pushing through that crap. It's like eating super hot peppers: first one suffers, then enjoys the knowledge that they are a tough one. And maybe an endorphin rush, if they have suffered intensely enough.
I've recently wanted to start building Linux kernels and play with device drivers again. Wanted to build and then execute on top of QEMU. Downloaded, configured, and built Linux kernel 6.6 following recipes from arch and a few other places. Doing a reasonable .config was a painful process (picking what to include etc.) The kernel build took several hours. The resulting image refused to boot under Virtual Machine Manager, to say nothing of using gdb etc. to interactively debug the kernel.
So, I figured I was looking at a solid weekend of googling, stack-overflowing, swearing, reading kernel Documentation files etc. and finally getting everything set up and working happily.
NOT! The easylkb script did the entire thing completely automatically, culminating in a login prompt on a running virtual machine!! The Linux kernel .config was one of the standard ones I believe, but I hadn't yet found it. The complete kernel build took on the order of several minutes instead of several hours.
There were a couple of minor issues I had with easylkb, which I will share as comments on github.