Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Linux-factory: A framework used to create custom Linux Debian operating systems (github.com/clayrisser)
189 points by nixcraft on Feb 27, 2023 | hide | past | favorite | 82 comments


I prefer the approach of AntiX and MX Linux. Start from a clean live distro, mod it to your taste, master it to a new Live ISO. You can also start from a normal MX/antiX system and make a live ISO backup. You can also install those ISOs. Systemd is optional on MX and disabled by default. LOTS of live and remaster options on those systems. I find myself more on live than on installed systems now. It's basically like working with FS snapshots.

A similar distro with remaster tools based on Devuan is Refracta.

There used to be the Debian Live project but it looks obsolete. EDIT: sorry I misread the bug reports, it is OK https://live-team.pages.debian.net/live-manual/html/live-man...


There's almost no circumstance I can imagine where giving up a solid usable well known OS sounds tempting to me. Getting Debian running on PogoPlugs and OpenWRT devices has been a repeated ordeal I'vd gladly traversed. Having a well behaved normal OS that meets many user's expectations (but has your special stuff regularly installed!) is divine.

I used to run Debian Live, and appreciated the tiny amount of shell hacking they have to layer persistent storage filesystem atop a ro filesystem- identical to how theres a ro container image then the mutable rw layer atop it... but a decade older!

I find that the pretense of running "live" is largely gone. It's basically become a meaningless difference of what medium you run on but I've been using the same scripts to create the necessary root OS & UEFI EFI filesystem (FAT32 with a magic partition-type, populated with the bootloader & config). often I spin up systems by just copying a btrfs snapshot & rsync'ing efi directory (and updating some partition uuid's, regenerate machine-id), then I have two systems. Live feels meaningless, non-distinct from regular to me.

All these attempts to treat things different & distinctly, to make special cases for ourself- oh we need a network/router oriented OS, we need a bare OS, we need a consumer OS, we need a NAS os- so rarely have I ever felt like these attempts to flee from the fold & venture out to special really have rewards. This behavior of making exceptions for your problem, talking yourself out of doing the easy normal base thing, seems so rarely to me to have rewards. Run Debian, run systemd, run normal well-known tools! Unless you're really certain getting off the base path really is essential, has huge specific merit.

That's why I like this effort. It allows customization & baking your own thing, but it's much more routine & regular & normal than so many "make your own OS" projects, where folks tend to be quite "back to the land" in motive. Let's make our own OS from sticks! Yeah! But no. You almost certainly should have an ok userland base in my view. Debian is a pretty great one.


If anyone wants to build with/atop Debian Live, I did a couple projects years ago where I wrote wrappers that might still be good starting points:

* LilDeb, a USB multi-tool designed for 256MB (like when you couldn't find a Linux box, but you could find a Windows PC), with careful separation between read-only OS, ephemeral data, and persistent data. One shell script builds the entire thing, including the various config files. https://www.neilvandyke.org/lildeb/

* Rackout, start to a living room media appliance project that was going to provide some examples for my (abandoned) Racket book. If you happen to want to use Racket to do things like repartition a disk or build a distro image, some of the libraries I wrote might come in handy. https://www.neilvandyke.org/rackout/


How do you maintain this as new base releases are made?

Ie: Do you just manually reapply your changes every time? Or just do in-place upgrade and then snapshot that?

Personally, by the second time I did this, I'd be writing a script. Then I'd want to script the running of the script.. and before you know it you end up with a framework..


It’s essentially an easy to use tool around live-build. Live build is maintained by the Debian team and will always remain up to date.

This means Linux Factory is future proof.


This project is a product of 4 years of work, mostly learning the Debian ecosystem. I started with Ubuntu, I’ve tried Arch, I’ve tried Gentoo, but nothing beats the ease of Debian when it comes to custom distributions. Debian was almost built to fork. Their ecosystem is amazing. Their live build tool is super.

I built Linux Factory so everyone else can build custom Debian distributions easily without having to know the live-build tool. It’s really easy to configure and get started. Please give me feedback on how I can make it better.


Why not just use mkosi? There are so many tools that do this already and mkosi is cross distro and can do super modern stuff like use UKIs.


Since I've been looking at mkosi for building Debian images recently, I would love a section which clarifies/differentiates features (and goals/non-goals) of linux factory and mkosi.


always when somebody says "why not use use X?" i always need to thing at https://news.ycombinator.com/item?id=8863

sometimes the answer is just, why not? - it's not forbidden to learn something or try a different approach.


I hate this reddit style comment of linking a barely related thread, not to make any sort of point, but to just try to show you are in the know of some in joke.


1. Having N tools that are 95% the same is just frustrating for everyone trying to find a tool that will be maintained.

2. Contributing to a project that almost does what you want leads to better tools that are also maintained so they can be relied upon to build new things.

3. Instead of building the Nth version of something, you could be working on something novel.

Theres no value from building something new if you can just learn how something that exists works and contribute to it instead. NIH syndrome is actively harmful to open source.


Thanks for the tool! Found a docu bug: the link in the "Grub" section does not point to the grub overlay :)


Thanks for tool. Will there be GUIed version in future ? Like Suse had previously ?


> Like Suse had previously ?

Do you mean SUSE Studio? That functionality has been integrated into the openSUSE Build Service at https://build.opensuse.org/ .


Hey just noticed all the activity here. I’m the author of Linux factory. I just opened up issues on GitHub so more discussion can happen there.


Congrats and thanks. Not sure people interested in Debian want to contribute to the borg producing copilot and chatgpt based on their work on gh, among other things.


This sounds like something I'd be very interested in, though I'm not sure what it is.

It'd be great if I could easily customize Debian (i3, vim, zsh, chrome, etc.) and distribute / share it with others.

But I'm not sure what this does and cannot tell even after reading the first few sections of the README.


As the author of Linux Factory, that’s exactly what I built it for. I would love to help guide you and get you started on your project. Please create an issue in the GitHub project and I’ll further guide you there.



The reason I didn’t use refracta is because I wanted a reproducible approach.


Ah, very cool. Thank you :)


There are a ton of Debian system build tools out there and people keep making new ones:

https://wiki.debian.org/SystemBuildTools


I hope this is the easiest to use.


There's nothing wrong with making new tools. I love making my own tools, both physical tools and software tools.

The feeling you get from it is great! Plus, this is often how new things (or ways to do things) are discovered.


I think I'd rather have one featureful well-maintained tool backed by a community of people than a hundred of them with different features and single maintainers for each one.


I recently had the opportunity to use Linux-Factory, and I must say that I'm thoroughly impressed with this operating system. It's clear that the Author behind Linux-Factory have put a lot of thought and effort into creating a streamlined and user-friendly experience that makes it easy to get things done.

One of the standout features of Linux-Factory is its speed and performance. This operating system is lightning-fast, and it runs smoothly even on older hardware. Additionally, the user interface is clean and intuitive, making it easy to navigate and customize to suit your preferences.

Another great thing about Linux-Factory is its versatility. Whether you're a developer, a gamer, or just someone who wants a reliable operating system for everyday use, Linux-Factory has something to offer. The software package selection is robust and well-curated, with all the tools and applications you need to get work done or enjoy your leisure time.

Overall, I highly recommend Linux-Factory to anyone who is looking for a fast, reliable, and versatile operating system. The Author have done an excellent job creating an operating system that surpasses expectations, and I look forward to seeing what they have in store for future updates. Thank you for creating such an amazing product......!


Alternatives that support more than just Debian: https://github.com/buildroot/buildroot https://github.com/skiffos/SkiffOS (Based on buildroot) https://www.yoctoproject.org/


these are for embedded boards though


SkiffOS also supports desktop PCs (with the intel/desktop layer) + EFI with rEFInd / GRUB. Also supports the Steam Deck.


Does that mean it can't be used for desktop PCs? I remember using it in the past but on a vm.


I won't use it myself, but it's linux, it's technically possible to use it in and out a vm anywhere


Linux-factory: A framework used to create custom Linux Debian ~~operating systems~~ installers

FTFY


Linux Factory sounds like a fantastic tool that could benefit a lot of people who want to build their own custom Debian-based distributions. It's awesome that you've made it accessible even for those who aren't familiar with the live-build tool.

In terms of feedback, it might be helpful to include some tutorials or guides for those who are completely new to the process of building custom distributions. Additionally, having a user community where people can share their experiences and ask questions could be really valuable.


Linux-factory is a powerful and innovative open-source project that is making it easier than ever before for developers and Linux enthusiasts to create custom Debian-based operating systems. Whether you're a seasoned developer or a newcomer to the Linux world, Linux-factory is definitely worth checking out!Great work @clayrisser sir...!! Hope you will reach many more milestones in future sir..!!


No offense, but this from a new account makes it look suspicious.


Hey groestel..! Iam not promoting any product or service, but rather expressing appreciation for an open-source project.I genuinely believes in the value of the project and wants to show my support. And if you want to support you can..!! No offense, Thank you...!!


Why are these things always for Debian based distros, but never for Fedora ones?


Why not use a distro that has the literally built in?


What do you mean?


What would you like to see for something like that?


Oh, hey!!! Did you know that RHEL, CentOS Stream, & Fedora has been providing an image builder for a couple of years? All of these links below demonstrate functionality that you can do with CentOS Stream & Fedora, and even with the no-cost developer subscription for RHEL which unlocks 16 RHEL subs with tons of extra functionality. Many links below to help you easily learn to use it. We are actually using this tooling internally to beging building all of the image artifacts for all three of these linux distros. Eating our own dogfood; this is the future.

Also, below are links and some screenshots of the wicked cool new customizations that just landed in CentOS Stream and should come to RHEL 9.2 and Fedora soon.

- Install in your local Fedora/CentOS Stream box

dnf install -y osbuild-composer composer-cli cockpit-composer bash-completion systemctl enable --now osbuild-composer.socket systemctl enable --now cockpit.socketfirewall-cmd --add-service=cockpit && firewall-cmd --add-service=cockpit --permanent source /etc/bash_completion.d/composer-cli

- RHEL Image Builder | Into the Terminal 53 youtube podcast https://www.youtube.com/watch?v=H-mv_WLmQdA - Blog: https://www.redhat.com/en/blog/using-no-cost-developer-subsc... - Blog: https://www.redhat.com/en/blog/announcing-full-support-new-r... - Upstream docs https://www.osbuild.org/guides/blueprint-reference/blueprint... - RHEL 9 docs https://access.redhat.com/documentation/en-us/red_hat_enterp...

Screenshots - https://github.com/tabowling/Fedora-Virt-Lab/blob/master/cs9... - https://github.com/tabowling/Fedora-Virt-Lab/blob/master/cs9...

Please give us feedback, feature requests, tell us how you're using it, pull requests at our upstream project https://github.com/osbuild Happy building!


I use image builder exclusively for my home lab and work-based demos. Its an amazing project!


Debian and Ubuntu are nice, but not exactly bleeding edge.

IMHO arch is far better, and comes with an extra bonus: you can use pacman just like on MSYS2 (Windows)


In a lot of situations, stable is better than bleeding edge. It depends on the use case.

For example, my personal computer is running Debian Sid (i.e bleeding edge, dev track) while my computer for work is using the latest Debian stable.


I use arch btw. It’s better because we the arch users say so. I’m an arch user btw.


That's not arguing in good faith.

Here's a simple example: you want hyprland, patheon and interception-caps2esc on ubuntu? Good luck!

On arch, it's just a pacman away.

You don't like how arch is perceived as elitist? Use manjaro.

Ubuntu had its place in the last century, but with most development happening on git you need a rolling release.


It’s not funny if you have to explain it but it was mostly a joke, without rolling eyes. Also making fun a bit about how easy it is to make a claim that something is better than another thing and leave it at that, which is very common in every population but in particular something you hear from the arch one. I use Arch btw.


> It’s not funny if you have to explain it but it was mostly a joke, without rolling eyes.

Poe law strikes again! My apologies then, because I known how Arch can have this reputation.

But Arch is just so good! Ubuntu and RedHat seem to be stuck in the past: it's not just about rolling releases, but many small details that pile up.

FYI I also find Alpine very interesting.


> Ubuntu and RedHat seem to be stuck in the past

Like offering users a GUI click-through installer?


> Like offering users a GUI click-through installer?

I can't say if it's a joke or if you are serious. But if you like an installer, there's manjaro.

Personally, I think install shouldn't need a click-through installer: it should be done by a commandline, to have a finer control over all the details.

With Ubuntu, I only used the installer iso to type Control-Shift-A to get a terminal, install rsync and do my tinkering.


I was obviously being sarcastic, but the sarcasm was earnest. For the most part, there shouldn't be any details to control. Just pick defaults that work for 99% users and let them get on with their life.

>But if you like an installer, there's manjaro.

Manjaro is not Arch. My comment was a criticism of Arch Linux, not Manjaro.


you can't control details with a GUI?


Then can you tell me how to use the ubuntu GUI install to a ZFS encrypted root, without zsys, without grub but with a .efi packed kernel?

Or how can you mount the efi partition to new systemd location /efi instead of /boot/efi (ex: if you're not using grub) without breaking the install?

There are just too many choices for too many people. It's best not to play, and let it be a commandline problem. You rarely have to reinstall without another OS or boot media, so doing a chroot and copy-pasting the right commands (or rsync or untar a working master image) is just simpler.


If you want bleeding edge Debian as a rolling release, sid is there. It's all a matter of preference.


and sid has proved to be very "stable" for me


I wonder how it compares to ELBE. https://elbe-rfs.org/


Seems vaguely similar to Armbian?


Er... in what way?


To be fair, there are tens of projects out there dedicated to creating custom rootfs...


Is there anything that makes it worth to switch from debian live-config scripts ?


- overlays (basically you can separate configuration into separate units)

- templating support

- configuration in yaml files

- it still fully supports live-build support, so it should be super easy to migrate. You just put the live build files in os/lb or in the lb folder of an overlay.


It's like Yocto but made saner (although with less options)


The real value with Yocto is the community and vendor supported/supplied BSPs (board support packages). The gains from these (typically extensive and robust) BSPs usually more than offsets other rough edges with Yocto.


I'd be interested in knowing what you consider not sane in Yocto?


Anything with a learning curve steeper than "throw your files in a Dockerfile and call it a day" unfortunately tends to be considered "not sane". There is a whole market when it comes to making new tools for people who "don't have the time to RTFM". And then for tools helping to use those tools.

(To be clear: I really like Yocto)


Any recommendations on getting started with Yocto? I want to build a custom image for a Jetson, and Yocto seems like the only good option.


IMO, start small, maybe with a RPi. Possibly Nvidia provides layers for the Jetson (I haven't seen that), but in my experience the risk is that those layers are super complicated (and fairly messy). I think it's better to learn how Yocto works first, rather than learning how to hack the layers provided by the manufacturer.

Also it helped me to learn a bit how other distros are built: I tried Linux from scratch, then got interested in Alpine, and then in how PostmarketOS got started based on Alpine tooling.

After all, a distro is "just" a bunch of packages, which are archives containing files that get decompressed in the right place. Yocto provides tools to create those packages (just like mainstream Linux distro). What Yocto adds on top is tools to manage cross-compiling (toolchains, a way to create an SDK, etc) and a system of layering that helps organizing them ("those packages go specifically with the RPi, so I will put them in a layer called "RPi" and I will call that a "BSP"", etc).

Don't get me wrong, there is quite a lot. But there is no magic, the goal is always to create packages that will put the right files in the right place :-).

Good luck!


I just started learning Yocto a few weeks ago, and just shipped my first embedded image. Here are a couple of learnings and thoughts:

1. Start with an empty image and build your way up. There can be some cross-effects between packages depending on what you exactly do, it's a lot easier to debug if you go step by step. The base image recipes are all listed in the documentation, start with one of them.[1]

2. Lean into Bitbake and the whole thing. The more you try to fight it, the more it fights back. I liked this guide as a starting point to get a feel for it.[2]

3. Find a way to quickly iterate on your images. Nothing worse than losing your train of thought! Since I wanted to include update functionality I added swupdate, which integrates reasonably well with Yocto[3] - although the documentation is a bit difficult to read at times. But being able to simply push an update through WiFi is really, really nice.

4. If you do some lower-level stuff you'll inevitably need to create configuration fragments. This process was REALLY opaque to me. Say you want to set build options for U-Boot - you'll have to: 1) run `bitbake -c menuconfig u-boot` to open the config menu; 2) when closing the menu, you save the configuration in the default location; 3) then you run `bitbake -c diffconfig u-boot` to generate a config fragment; 4) you copy the config fragment into your recipe-append-folder and add it to the SRC_URI variable; 5) you run `bitbake -c clean u-boot`. When U-Boot is built the next time it will apply your configuration fragment to the build process.

These were the most important things that came to mind. Hope they help, and good luck on your journey! :)

[1] https://docs.yoctoproject.org/ref-manual/images.html

[2] https://a4z.gitlab.io/docs/BitBake/guide.html

[3] https://sbabic.github.io/swupdate/building-with-yocto.html


Word of warning - I don't know how much you've looked into it yet but depending on your needs from Jetson the Nvidia stack gets really complicated really quickly.

I know people have frustrations with Nivida supplied Jetpack but when you end up trying to figure out how to roll your own support for TensorRT, DLA, cuDNN, etc (on ARM64, of course) many people bail and run screaming back to Jetpack.



yocto is essentially gentoo for embedded boards, definitely over-engineered, but no better alternatives


Yocto is fine if you are using a SOM and the vendor provides a yocto BSP.

If are building your own board and developing a BSP, Yocto is a nightmare peeling away the onion layers.

I find Buildroot much saner and easier to work with.


I completely agree; I worked on a project with a custom board and we tried Yocto at first because the silicon vendor pushed it, and it was indeed a complete nightmare. We finally switched to Buildroot and life was much easier.


IMO those are quite different.

I see buildroot for making a firmware, in the sense that I ship a bunch of tools that will run on a device (and I can flash updates).

Yocto is there to build a distro, where you have to manage packages, and potentially other people will be developing for it.

Of course if Yocto is overkill in your case, then buildroot will probably be a better fit.


How is this different / better than live-build?


It’s easier to use. It actually uses live-build under the covers, and even supports enabling you to tap into the live build system directly.

A couple of things it supports that live build does not support.

- overlays (basically configuration modules)

- templating

- python hooks


Any color you want, as long as it's black.


Finally I can have my own distro complete with bugs.


default.nix in this repo doesn’t seems to work since it doesn’t include poetry and many other requirements.


Yes, sorry about that. I need to work a bit more on the nix compatibility. Please file an issue on this.


i have already converted to ZFS/btrfs snapshot to publish my project


or use NixOS which has this built in as a first class citizen.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: