I think articles like this are important... we need a good way of collecting knoweldge (and especially frustrations) like this in a compact format. Everyone who thinks, "I'll use tar," in response to some problem involving archives, backups, package distribution, container distribution, etc. will at some point run head-first into one of the things that makes tar weird. Without good collections of complaints, somebody comes in to solve the problem and ends up only solving the corner of it that they personally care about.
Something I personally find frustrating is that I'd like to be able to create a tar archive without recording UIDs, usernames, or timestamps. The command-line tools aren't really built for that, and yet this seems pretty reasonable, and libraries which create tar files do this by default. (This is covered under the reproducible builds section.)
The next thing that you're likely to want is some kind of random-access archive, for which you'll use either zip (which sticks the directory at the end) or squashfs, maybe.
Finally, the last thing that I often want is just a very binary key-value format which I can distribute and randomly access with reasonable efficiency.
Looking at things from the opposite perspective, I'm personally frustrated by the cross-platform personal backup options. Currently, I'm using Duplicity or some variant, but I find it a bit of and as a result my backup schedule is rather sporadic.
I recently learned about [SQLite archives](https://www.sqlite.org/sqlar.html) which feel like an interesting option to consider, especially with your comment around sometimes wanting a "binary key-value format" in mind.
Because it's "just sqlite" it's possible to use the archive functionality in archive-like ways, while also using the file as a sqlite db (because that's all it is).
It's not a perfect tool for all scenarios (and I'm thinking it wouldn't be good for backups), but I feel there are definitely some very useful scenarios for it.
> Looking at things from the opposite perspective, I'm personally frustrated by the cross-platform personal backup options. Currently, I'm using Duplicity or some variant, but I find it a bit of and as a result my backup schedule is rather sporadic.
It's funny you mention backups -- the similarity is not a coincidence, as many of the problems we have in container images are related to problems the backup community had in the past. As for a backup tool recommendation, I use restic[1] which is quite neat.
Something I personally find frustrating is that I'd like to be able to create a tar archive without recording UIDs, usernames, or timestamps. The command-line tools aren't really built for that, and yet this seems pretty reasonable, and libraries which create tar files do this by default. (This is covered under the reproducible builds section.)
The next thing that you're likely to want is some kind of random-access archive, for which you'll use either zip (which sticks the directory at the end) or squashfs, maybe.
Finally, the last thing that I often want is just a very binary key-value format which I can distribute and randomly access with reasonable efficiency.
Looking at things from the opposite perspective, I'm personally frustrated by the cross-platform personal backup options. Currently, I'm using Duplicity or some variant, but I find it a bit of and as a result my backup schedule is rather sporadic.