Nice to see another approach to this subject! Kudos to the author.
I have a couple of responses to specific points brought up in the article.
The author suggests that the original 45-byte executable no longer works on modern systems. If so, this is news to me. Admittedly my current machine is a bit behind the cutting edge (4.15), but what's there should still work. If people are finding the current version to fail for them, I'd appreciate some details on their setup.
* I respectfully disagree that 32-bit executables are "less relevant" today; I suspect they will continue to be supported for many, many years to come. Of course for a new explorer 64-bit executables are far more interesting, but when you're shaving bytes at a time, you can't beat a 32-bit executable.
* Many people are unaware that my original essay is only the first of a series that I wrote. (All of the essays are linked at the bottom of the original.) I note that my smallest 64-bit ELF executable without introducing invalid fields is also 120 bytes, so that's cool.
* However, by taking advantage of unvalidated fields, I was able to produce a working 64-bit ELF executable that is 84 bytes in size. The overlapping is a bit tricky, but I've verified that it continues to work on my box. See http://www.muppetlabs.com/~breadbox/software/tiny/return42.h... -- all variations of my return-42 executables are collected there.
* My smallest 64-bit ELF executable that prints "hello, world\n" (no punctuation: I always use the string from K&R) is 98 bytes. I don't have the assembly for that one posted on my site, but it uses the same layout as the 86-byte executable.
I find I need to quote myself from 2013, as I can't really add anything further to it:
> Wow. I honestly wasn't expecting to ever see this on the front page of HN again, given the current ubiquity of 64-bit Linux. (And yes, before anyone asks, I've played around with minimizing 64-bit executables. Unfortunately they are both larger and less forgiving of tomfoolery. The smallest 64-bit ELF I've created is 84 bytes.)
> Since it is here, though, I want to take the opportunity to say thanks to everyone who's expressed their appreciation of my essay. And I should note here that writing that essay, so many years ago now, is one of the better thing I've done for my career. Share what you have to learn the hard way; the effort won't be wasted.
> Is the author stating that _start is an interface to the OS then? It's interesting I never thought of this as being an interface.
Perhaps it's not the best term, but it's not really incorrect. I probably wouldn't phrase it that way today (20 years later).
> Is this just a long-winded way of saying _start sets up the stack for the new process? WIthout a stack you can't "set up" argc, argv etc.
Well -- to be precise, the stack is created by the kernel during the exec system call. The kernel also copies the argument and environment strings into the process's memory. However, the kernel does not create the argv array of pointers, or the envp array. Building those arrays is the job of _start(), and that is what I meant by "setting up argc and argv".
Please note: That was the case back in 1999. Things are different now -- I think it changed with the 64-bit kernel? In any case, nowadays the ELF loader in the kernel also builds the arrays.
> I would also be curious what "among other things" might include.
As before, it depends on the C runtime. The _start() function might ensure proper initialization of libc, for example. (Again, with current binaries that's usually done via an `.init` section, but in 1999 that was not the norm.)
(Note: I'm assuming from the reference to "map layout" that this comment is directed at the original article, and not the video mentioned in the parent comment.)
Reading the assembly language is what many (perhaps most?) people would do at that point. But not everyone is as comfortable with reading assembly, and I wanted to show that a lot can be done without taking that step.
Also, "easier" is a relative term. It was far, far easier for me to examine the data files sitting comfortably in front of my Linux box, than to try to pull together a decent debugging setup inside of a dosbox before making any headway. There's always more than one way to do it.
Horrible article. It had me glued to the screen in rapt interest and I ended up losing track of time, causing me to be late for a meeting.
Actually, I have been biting at the bit to get into reverse engineering after discovering some of Chris Domas' [0]. However, my hand has been stayed simply by a lack of somebody to learn from and nerd out with on the topic. Would you be able to share any communities you are aware of in this regard?
Really though, awesome article. Thank you for taking the time to write and share it!
Thanks for the kind feedback. Unfortunately I don't know much about communities. Most people who do reverse engineering focus on examining code/assembly, but my limited experience is solely with data files.
This was a great post with references to many tricks and tools I was not aware of that might help me in similar endeavors. Do you have any particular advice for reverse engineering image data files (not encrypted, of a completely proprietary format)?
I'm afraid that's something I don't have any experience with. However, another commenter mentioned a cool video he did on just this subject, at https://news.ycombinator.com/item?id=21728298 -- so maybe check that out?
Yes, part of the usual process of making a tiny binary is jettisoning much of the overhead that debuggers depend on in order to find their way through an external program. Not just the debugging symbols themselves, but things like section header tables and maintenance of linked stack frames in the running code. Dumping all of that saves space, at the expense of making the resulting binary more and more opaque.
Fascinating! Thanks so much to the company for writing this followup -- every year or so I've wondered how this company was doing and if the experiment was still going. It's heartening to learn that it is.
I have a couple of responses to specific points brought up in the article.
The author suggests that the original 45-byte executable no longer works on modern systems. If so, this is news to me. Admittedly my current machine is a bit behind the cutting edge (4.15), but what's there should still work. If people are finding the current version to fail for them, I'd appreciate some details on their setup.
* I respectfully disagree that 32-bit executables are "less relevant" today; I suspect they will continue to be supported for many, many years to come. Of course for a new explorer 64-bit executables are far more interesting, but when you're shaving bytes at a time, you can't beat a 32-bit executable.
* Many people are unaware that my original essay is only the first of a series that I wrote. (All of the essays are linked at the bottom of the original.) I note that my smallest 64-bit ELF executable without introducing invalid fields is also 120 bytes, so that's cool.
* However, by taking advantage of unvalidated fields, I was able to produce a working 64-bit ELF executable that is 84 bytes in size. The overlapping is a bit tricky, but I've verified that it continues to work on my box. See http://www.muppetlabs.com/~breadbox/software/tiny/return42.h... -- all variations of my return-42 executables are collected there.
* My smallest 64-bit ELF executable that prints "hello, world\n" (no punctuation: I always use the string from K&R) is 98 bytes. I don't have the assembly for that one posted on my site, but it uses the same layout as the 86-byte executable.