C programmers expect dead code removal. Especially when the compiler also inlines functions (and, of course, inlining makes the biggest impact on short functions; and one way to get short functions is to have aggressive dead code removal). And macros can expand into very weird, but valid, code; so the statement that "nobody would ever write code like that" isn't relevant. The compiler may well have to handle unnatural looking code.
As others have stated, compilers generally don't actually have special case code to create unintuitive behavior if it looks like the programmer goofed.
It's possible and desirable for a compiler to remove branches of "if" statements that it knows at compile time won't ever be true. And, of course, one special case of statically known "if" statements are checks for NULL or not-NULL pointers in cases where the compiler knows that a pointer will never be NULL (e.g., it points to the stack) or will always be NULL (e.g., it was initialized to NULL and passed to a function or macro).
So the standard allows the compiler to say "this pointer cannot be NULL at this point because it was already dereferenced." Either the compiler is right because the pointer couldn't be NULL, or dereferencing the pointer already triggered undefined behavior, in which case unexpected behavior is perfectly acceptable. Some programmers will complain because the compiler won't act sensibly in this case, but C doesn't have any sensible option for what the compiler should do when you dereference a NULL pointer (yes, your operating system may give you a SEGFAULT, but the rules are written by a committee that can't guarantee that there will be an operating system).
I have no idea if it's true, but compiler implementers swear that their compilers perform so many optimizations, and that those optimizations allow additional optimizations, that this kind of approach would bury you in messages.
C programmers should be able to expect that "optimizations" will not transform program meaning. And because C is so low level, certain types of optimizations may be more difficult or impossible. If the pointer was explicitly set to NULL, the compiler can justifiably deduce the branch will not be taken but the deduction "if the programmer dereferenced the pointer it must not be NULL" is not based on a sound rule. In fact, the whole concept that the compiler can make any transformation it wants in the presence of UB is wacky. Optimization should always be secondary to correctness.
Ok, so you want compilers to generate a translation for program #2 that works "correctly" to your mind when x is null.
Please explain to the class the meaning of the construct int y = *x; when x is null, so that all conforming C compilers can be updated to generate code for this case "correctly".
is probably closer to the intended meaning. Checking if(!x) afterwards is still dead code, but at least the program is guaranteed to fail in a defined manner.
Of course, if implemented this way, every dereference would have to be paired with an assert call, bringing the performance down to the level of Java. (While bringing the memory safety up to the level of Java.)
foo(int*): # @foo(int*)
test rdi, rdi
je .LBB0_1
jmp bar() # TAILCALL
.LBB0_1:
ret
(You might like to compare this with the article's claims.)
More the sort of thing that I'd expect to be generated (since it's a more accurate rendering of the C code):
foo(int*): # @foo(int*)
mov eax, [rdi]
test rdi, rdi
je .LBB0_1
jmp bar() # TAILCALL
.LBB0_1:
ret
I know that NULL is a valid address on x64 (it's zero), and on any system that doesn't completely suck it will be unmapped. (If I'm using one of the ones that sucks, I'll already be prepared for this. But luckily I'm not.) So I'd like to feel confident that the compiler will just leave the dereferences in - that way, when I make some horrid mistake, I'll at least get an error at runtime.
But rather than compile my mistakes as written, it seems that the compiler would prefer to double down on them.
If you want the compiler to leave the dereferences in, use `-fno-delete-null-pointer-checks` or ask your compiler vendor for an equivalent option. Compilers delete null pointer checks by default (on typical platforms) because it's what most users want.
That's because y is never used, so why should the pointer be dereferenced? If you call bar(y) instead of just bar(), "x86-64 clang 3.9.1" with -O3 does the load as well (but after the check)
foo(int*): # @foo(int*)
test rdi, rdi
je .LBB0_1
mov edi, dword ptr [rdi]
jmp bar(int) # TAILCALL
.LBB0_1:
ret
Only GCC does the kind of aggressive optimization the article mentions (and might need to be tamed by -fno-delete-null-pointer-checks).
Ha... yes, a good point. That's a reasonable reason not to generate the load, and stupid me for not noticing. What's also dumb is that my eyes just glossed over the ud2 instruction that both put in main too. The program (not unreasonably) won't even run properly anyway.
gcc does seem to be keener than clang to chop bits out - I think I prefer clang here. But let's see how I feel if I encounter this in anger in a non-toy example ;) I must say I'm still a bit suspicious, but I can't really argue that this behaviour is especially surprising here, or difficult to explain.
That's the opposite of what he wants. He wants a compiler that produces a translation which derefrences the null and follows the true branch.
He wants null to equal a valid addressable address, which is completely nonstandard and not portable to everywhere C is used, to be standard that portable compilers emit code for.
He wants to imagine that null and zero are the same thing.
> In fact, the whole concept that the compiler can make any transformation it wants in the presence of UB is wacky.
That's the way it's often explained but it's not really what happens--the compiler doesn't scan for undefined behavior and then replace it with random operations. Instead, it's applying a series of transformations that preserve the program's semantics if the program stays "in bounds", avoiding invoking undefined behaviors.
I agree that equating "if the programmer dereferenced the pointer" and "the pointer must not be NULL" betrays a....touching naivety about the quality of a lot of code, but if you start from the premise that the programmer shouldn't be doing that, the results aren't totally insane.
> C programmers should be able to expect that "optimizations" will not transform program meaning.
That's the official rule, but it's "program meaning as defined by the standard." It's not perfect, but nobody's come up with a better alternative. We get bugs because programmers expect some meaning that's not in the standard. But compilers are written according to the standard, not according to some folklore about what reasonable or experienced programmers expect.
*
Again, the idea isn't that the compiler found a mistake and will do its best to make you regret it. Derefencing a pointer is a clear statement that the programmer believes the pointer isn't NULL. The standard allows the compiler to believe that statement. Partly because the language doesn't define what to do if the statement is false.
> But compilers are written according to the standard.
Written to the writers _interpretation_ of the standard. I bet money that every compiler written from a text standard hasn't followed said standard. It would be nice if a standard included code fragments used to show/test the validity of what is stated.
Actually that's not correct. The standard says the behavior is up to the compiler. The compiler author took that as a license to produce a non truth preserving transformation of the code. The actual current clang behavior also satisfies the standard as written.
> The standard says the behavior is up to the compiler.
I think this statement is correct, but it's the kind of thing people say when they confuse implementation defined behavior and undefined behavior. And that distinction is key.
Implementation defined behavior means the compiler gets to choose what it will do, document the choice, and then stick to it.
Undefined behavior means that the program is invalid, but the compiler isn't expected to notice the error. Whatever the compiler spits out is acceptable by definition. The compiler can generate a program that doesn't follow the rules of C; or that only does something weird when undefined behavior is triggered, but the weird behavior doesn't take place on the same line as the undefined behavior; etc.
It's certainly true that "the compiler isn't expected to notice the error" doesn't prohibit a compiler from noticing the error. A compiler can notice, but it's standard conforming even if it doesn't.
I should probably mention that when I say "the standard" I mean the C language standard; the POSIX standard may add requirements such that something undefined according to the language is well defined on POSIX.
So the standard does not require the compiler to e.g. remove the "redundant" check for null or assume that signed integers don't cycle values on overflow, but _permits_ the compiler to do so. Thus, we have two problems: a poorly thought out standard which permits dangerous compiler behavior and poorly thought out compiler that jumps at the chance.
I never said the standard required the compiler to remove the redundant check; just that it is allowed to, and I gave an example of why it might.
But since the compiler is allowed to, programmers have to act accordingly.
*
Yes, this does lead to a situation similar to what C.A.R. Hoare described:
"Now let me tell you about yet another overambitious language project. ... I was a member and even chairman of the Technical Committee No. 10 of the European Computer Manufacturers Association. We were charged ... with ... the standardization of a language to end all languages. ... I had studied with interest and amazement, even a touch of amusement, the four initial documents describing a language called NPL. ... Each was more ambitious and absurd than the last in its wishful speculations. Then the language began to be implemented and a new series of documents began to appear at six-monthly intervals, each describing the final frozen version of the language, under its final frozen name PL/1.
"But to me, each revision of the document simply showed how far the initial Flevel implementation had progressed. Those parts of the language that were not yet implemented were still described in free-flowing flowery prose giving promise of unalloyed delight. In the parts that had been implemented, the flowers had withered; they were choked by an undergrowth of explanatory footnotes, placing arbitrary and unpleasant restrictions on the use of each feature and loading upon a programmer the responsibility for controlling the complex and unexpected side-effects and interaction effects with all the other features of the language" ( http://zoo.cs.yale.edu/classes/cs422/2014/bib/hoare81emperor... , pg. 10).
In The Design and Evolution of C++, Stroustrup mentioned that occasionally when designing a feature, he could think of multiple possibilities: often one would involve convoluted rules that the compiler could enforce, and the other would be a simple rule (or set of rules) that the compiler couldn't necessarily enforce. He said he generally chose the simple rules, even if he didn't know how to have the compiler detect violations. So C++ ended up with things like the One Definition Rule (and violation is undefined behavior). I've never seen any similar statements from Ritchie or Thompson, but I suspect they followed a similar approach. Of course, today both languages are governed by committees, so how they balance trade-offs may have changed.
As others have stated, compilers generally don't actually have special case code to create unintuitive behavior if it looks like the programmer goofed.
It's possible and desirable for a compiler to remove branches of "if" statements that it knows at compile time won't ever be true. And, of course, one special case of statically known "if" statements are checks for NULL or not-NULL pointers in cases where the compiler knows that a pointer will never be NULL (e.g., it points to the stack) or will always be NULL (e.g., it was initialized to NULL and passed to a function or macro).
So the standard allows the compiler to say "this pointer cannot be NULL at this point because it was already dereferenced." Either the compiler is right because the pointer couldn't be NULL, or dereferencing the pointer already triggered undefined behavior, in which case unexpected behavior is perfectly acceptable. Some programmers will complain because the compiler won't act sensibly in this case, but C doesn't have any sensible option for what the compiler should do when you dereference a NULL pointer (yes, your operating system may give you a SEGFAULT, but the rules are written by a committee that can't guarantee that there will be an operating system).