I Do Not Know C: Short quiz on undefined behavior (2015) (opens in new tab)

It's quite common in embedded systems to have the fault handler end with an infinite loop, to give the programmer a chance to attach a debugger an inspect the call stack. Sometimes this behavior is turned on or off with a debug flag, which can trigger this unexpected optimization if the flag is not a compile time constant.

> If you really want to go to sleep, use pause() or similar. An infinite loop eats up CPU cycles

Yes but an infinite loop + sleep is okay, right?

If the optimizer can determine that "nothing happens" in the loop, it can optimize the loop away without attempting to determine whether or not the loop terminates.

It allows the optimizer to assume away the halting problem; all nontrivial loops are obligated to halt.

C programmers expect dead code removal. Especially when the compiler also inlines functions (and, of course, inlining makes the biggest impact on short functions; and one way to get short functions is to have aggressive dead code removal). And macros can expand into very weird, but valid, code; so the statement that "nobody would ever write code like that" isn't relevant. The compiler may well have to handle unnatural looking code.

As others have stated, compilers generally don't actually have special case code to create unintuitive behavior if it looks like the programmer goofed.

It's possible and desirable for a compiler to remove branches of "if" statements that it knows at compile time won't ever be true. And, of course, one special case of statically known "if" statements are checks for NULL or not-NULL pointers in cases where the compiler knows that a pointer will never be NULL (e.g., it points to the stack) or will always be NULL (e.g., it was initialized to NULL and passed to a function or macro).

So the standard allows the compiler to say "this pointer cannot be NULL at this point because it was already dereferenced." Either the compiler is right because the pointer couldn't be NULL, or dereferencing the pointer already triggered undefined behavior, in which case unexpected behavior is perfectly acceptable. Some programmers will complain because the compiler won't act sensibly in this case, but C doesn't have any sensible option for what the compiler should do when you dereference a NULL pointer (yes, your operating system may give you a SEGFAULT, but the rules are written by a committee that can't guarantee that there will be an operating system).

> Under what set of logic does being able to de-reference a pointer confer that it's value is not 0 (which is what the test equates to)?

You're conflating null and zero (which C encourages you to do for various terrible reasons). The test does not test that x is not zero; it tests that x is not null (null, like zero, is falsey, but again, null is not to be mistaken for zero), which in C is sometimes represented by the character '0' but which legally can be totally distinct from the bit-pattern zero and which should be thought of as totally distinct. Zero can be a valid address in memory. Null is never a valid address in memory. The integral type with value zero when cast to a pointer is guaranteed to be the null pointer (which may have a different bit pattern than zero!). Casting non-integral types that happen to have the value zero to a pointer is not guaranteed to produce a null pointer. Confused yet?

The compiler isn't. It knows that you're testing that a pointer is not null.

Since x has already been derefrenced, and since derefrencing x has no translatable meaning if x is null, it follows that we can only produce a meaningful translation of this program iff x is not null.

It therefore follows that x must not be null in the test, since x has not changed.

> Under what set of logic does being able to de-reference a pointer confer that it's value is not 0 (which is what the test equates to)?

Simple: undefined behavior makes all physically possible behaviors permissible.

In reality though, such an elimination would only be correct if the compiler was able to prove that the function is ever called with NULL, and if the compiler is smart enough to do that, hopefully the compiler writers are not A-holes and will warn instead of playing silly-buggers.

You're thinking about it in the context of actual computers. The C standard says absolutely nothing about what NULL has to be, besides that the integer value 0 is considered to be the NULL address and that dereferencing it is considered invalid. The NULL address does not have to be all 0 bits. Architectures are generally free to define it to any invalid address they want to be NULL, 0 just happens to be a common and easy one. The catch you're pointing out is that on x86 there are technically no 'invalid' addresses, so we just use 0 and assume you won't ever attempt to use the stuff there (Which in practice on x86, nobody puts anything there).

  Under what set of logic does being able to de-reference a pointer confer that it's value is not 0 (which is what the test equates to)?

Normal deductive logic?

  * No NULL pointer can be dereferenced. 
  * x is dereferenced.
  * Therefore, x is not a NULL pointer.

Of course, the compiler is presuming that your code is correct. That's a reasonable presumption when dealing with computer programming languages. Programming languages would be rather hard to interpret and translate--not to mention impossible to optimize--if you couldn't apply basic deductive logic to their statements.

Imagine the routine had this code, instead:

  void foo (int *x) {
    if (*x != *x) {
    {
      return;
    }
    bar();
    return;
  }

wouldn't you expect the compiler to apply the same optimizations? Or would you be upset that eliding the check broke some code that depended on a race condition somewhere else in your program?

Also, pointing out that the "value is not 0 (which is what the test equates to)" is a non-sequitur. During compilation the literal 0 can behave as a NULL pointer constant. But the machine representation of a NULL pointer does not need to be all-bits 0, and such machine still exist today. And usually, as in this case, the distinction is irrelevant. It doesn't matter that the 0th page is mappable on your hardware. What matters is that the C specification says that a NULL pointer cannot be dereferenced; that dereferencing a NULL pointer is non-sense code.

There's an argument that compilers should be careful about the optimizations they make. Not all programs are correct, and taking that presumption too far can be detrimental. But it's not always trivial to implement an optimizing compiler to "do what I say, not what I mean". Optimizations depend on the soundness of being able to apply deductive logic to a program--that is, being able to string together a series of simple predicates to reach a conclusion about program behavior. You often have to add _more_ complexity to a compiler to _not_ optimize certain syntactic constructs. Recognizing the larger construct, especially only the subset that are pathological, without optimizing the ones everybody expects to actually be optimized, can be more difficult than simply applying a series of very basic deductive rules. So it's no wonder that most compiler implementations, especially high-performance compilers, tend to push back on this front.

What would be nice is for compilers to attempt to generate diagnostics when they elide code like that. An optimizer needs to be 100% correct all the time, every time. A diagnostic can be wrong some amount of time, which means it's easier to implement and the implementation of a particular check doesn't ripple through the entire code base.

GCC and clang implement many good diagnostics. But with -Wall -Wextra they also tend to generate alot of noise. Nothing is more annoying than GCC or clang complaining about perfectly compliant code for which there's no chance of it hiding a bug. For example, I used to often wrote initializer macros like:

  #define OPTIONS_INIT(...) { .foo = 1, .bar = 3, __VA_ARGS__ }
  struct options {
    int foo;
    int bar;
  };

allowing applications to have code like:

  struct options opts = OPTIONS_INIT(.bar = 0);

But with -Wall GCC and clang will complain about the second .bar definition overriding the first. (Because the macro expands to { .foo = 1, .bar = 3, .bar = 0 }). The C specification guarantees in no unambiguous terms that the last definition of .bar wins. And presumably they guarantee that precisely to make writing such macros feasible. I've never once had a problem with unintentionally redefine a struct field in an initializer list. Yet GCC and clang are adamant about complaining. It's so annoying especially because 1) there's absolutely nothing wrong with the code and 2) disabling the warning requires a different flag for clang than for GCC.

(I realize that for such option types you usually want define the semantics so that the default, most common value is 0. But it's not always desirable, and certainly not always practical, to be able to stick that mode. And that's just one example of that construct.)

I think the catch with #2 is that we probably run into it more in the opposite way - a unnecessary NULL check put in from, say, inlining a function or expanding a macro is removed. On that note though, even in OS code address 0 is usually setup to cause some type of fault to catch errors. I think the issue happens when the compiler removes or moves the deference somewhere else - though obviously the cases where this is legal are limited. But in the posted code, for example, the `y` variable is unused and thus could be removed entirely, which would also remove the NULL dereference (but still remove the NULL check).

> What's the best way to "fix" strict aliasing without disabling the undefined behavior around it? Using a union?

I was just talking about `-fno-strict-aliasing`, which is a flag for `gcc` (And `clang` I assume), but it does remove all UB like you're saying by simply allowing all pointers to alias.

The other options are unions like you're thinking (Though that's also technically UB, since writing to one union member and reading from another is undefined, though most compilers allow it without incident), or extensions like `gcc`s `may_alias` attribute. The `may_alias` is really the cleanest way to do it, but for something like OS code the aliasing happens in such strange places that just disabling strict-aliasing completely tends to be the way to go.

> What's the best way to "fix" strict aliasing without disabling the undefined behavior around it? Using a union?

I had this discussion with another C++ programmer and we came to the conclusion that, if you care to avoid that particular UB, any time you cast pointers between unrelated or basic types and you're going to write to one pointer and read from the other, you need to go through a union, as annoying as it is.

I would agree that strict-aliasing is a hurt point for a lot of C devs, which is unfortunate. I'd only suggest that in general, if the strict-aliasing rule is coming into play you're probably already doing some really shady to begin with. Like in this example, casting a `long ` to an `int ` is likely a bad way to go about things even without worrying about strict-aliasing. In a lot of ways, I'd say that problems with the strict-aliasing rule are a symptom of a larger problem. If you can convince them that what they're doing is just bad coding practice to begin with, you might have a better time making them write correct code in the long run.

Now if you're working more directly with hardware (Which is of course possible/likely with C) then it might just be easier to disable strict-aliasing all together if you can, since identifying all the spots where it might be a problem tends to be an issue.

This is not really correct. The kinds of implementation dependencies usually encountered reflected processor architecture. The C standards committee and compiler community have created a situation in which different levels of "optimization" can change the logical behavior of the code! Truly a ridiculous state of affairs. The standards committee has some mysterious idea I suppose, but the compiler writers who want to do program transformation should work on mathematica or prolog, not C.

In the large, no, they don't care about the 90-95% of the code base that's not performance critical. And these days, the stuff that is critical will be #ifdef and asm(...) stew.

I can't tell you how many projects I have been on where disabling optimization made no measurable difference in performance.

This being said, I cannot speak for game devs nor video device driver developers.

In some sense the language is the compiler and the compiler is the language; the language is much like a human language, used for its utility in expressing things (ideas, programs). You can tell if your human language words work by determining if people understand you. If people start being obtuse and refusing to understand you because of an arbitrary grammar rule that isn't really enforced, you'd be right to be upset with the people just as much as the grammar.

It in fact doesn't matter at all what the standard says if GCC and LLVM say something different, because you can't use the standard to generate assembly code.

The standard doesn't have anything to say about UB, so it's the compiler's responsibility to do the most reasonable, non-shocking thing with it possible: if I'm a GCC developer and you ran GCC on one of these fairly mundane examples and it compiled without error then ran rm -rf / or stole your private RSA keys and posted them on 4chan and I said "well, you can't be mad because it's undefined, it's the standard's fault" you'd probably punch me in the face after some quick damage control.

If it deletes an if loop or terminates a spinlock early that's potentially even worse than those two examples.

On the other hand, the expected and desirable behavior in one platform might be different from that in another platform. It's possible to overspecify and end up requiring extra code when performing ordinary arithmetic operations, or lock yourself out of useful optimizations.

It's a stupid convention of compiler writers and standards writers at the expense of common sense and engineering standards. In fact there are many thousands of lines of C code that depend on compilers doing something sensible with UB. For example 0 is a valid address in many cases (even in some versions of UNIX). The decision to allow compiler writers to make counter-factual assumptions on the basis of UB is the kind of decision one expects from petty bureaucrats.

Strict aliasing and ODR violations are extremely difficult to detect; these are the poster children for "undefined behavior that's hard to avoid and could seriously ruin your day if the compiler gets wind of it."

There does appear to finally be a strict aliasing checker, but I have no experience with it.

Right. Except for a few optimizer edge cases, you generally know what "undefined behavior" is going to spit out on a particular machine. Signed integer overflow, for example, almost always happens exactly the way you'd expect.

Yes, in C a plain

  int i;

at file scope is a tentative definition - if, by the end of the compilation unit, no definition has been seen, one of them will become a definition, otherwise it is just a declaration.

On the other hand, this:

  int i = 0;

is a definition, and you can't have two of those.

That was unexpected.

facepalm of course, even if I use gcc, if I compile a.cc it switches to the c++ compiler. Thanks.