memcmp(3) memory containing invalid values

Regarding undefined behavior/uninitialized variables of automatic storage duration

First of all there's some misconceptions here.

if (x == 0) is UB only because x was declared as a local variable (automatic storage) without having its address taken - "could have been declared as register" (C17 6.3.2.1).

It is not UB because it's a read of an uninitialized variable with an indeterminate value. Reading an indeterminate value is merely unspecified behavior most of the time, meaning no assumptions about the value read can be made, but at least the program won't crash or go haywire as with undefined behavior.

Here it is important to recognize that just because something isn't undefined behavior, that doesn't mean it turns well-defined. Any language feature in C can either be well-defined or poorly-defined. Where poorly defined behavior can be one or several of: undefined, unspecified, implementation-defined or locale-specific behavior.

Details and sources here: (Why) is using an uninitialized variable undefined behavior? From my answer there:

• In case the variable has automatic storage duration and does not have its address taken, the code always invokes undefined behavior [1].

• Otherwise, in case the system supports trap representations for the given variable type, the code always invokes undefined behavior [2].

• Otherwise if there are no trap representations, the variable takes an unspecified value. There is no guarantee that this unspecified value is consistent each time the variable is read. However, it is guaranteed not to be a trap representation and it is therefore guaranteed not to invoke undefined behavior [3].

Trap representations depend on the type used. Non-two's complement types may have trap representations, as may floating point variables or pointers etc.

However the stdint.h types are guaranteed to use two's complement and it doesn't make sense to implement trap representations for two's complement. All possible binary representations are possible to use as values, there's no negative zero and similar crap. The only possible way to implement trap representations for int32_t etc would have been to extend the size and include padding bits. This is actually allowed for the default types of C (as of today's date at least) but it is explicitly not allowed for the stdint.h types (C17 7.20.1.1).

C23 will finally remove all integer types that aren't two's complement from the language, so a lot of the artificial nonsense like padding bits and trap representations of integers will get removed as well - good riddance.

Now to answer the question:

What does it mean that we can use memcmp(3) on invalid values?

It likely means that memcmp only looks at raw data, it does not care about types. It can't know if the data being compared is an invalid float or an invalid one's complement int etc.

Also, everything passed to memcmp explicitly has its address taken. So by design, it rules out that form of UB.

Although I wouldn't say that using memcmp is always safe. One of the few real-life examples of trap representations I've encountered is where the CPU/MMU explicitly looks at pointer types/addresses and will give you a hardware exception if you claim that something that's an invalid/misaligned address is a pointer. So if you pass such a pointer to memcmp, the core might trap before even entering the function.

But does that memcmp(3) call give us any guarantees about the value stored in x? That is, if we know that the representation is 0, can we assume that the value of x is precisely 0? Or is it still UB to use the value after knowing its representation?

It is unspecified behavior so in your example, the compiler is free to optimize the code either as if(1) or if(0). And it need not do that optimization in a consistent way across the same program either. Nothing can be assumed about the value of x. The memory location does not have to actually get read unless x was declared volatile.

Therefore return (memcmp(&x, &x, sizeof(x)) == 0); need not return true either, since each read of an indeterminate value is not required to result in the same value. (Also see DR260 for some clarifications by the Committee back in the C99 days.)

Similarly, free(ptr) makes ptr turn into an indeterminate value. However, it is extremely likely that any given implementation will just preserve whatever address ptr happened to hold previously.

An amusing example:

#include <string.h>
#include <stdio.h>

int main (void)
{
  int apples;
  int oranges;
  if(memcmp(&apples, &oranges, sizeof(int))==0)
  {
    puts("Apples are oranges.");
    printf("%d == %d\n", apples, oranges);
  }
  else
  {
    puts("Apples are not oranges.");
    printf("%d != %d\n", apples, oranges);
  }
}

• clang 15.0.0 -O3 gives output:

  Apples are oranges.
  -1853976553 == 0
  

• icx 2022.2.1 -O3 gives output:

  Apples are oranges.
  4202519 == 0
  

• gcc 12.2 -O3 actually stores both string literals in memory (missed optimization bug?) then gives output:

  Apples are oranges.
  0 == 0
  

None of these 3 compilers actually called memcmp. This is conforming behavior from all 3 compilers tested.

posted about 1 year ago

CC BY-SA 4.0

1y ago

Lundin‭

8077 reputation 33 109 919 152

MEMCMP simply compares the memory bits between two locations. This has nothing to do with whatever those bits might mean.

Your first example, on the other hand, compares the contents of variable X with the value 0. Doing a MEMCMP with an area of memory with all bits 0 is the same thing, only if you know the representation of the value 0 in X is all bits 0.

The distinction becomes more obvious when using a value other than 0. Let's say you know X is a 32 bit integer and you want to check whether its value is 7. The expression (x == 7) will always work, since it's directly asking what you want to know. It does not suppose a particular representation for the value 7. Using MEMCMP, do you compare the bits of X with the byte sequence 00 00 00 07, or 07 00 00 00? Both are quite possible on common modern machines. The difference in this example is whether the machine is little or big endian.

I used endianess as an example only. There are other possible reasons that make this kind of use of MEMCMP machine and possibly compiler specific. For example, the representation of 1.0 in a common 32 bit floating point format is 3F800000h. And that's only the value of the raw bits expressed as a 32 bit integer. Endianess still applies.

posted about 1 year ago

CC BY-SA 4.0

1y ago

Olin Lathrop‭

1025 reputation 1 22 88 11