Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Welcome to Software Development on Codidact!

Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.

Comments on memcmp(3) memory containing invalid values

Parent

memcmp(3) memory containing invalid values

+9
−0

What does it mean that we can use memcmp(3) on invalid values?

ISO C allows comparing an invalid value through memcmp(3), because it doesn't read the value, but rather its representation, and "reading the representation is never Undefined Behavior" (or so I've been told).

The following code invokes UB:

int foo(void)
{
    int32_t  x;

    if (x == 0)  // <- UB
        return 0;
    return 1;
}

However, the following seems to be allowed:

int bar(void)
{
    int32_t  x;

    if (memcmp(&x, "\0\0\0\0", sizeof(x)) == 0) {
        /*
         * Could we do the following?
         *
         *     return x;
         */
        return 0;
    }
    return 1;
}

But does that memcmp(3) call give us any guarantees about the value stored in x? That is, if we know that the representation is 0, can we assume that the value of x is precisely 0? Or is it still UB to use the value after knowing its representation?

And if it's still UB to read the variable even when we know that the representation is a valid one, what's the point in allowing one to memcmp(3) a variable with an invalid value, such as an uninitialized local variable, or a freed pointer?

Being able to know the representation of an invalid value doesn't match very well with the concept of the abstract machine, it seems to me. It's like being able to look outside of the cave, but not being allowed to use that information.


Moreover, while the following is UB:

bool foo2(void)
{
    int32_t  x;

    return (x == x);  // <- UB
}

the following would have defined behavior, I guess:

bool bar2(void)
{
    int32_t  x;

    return (memcmp(&x, &x, sizeof(x)) == 0);
}

The above should always return true, I guess, since the representation of a value, even if it's invalid, is the same, independent of how many times you read it (so far, I don't think ISO C considers quantum variables ;), as long as we don't invoke UB.


This question came to my mind while discussing about use-after-free issues with realloc(3): https://inbox.sourceware.org/gcc/3098fd18-9dbf-b4e9-bae5-62ec6fea74cd@opteya.com/T/

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.
Why should this post be closed?

0 comment threads

Post
+8
−0

Regarding undefined behavior/uninitialized variables of automatic storage duration

First of all there's some misconceptions here.

if (x == 0) is UB only because x was declared as a local variable (automatic storage) without having its address taken - "could have been declared as register" (C17 6.3.2.1).

It is not UB because it's a read of an uninitialized variable with an indeterminate value. Reading an indeterminate value is merely unspecified behavior most of the time, meaning no assumptions about the value read can be made, but at least the program won't crash or go haywire as with undefined behavior.

Here it is important to recognize that just because something isn't undefined behavior, that doesn't mean it turns well-defined. Any language feature in C can either be well-defined or poorly-defined. Where poorly defined behavior can be one or several of: undefined, unspecified, implementation-defined or locale-specific behavior.

Details and sources here: (Why) is using an uninitialized variable undefined behavior? From my answer there:

  • In case the variable has automatic storage duration and does not have its address taken, the code always invokes undefined behavior [1].

  • Otherwise, in case the system supports trap representations for the given variable type, the code always invokes undefined behavior [2].

  • Otherwise if there are no trap representations, the variable takes an unspecified value. There is no guarantee that this unspecified value is consistent each time the variable is read. However, it is guaranteed not to be a trap representation and it is therefore guaranteed not to invoke undefined behavior [3].

Trap representations depend on the type used. Non-two's complement types may have trap representations, as may floating point variables or pointers etc.

However the stdint.h types are guaranteed to use two's complement and it doesn't make sense to implement trap representations for two's complement. All possible binary representations are possible to use as values, there's no negative zero and similar crap. The only possible way to implement trap representations for int32_t etc would have been to extend the size and include padding bits. This is actually allowed for the default types of C (as of today's date at least) but it is explicitly not allowed for the stdint.h types (C17 7.20.1.1).

C23 will finally remove all integer types that aren't two's complement from the language, so a lot of the artificial nonsense like padding bits and trap representations of integers will get removed as well - good riddance.


Now to answer the question:

What does it mean that we can use memcmp(3) on invalid values?

It likely means that memcmp only looks at raw data, it does not care about types. It can't know if the data being compared is an invalid float or an invalid one's complement int etc.

Also, everything passed to memcmp explicitly has its address taken. So by design, it rules out that form of UB.

Although I wouldn't say that using memcmp is always safe. One of the few real-life examples of trap representations I've encountered is where the CPU/MMU explicitly looks at pointer types/addresses and will give you a hardware exception if you claim that something that's an invalid/misaligned address is a pointer. So if you pass such a pointer to memcmp, the core might trap before even entering the function.

But does that memcmp(3) call give us any guarantees about the value stored in x? That is, if we know that the representation is 0, can we assume that the value of x is precisely 0? Or is it still UB to use the value after knowing its representation?

It is unspecified behavior so in your example, the compiler is free to optimize the code either as if(1) or if(0). And it need not do that optimization in a consistent way across the same program either. Nothing can be assumed about the value of x. The memory location does not have to actually get read unless x was declared volatile.

Therefore return (memcmp(&x, &x, sizeof(x)) == 0); need not return true either, since each read of an indeterminate value is not required to result in the same value. (Also see DR260 for some clarifications by the Committee back in the C99 days.)

Similarly, free(ptr) makes ptr turn into an indeterminate value. However, it is extremely likely that any given implementation will just preserve whatever address ptr happened to hold previously.


An amusing example:

#include <string.h>
#include <stdio.h>

int main (void)
{
  int apples;
  int oranges;
  if(memcmp(&apples, &oranges, sizeof(int))==0)
  {
    puts("Apples are oranges.");
    printf("%d == %d\n", apples, oranges);
  }
  else
  {
    puts("Apples are not oranges.");
    printf("%d != %d\n", apples, oranges);
  }
}
  • clang 15.0.0 -O3 gives output:

    Apples are oranges.
    -1853976553 == 0
    
  • icx 2022.2.1 -O3 gives output:

    Apples are oranges.
    4202519 == 0
    
  • gcc 12.2 -O3 actually stores both string literals in memory (missed optimization bug?) then gives output:

    Apples are oranges.
    0 == 0
    

None of these 3 compilers actually called memcmp. This is conforming behavior from all 3 compilers tested.

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.

1 comment thread

`free(3)`d pointer instead of uninitialized variable (5 comments)
`free(3)`d pointer instead of uninitialized variable
alx‭ wrote almost 2 years ago

I guess using a free(3)d pointer in the same way, instead of an uninitialized int32_t local, would still be unspecified behavior, since we can't trap because memcmp(3) uses char to read the representation. Am I correct? I used int32_t just to simplify the question, but it's true that int32_t can't trap (unlike pointers). Anyway, it made your answer even more interesting, since you explained more than I expected :)

Lundin‭ wrote almost 2 years ago

alx‭ Yes, from C17 6.2.4: "The value of a pointer becomes indeterminate when the object it points to (or just past) reaches the end of its lifetime." In case of allocated storage, this happens when you call free(). For example int* ptr = original; free(original); if(ptr == original) is not required to evaluate to true, even though it's very likely to happen on all implementations.

Lundin‭ wrote almost 2 years ago

Btw how memcmp is implemented internally isn't relevant since that implementation in itself need not be done in C or be conforming C - it is a standard library function. For example it could be implemented in assembler.

alx‭ wrote almost 2 years ago

int* ptr = original; free(original); if(ptr == original): That would actually be different. That one is in fact Undefined Behavior, since you're reading the pointer value and not its representation. To make it unspecified behavior, you's need to call memcmp(3).

Lundin‭ wrote almost 2 years ago

alx‭ Only in case pointers have trap representations. Although... in case someone is curious, because of this very example, I managed to kill the clang 15 compiler's conformance in horrible ways. clang 15 miscompiles code accessing indeterminate values It appears to stop generating code halfways through the executable and instead lies down to die... no other compiler including clang 14 does. And nobody including myself seems to understand why it does this. Writing questionable code is a great way to expose compiler bugs for sure.