Auto merge of rust-lang#144081 - RalfJung:const-ptr-fragments, r=oli-obk

const-eval: full support for pointer fragments

This fixes rust-lang/const-eval#72 and makes `swap_nonoverlapping` fully work in const-eval by enhancing per-byte provenance tracking with tracking of *which* of the bytes of the pointer this one is. Later, if we see all the same bytes in the exact same order, we can treat it like a whole pointer again without ever risking a leak of the data bytes (that encode the offset into the allocation). This lifts the limitation that was discussed quite a bit in rust-lang#137280.

For a concrete piece of code that used to fail and now works properly consider this example doing a byte-for-byte memcpy in const without using intrinsics:
```rust
use std::{mem::{self, MaybeUninit}, ptr};

type Byte = MaybeUninit<u8>;

const unsafe fn memcpy(dst: *mut Byte, src: *const Byte, n: usize) {
    let mut i = 0;
    while i < n {
        *dst.add(i) = *src.add(i);
        i += 1;
    }
}

const _MEMCPY: () = unsafe {
    let ptr = &42;
    let mut ptr2 = ptr::null::<i32>();
    // Copy from ptr to ptr2.
    memcpy(&mut ptr2 as *mut _ as *mut _, &ptr as *const _ as *const _, mem::size_of::<&i32>());
    assert!(*ptr2 == 42);
};
```
What makes this code tricky is that pointers are "opaque blobs" in const-eval, we cannot just let people look at the individual bytes since *we don't know what those bytes look like* -- that depends on the absolute address the pointed-to object will be placed at. The code above "breaks apart" a pointer into individual bytes, and then puts them back together in the same order elsewhere. This PR implements the logic to properly track how those individual bytes relate to the original pointer, and to recognize when they are in the right order again.

We still reject constants where the final value contains a not-fully-put-together pointer: I have no idea how one could construct an LLVM global where one byte is defined as "the 3rd byte of a pointer to that other global over there" -- and even if LLVM supports this somehow, we can leave implementing that to a future PR. It seems unlikely to me anyone would even want this, but who knows.^^

This also changes the behavior of Miri, by tracking the order of bytes with provenance and only considering a pointer to have valid provenance if all bytes are in the original order again. This is related to rust-lang/unsafe-code-guidelines#558. It means one cannot implement XOR linked lists with strict provenance any more, which is however only of theoretical interest. Practically I am curious if anyone will show up with any code that Miri now complains about - that would be interesting data. Cc `@rust-lang/opsem`

Author: bors

core/src/ptr/mod.rs

@@ -1347,40 +1347,6 @@ pub const unsafe fn swap<T>(x: *mut T, y: *mut T) {
 /// assert_eq!(x, [7, 8, 3, 4]);
 /// assert_eq!(y, [1, 2, 9]);
 /// ```
-///
-/// # Const evaluation limitations
-///
-/// If this function is invoked during const-evaluation, the current implementation has a small (and
-/// rarely relevant) limitation: if `count` is at least 2 and the data pointed to by `x` or `y`
-/// contains a pointer that crosses the boundary of two `T`-sized chunks of memory, the function may
-/// fail to evaluate (similar to a panic during const-evaluation). This behavior may change in the
-/// future.
-///
-/// The limitation is illustrated by the following example:
-///
-/// ```
-/// use std::mem::size_of;
-/// use std::ptr;
-///
-/// const { unsafe {
-///     const PTR_SIZE: usize = size_of::<*const i32>();
-///     let mut data1 = [0u8; PTR_SIZE];
-///     let mut data2 = [0u8; PTR_SIZE];
-///     // Store a pointer in `data1`.
-///     data1.as_mut_ptr().cast::<*const i32>().write_unaligned(&42);
-///     // Swap the contents of `data1` and `data2` by swapping `PTR_SIZE` many `u8`-sized chunks.
-///     // This call will fail, because the pointer in `data1` crosses the boundary
-///     // between several of the 1-byte chunks that are being swapped here.
-///     //ptr::swap_nonoverlapping(data1.as_mut_ptr(), data2.as_mut_ptr(), PTR_SIZE);
-///     // Swap the contents of `data1` and `data2` by swapping a single chunk of size
-///     // `[u8; PTR_SIZE]`. That works, as there is no pointer crossing the boundary between
-///     // two chunks.
-///     ptr::swap_nonoverlapping(&mut data1, &mut data2, 1);
-///     // Read the pointer from `data2` and dereference it.
-///     let ptr = data2.as_ptr().cast::<*const i32>().read_unaligned();
-///     assert!(*ptr == 42);
-/// } }
-/// ```
 #[inline]
 #[stable(feature = "swap_nonoverlapping", since = "1.27.0")]
 #[rustc_const_stable(feature = "const_swap_nonoverlapping", since = "1.88.0")]
@@ -1409,9 +1375,7 @@ pub const unsafe fn swap_nonoverlapping<T>(x: *mut T, y: *mut T, count: usize) {
     const_eval_select!(
         @capture[T] { x: *mut T, y: *mut T, count: usize }:
         if const {
-            // At compile-time we want to always copy this in chunks of `T`, to ensure that if there
-            // are pointers inside `T` we will copy them in one go rather than trying to copy a part
-            // of a pointer (which would not work).
+            // At compile-time we don't need all the special code below.
             // SAFETY: Same preconditions as this function
             unsafe { swap_nonoverlapping_const(x, y, count) }
         } else {

coretests/tests/ptr.rs

@@ -936,22 +936,18 @@ fn test_const_swap_ptr() {
         assert!(*s1.0.ptr == 666);
         assert!(*s2.0.ptr == 1);
 
-        // Swap them back, again as an array.
+        // Swap them back, byte-for-byte
         unsafe {
             ptr::swap_nonoverlapping(
-                ptr::from_mut(&mut s1).cast::<T>(),
-                ptr::from_mut(&mut s2).cast::<T>(),
-                1,
+                ptr::from_mut(&mut s1).cast::<u8>(),
+                ptr::from_mut(&mut s2).cast::<u8>(),
+                size_of::<A>(),
             );
         }
 
         // Make sure they still work.
         assert!(*s1.0.ptr == 1);
         assert!(*s2.0.ptr == 666);
-
-        // This is where we'd swap again using a `u8` type and a `count` of `size_of::<T>()` if it
-        // were not for the limitation of `swap_nonoverlapping` around pointers crossing multiple
-        // elements.
     };
 }