Teach transmute_{ref,mut}! to handle slice DSTs #2428

joshlf · 2025-03-07T22:27:19Z

We generalize our existing support in util::macro_util, now relying on
PMEs to detect when transmutes cannot be supported. In order to continue
to support sized reference transmutation in a const context, we use
the autoref specialization trick to fall back to our old (const-safe)
implementation when both the source and destination type are Sized.

The main challenge in generalizing this support is making a trait
implementation available conditional on a PME. We do this using the
unsafe_with_size_eq! helper macro, which introduces
#[repr(transparent)] wrapper types, Src and Dst. Internal to this
macro, we implement SizeEq<Src<T>> for Dst<U>, with the implementation
of SizeEq::cast_from_raw containing the PME logic for size equality.
The macro's caller must promise to only use the Src and Dst types
to wrap the T and U types for which this PME logic has been applied
(or else the SizeEq impl could "leak" to types for which it is
unsound).

In addition, we generalize our prior support for transmuting between
unsized types. In particular, we previously used the SizeEq trait to
denote that two types have equal sizes in the face of a cast operation
(in particular, that *const T as *const U preserves referent size). In
this commit, we add support for metadata fix-up, which means that we
support casts for which *const T as *const U does not preserve
referent size. Instead, we compute an affine function at compile time
and apply it at runtime - computing the destination type's metadata as a
function of the source metadata, dst_meta = A + src_meta * B. A and
B are computed at compile time.

We generalize SizeEq to permit its cast_from_raw method to perform a
runtime metadata fix-up operation.

Makes progress on #1817

Co-authored-by: Jack Wrenn [email protected]

This PR is on branch transmute-ref-dst.

src/layout.rs

src/macros.rs

src/util/macros.rs

src/pointer/transmute.rs

codecov-commenter · 2025-03-25T20:16:03Z

Codecov Report

Attention: Patch coverage is 73.20000% with 67 lines in your changes missing coverage. Please review.

Project coverage is 88.85%. Comparing base (f9c9703) to head (f5fcda2).

Files with missing lines	Patch %	Lines
src/layout.rs	33.33%	28 Missing ⚠️
src/macros.rs	80.64%	18 Missing ⚠️
src/impls.rs	0.00%	8 Missing ⚠️
src/doctests.rs	0.00%	4 Missing ⚠️
src/lib.rs	55.55%	4 Missing ⚠️
src/pointer/transmute.rs	40.00%	3 Missing ⚠️
src/util/macros.rs	88.23%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2428      +/-   ##
==========================================
- Coverage   89.25%   88.85%   -0.40%     
==========================================
  Files          19       20       +1     
  Lines        5126     5312     +186     
==========================================
+ Hits         4575     4720     +145     
- Misses        551      592      +41

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joshlf · 2025-03-25T20:35:58Z

src/pointer/transmute.rs

        t
    }
 }

+// TODO: Update all `TransmuteFrom` safety proofs.


Dismissed because we forgot this important TODO: #2428 (comment)

src/util/macros.rs

jswrenn · 2025-06-03T16:47:22Z

src/util/macros.rs

+        // so permits interior mutation exactly when `T` does.
+        unsafe_impl!(T: ?Sized + Immutable => Immutable for $dst<T>);
+
+        $blk


Wonder if the outer unsafe required to currently call this macro leaks into cover $blk.

Tried offline; turned out to be very difficult, so we're going to punt for now.

jswrenn · 2025-06-03T16:49:46Z

src/util/macros.rs

+        if 1 == 0 {
+            let ptr = <$t as KnownLayout>::raw_dangling();
+            #[allow(unused_unsafe)]
+            // SAFETY: This call is never executed.
+            let ptr = unsafe { crate::pointer::PtrInner::new(ptr) };
+            #[allow(unused_unsafe)]
+            // SAFETY: This call is never executed.
+            let ptr = unsafe { cast!(ptr) };
+            let _ = <$dst<$u> as SizeEq<$src<$t>>>::cast_from_raw(ptr);
+        }


We should drop a comment as to why this block exists.

Is this okay? Can we rely on this PME occuring despite this code being dead?

Comment: Done.

Can we rely on the PME: Maybe not, but the follow-up PR will ensure that it's sound since SizeEq will no longer convey a safety post-condition that can be consumed by unsafe code - the only way to use it will be via SizeEq::cast_from_raw, which will guarantee that the PME is triggered.

We generalize our existing support in `util::macro_util`, now relying on PMEs to detect when transmutes cannot be supported. In order to continue to support sized reference transmutation in a `const` context, we use the autoref specialization trick to fall back to our old (`const`-safe) implementation when both the source and destination type are `Sized`. The main challenge in generalizing this support is making a trait implementation available conditional on a PME. We do this using the `unsafe_with_size_eq!` helper macro, which introduces `#[repr(transparent)]` wrapper types, `Src` and `Dst`. Internal to this macro, we implement `SizeEq<Src<T>> for Dst<U>`, with the implementation of `SizeEq::cast_from_raw` containing the PME logic for size equality. The macro's caller must promise to only use the `Src` and `Dst` types to wrap the `T` and `U` types for which this PME logic has been applied (or else the `SizeEq` impl could "leak" to types for which it is unsound). In addition, we generalize our prior support for transmuting between unsized types. In particular, we previously used the `SizeEq` trait to denote that two types have equal sizes in the face of a cast operation (in particular, that `*const T as *const U` preserves referent size). In this commit, we add support for metadata fix-up, which means that we support casts for which `*const T as *const U` does *not* preserve referent size. Instead, we compute an affine function at compile time and apply it at runtime - computing the destination type's metadata as a function of the source metadata, `dst_meta = A + src_meta * B`. `A` and `B` are computed at compile time. We generalize `SizeEq` to permit its `cast_from_raw` method to perform a runtime metadata fix-up operation. Makes progress on #1817 Co-authored-by: Jack Wrenn <[email protected]> gherrit-pr-id: Ib4bc62202e0b3b09d155333b525087f7aa8f02c2

This should have been changed in #2428, where we changed the in-practice safety invariant, but forgot to update the doc comment. Makes progress on #1817 gherrit-pr-id: I910c77e45532313e316ee443f9cf70d0d74cff67

This was referenced Mar 7, 2025

Remove obsolete safety proofs #2423

Merged

Implement traits for Cell #2421

Merged

[pointer] SizeEq supports raw pointer transmutes #2427

Merged

Teach transmute_{ref,mut}! to handle slice DSTs #1924

Closed

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch 6 times, most recently from 829b199 to a7c40db Compare March 8, 2025 16:58

joshlf force-pushed the I70d5aa5ace6bd2e39e679eac7f00a66d4b843d57 branch from 4d09d7c to f2ec335 Compare March 8, 2025 16:58

Base automatically changed from I70d5aa5ace6bd2e39e679eac7f00a66d4b843d57 to main March 8, 2025 19:16

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch 3 times, most recently from 98bbab7 to 5196fac Compare March 9, 2025 01:10

joshlf commented Mar 9, 2025

View reviewed changes

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch 2 times, most recently from 398966d to 976b234 Compare March 9, 2025 03:03

joshlf changed the base branch from main to I691b42ce8c0c3c6e5990e7684fc66f8f5dd73d85 March 9, 2025 03:03

joshlf mentioned this pull request Mar 9, 2025

[layout] Move Kani proofs from crate root #2430

Merged

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch 4 times, most recently from bd3b50a to e972a2c Compare March 9, 2025 06:47

joshlf commented Mar 9, 2025

View reviewed changes

src/pointer/transmute.rs Outdated Show resolved Hide resolved

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch 2 times, most recently from e2b7f7d to 459de15 Compare March 11, 2025 18:51

Base automatically changed from I691b42ce8c0c3c6e5990e7684fc66f8f5dd73d85 to main March 11, 2025 19:21

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch from 459de15 to 424ddb4 Compare March 11, 2025 20:19

joshlf changed the base branch from main to Icdb256acfdb274f34312cf5b216a02ca426338a1 March 11, 2025 20:19

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch from 3931fcf to db9d6c0 Compare March 25, 2025 20:11

joshlf commented Mar 25, 2025

View reviewed changes

src/pointer/transmute.rs Outdated

t

}

}

// TODO: Update all `TransmuteFrom` safety proofs.

Copy link

Member Author

joshlf Mar 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch from db9d6c0 to d170d54 Compare March 25, 2025 20:40

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch from d170d54 to 5d7c23b Compare March 28, 2025 00:11

joshlf mentioned this pull request Mar 28, 2025

[WIP] Update TransmuteFrom safety proofs #2469

Closed

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch 2 times, most recently from 85b4437 to 522d450 Compare March 28, 2025 19:41

joshlf mentioned this pull request Mar 28, 2025

Generalize SizeEq to SizeFrom, support shrinking #2472

Closed

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch 3 times, most recently from fbb37b8 to bb9b06d Compare March 28, 2025 23:31

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch from bb9b06d to 38c910a Compare April 7, 2025 22:33

joshlf mentioned this pull request Apr 7, 2025

[macros] Support shrinking reference transmutes #2487

Open

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch from 38c910a to 454f974 Compare April 7, 2025 23:35

jswrenn reviewed Jun 3, 2025

View reviewed changes

src/util/macros.rs Show resolved Hide resolved

jswrenn reviewed Jun 3, 2025

View reviewed changes

joshlf force-pushed the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch from 454f974 to f5fcda2 Compare June 3, 2025 19:13

jswrenn approved these changes Jun 3, 2025

View reviewed changes

jswrenn enabled auto-merge June 3, 2025 19:15

jswrenn added this pull request to the merge queue Jun 3, 2025

Merged via the queue into main with commit bc20681 Jun 3, 2025
89 checks passed

jswrenn deleted the Ib4bc62202e0b3b09d155333b525087f7aa8f02c2 branch June 3, 2025 19:50

joshlf mentioned this pull request Jun 4, 2025

Update SizeEq safety invariant #2564

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Teach transmute_{ref,mut}! to handle slice DSTs #2428

Teach transmute_{ref,mut}! to handle slice DSTs #2428

Uh oh!

joshlf commented Mar 7, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Mar 25, 2025 •

edited

Loading

Uh oh!

joshlf Mar 25, 2025

Uh oh!

Uh oh!

jswrenn Jun 3, 2025

Uh oh!

joshlf Jun 3, 2025

Uh oh!

jswrenn Jun 3, 2025

Uh oh!

jswrenn Jun 3, 2025

Uh oh!

joshlf Jun 3, 2025

Uh oh!

Uh oh!

Uh oh!

Teach transmute_{ref,mut}! to handle slice DSTs #2428

Teach transmute_{ref,mut}! to handle slice DSTs #2428

Uh oh!

Conversation

joshlf commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

joshlf Mar 25, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jswrenn Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

joshlf Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

jswrenn Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

jswrenn Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

joshlf Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

joshlf commented Mar 7, 2025 •

edited

Loading

codecov-commenter commented Mar 25, 2025 •

edited

Loading