don't apply temporary lifetime extension rules to non-extended super let

[dianne]

This changes the semantics for super let (and macros implemented in terms of it, such as pin!, format_args!, write!, and println!) as suggested by @theemathas in #145784 (comment), making super let initializers only count as extending expressions when the super let itself is within an extending block. Since super let initializers aren't temporary drop scopes, their temporaries outside of inner temporary scopes are effectively always extended, even when not in extending positions; this only affects two cases as far as I can tell:

• Block tail expressions in Rust 2024. This PR makes f(pin!({ &temp() })) drop temp() at the end of the block in Rust 2024, whereas previously it would live until after the call to f because syntactically the temp() was in an extending position as a result of super let in pin!'s expansion.
• super let nested within a non-extended super let is no longer extended. i.e. a normal let is required to treat super lets as extending (in which case nested super lets will also be extending).

Closes #145784

This is a breaking change. Both static and dynamic semantics are affected. The most likely breakage is for programs to stop compiling, but it's technically possible for drop order to silently change as well (as in #145784). Since this affects stable macros, it probably would need a crater run.

Nominating for discussion alongside #145784: @rustbot label +I-lang-nominated +I-libs-api-nominated

Tracking issue for super let: #139076

[rustbot]

r? @jackh726

rustbot has assigned @jackh726.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[dianne]

Copied from #145784 (comment), since I think this is a notable caveat of this PR and worth considering before approving it:

This comes with a bit of a gotcha in terms of temporary lifetimes: it might be strange that the temp() would live longer in

non_extending({ let x = { &temp() }; f(x) }); // ok

than in

non_extending({ super let x = { &temp() }; f(x) }); // error

Though the case for if let is similar: its scrutinee isn't a temporary scope and it doesn't have lifetime extension rules that can make block tail expressions' temporaries live longer.

[dianne]

Also copied since it motivates this PR: I think something like this may be necessary for the identity

&EXPR === { super let x = &EXPR; x }

to hold in both extending and non-extending contexts. Substituting, e.g. { &temp() } in for EXPR, the identity only currently holds in extending contexts in Rust 2024. In non-extending contexts,

&{ &temp() }

drops temp() when leaving the block, but

{ super let x = &{ &temp() }; x }

would extend it to outlive x. This PR shortens the lifetime of temp() such that it's dropped at the end of the block in both cases. I haven't done any rigorous proof or extensive testing that this PR together with #145342 makes the identity always hold, however.

[traviscross]

To confirm, with this PR, does this behavior hold (in Rust 2024)?:

fn f<T>(_: LogDrop<'_>, x: T) -> T { x }

// These two should be the same.
assert_drop_order(1..=3, |e| {
    let _v = f(e.log(2), &{ &raw const *&e.log(1) });
    drop(e.log(3));
});
assert_drop_order(1..=3, |e| {
    let _v = f(e.log(2), {
        super let v = &{ &raw const *&e.log(1) };
        v
    });
    drop(e.log(3));
});
// These two should be the same.
assert_drop_order(1..=3, |e| {
    let _v = f(e.log(1), &&raw const *&e.log(2));
    drop(e.log(3));
});
assert_drop_order(1..=3, |e| {
    let _v = f(e.log(1), {
        super let v = &&raw const *&e.log(2);
        v
    });
    drop(e.log(3));
});
// These two should be the same.
assert_drop_order(1..=2, |e| {
    let _v = &{ &raw const *&e.log(2) };
    drop(e.log(1));
});
assert_drop_order(1..=2, |e| {
    let _v = {
        super let v = &{ &raw const *&e.log(2) };
        v
    };
    drop(e.log(1));
});

Playground link

(If any of these are missing, please add them as tests.)

[theemathas]

super let nested within a non-extended super let is no longer extended

Does this PR affect any edition 2021 code?

[dianne]

To confirm, with this PR, does this behavior hold (in Rust 2024)?:

Those all hold under this PR, yes. I've added them all as tests (with some additional versioning to account for the Edition-dependent drop order in the first one); thanks!

super let nested within a non-extended super let is no longer extended

Does this PR affect any edition 2021 code?

Not that I'm aware of. That detail matters in Edition 2024, since the nested super let could have an extending borrow operator within a block tail expression in its initializer, which would previously extend the borrowed temporary's lifetime to outlive the outer super let's binding. In Edition 2021, that temporary outlives the outer super let's binding regardless, since block tail expressions (and super let initializers) aren't temporary scopes.

[traviscross]

Suggested change

	// We have extending borrow expressions within the initializer
	// expression.
	// We have extending borrow expressions within a non-extending
	// expression within the initializer expression.

(Revising my earlier text here.)

[traviscross]

This is correct, I believe. Let's propose to do it.

@rfcbot fcp merge

@dianne, let's document this in the Reference.

cc @rust-lang/lang-docs

[rfcbot]

Team member @traviscross has proposed to merge this. The next step is review by the rest of the tagged team members:

• @joshtriplett
• @nikomatsakis
• @scottmcm
• @tmandry
• @traviscross

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

[traviscross]

cc @m-ou-se @rust-lang/libs-api

[dianne]

With regard to the Reference docs, the ways this is exposed (as far as I'm aware) are through pin! and format_args!, neither of which appear to be documented there.

If pin! was in the Reference, I'd add a point to destructors.scope.lifetime-extension.exprs specifying the operand of an extending pin! to be an extending expression¹. Is there a plan to document pin! in the Reference, or would it make more sense to document its lifetime extension behavior in the standard library docs?

The same goes for format_args!, but that has an additional complication in that its behavior is affected by #145422 as well: currently whether or not the arguments to an extending format_args! are extending expressions depends on how many arguments there are.

This extends to write! and println! with multiple arguments as well (playground):

fn main() {
    assert_drop_order(1..=2, |o| {
       println!("{:?}{:?}", { &o.log(2) }, drop(o.log(1))); 
    });
}

This didn't compile before 1.89, but in 1.89, 1.90-beta, and nightly the o.log(2) temporary outlives the block (presumably due to #140748 landing in 1.89). Under this PR, it goes back to not compiling in Edition 2024. I've added a test for that: (diff).

It also doesn't compile in 1.89 with only a single { &temp() } format argument, due to how format_args! expands (#145422 (comment)). I'm hoping that means it'll be less likely to show up as a breakage in a crater run, but I wouldn't be surprised at all if something has started relying on it in the multiple-format-args case.

Footnotes

1. The real change this PR makes is that normal temporary scoping rules now apply to pin!'s operand, with lifetime extension only happening in extending expressions, so that its operand's temporary lifetime is like that of a borrow operator. Maybe that deserves an explicit note as well? ↩

[traviscross]

Is there a plan to document pin! in the Reference, or would it make more sense to document its lifetime extension behavior in the standard library docs?

It'd be best to (at least) document this in the Reference. E.g., we could say that certain built-in macros are extending expressions, and then we could list them and describe the caveats about when they are or are not extending expressions.

[m-ou-se]

This looks correct to me. It results in the right behavior for pin!() and format_args!() (the only stable uses of super let).

It does make super let as a statement even weirder, but that's not surprising. We already concluded that { super let _ = ..; ..} should be some sort of expression/operator/syntax of its own, rather than a block with a statement. This is just an extra argument for that.

View changes since this review