From d051b98a833ce8421e37ad6db689c384f68f8a2b Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Thu, 14 Aug 2014 18:07:33 +0200 Subject: [PATCH 01/38] first draft of non-zeroing drop RFC. --- active/0000-remove-drop-flag-and-zeroing.md | 529 ++++++++++++++++++++ 1 file changed, 529 insertions(+) create mode 100644 active/0000-remove-drop-flag-and-zeroing.md diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md new file mode 100644 index 00000000000..c7572941669 --- /dev/null +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -0,0 +1,529 @@ +- Start Date: 2014-08-14 +- RFC PR: (leave this empty) +- Rust Issue: (leave this empty) + +# Summary + +Revise language semantics for drop so that all branches move or drop +the same pieces of state ("value-paths"). Add lint(s) to inform the +programmer of situations when this new drop-semantics could cause +side-effects of RAII-style code (e.g. releasing locks, flushing +buffers) to occur sooner than expected. Remove the dynamic tracking +of whether a value has been dropped or not; in particular, (1.) remove +implicit addition of a drop-flag by `Drop` impl, and (2.) remove +implicit zeroing of the memory that occurs when values are dropped. + +# Motivation + +Currently, implementing `Drop` on a struct (or enum) injects a hidden +bit, known as the "drop-flag", into the struct (and likewise, each of +the the enum variants). The drop-flag, in tandem with Rust's implicit +zeroing of dropped values, tracks whether a value has already been +moved to another owner or been dropped. (See "How dynamic drop +semantics works" for more details.) + +Here are some problems with this: + + * Most important: implicit memory zeroing is a hidden cost that all + Rust programs are paying. With the removal of the drop flag, we + can remove implicit memory zeroing (or at least revisit its utility + -- there may be other motivations for implicit memory zeroing, + e.g. to try to keep secret data from being exposed to unsafe code). + + * Hidden bits are bad, part I: Users coming from a C/C++ background + expect `struct Foo { x: u32, y: u32 }` to occupy 8 bytes, but if + `Foo` implements `Drop`, the hidden drop flag will cause it to + double in size (16 bytes). + + * Hidden bits are bad, part II: Some users have expressed an + expectation that the drop-flag only be present for individual local + variables, but that is not what Rust does currently: when `Foo` + implements `Drop`, each instance of `Foo` carries a drop-flag, even + in contexts like a `Vec` or `[Foo, ..100]` where a program + cannot actually move individual values out of the collection. + Thus, the amount of extra memory being used by drop-flags is not + bounded by program stack growth; the memory wastage is strewn + throughout the heap. + +So, those are the main motivations for removing the drop flag. + +But, how do we actually remove the drop flag? The answer: By replacing +the dynamic drop semantics (that implicitly checks the flag to +determine if a value has already been dropped) with a static drop +semantics (that performs drop of certain values more eagerly, +i.e. before the end of their owner's lexical scope). + +A static drop semantics essentially works by inserting implicit calls +to `mem::drop` at certain points in the control-flow to ensure that +the set of values to drop is statically known at compile-time. +(See "How static drop semantics works" for more details.) + +There are two important things to note about a static drop semantics: + + 1. It is *equal* in expressive power to the Rust language as we know + it today. This is because, if the user is actually relying on the + drop-flag today in some variable or field declaration `x: T`, they + can replace that declaration with `x: Option` and thus recreate + the effect of the drop-flag. (Note that formal comparisons of + expressiveness typically say nothing about *convenience*.) + + 2. Static drop semantics could be *surprising* to Rust programmers + who are used to dynamic drop semantics, in terms of it potentially + invalidating certain kinds of + resource-acquisition-is-initialization (RAII) patterns. + In particular, an implicit early drop could lead to unexpected + side-effects occurring earlier than expected. Such side-effects + include: + + * Memory being deallocated (probably harmless, but some may care) + + * Output buffers being flushed in an output port. + + * Mutex locks being released (more worrisome potential change + in timing semantics). + +The bulk of the Detailed Design is dedicated to mitigating that second +observation, in order to reduce the expected number of surprises for +Rust programmers. The main idea for this mitigation is the addition +of one or more lints that report to the user when an side-effectful +early-drop will be implicitly injected into the code, and suggest to +them that they revise their code to remove the implicit injection +(e.g. by explicitly dropping the path in question, or by +re-establishing the drop obligation on the other control-flow paths, +or by rewriting the code to put in a manual drop-flag via +`Option`). + + +# Detailed design + +TODO + +This is the bulk of the RFC. Explain the design in enough detail for somebody familiar +with the language to understand, and for somebody familiar with the compiler to implement. +This should get into specifics and corner-cases, and include examples of how the feature is used. + +# Drawbacks + +* The lint may be annoying to users whose programs are not affected by + the early drops. (We mitigate this by providing ways for users to + opt-out of the lint `#[allow(unmarked_early_drops)]`, both in a + lexically scoped fashion, like other lints, and in a type-based + fashion via a `QuietEarlyDrop` trait.) + +* The early drops may surprise the users who are used to the dynamic + drop semantics. (We mitigate this by providing warnings via the + lint, a clear path for rewriting code in terms of `Option` to + emulate a drop-flag, and a way for users to enable a stricter lint: + `#[warn(quiet_early_drops)]` that reports all early drops, including + those hidden via `QuietEarlyDrop`.) + +* There may be benefits to implicit memory-zeroing that are not + accounted for in this RFC, in which case we may end up only removing + the drop-flag but not the implicit memory-zeroing. Still, even if + the only benefit is removing the drop-flag, it may still be worth + the pain of static drop semantics. + +# Alternatives + +## Do nothing + +Keep dynamic drop semantics, the drop flag, and the implicit memory +zeroing, paying their hidden costs in time and space. + +## Require explicit drops rather than injecting them + +Rather than injecting implicit drops where necessary, we could just +reject programs that have control-flow merge points with an +inconsistent set of incoming drop-obligations. + +This would be equivalent to doing `#[deny(unmarked_early_drops)]`. + +Felix (the author) was originally planning to take this approach, but +it became clear after a little experimentation that the annoyance +implied here made this a non-starter. + +## Associate drop flags with stack-0local variables alone + +I mentioned in "Hidden bits are bad, part II" that some users have +said they thought that the drop flag was only part of a local +variable's state, not part of every occurrence of a struct/enum. + +We could try to explore this option, but it seems potentially very +complicated to me. E.g. would each droppable structs embedded within a +type get its own drop flag when stack-allocated (thus making the +layout of a type potentially depend on where it is allocated; that, or +the drop flags would have to be stored out-of-band compared to the +location of the struct itself in memory). + +Besides, we would still need to do something about droppable types +that are *not* stack-allocated, which implies that we would still need +to do some sort of static drop semantics for those values. And if we +are going to do that anyway, we might as well apply it across the +board. + +# Unresolved questions + +I am not certain of all the implementation details of changes to the +`trans` module in `rustc`. + +# Appendix + +## Program illustrating space impact of hidden drop flag + + +```rust +#![feature(unsafe_destructor, macro_rules)] + +use std::mem; + +struct S0; +struct D0; +struct S3u8 { _x: [u8, ..3] } +struct D3u8 { _x: [u8, ..3] } +struct Si32 { _x: i32, } +struct Di32 { _x: i32, } +struct Si64 { _x: i64, } +struct Di64 { _x: i64, } + +macro_rules! show_me { + ($e:expr) => { println!("{:>#50s}: {}", stringify!($e), $e); } +} + +macro_rules! empty_drops { + ($($i:ident)*) => { $(impl Drop for $i { fn drop(&mut self) { } })* } +} + +empty_drops!(D0 D3u8 Di32 Di64) + +fn main() { + show_me!(mem::size_of::()); + show_me!(mem::size_of::()); + show_me!(mem::size_of::()); + show_me!(mem::size_of::()); + show_me!(mem::size_of::()); + show_me!(mem::size_of::()); + show_me!(mem::size_of::()); + show_me!(mem::size_of::()); + show_me!(mem::size_of::<[S3u8, ..100]>()); + show_me!(mem::size_of::<[D3u8, ..100]>()); + show_me!(mem::size_of::<[Si32, ..100]>()); + show_me!(mem::size_of::<[Di32, ..100]>()); +} +``` + +## How dynamic drop semantics works + +(This section is just presenting background information on the +semantics of `drop` and the drop-flag as it works in Rust today; it +does not contain any discussion of the changes being proposed by this +RFC.) + +A struct or enum implementing `Drop` will have its drop-flag +automatically set to a non-zero value when it is constructed. When +attempting to drop the struct or enum (i.e. when control reaches the +end of the lexical scope of its owner), the injected glue code will +only execute its associated `fn drop` if its drop-flag is non-zero. + +In addition, the compiler injects code to ensure that when a value is +moved to a new location in memory or dropped, then the original memory +is entirely zeroed. + +A struct/enum definition implementing `Drop` can be tagged with the +attribute `#[unsafe_no_drop_flag]`. When so tagged, the struct/enum +will not have a hidden drop flag embedded within it. In this case, the +injected glue code will execute the associated glue code +unconditionally, even though the struct/enum value may have been moved +to a new location in memory or dropped (in either case, the memory +representing the value will have been zeroed). + +The above has a number of implications: + + * A program can manually cause the drop code associated with a value + to be skipped by first zeroing out its memory. + + * A `Drop` implementation for a struct tagged with `unsafe_no_drop_flag` + must assume that it will be called more than once. (However, every + call to `drop` after the first will be given zeroed memory.) + +### Program illustrating semantic impact of hidden drop flag + +```rust +#![feature(macro_rules)] + +use std::fmt; +use std::mem; + +#[deriving(Clone,Show)] +struct S { name: &'static str } + +#[deriving(Clone,Show)] +struct Df { name: &'static str } + +#[deriving(Clone,Show)] +struct Pair{ x: X, y: Y } + +static mut current_indent: uint = 0; + +fn indent() -> String { + String::from_char(unsafe { current_indent }, ' ') +} + +impl Drop for Df { + fn drop(&mut self) { + println!("{}dropping Df {}", indent(), self.name) + } +} + +macro_rules! struct_Dn { + ($Dn:ident) => { + + #[unsafe_no_drop_flag] + #[deriving(Clone,Show)] + struct $Dn { name: &'static str } + + impl Drop for $Dn { + fn drop(&mut self) { + if unsafe { (0,0) == mem::transmute::<_,(uint,uint)>(self.name) } { + println!("{}dropping already-zeroed {}", + indent(), stringify!($Dn)); + } else { + println!("{}dropping {} {}", + indent(), stringify!($Dn), self.name) + } + } + } + } +} + +struct_Dn!(DnA) +struct_Dn!(DnB) +struct_Dn!(DnC) + +fn take_and_pass(t: T) { + println!("{}t-n-p took and will pass: {}", indent(), &t); + unsafe { current_indent += 4; } + take_and_drop(t); + unsafe { current_indent -= 4; } +} + +fn take_and_drop(t: T) { + println!("{}t-n-d took and will drop: {}", indent(), &t); +} + +fn xform(mut input: Df) -> Df { + input.name = "transformed"; + input +} + +fn foo(b: || -> bool) { + let mut f1 = Df { name: "f1" }; + let mut n2 = DnC { name: "n2" }; + let f3 = Df { name: "f3" }; + let f4 = Df { name: "f4" }; + let f5 = Df { name: "f5" }; + let f6 = Df { name: "f6" }; + let n7 = DnA { name: "n7" }; + let _fx = xform(f6); // `f6` consumed by `xform` + let _n9 = DnB { name: "n9" }; + let p = Pair { x: f4, y: f5 }; // `f4` and `f5` moved into `p` + let _f10 = Df { name: "f10" }; + + println!("foo scope start: {}", (&f3, &n7)); + unsafe { current_indent += 4; } + if b() { + take_and_pass(p.x); // `p.x` consumed by `take_and_pass`, which drops it + } + if b() { + take_and_pass(n7); // `n7` consumed by `take_and_pass`, which drops it + } + + // totally unsafe: manually zero the struct, including its drop flag. + unsafe fn manually_zero(s: &mut S) { + let len = mem::size_of::(); + let p : *mut u8 = mem::transmute(s); + for i in range(0, len) { + *p.offset(i as int) = 0; + } + } + unsafe { + manually_zero(&mut f1); + manually_zero(&mut n2); + } + println!("foo scope end"); + unsafe { current_indent -= 4; } + + // here, we drop each local variable, in reverse order of declaration. + // So we should see the following drop sequence: + // drop(f10), printing "Df f10" + // drop(p) + // ==> drop(p.y), printing "Df f5" + // ==> attempt to drop(and skip) already-dropped p.x, no-op + // drop(_n9), printing "DnB n9" + // drop(_fx), printing "Df transformed" + // attempt to drop already-dropped n7, printing "already-zeroed DnA" + // no drop of `f6` since it was consumed by `xform` + // no drop of `f5` since it was moved into `p` + // no drop of `f4` since it was moved into `p` + // drop(f3), printing "f3" + // attempt to drop manually-zeroed `n2`, printing "already-zeroed DnC" + // attempt to drop manually-zeroed `f1`, no-op. +} + +fn main() { + foo(|| true); +} +``` + +## How static drop semantics works + +(This section is presenting a detailed outline of how static drop +semantics, which is part of this RFC proposal, looks from the view +point of someone trying to understand a Rust program.) + +No struct or enum has an implicit drop-flag. When a local variable is +initialized, that establishes a set of "drop obligations": a set of +structural paths (e.g. a local `a`, or a path to a field `b.f.y`) that +need to be dropped (or moved away to a new owner). + +The drop obligations for a local variable `x` of struct-type `T` are +computed from analyzing the structure of `T`. If `T` itself +implements `Drop`, then `x` is a drop obligation. If `T` does not +implement `Drop`, then the set of drop obligations is the union of the +drop obligations of the fields of `T`. + +When a path is moved to a new location or consumed by a function call, +it is removed from the set of drop obligations. + +For example: + +```rust + +struct Pair{ x:X, y:Y } + +struct D; struct S; + +impl Drop for D { ... } + +fn test() -> bool { ... } + +fn xform(d:D) -> D { ... } + +fn f1() { + // At the outset, the set of drop obligations is + // just the set of moved input parameters (empty + // in this case). + + // DROP OBLIGATIONS + // ------------------------ + // { } + let pDD : Pair = ...; + // {pDD.x, pDD.y} + let pDS : Pair = ...; + // {pDD.x, pDD.y, pDS.x} + let some_d : Option; + // {pDD.x, pDD.y, pDS.x} + if test() { + // {pDD.x, pDD.y, pDS.x} + { + let temp = xform(pDD.x); + // { pDD.y, pDS.x, temp} + some_d = Some(temp); + // { pDD.y, pDS.x, temp, some_d} + } // END OF SCOPE for `temp` + // { pDD.y, pDS.x, some_d} + } else { + { + // {pDD.x, pDD.y, pDS.x} + let z = D; + // {pDD.x, pDD.y, pDS.x, z} + + // This drops `pDD.y` before + // moving `pDD.x` there. + pDD.y = pDD.x; + + // { pDD.y, pDS.x, z} + some_d = None; + // { pDD.y, pDS.x, z, some_d} + } // END OF SCOPE for `z` + // { pDD.y, pDS.x, some_d} + } + + // MERGE POINT: set of drop obligations must + // match on all incoming control-flow paths... + // + // ... which they do in this case. + + // (... some code that does not change drop obligations ...) + + // { pDD.y, pDS.x, some_d} +} + +// `f2` is similar to `f1`, except that it will have differing set +// of drop obligations at the merge point, necessitating a hidden +// drop call. +fn f2() { + // At the outset, the set of drop obligations is + // just the set of moved input parameters (empty + // in this case). + + // DROP OBLIGATIONS + // ------------------------ + // { } + let pDD : Pair = ...; + // {pDD.x, pDD.y} + let pDS : Pair = ...; + // {pDD.x, pDD.y, pDS.x} + let some_d : Option; + // {pDD.x, pDD.y, pDS.x} + if test() { + // {pDD.x, pDD.y, pDS.x} + { + let temp = xform(pDD.y); + // {pDD.x, pDS.x, temp} + some_d = Some(temp); + // {pDD.x, pDS.x, temp, some_d} + } // END OF SCOPE for `temp` + // {pDD.x, pDS.x, some_d} + + // MERGE POINT PREDECESSOR 1 + + // implicit drops injected: drop(pDD.y) + } else { + { + // {pDD.x, pDD.y, pDS.x} + let z = D; + // {pDD.x, pDD.y, pDS.x, z} + + // This drops `pDD.y` before + // moving `pDD.x` there. + pDD.y = pDD.x; + + // { pDD.y, pDS.x, z} + some_d = None; + // { pDD.y, pDS.x, z, some_d} + } // END OF SCOPE for `z` + // { pDD.y, pDS.x, some_d} + + // MERGE POINT PREDECESSOR 2 + + // implicit drops injected: drop(pDD.y) + } + + // MERGE POINT: set of drop obligations must + // match on all incoming control-flow paths. + // + // For the original user code, they did not + // in this case. + // + // Therefore, implicit drops are injected up + // above, to ensure that the set of drop + // obligations match. + + // After the implict drops, the resulting + // remaining drop obligations are the + // following: + + // { pDS.x, some_d} +} +``` + From c12de48356c7b44d0dab71f6f1be53afbd838845 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Thu, 14 Aug 2014 18:21:35 +0200 Subject: [PATCH 02/38] Threw a new idea into the "Alternatives" section. --- active/0000-remove-drop-flag-and-zeroing.md | 32 ++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index c7572941669..16f2e8c5f06 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -142,7 +142,7 @@ Felix (the author) was originally planning to take this approach, but it became clear after a little experimentation that the annoyance implied here made this a non-starter. -## Associate drop flags with stack-0local variables alone +## Associate drop flags with stack-local variables alone I mentioned in "Hidden bits are bad, part II" that some users have said they thought that the drop flag was only part of a local @@ -161,6 +161,36 @@ to do some sort of static drop semantics for those values. And if we are going to do that anyway, we might as well apply it across the board. +## Separate individual and grouped instances of a type + +(This is a variant on "Associate drop flags with stack-local variables +alone" above, but with a clearer vision for how it would work.) + +Instead of having a single `size_of` value for a given type, treat +each type as having two different sizes: `indiv_size_of::` and +`group_size_of::`. + +An individual instance can be moved on its own. It gets a drop-flag. + +An instance that is part of a group cannot be moved on its own. The +whole group gets a drop flag, but each instance within it does not. +(I am assuming the group is itself allocated as an individual +instance, though perhaps one could recursively structure a group made +up of groups; that is an unclear aspect of this thought-exercise.) + +When looking at a slice `[T]`, the instances are part of the group, +and one uses `group_size_of::` for offset calculations. + +For enums, structs, and tuples, the fields are considered individuals. +(Though perhaps we could have "atomic" enums/structs/tuples where the +fields are considered part of the group as a whole and cannot be +independently moved to new owners on control flow branches.) + +This is an interesting thought exercise, but a pretty serious +language/library change, and it is not clear whether it would be any +better than static drop semantics in terms of Rust's language +complexity budget. + # Unresolved questions I am not certain of all the implementation details of changes to the From b449d0e0f740df4e2b297adc40cb581e55b59497 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Thu, 14 Aug 2014 23:01:46 +0200 Subject: [PATCH 03/38] Started the Detailed Design by using a former appendix as its basis. Added an unresolved Q regarding wildcards. --- active/0000-remove-drop-flag-and-zeroing.md | 389 ++++++++++++-------- 1 file changed, 233 insertions(+), 156 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 16f2e8c5f06..666c92a391f 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -56,7 +56,8 @@ i.e. before the end of their owner's lexical scope). A static drop semantics essentially works by inserting implicit calls to `mem::drop` at certain points in the control-flow to ensure that the set of values to drop is statically known at compile-time. -(See "How static drop semantics works" for more details.) +(See "How static drop semantics works" in the detailed design for more +discussion of how this is done.) There are two important things to note about a static drop semantics: @@ -102,6 +103,161 @@ This is the bulk of the RFC. Explain the design in enough detail for somebody fa with the language to understand, and for somebody familiar with the compiler to implement. This should get into specifics and corner-cases, and include examples of how the feature is used. +## How static drop semantics works + +(This section is presenting a detailed outline of how static drop +semantics, which is part of this RFC proposal, looks from the view +point of someone trying to understand a Rust program.) + +No struct or enum has an implicit drop-flag. When a local variable is +initialized, that establishes a set of "drop obligations": a set of +structural paths (e.g. a local `a`, or a path to a field `b.f.y`) that +need to be dropped (or moved away to a new owner). + +The drop obligations for a local variable `x` of struct-type `T` are +computed from analyzing the structure of `T`. If `T` itself +implements `Drop`, then `x` is a drop obligation. If `T` does not +implement `Drop`, then the set of drop obligations is the union of the +drop obligations of the fields of `T`. + +When a path is moved to a new location or consumed by a function call, +it is removed from the set of drop obligations. + +For example: + +```rust + +struct Pair{ x:X, y:Y } + +struct D; struct S; + +impl Drop for D { ... } + +fn test() -> bool { ... } + +fn xform(d:D) -> D { ... } + +fn f1() { + // At the outset, the set of drop obligations is + // just the set of moved input parameters (empty + // in this case). + + // DROP OBLIGATIONS + // ------------------------ + // { } + let pDD : Pair = ...; + // {pDD.x, pDD.y} + let pDS : Pair = ...; + // {pDD.x, pDD.y, pDS.x} + let some_d : Option; + // {pDD.x, pDD.y, pDS.x} + if test() { + // {pDD.x, pDD.y, pDS.x} + { + let temp = xform(pDD.x); + // { pDD.y, pDS.x, temp} + some_d = Some(temp); + // { pDD.y, pDS.x, temp, some_d} + } // END OF SCOPE for `temp` + // { pDD.y, pDS.x, some_d} + } else { + { + // {pDD.x, pDD.y, pDS.x} + let z = D; + // {pDD.x, pDD.y, pDS.x, z} + + // This drops `pDD.y` before + // moving `pDD.x` there. + pDD.y = pDD.x; + + // { pDD.y, pDS.x, z} + some_d = None; + // { pDD.y, pDS.x, z, some_d} + } // END OF SCOPE for `z` + // { pDD.y, pDS.x, some_d} + } + + // MERGE POINT: set of drop obligations must + // match on all incoming control-flow paths... + // + // ... which they do in this case. + + // (... some code that does not change drop obligations ...) + + // { pDD.y, pDS.x, some_d} +} + +// `f2` is similar to `f1`, except that it will have differing set +// of drop obligations at the merge point, necessitating a hidden +// drop call. +fn f2() { + // At the outset, the set of drop obligations is + // just the set of moved input parameters (empty + // in this case). + + // DROP OBLIGATIONS + // ------------------------ + // { } + let pDD : Pair = ...; + // {pDD.x, pDD.y} + let pDS : Pair = ...; + // {pDD.x, pDD.y, pDS.x} + let some_d : Option; + // {pDD.x, pDD.y, pDS.x} + if test() { + // {pDD.x, pDD.y, pDS.x} + { + let temp = xform(pDD.y); + // {pDD.x, pDS.x, temp} + some_d = Some(temp); + // {pDD.x, pDS.x, temp, some_d} + } // END OF SCOPE for `temp` + // {pDD.x, pDS.x, some_d} + + // MERGE POINT PREDECESSOR 1 + + // implicit drops injected: drop(pDD.y) + } else { + { + // {pDD.x, pDD.y, pDS.x} + let z = D; + // {pDD.x, pDD.y, pDS.x, z} + + // This drops `pDD.y` before + // moving `pDD.x` there. + pDD.y = pDD.x; + + // { pDD.y, pDS.x, z} + some_d = None; + // { pDD.y, pDS.x, z, some_d} + } // END OF SCOPE for `z` + // { pDD.y, pDS.x, some_d} + + // MERGE POINT PREDECESSOR 2 + + // implicit drops injected: drop(pDD.y) + } + + // MERGE POINT: set of drop obligations must + // match on all incoming control-flow paths. + // + // For the original user code, they did not + // in this case. + // + // Therefore, implicit drops are injected up + // above, to ensure that the set of drop + // obligations match. + + // After the implict drops, the resulting + // remaining drop obligations are the + // following: + + // { pDS.x, some_d} +} +``` + + + # Drawbacks * The lint may be annoying to users whose programs are not affected by @@ -193,8 +349,82 @@ complexity budget. # Unresolved questions -I am not certain of all the implementation details of changes to the -`trans` module in `rustc`. +## How should moving into wildcards be handled? + +In an example like: + +```rust +enum Pair { Two(X,Y), One(X), Zed } +let x : Pair = ...; // FD is some eFfectful Drop thing. +match x { + Two(payload, _) => { + ... code working with, then dropping, payload ... + } + One(payload) => { + ... code working with, then dropping, payload ... + } + Zed => { + } +} +``` + +In the above example, when the first match arm matches, we move `x` +into it, binding its first tuple component as `payload`. But how +should the second tuple component of `x` be handled? We need to drop +it at some point, since we need for all of `x` to be dropped at +the point in the control-flow where all of the match arms meet. + +The most obvious options are: + + 1. In any given arm, implicitly drop state bound to `_` at the end of the + arm's scope. + + This would be as if the programmer had actually written: + + ```rust + Two(payload, _ignore_me) => { + ... code working with, then dropping, payload ... + } + ``` + + 2. In any given arm, implicitly drop state bound to `_` at the + beginning of the arm's scope. + + This would be as if the programmer had actually written: + + ```rust + Two(payload, _ignore_me) => { + ... code working with, then dropping, payload ... + } + ``` + + 3. Disallow moving into `_` patterns; force programmer to name them + and deal with them. + + While this is clearly a clean conservative solution, it is also + awkward when you consider attempting to simplify the code above + like so (illegal under the suggested scheme): + +```rust +enum Pair { Two(X,Y), One(X), Zed } +let x : Pair = ...; // FD is some eFfectful Drop thing. +match x { + Two(payload, _) | + One(payload) => { + ... code working with, then dropping, payload ... + } + Zed => { + } +} +``` + + +## Implementation gotchas in `trans` + +I implemented the `borrowck` and `dataflow` changes necessary to +compute the set of drop obligations accurately, but I am not certain +of all the implementation details of changes to the `trans` module in +`rustc`. # Appendix @@ -404,156 +634,3 @@ fn main() { } ``` -## How static drop semantics works - -(This section is presenting a detailed outline of how static drop -semantics, which is part of this RFC proposal, looks from the view -point of someone trying to understand a Rust program.) - -No struct or enum has an implicit drop-flag. When a local variable is -initialized, that establishes a set of "drop obligations": a set of -structural paths (e.g. a local `a`, or a path to a field `b.f.y`) that -need to be dropped (or moved away to a new owner). - -The drop obligations for a local variable `x` of struct-type `T` are -computed from analyzing the structure of `T`. If `T` itself -implements `Drop`, then `x` is a drop obligation. If `T` does not -implement `Drop`, then the set of drop obligations is the union of the -drop obligations of the fields of `T`. - -When a path is moved to a new location or consumed by a function call, -it is removed from the set of drop obligations. - -For example: - -```rust - -struct Pair{ x:X, y:Y } - -struct D; struct S; - -impl Drop for D { ... } - -fn test() -> bool { ... } - -fn xform(d:D) -> D { ... } - -fn f1() { - // At the outset, the set of drop obligations is - // just the set of moved input parameters (empty - // in this case). - - // DROP OBLIGATIONS - // ------------------------ - // { } - let pDD : Pair = ...; - // {pDD.x, pDD.y} - let pDS : Pair = ...; - // {pDD.x, pDD.y, pDS.x} - let some_d : Option; - // {pDD.x, pDD.y, pDS.x} - if test() { - // {pDD.x, pDD.y, pDS.x} - { - let temp = xform(pDD.x); - // { pDD.y, pDS.x, temp} - some_d = Some(temp); - // { pDD.y, pDS.x, temp, some_d} - } // END OF SCOPE for `temp` - // { pDD.y, pDS.x, some_d} - } else { - { - // {pDD.x, pDD.y, pDS.x} - let z = D; - // {pDD.x, pDD.y, pDS.x, z} - - // This drops `pDD.y` before - // moving `pDD.x` there. - pDD.y = pDD.x; - - // { pDD.y, pDS.x, z} - some_d = None; - // { pDD.y, pDS.x, z, some_d} - } // END OF SCOPE for `z` - // { pDD.y, pDS.x, some_d} - } - - // MERGE POINT: set of drop obligations must - // match on all incoming control-flow paths... - // - // ... which they do in this case. - - // (... some code that does not change drop obligations ...) - - // { pDD.y, pDS.x, some_d} -} - -// `f2` is similar to `f1`, except that it will have differing set -// of drop obligations at the merge point, necessitating a hidden -// drop call. -fn f2() { - // At the outset, the set of drop obligations is - // just the set of moved input parameters (empty - // in this case). - - // DROP OBLIGATIONS - // ------------------------ - // { } - let pDD : Pair = ...; - // {pDD.x, pDD.y} - let pDS : Pair = ...; - // {pDD.x, pDD.y, pDS.x} - let some_d : Option; - // {pDD.x, pDD.y, pDS.x} - if test() { - // {pDD.x, pDD.y, pDS.x} - { - let temp = xform(pDD.y); - // {pDD.x, pDS.x, temp} - some_d = Some(temp); - // {pDD.x, pDS.x, temp, some_d} - } // END OF SCOPE for `temp` - // {pDD.x, pDS.x, some_d} - - // MERGE POINT PREDECESSOR 1 - - // implicit drops injected: drop(pDD.y) - } else { - { - // {pDD.x, pDD.y, pDS.x} - let z = D; - // {pDD.x, pDD.y, pDS.x, z} - - // This drops `pDD.y` before - // moving `pDD.x` there. - pDD.y = pDD.x; - - // { pDD.y, pDS.x, z} - some_d = None; - // { pDD.y, pDS.x, z, some_d} - } // END OF SCOPE for `z` - // { pDD.y, pDS.x, some_d} - - // MERGE POINT PREDECESSOR 2 - - // implicit drops injected: drop(pDD.y) - } - - // MERGE POINT: set of drop obligations must - // match on all incoming control-flow paths. - // - // For the original user code, they did not - // in this case. - // - // Therefore, implicit drops are injected up - // above, to ensure that the set of drop - // obligations match. - - // After the implict drops, the resulting - // remaining drop obligations are the - // following: - - // { pDS.x, some_d} -} -``` - From 56bc05da6c28b2f547695c04611f8e4ce5613c4f Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 01:53:04 +0200 Subject: [PATCH 04/38] enough written that I am tempted to advertise. --- active/0000-remove-drop-flag-and-zeroing.md | 389 ++++++++++++++++++-- 1 file changed, 355 insertions(+), 34 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 666c92a391f..84373d02c37 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -69,19 +69,9 @@ There are two important things to note about a static drop semantics: expressiveness typically say nothing about *convenience*.) 2. Static drop semantics could be *surprising* to Rust programmers - who are used to dynamic drop semantics, in terms of it potentially - invalidating certain kinds of - resource-acquisition-is-initialization (RAII) patterns. - In particular, an implicit early drop could lead to unexpected - side-effects occurring earlier than expected. Such side-effects - include: - - * Memory being deallocated (probably harmless, but some may care) - - * Output buffers being flushed in an output port. - - * Mutex locks being released (more worrisome potential change - in timing semantics). + who are used to dynamic drop semantics. In particular, an implicit early + drop could lead to unexpected side-effects occurring earlier than + expected. The bulk of the Detailed Design is dedicated to mitigating that second observation, in order to reduce the expected number of surprises for @@ -97,17 +87,27 @@ or by rewriting the code to put in a manual drop-flag via # Detailed design -TODO +The change suggested by this RFC has three parts: + +1. Change from a dynamic drop semantics to a static drop semantics, + +2. Provide one or more lints to inform the programmer of potential + suprises that may arise from earlier drops that are caused by the + static drop semantics, and + +3. Remove the implicitly added drop-flag, and the implicit zeroing of + the memory for dropped values. -This is the bulk of the RFC. Explain the design in enough detail for somebody familiar -with the language to understand, and for somebody familiar with the compiler to implement. -This should get into specifics and corner-cases, and include examples of how the feature is used. + +Each of the three parts is given its own section below. ## How static drop semantics works -(This section is presenting a detailed outline of how static drop -semantics, which is part of this RFC proposal, looks from the view -point of someone trying to understand a Rust program.) +This section presents a detailed outline of how static drop semantics +looks from the view point of someone trying to understand a Rust +program. + +### Drop obligations No struct or enum has an implicit drop-flag. When a local variable is initialized, that establishes a set of "drop obligations": a set of @@ -123,6 +123,8 @@ drop obligations of the fields of `T`. When a path is moved to a new location or consumed by a function call, it is removed from the set of drop obligations. +### Example of code with unchanged behavior under static drop semantics + For example: ```rust @@ -146,35 +148,35 @@ fn f1() { // ------------------------ // { } let pDD : Pair = ...; - // {pDD.x, pDD.y} + // { pDD.x, pDD.y } let pDS : Pair = ...; - // {pDD.x, pDD.y, pDS.x} + // { pDD.x, pDD.y, pDS.x } let some_d : Option; - // {pDD.x, pDD.y, pDS.x} + // { pDD.x, pDD.y, pDS.x } if test() { - // {pDD.x, pDD.y, pDS.x} + // { pDD.x, pDD.y, pDS.x } { let temp = xform(pDD.x); - // { pDD.y, pDS.x, temp} + // { pDD.y, pDS.x, temp } some_d = Some(temp); - // { pDD.y, pDS.x, temp, some_d} + // { pDD.y, pDS.x, temp, some_d } } // END OF SCOPE for `temp` - // { pDD.y, pDS.x, some_d} + // { pDD.y, pDS.x, some_d } } else { { - // {pDD.x, pDD.y, pDS.x} + // { pDD.x, pDD.y, pDS.x } let z = D; - // {pDD.x, pDD.y, pDS.x, z} + // { pDD.x, pDD.y, pDS.x, z } // This drops `pDD.y` before // moving `pDD.x` there. pDD.y = pDD.x; - // { pDD.y, pDS.x, z} + // { pDD.y, pDS.x, z } some_d = None; - // { pDD.y, pDS.x, z, some_d} + // { pDD.y, pDS.x, z, some_d } } // END OF SCOPE for `z` - // { pDD.y, pDS.x, some_d} + // { pDD.y, pDS.x, some_d } } // MERGE POINT: set of drop obligations must @@ -182,11 +184,16 @@ fn f1() { // // ... which they do in this case. + // { pDD.y, pDS.x, some_d } + // (... some code that does not change drop obligations ...) - // { pDD.y, pDS.x, some_d} + // { pDD.y, pDS.x, some_d } } + +### Example of code with changed behavior under static drop semantics + // `f2` is similar to `f1`, except that it will have differing set // of drop obligations at the merge point, necessitating a hidden // drop call. @@ -253,10 +260,256 @@ fn f2() { // following: // { pDS.x, some_d} + + // (... some code that does not change drop obligations ...) + + // { pDS.x, some_d} +} +``` + +### Control-flow sensitivity + +Note that the static drop semantics is based on a control-flow +analysis, *not* just the lexical nesting structure of the code. + +In particular: If control flow splits at a point like an if-expression, +but the two arms never meet + +### match expressons and enum variants + +The examples above used just structs and `if` expressions, but there +is an additional twist introduced by `enum` types. The examples above +showed that a struct type can be partially moved: one of its fields +can be moved away while the other is still present, and this can be +faithfully represented in the set of drop oblgiations. +But consider an `enum` and `match`: + +```rust +pub enum Pairy { Two(X,X), One(X,X) } +pub fn foo(c: || -> Pairy, + dA: |A| -> i8, + dR: |&A| -> i8) -> i8 { + let s = c(); + let ret = match s { + Two(ref r1, ref r2) => { + dR(r1) + dR(r2) + } + One(a1, a2) => { + dA(a1) + dA(a2) + } + }; + c(); + ret } ``` +In the above code, which is legal today in Rust, we have an arm for +`Two` that matches the input `s` by reference, while the arm for `One` +moves `s` into the match. That is, if the `Two` arm matches, then `s` +is left in place to be dropped at the end of the execution of `foo()`, +while if the `One` arm matches, then the `s` is deconstructed and +moved in pieces into `a1` and `a2`, which are themselves then consumed +by the calls to `dA`. + +While we *could* attempt to continue supporting this style of code +(see "variant-predicated drop-obligations" in the Alternatives +section), it seems simpler if we just disallow it. This RFC +proposes the following rule: if any arm in a match consumes +the input via `move`, then *every* arm in the match must consume the +input *by the end of each arm's associated body*. + +That last condition is crucial, because it enables patterns like +this to continue working: +```rust + match s { + One(a1, a2) => { // a moving match here + dA(a1) + dA(a2) + } + Two(_, _) => { // a non-binding match here + helper_function(s) + } + }; + +``` + +### Type parameters + +With respect to static drop semantics, there is not much to say about +type parameters: unless they have the `Copy` bound, we must assume +that they implement `Drop`, and therefore introduce drop obligations +(the same as types that actually do implement `Drop`, as illustrated +above). + +## Early drop lints + +Some users may be surprised by the implicit drops that are injected +by static drop semantics, especially if the drop code being executed +has observable side-effects. + +Such side-effects include: + + * Memory being deallocated earlier than expected (probably harmless, + but some may care) + + * Output buffers being flushed in an output port earlier than + expected. + + * Mutex locks being released earlier than expected (a worrisome + change in timing semantics when writing concurrent algorithms). + +In particular, the injection of the implcit drops could silently +invalidate certain kinds of "resource acquisition is initialization" +(RAII) patterns. + +It is important to keep in mind that one can always recreate the +effect of the former drop flag by wrapping one's type `T` in an +`Option`; therefore, the problem is *not* that such RAII patterns +cannot be expresed. It is merely that a user may assume that a +variable binding induces RAII-style effect, and that assumpion is then +invalidated due to a errant move on one control-flow branch. + +Therefore, to defend against users being surprised by the early +implicit drops induced by static drop semantics, this RFC proposes +adding lints that tell the user about the points in the control-flow +where implcit drops are injected. The specific proposal is to add two +lints, named `quiet_early_drop` and `unmarked_early_drop`, with the +default settings `#[allow(quiet_early_drop)]` and +`#[warn(unmarked_early_drop)]`. (The two lints are similar in name +because they provide very similar functionality; the only difference +is how aggressively each reports injected drop invocations.) + +### The `unmarked_early_drop` lint + +Here is an example piece of code (very loosely adapted from the Rust +`sync` crate): + +```rust + let (guard, state) = self.lock(); // (`guard` is mutex `LockGuard`) + ... + if state.disconnected { + ... + } else { + ... + match f() { + Variant1(payload) => g(payload, guard), + Variant2 => {} + } + + ... // (***) + + Ok(()) + } +``` + +In the `Variant1` arm above, `guard` is consumed by `g`. Threfore, +when the bit of code labelled with a `(***)` represents a span that, +when reached via the `Variant2` branch of the match statement, has the +`guard` still held under dynamic drop semantics, but the `guard` is +*released* under static drop semantics. + +The `unmwarked_early_drop` lint is meant to catch cases such as this, +where the user has inadvertantly written code where static drop +semantics injects an implicit call to a side-effectful `drop` method. + +Assuming that the `LockGuard` has a `Drop` impl but does not implement +the `QuietEarlyDrop` trait (see below `quiet_early_drop` lint below), +then the #[warn(unmarked_early_drop)]` lint will report a warning for +the code above, telling the user that the `guard` is moved away on the +`Variant1` branch but not on the other branches. In general the lint +cannot know what the actual intention of the user was. Therefore, the +lint suggests that the user either (1.) add an explicit drop call, for +clarity, or (2.) reinitialize `guard` on the `Variant1` arm, or (3.) +emulate a drop-flag by using `Option` instead of +`LockGuard` as the type of `guard`. + +### The `quiet_early_drop` lint and `QuietEarlyDrop` trait + +To be effective, a lint must not issue a significant number of false +positives: i.e., we do not want to tell the user about every site in +their crate where a `String` was dropped on one branch but not +another. + +More generally, it is likely that most sites where `Vec` is +dropped are not of interest either. Instead, the user is likely to +want to focus on points in the control flow where *effectful* drops +are executed (such as flushing output buffers or releasing locks). + +Meanwhile, some users may still care about every potential +side-effect, even those that their libraries have deemed "pure". Some +users may just want, out of principle, to mark every early drop +explcitly, in the belief that such explicitness improves code +comprehension. + +Therefore, rather than provide just a single lint for warning about +all occurrences of injected early drops, this proposal suggests a +simple two-tier structure: by default, `Drop` implementations are +assumed to have significant side-effects, and thus qualify for warning +via the aforementioned `unmarked_early_drop` trait. However, when +defining a type, one can implement the `QuietEarlyDrop` trait, which +recategorizes the type as having a `Drop` implementation that "pure" +(i.e. does not exhibit side-effects that the client of the crate is +likely to care about). + +An easy example of such a type whose `drop` method is likely to be +considered pure is `Vec`, since the only side-effect of dropping a +`Vec` is deallocation of its backing buffer. More generally, +`Vec` should be `QuietEarlyDrop` for any `T` that is also +`QuietEarlyDrop`. + +If a type implements `QuietEarlyDrop`, then early implicit drops of +that type will no longer be reported by `#[warn(unmarked_early_drop)]` +(instead, such a type becomes the reponsibility of the +`#[allow(quiet_early_drop)]` lint). Thus, the former lint will +hopefully provide well-focused warnings with a low false-positive +rate, while the latter, being set to `allow` by default, will +not generate much noise. + +Meanwhile, to ensure that a particular fn item has no hidden early +drops, one can turn on `#[deny(quiet_early_drop)]` and +`#[deny(unmarked_early_drop)]`, and then all statically injected drops +are reported (and the code rejected if any are present), regardless of +whether the types involved implement `QuietEarlyDrop` or not. + +### Scope end for owner can handle mismatched drop obligations + +Consider again the long examples from the "How static drop semantics +works" section above. Both examples took care to explcitly include a +bit at the end marked with the comment "(... some code that does not +change drop obligations ...)". This represents some potentially +side-effectful code that comes between a merge-point and the end of +the procedure (or more generally, the end of the lexical scope for +some local varibles that own paths in the set of drop obligations). + +If you hit a merge-point with two sets of drop obligations like `{ +pDD.x, pDD.y }` and `{ pDD.x, z }`, and there is no side-effectful +computation between that merge-point and the end of the scope for +`pDD` and `z`, then there is no problem with the mismatches between +the set of drop obligations, and neither lint should report anything. + +### Type parameters, revisited + +We noted in the "How static drop semantics works" section that +type parameters are not particularly special with respect t +static drop semantics. + +However, with the lints there is potential for type parameters to be +treated specially. + +Nonetheless, this RFC currently proposes that type parameters not be +treated specially by the lints: if you have mismatched drop +obligations, then that represents a hidden implicit drop that you may +not have known was there, and it behooves you to make an explicit call +to `drop`. + +(See further discussion in the "Unresolved Questions.") + +## Removing the drop-flag; removing memory zeroing + +With the above two pieces in place, the remainder is trivial. Namely: +once static drop semantics is actually implemented, then the compiler +can be revised to no longer inject a drop flag into structs and enums that +implement `Drop`, and likewise memory zeroing can be removed. # Drawbacks @@ -298,6 +551,29 @@ Felix (the author) was originally planning to take this approach, but it became clear after a little experimentation that the annoyance implied here made this a non-starter. +## Do this, but add support for variant-predicated drop-obligations + +In "match expressons and enum variants" above, this RFC proposed a +rule that if any arm in a match consumes the input via `move`, then +every arm in the match must consume the input (by the end of its +body). + +There is an alterative, however. We could enrich the structure of +drop-obligations to include paths that are predicated on enum +variants, like so: `{(s is Two => s#0), (s is Two => s#1)}`. This +represents the idea that (1.) all control flows where `s` is the `One` +variant dropped `s` entirely but also, (2.) all control flows where +`s` is the `Two` variant still has the 0'th and 1'st tuple compoents +remaining as drop obligations. + +I do not currently know how to efficiently implement such an enriched +drop-obligation representation. It seems it would get nasty when +one considers that these predicates can accumulate. + +Also, if we do figure out how to implement this, we could add this +later backward compatibly. I do not want to attempt to implement it +in the short-term. + ## Associate drop flags with stack-local variables alone I mentioned in "Hidden bits are bad, part II" that some users have @@ -349,6 +625,51 @@ complexity budget. # Unresolved questions +## Names (bikeshed welcome) + +I know there must be better names for lints and the traits being added +ahere. It took me a while to come up with `unmarked_early_drop` to +categorized the types that implement `Drop` but do not have a +`QuietEarlyDrop` impl, and `quiet_early_drop` to categorize +(obviously) the types that do implement both traits. + +## Which library types should be `QuietEarlyDrop`. + +Side-effectfulness is in the eye of the beholder. In particular, +I wonder how to handle `rc`; should it be: + +```rust +impl QuietEarlyDrop for Rc +``` + +or should it be like other container types, like so: +```rust +impl QuietEarlyDrop for Rc +``` + +One school of thought says that when you use `Rc`, you have +effectively given up on RAII for that type, at least when RAII is +viewed as being tied to a particular lexical scope, and therefore +all instances of `Rc` should be considered to have pure drop +behavior, regardless of their contents. + +But another school of thought says that since the destruction timing +of `Rc` is predictable (compared to `Gc` which in principle is +not predictable), then it would make sense to continue using the same +bubble-up semantics that the other collection types use. + + +## Should type parameters be treated specially + +It is possible that generic code in general does not +need to be written with the same sort of care about drop timing that +code specific to a particular effectful type needs to be. (Or rather, +it is possible that someone writing generic code will either want to +opt into receiving warnings about merge-points with mismatched drop +obligations.) + + + ## How should moving into wildcards be handled? In an example like: @@ -426,7 +747,7 @@ compute the set of drop obligations accurately, but I am not certain of all the implementation details of changes to the `trans` module in `rustc`. -# Appendix +# Appendices ## Program illustrating space impact of hidden drop flag From 5e73f426d8cd4c23f33d0c6a197ec7aa2035fdf6 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 01:55:04 +0200 Subject: [PATCH 05/38] revise presentation of summary --- active/0000-remove-drop-flag-and-zeroing.md | 22 +++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 84373d02c37..1cd5e3675a1 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -4,14 +4,20 @@ # Summary -Revise language semantics for drop so that all branches move or drop -the same pieces of state ("value-paths"). Add lint(s) to inform the -programmer of situations when this new drop-semantics could cause -side-effects of RAII-style code (e.g. releasing locks, flushing -buffers) to occur sooner than expected. Remove the dynamic tracking -of whether a value has been dropped or not; in particular, (1.) remove -implicit addition of a drop-flag by `Drop` impl, and (2.) remove -implicit zeroing of the memory that occurs when values are dropped. +Three step plan: + + 1. Revise language semantics for drop so that all branches move or drop + the same pieces of state ("value-paths"). + + 2. Add lint(s) to inform the programmer of situations when this new + drop-semantics could cause side-effects of RAII-style code + (e.g. releasing locks, flushing buffers) to occur sooner than + expected. + + 3. Remove the dynamic tracking of whether a value has been dropped or + not; in particular, (a) remove implicit addition of a drop-flag by + `Drop` impl, and (b) remove implicit zeroing of the memory that + occurs when values are dropped. # Motivation From 6bcd38b8e8bf5bd3ba3804b226f991cc28d27fe0 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 01:55:57 +0200 Subject: [PATCH 06/38] replace undef term with one I actually used. --- active/0000-remove-drop-flag-and-zeroing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 1cd5e3675a1..d7e670f99ed 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -7,7 +7,7 @@ Three step plan: 1. Revise language semantics for drop so that all branches move or drop - the same pieces of state ("value-paths"). + the same pieces of state ("drop obligations"). 2. Add lint(s) to inform the programmer of situations when this new drop-semantics could cause side-effects of RAII-style code From fc512c714ac1ce1276368734aaa4abbc64e62f79 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 02:01:29 +0200 Subject: [PATCH 07/38] Fixed spelling errors. --- active/0000-remove-drop-flag-and-zeroing.md | 42 ++++++++++----------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index d7e670f99ed..9a129ceeab2 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -98,7 +98,7 @@ The change suggested by this RFC has three parts: 1. Change from a dynamic drop semantics to a static drop semantics, 2. Provide one or more lints to inform the programmer of potential - suprises that may arise from earlier drops that are caused by the + surprises that may arise from earlier drops that are caused by the static drop semantics, and 3. Remove the implicitly added drop-flag, and the implicit zeroing of @@ -261,7 +261,7 @@ fn f2() { // above, to ensure that the set of drop // obligations match. - // After the implict drops, the resulting + // After the implicit drops, the resulting // remaining drop obligations are the // following: @@ -281,13 +281,13 @@ analysis, *not* just the lexical nesting structure of the code. In particular: If control flow splits at a point like an if-expression, but the two arms never meet -### match expressons and enum variants +### match expressions and enum variants The examples above used just structs and `if` expressions, but there is an additional twist introduced by `enum` types. The examples above showed that a struct type can be partially moved: one of its fields can be moved away while the other is still present, and this can be -faithfully represented in the set of drop oblgiations. +faithfully represented in the set of drop obligations. But consider an `enum` and `match`: ```rust @@ -364,21 +364,21 @@ Such side-effects include: * Mutex locks being released earlier than expected (a worrisome change in timing semantics when writing concurrent algorithms). -In particular, the injection of the implcit drops could silently +In particular, the injection of the implicit drops could silently invalidate certain kinds of "resource acquisition is initialization" (RAII) patterns. It is important to keep in mind that one can always recreate the effect of the former drop flag by wrapping one's type `T` in an `Option`; therefore, the problem is *not* that such RAII patterns -cannot be expresed. It is merely that a user may assume that a -variable binding induces RAII-style effect, and that assumpion is then +cannot be expressed. It is merely that a user may assume that a +variable binding induces RAII-style effect, and that assumption is then invalidated due to a errant move on one control-flow branch. Therefore, to defend against users being surprised by the early implicit drops induced by static drop semantics, this RFC proposes adding lints that tell the user about the points in the control-flow -where implcit drops are injected. The specific proposal is to add two +where implicit drops are injected. The specific proposal is to add two lints, named `quiet_early_drop` and `unmarked_early_drop`, with the default settings `#[allow(quiet_early_drop)]` and `#[warn(unmarked_early_drop)]`. (The two lints are similar in name @@ -408,14 +408,14 @@ Here is an example piece of code (very loosely adapted from the Rust } ``` -In the `Variant1` arm above, `guard` is consumed by `g`. Threfore, -when the bit of code labelled with a `(***)` represents a span that, +In the `Variant1` arm above, `guard` is consumed by `g`. Therefore, +when the bit of code labeled with a `(***)` represents a span that, when reached via the `Variant2` branch of the match statement, has the `guard` still held under dynamic drop semantics, but the `guard` is *released* under static drop semantics. -The `unmwarked_early_drop` lint is meant to catch cases such as this, -where the user has inadvertantly written code where static drop +The `unmarked_early_drop` lint is meant to catch cases such as this, +where the user has inadvertently written code where static drop semantics injects an implicit call to a side-effectful `drop` method. Assuming that the `LockGuard` has a `Drop` impl but does not implement @@ -444,7 +444,7 @@ are executed (such as flushing output buffers or releasing locks). Meanwhile, some users may still care about every potential side-effect, even those that their libraries have deemed "pure". Some users may just want, out of principle, to mark every early drop -explcitly, in the belief that such explicitness improves code +explicitly, in the belief that such explicitness improves code comprehension. Therefore, rather than provide just a single lint for warning about @@ -453,7 +453,7 @@ simple two-tier structure: by default, `Drop` implementations are assumed to have significant side-effects, and thus qualify for warning via the aforementioned `unmarked_early_drop` trait. However, when defining a type, one can implement the `QuietEarlyDrop` trait, which -recategorizes the type as having a `Drop` implementation that "pure" +re-categorizes the type as having a `Drop` implementation that "pure" (i.e. does not exhibit side-effects that the client of the crate is likely to care about). @@ -465,7 +465,7 @@ considered pure is `Vec`, since the only side-effect of dropping a If a type implements `QuietEarlyDrop`, then early implicit drops of that type will no longer be reported by `#[warn(unmarked_early_drop)]` -(instead, such a type becomes the reponsibility of the +(instead, such a type becomes the responsibility of the `#[allow(quiet_early_drop)]` lint). Thus, the former lint will hopefully provide well-focused warnings with a low false-positive rate, while the latter, being set to `allow` by default, will @@ -480,12 +480,12 @@ whether the types involved implement `QuietEarlyDrop` or not. ### Scope end for owner can handle mismatched drop obligations Consider again the long examples from the "How static drop semantics -works" section above. Both examples took care to explcitly include a +works" section above. Both examples took care to explicitly include a bit at the end marked with the comment "(... some code that does not change drop obligations ...)". This represents some potentially side-effectful code that comes between a merge-point and the end of the procedure (or more generally, the end of the lexical scope for -some local varibles that own paths in the set of drop obligations). +some local variables that own paths in the set of drop obligations). If you hit a merge-point with two sets of drop obligations like `{ pDD.x, pDD.y }` and `{ pDD.x, z }`, and there is no side-effectful @@ -559,17 +559,17 @@ implied here made this a non-starter. ## Do this, but add support for variant-predicated drop-obligations -In "match expressons and enum variants" above, this RFC proposed a +In "match expressions and enum variants" above, this RFC proposed a rule that if any arm in a match consumes the input via `move`, then every arm in the match must consume the input (by the end of its body). -There is an alterative, however. We could enrich the structure of +There is an alternative, however. We could enrich the structure of drop-obligations to include paths that are predicated on enum variants, like so: `{(s is Two => s#0), (s is Two => s#1)}`. This represents the idea that (1.) all control flows where `s` is the `One` variant dropped `s` entirely but also, (2.) all control flows where -`s` is the `Two` variant still has the 0'th and 1'st tuple compoents +`s` is the `Two` variant still has the 0'th and 1'st tuple components remaining as drop obligations. I do not currently know how to efficiently implement such an enriched @@ -634,7 +634,7 @@ complexity budget. ## Names (bikeshed welcome) I know there must be better names for lints and the traits being added -ahere. It took me a while to come up with `unmarked_early_drop` to +here. It took me a while to come up with `unmarked_early_drop` to categorized the types that implement `Drop` but do not have a `QuietEarlyDrop` impl, and `quiet_early_drop` to categorize (obviously) the types that do implement both traits. From 29066c075a7dc63920de24d113b5f595126295a4 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 02:01:48 +0200 Subject: [PATCH 08/38] removed "trans gotchas" note. Waste of space. --- active/0000-remove-drop-flag-and-zeroing.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 9a129ceeab2..743d57eeb97 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -746,13 +746,6 @@ match x { ``` -## Implementation gotchas in `trans` - -I implemented the `borrowck` and `dataflow` changes necessary to -compute the set of drop obligations accurately, but I am not certain -of all the implementation details of changes to the `trans` module in -`rustc`. - # Appendices ## Program illustrating space impact of hidden drop flag From eb661a2a59532c29a2f77e4aa2ffebc972b15e61 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 02:04:08 +0200 Subject: [PATCH 09/38] proper reference to the appendix. --- active/0000-remove-drop-flag-and-zeroing.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 743d57eeb97..8e6f82f378f 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -25,8 +25,9 @@ Currently, implementing `Drop` on a struct (or enum) injects a hidden bit, known as the "drop-flag", into the struct (and likewise, each of the the enum variants). The drop-flag, in tandem with Rust's implicit zeroing of dropped values, tracks whether a value has already been -moved to another owner or been dropped. (See "How dynamic drop -semantics works" for more details.) +moved to another owner or been dropped. (See the "How dynamic drop +semantics works" appendix for more details if you are unfamiliar +with this part of Rust's current implementation.) Here are some problems with this: From 5d94a569959d60cd3f4c6b7813653a7dc2b97d90 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 02:09:14 +0200 Subject: [PATCH 10/38] add semi-troubling question about expressiveness. --- active/0000-remove-drop-flag-and-zeroing.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 8e6f82f378f..f54ec801040 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -68,7 +68,7 @@ discussion of how this is done.) There are two important things to note about a static drop semantics: - 1. It is *equal* in expressive power to the Rust language as we know + 1. It should be *equal* in expressive power to the Rust language as we know it today. This is because, if the user is actually relying on the drop-flag today in some variable or field declaration `x: T`, they can replace that declaration with `x: Option` and thus recreate @@ -581,6 +581,20 @@ Also, if we do figure out how to implement this, we could add this later backward compatibly. I do not want to attempt to implement it in the short-term. +## Does the match-arm rule break expressiveness claim? + +I made the claim in a number of places that a static drop semantics +should be *equal* in expressive power to the Rust language as we know +it today. + +However, when I made that claim, I did not think carefully +about the impliciations of the simple match arm rule. +Being forced to move out of the original owner in every arm +might imply that you cannot perform a mechanical transformation +on the program to reencode the prior behavior. +(I am not sure, it is too late at night right now for me to +be sure one way or another about this.) + ## Associate drop flags with stack-local variables alone I mentioned in "Hidden bits are bad, part II" that some users have From 4cdd48b9136cb729a0c3940ef914cf24e7835374 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 02:52:45 +0200 Subject: [PATCH 11/38] Some revisions to the text while I was reviewing its web appearance. Added a hasty section "match expressions and enum variants that copy (or do not bind)" but while I was writing it I realized that it does not make much sense as written and does not actually reflect my current strategy, so I am going to remove it now. --- active/0000-remove-drop-flag-and-zeroing.md | 81 +++++++++++++++++++-- 1 file changed, 75 insertions(+), 6 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index f54ec801040..69eda05ca02 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -81,11 +81,11 @@ There are two important things to note about a static drop semantics: expected. The bulk of the Detailed Design is dedicated to mitigating that second -observation, in order to reduce the expected number of surprises for +observation, in order to reduce the number of surprises for Rust programmers. The main idea for this mitigation is the addition of one or more lints that report to the user when an side-effectful early-drop will be implicitly injected into the code, and suggest to -them that they revise their code to remove the implicit injection +them that they revise their code to remove the implicit `drop` injection (e.g. by explicitly dropping the path in question, or by re-establishing the drop obligation on the other control-flow paths, or by rewriting the code to put in a manual drop-flag via @@ -127,12 +127,15 @@ implements `Drop`, then `x` is a drop obligation. If `T` does not implement `Drop`, then the set of drop obligations is the union of the drop obligations of the fields of `T`. -When a path is moved to a new location or consumed by a function call, -it is removed from the set of drop obligations. +When a path is moved to a new location, or consumed by a function call, +or when control flow reaches the end of its owner's lexical scope, +the path is removed from the set of drop obligations. ### Example of code with unchanged behavior under static drop semantics -For example: +Consider the following example, where `D` represents some struct that +introduces a drop-obligation, while `S` represents some struct that +does not. ```rust @@ -197,10 +200,24 @@ fn f1() { // { pDD.y, pDS.x, some_d } } +``` + +Some notes about the example above: +It may seem silly that the line `some_d = None;` introduces a +drop-obligation for `some_d`, since `None` itself contains nothing to +drop. The analysis infers whether such an assignment introduces a +drop-obligation based on the type of `some_d` (`Option`, which +represents a drop-obligation, or at least a potential one). Anyway, +the point is that having this assignment introduce a drop-obligation +there makes things happier at the merge point that follows it in the +control flow. (There is further discussion of subtlety here +in the "match expressions and enum variants that copy (or do not bind)" +section below.) ### Example of code with changed behavior under static drop semantics +```rust // `f2` is similar to `f1`, except that it will have differing set // of drop obligations at the merge point, necessitating a hidden // drop call. @@ -282,7 +299,7 @@ analysis, *not* just the lexical nesting structure of the code. In particular: If control flow splits at a point like an if-expression, but the two arms never meet -### match expressions and enum variants +### match expressions and enum variants that move The examples above used just structs and `if` expressions, but there is an additional twist introduced by `enum` types. The examples above @@ -494,6 +511,58 @@ computation between that merge-point and the end of the scope for `pDD` and `z`, then there is no problem with the mismatches between the set of drop obligations, and neither lint should report anything. +### match expressions and enum variants that copy (or do not bind) + +In the "Example of code with unchanged behavior under static drop +semantics", we noted that it may have seemed silly to have the +assignment `some_d = None;` introduce a drop-obligation for `some_d`. + +And in fact, there are clearly silly instances of this; while it +happened to work out that it made the set of drop obligations match at +the merge point above, consider now this (artificial) example: + +```rust + let some_d : Option + match { + Some(d) => foo(d), + None => + } + if test() { + println!("no assignments"); + } else { + some_d = None; + } +``` + +Blind adherence to the rules outlined in "Drop obligations" and +the "Example of code with unchanged behavior under static drop semantics" +would lead one to think that our lints should report a mismatch +at the merge point here, since on one branch we have assigned to +`some_d : Option`, while on another we did no such assignment, +and therefore we will insert an implicit drop of `some_d` on the +branch that assigns `some_d = None;` + +Of course, such blind adherence is bad. In particular, it would be +silly to warn about an implicit drop of `some_d` here, since we can +easily tell that on this arm, `some_d` will always be something that +has no side-effect when dropped. + +The lints can handle this situation just fine: Just as +we made a dataflow analysis that determines the set of drop obligations, +we can make a dataflow analysis that determines which drop obligations +are in fact *ignorable* because the particular variants involved +on those control flow paths are made up entirely of non-moved data. +(In the `some_d` example, this trivially holds because there was +no data moved nor copied; more generally though we want this analysis +to also handle cases like: +```rust +enum Pairy2{ Two(X,Y), One(X,X) } +let s : Pairy2 = ...; +match s { + +} +``` + ### Type parameters, revisited We noted in the "How static drop semantics works" section that From 0a7f3135d7568cd54329e5accc4f59501d291bc4 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 02:53:52 +0200 Subject: [PATCH 12/38] Removed sketch "match expressions and enum variants that copy (or do not bind)" text. --- active/0000-remove-drop-flag-and-zeroing.md | 56 +-------------------- 1 file changed, 1 insertion(+), 55 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 69eda05ca02..39330c42781 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -211,9 +211,7 @@ drop-obligation based on the type of `some_d` (`Option`, which represents a drop-obligation, or at least a potential one). Anyway, the point is that having this assignment introduce a drop-obligation there makes things happier at the merge point that follows it in the -control flow. (There is further discussion of subtlety here -in the "match expressions and enum variants that copy (or do not bind)" -section below.) +control flow. ### Example of code with changed behavior under static drop semantics @@ -511,58 +509,6 @@ computation between that merge-point and the end of the scope for `pDD` and `z`, then there is no problem with the mismatches between the set of drop obligations, and neither lint should report anything. -### match expressions and enum variants that copy (or do not bind) - -In the "Example of code with unchanged behavior under static drop -semantics", we noted that it may have seemed silly to have the -assignment `some_d = None;` introduce a drop-obligation for `some_d`. - -And in fact, there are clearly silly instances of this; while it -happened to work out that it made the set of drop obligations match at -the merge point above, consider now this (artificial) example: - -```rust - let some_d : Option - match { - Some(d) => foo(d), - None => - } - if test() { - println!("no assignments"); - } else { - some_d = None; - } -``` - -Blind adherence to the rules outlined in "Drop obligations" and -the "Example of code with unchanged behavior under static drop semantics" -would lead one to think that our lints should report a mismatch -at the merge point here, since on one branch we have assigned to -`some_d : Option`, while on another we did no such assignment, -and therefore we will insert an implicit drop of `some_d` on the -branch that assigns `some_d = None;` - -Of course, such blind adherence is bad. In particular, it would be -silly to warn about an implicit drop of `some_d` here, since we can -easily tell that on this arm, `some_d` will always be something that -has no side-effect when dropped. - -The lints can handle this situation just fine: Just as -we made a dataflow analysis that determines the set of drop obligations, -we can make a dataflow analysis that determines which drop obligations -are in fact *ignorable* because the particular variants involved -on those control flow paths are made up entirely of non-moved data. -(In the `some_d` example, this trivially holds because there was -no data moved nor copied; more generally though we want this analysis -to also handle cases like: -```rust -enum Pairy2{ Two(X,Y), One(X,X) } -let s : Pairy2 = ...; -match s { - -} -``` - ### Type parameters, revisited We noted in the "How static drop semantics works" section that From a2639b59e7c03a36740ddcd5b1715e3ca831e931 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 02:58:17 +0200 Subject: [PATCH 13/38] Flesh out control-flow sensitivity section.j --- active/0000-remove-drop-flag-and-zeroing.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 39330c42781..fe3c7cf6d22 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -295,7 +295,17 @@ Note that the static drop semantics is based on a control-flow analysis, *not* just the lexical nesting structure of the code. In particular: If control flow splits at a point like an if-expression, -but the two arms never meet +but the two arms never meet, then they can have completely +sets of drop obligations. + +This is important, since in coding patterns like loops, one +often sees different sets of drop obligations prior to a `break` +compared to a `continue` or loop end. + +Likewise, a `return` statement represents another control flow jump +where the set of drop obligations can be completely different from +elsewhere in the code (this ties into a related topic discussed in +"Scope end for owner can handle mismatched drop obligations"). ### match expressions and enum variants that move From 852039c3097af850cec2570cf01413c5d0f44cc6 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 03:08:16 +0200 Subject: [PATCH 14/38] Improve presentation by labelling the big idea parts with "Part N" for N in 1..3. --- active/0000-remove-drop-flag-and-zeroing.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index fe3c7cf6d22..76b610b1d40 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -108,7 +108,7 @@ The change suggested by this RFC has three parts: Each of the three parts is given its own section below. -## How static drop semantics works +## Part 1: How static drop semantics works This section presents a detailed outline of how static drop semantics looks from the view point of someone trying to understand a Rust @@ -373,7 +373,7 @@ that they implement `Drop`, and therefore introduce drop obligations (the same as types that actually do implement `Drop`, as illustrated above). -## Early drop lints +## Part 2: Early drop lints Some users may be surprised by the implicit drops that are injected by static drop semantics, especially if the drop code being executed @@ -536,7 +536,7 @@ to `drop`. (See further discussion in the "Unresolved Questions.") -## Removing the drop-flag; removing memory zeroing +## Part 3: Removing the drop-flag; removing memory zeroing With the above two pieces in place, the remainder is trivial. Namely: once static drop semantics is actually implemented, then the compiler From a90db5726f2c82017f630c83c28eb5c03d435c51 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 03:11:03 +0200 Subject: [PATCH 15/38] Explicitly point out that the end-of-arm-body trick has its limits. --- active/0000-remove-drop-flag-and-zeroing.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 76b610b1d40..5bb4e571af6 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -365,6 +365,22 @@ this to continue working: ``` +Unfortunately, the same property does not hold +for a ref-binding match: we cannot write code like this: +```rust + match s { + One(a1, a2) => { // a moving match here + dA(a1) + dA(a2) + } + Two(ref r1, ref r2) => { // a ref-binding match here + let ret = helper_function(r1, r2); + mem::drop(s); // <-- oops, `s` is still borrowed. + ret + } + }; +``` + + ### Type parameters With respect to static drop semantics, there is not much to say about From d0c6258f19639f62a344291925ec7b8ab0fbac5e Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 03:12:43 +0200 Subject: [PATCH 16/38] fix typo. --- active/0000-remove-drop-flag-and-zeroing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 5bb4e571af6..8d18f69f8e5 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -462,7 +462,7 @@ semantics injects an implicit call to a side-effectful `drop` method. Assuming that the `LockGuard` has a `Drop` impl but does not implement the `QuietEarlyDrop` trait (see below `quiet_early_drop` lint below), -then the #[warn(unmarked_early_drop)]` lint will report a warning for +then the `#[warn(unmarked_early_drop)]` lint will report a warning for the code above, telling the user that the `guard` is moved away on the `Variant1` branch but not on the other branches. In general the lint cannot know what the actual intention of the user was. Therefore, the From f59ef8bdbbd2112fc8813fa7ba34d1afe236329b Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 03:13:48 +0200 Subject: [PATCH 17/38] improve presentation by inserting a paragraph break. --- active/0000-remove-drop-flag-and-zeroing.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 8d18f69f8e5..881caee6aff 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -464,7 +464,9 @@ Assuming that the `LockGuard` has a `Drop` impl but does not implement the `QuietEarlyDrop` trait (see below `quiet_early_drop` lint below), then the `#[warn(unmarked_early_drop)]` lint will report a warning for the code above, telling the user that the `guard` is moved away on the -`Variant1` branch but not on the other branches. In general the lint +`Variant1` branch but not on the other branches. + +In general the lint cannot know what the actual intention of the user was. Therefore, the lint suggests that the user either (1.) add an explicit drop call, for clarity, or (2.) reinitialize `guard` on the `Variant1` arm, or (3.) From 766a76e62c8fff31065d6c43cf7dc13de2d2efd7 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 03:17:59 +0200 Subject: [PATCH 18/38] Elaborate my reasoning on "Scope end for owner can handle mismatched drop obligations" --- active/0000-remove-drop-flag-and-zeroing.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 881caee6aff..f73cb942a33 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -537,6 +537,13 @@ computation between that merge-point and the end of the scope for `pDD` and `z`, then there is no problem with the mismatches between the set of drop obligations, and neither lint should report anything. +The reasoning is as follows: under dynamic drop semantics, the drop +would have run at the end of the scope. But there are no side-effects +between the merge-point being analyzed and the end of the scope; +therefore there is no harm in running the `drop` method, no matter what +the side-effects of the `drop` are, instead at the end of the incoming +branch to that merge point. + ### Type parameters, revisited We noted in the "How static drop semantics works" section that From 0b72ddce3a632789c2cb86844effa2fd54e99e0a Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 03:19:57 +0200 Subject: [PATCH 19/38] Cross-reference the space-impact appendix. --- active/0000-remove-drop-flag-and-zeroing.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index f73cb942a33..7991a026bcc 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -41,6 +41,8 @@ Here are some problems with this: expect `struct Foo { x: u32, y: u32 }` to occupy 8 bytes, but if `Foo` implements `Drop`, the hidden drop flag will cause it to double in size (16 bytes). + See the "Program illustrating space impact of hidden drop flag" + appendix for a concrete illustration. * Hidden bits are bad, part II: Some users have expressed an expectation that the drop-flag only be present for individual local From 4df7982583ea53fd256c1c6515c1753759b24fcc Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 03:28:25 +0200 Subject: [PATCH 20/38] Add some TODO's for the document. --- active/0000-remove-drop-flag-and-zeroing.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 7991a026bcc..4cbfab1fa9a 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -304,11 +304,15 @@ This is important, since in coding patterns like loops, one often sees different sets of drop obligations prior to a `break` compared to a `continue` or loop end. +TODO: add concrete example of `break` with different drop obligation. + Likewise, a `return` statement represents another control flow jump where the set of drop obligations can be completely different from elsewhere in the code (this ties into a related topic discussed in "Scope end for owner can handle mismatched drop obligations"). +TODO: add concrete example of `return` with different drop obligation. + ### match expressions and enum variants that move The examples above used just structs and `if` expressions, but there @@ -546,6 +550,14 @@ therefore there is no harm in running the `drop` method, no matter what the side-effects of the `drop` are, instead at the end of the incoming branch to that merge point. +### match expressions and enum variants that copy (or do-not-bind) + +TODO: There is some special handling of these in the current prototype +implementation (mostly to avoid getting spurious lint warnings about +data that obviously cannot have an effectful drop). I need to +document what the motivation is for having special handling here and +describe what that special handling is. + ### Type parameters, revisited We noted in the "How static drop semantics works" section that @@ -563,6 +575,13 @@ to `drop`. (See further discussion in the "Unresolved Questions.") +## Quality of output from the lints + +TODO: given all the fine-tuning modifications for the lints listed +here, I should put some data here on how well/poorly the lints do in +terms of false-positive rates depending on which modifications are +enabled/disabled. + ## Part 3: Removing the drop-flag; removing memory zeroing With the above two pieces in place, the remainder is trivial. Namely: From f08b515d1f0c61d62641c6c7be88639a3aaa710c Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Fri, 15 Aug 2014 08:28:27 +0200 Subject: [PATCH 21/38] Fix typo noted by Brian. As a drive-by, spell out what's happening explicitly, mostly so that the sets involved appear textually near each other. --- active/0000-remove-drop-flag-and-zeroing.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 4cbfab1fa9a..869dc1d213a 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -247,7 +247,7 @@ fn f2() { // MERGE POINT PREDECESSOR 1 - // implicit drops injected: drop(pDD.y) + // implicit drops injected: drop(pDD.x) } else { { // {pDD.x, pDD.y, pDS.x} @@ -273,11 +273,17 @@ fn f2() { // match on all incoming control-flow paths. // // For the original user code, they did not - // in this case. + // in this case. In the original code, + // Predecessor 1 has drop obligations + // {pDD.x, pDS.x, some_d} + // and Predecessor 2 has drop obligations + // { pDD.y, pDS.x, some_d}. // // Therefore, implicit drops are injected up // above, to ensure that the set of drop - // obligations match. + // obligations match, yielding the final + // set: + // { pDS.x, some_d}. // After the implicit drops, the resulting // remaining drop obligations are the From cf0707bd3873614e02afac32c9508affd57b31eb Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Tue, 19 Aug 2014 19:01:32 +0200 Subject: [PATCH 22/38] updates to dropflag RFC --- active/0000-remove-drop-flag-and-zeroing.md | 124 +++++++++++++++++++- 1 file changed, 119 insertions(+), 5 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index f54ec801040..b071331595f 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -197,10 +197,11 @@ fn f1() { // { pDD.y, pDS.x, some_d } } - +``` ### Example of code with changed behavior under static drop semantics +```rust // `f2` is similar to `f1`, except that it will have differing set // of drop obligations at the merge point, necessitating a hidden // drop call. @@ -321,7 +322,7 @@ by the calls to `dA`. While we *could* attempt to continue supporting this style of code (see "variant-predicated drop-obligations" in the Alternatives section), it seems simpler if we just disallow it. This RFC -proposes the following rule: if any arm in a match consumes +proposes the following so-called "match-arm rule": if any arm in a match consumes the input via `move`, then *every* arm in the match must consume the input *by the end of each arm's associated body*. @@ -560,8 +561,8 @@ implied here made this a non-starter. ## Do this, but add support for variant-predicated drop-obligations -In "match expressions and enum variants" above, this RFC proposed a -rule that if any arm in a match consumes the input via `move`, then +In "match expressions and enum variants" above, this RFC proposed the +match-arm rule that if any arm in a match consumes the input via `move`, then every arm in the match must consume the input (by the end of its body). @@ -588,7 +589,7 @@ should be *equal* in expressive power to the Rust language as we know it today. However, when I made that claim, I did not think carefully -about the impliciations of the simple match arm rule. +about the impliciations of the simple match-arm rule. Being forced to move out of the original owner in every arm might imply that you cannot perform a mechanical transformation on the program to reencode the prior behavior. @@ -760,7 +761,120 @@ match x { } ``` +## Should the match-arm rule be weakened to just a warning + +In principle we do not need to actually make it *illegal* to +write: +```rust + let ret = match s { + Two(ref r1, ref r2) => { + dR(r1) + dR(r2) + } + One(a1, a2) => { + dA(a1) + dA(a2) + } + }; +``` + +We could instead just treat this like another instance of a case where +there will be another early implciit drop (namely a drop of `s` at the +end of each arm where it has been accessed by reference) -- the +difference is that we cannot suggest that the user add an explicit +`drop` of `s` for such arms, since doing so would violate the +borrowing rules (since the references are still in scope). + +(But then again, if the borrowed references leak into the constructed +value that lives longer than the `match` itself, those implicit early +drops will be unsound. This scenario leads me to think that we should +strongly consider adopting the stronger form of the match-arm rule, +for simplicity in the compiler itself.) + +## Is the expressiveness claim just broken due to potential for borrows + +Consider the following code (TODO: check that this or some variant of +it actually passes borrowck today). + +```rust +let x; +let opt_ref = (if condition1 { + x = D1; + Some(&x.f) +} else { + None +}; +``` + +The above may work, but how does one port it to use `Option` to replace the drop-flag? + +In other words, does the below work? (TODO: check that below actually breaks in needsdrop branch.) + +```rust +let x = None; +let opt_ref = (if condition1 { + x = Some(D1); + Some(&x.as_ref().unwrap().f) +} else { + None +}; +``` + +## The most direct `Option` re-encoding of drop-flag yields dead_assignments + +When porting this old code: + +```rust +let x; +if condition1 { + if condition2 { + x = D1; + } else { + x = D2; + } + use_of(&x); +} +// no explicit uses of `x` here, though `x` may have an effect when dropped. +``` + +in a manner that preserves the spirit of the drop flag, +(including only dropping `x` at the end of its scope as +declared above), + +```rust +let mut x = None; +if condition1 { + if condition2 { + x = Some(D1); + } else { + x = Some(D2); + } + use_of(x.as_ref().unwrap()); +} +// no explicit uses of `x` here, though `x` may have an effect when dropped. +``` + +The problem here is that this causes the dead-assignment lint to fire, +since there is no uses of `x` that is not first preceded by an +assignment with a non-None value. + +Under this RFC as proposed, to have warning-free code, one would have to +write: + +```rust +let x; +if condition1 { + if condition2 { + x = Some(D1); + } else { + x = Some(D2); + } + use_of(&x); +} else { + x = None; +} +``` +(Maybe some people would regard the latter as an improvement on the first re-write.) + # Appendices ## Program illustrating space impact of hidden drop flag From be60ed3447620ac9a20ba902f2975c43546d9093 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 12:41:49 +0200 Subject: [PATCH 23/38] Various cleanups to the text presentation. --- active/0000-remove-drop-flag-and-zeroing.md | 93 +++++++++++++-------- 1 file changed, 60 insertions(+), 33 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 11f463e6915..a36da146850 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -7,13 +7,21 @@ Three step plan: 1. Revise language semantics for drop so that all branches move or drop - the same pieces of state ("drop obligations"). + the same pieces of state ("drop obligations"). To satisfy this + constraint, the compiler has freedom to move the drop code for + some state to earlier points in the control flow ("early drops"). - 2. Add lint(s) to inform the programmer of situations when this new + 2. Add lints to inform the programmer of situations when this new drop-semantics could cause side-effects of RAII-style code (e.g. releasing locks, flushing buffers) to occur sooner than expected. + Types that have side-effectful drop implement a marker trait, + `NoisyDrop`, that drives a warn-by-default lint; another marker + trait, `QuietDrop`, allows traits to opt opt. An allow-by-default + lint provides a way for programmers to request notification of all + auto-inserted early-drops. + 3. Remove the dynamic tracking of whether a value has been dropped or not; in particular, (a) remove implicit addition of a drop-flag by `Drop` impl, and (b) remove implicit zeroing of the memory that @@ -29,13 +37,16 @@ moved to another owner or been dropped. (See the "How dynamic drop semantics works" appendix for more details if you are unfamiliar with this part of Rust's current implementation.) -Here are some problems with this: +## Problems with dynamic drop semantics + +Here are some problems with this situation: - * Most important: implicit memory zeroing is a hidden cost that all - Rust programs are paying. With the removal of the drop flag, we - can remove implicit memory zeroing (or at least revisit its utility - -- there may be other motivations for implicit memory zeroing, - e.g. to try to keep secret data from being exposed to unsafe code). + * Most important: implicit memory zeroing is a hidden cost that today + all Rust programs pay, in both execution time and code size. + With the removal of the drop flag, we can remove implicit memory + zeroing (or at least revisit its utility -- there may be other + motivations for implicit memory zeroing, e.g. to try to keep secret + data from being exposed to unsafe code). * Hidden bits are bad, part I: Users coming from a C/C++ background expect `struct Foo { x: u32, y: u32 }` to occupy 8 bytes, but if @@ -54,12 +65,14 @@ Here are some problems with this: bounded by program stack growth; the memory wastage is strewn throughout the heap. -So, those are the main motivations for removing the drop flag. +The above are the main motivations for removing the drop flag. + +## Abandoning dynamic drop semantics -But, how do we actually remove the drop flag? The answer: By replacing +How do we actually remove the drop flag? The answer: By replacing the dynamic drop semantics (that implicitly checks the flag to -determine if a value has already been dropped) with a static drop -semantics (that performs drop of certain values more eagerly, +determine if a value has already been dropped) with a *static drop +semantics* (that performs drop of certain values more eagerly, i.e. before the end of their owner's lexical scope). A static drop semantics essentially works by inserting implicit calls @@ -71,27 +84,38 @@ discussion of how this is done.) There are two important things to note about a static drop semantics: 1. It should be *equal* in expressive power to the Rust language as we know - it today. This is because, if the user is actually relying on the - drop-flag today in some variable or field declaration `x: T`, they - can replace that declaration with `x: Option` and thus recreate - the effect of the drop-flag. (Note that formal comparisons of - expressiveness typically say nothing about *convenience*.) - - 2. Static drop semantics could be *surprising* to Rust programmers - who are used to dynamic drop semantics. In particular, an implicit early + it today. + + If the user is actually relying on the drop-flag today in some + variable or field declaration `x: T`, they can replace that + declaration with `x: Option` and thus recreate the effect of + the drop-flag. + + (Note that formal comparisons of expressiveness typically say + nothing about *convenience*; this RFC is explicitly sacrificing + the "convenience" of the implicit drop flag, under the assumption + that in the common case, programmers would choose an early-drop + over an `Option` wrapper, if given the choice.) + + 2. Static drop semantics may be *surprising* to programmers. + + Rust programmers may be used to dynamic drop semantics, and C++ + programmers may be used to destructors always being run at the end + of the scope (never earlier). In particular, an implicit early drop could lead to unexpected side-effects occurring earlier than expected. -The bulk of the Detailed Design is dedicated to mitigating that second -observation, in order to reduce the number of surprises for -Rust programmers. The main idea for this mitigation is the addition -of one or more lints that report to the user when an side-effectful -early-drop will be implicitly injected into the code, and suggest to -them that they revise their code to remove the implicit `drop` injection -(e.g. by explicitly dropping the path in question, or by -re-establishing the drop obligation on the other control-flow paths, -or by rewriting the code to put in a manual drop-flag via -`Option`). +The bulk of the Detailed Design is dedicated to mitigating the second +observation, to reduce the number of surprises for a programmer +encountering an early-drop injected by `rustc`. + +The main idea for this mitigation is the addition of one or more lints +that report to the user when an side-effectful early-drop will be +implicitly injected into the code, and suggest to them that they +revise their code to remove the implicit `drop` injection (e.g. by +explicitly dropping the path in question, or by re-establishing the +drop obligation on the other control-flow paths, or by rewriting the +code to put in a manual drop-flag via `Option`). # Detailed design @@ -152,6 +176,7 @@ fn test() -> bool { ... } fn xform(d:D) -> D { ... } fn f1() { + // At the outset, the set of drop obligations is // just the set of moved input parameters (empty // in this case). @@ -217,11 +242,13 @@ control flow. ### Example of code with changed behavior under static drop semantics +The function `f2` below is similar to `f1`, except that it will have differing set +of drop obligations at the merge point, necessitating a hidden +drop call. + ```rust -// `f2` is similar to `f1`, except that it will have differing set -// of drop obligations at the merge point, necessitating a hidden -// drop call. fn f2() { + // At the outset, the set of drop obligations is // just the set of moved input parameters (empty // in this case). From 96794161dca4ebc90ffebcd86e6d292a416cdb7f Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 14:03:02 +0200 Subject: [PATCH 24/38] Address TODOs. * Added examples for `break` and `return` * Renamed marker traits and lints to use "early"/"loud" terminology. * Removed unneeded text that should no longer be necessary now that "loud" is the default. --- active/0000-remove-drop-flag-and-zeroing.md | 322 ++++++++++++-------- 1 file changed, 190 insertions(+), 132 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index a36da146850..d8a24276227 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -335,16 +335,129 @@ sets of drop obligations. This is important, since in coding patterns like loops, one often sees different sets of drop obligations prior to a `break` -compared to a `continue` or loop end. +compared to a point where the loop repeats, such as a `continue` +or the end of a `loop` block. -TODO: add concrete example of `break` with different drop obligation. +```rust + // At the outset, the set of drop obligations is + // just the set of moved input parameters (empty + // in this case). + + // DROP OBLIGATIONS + // ------------------------ + // { } + let mut pDD : Pair = mk_dd(); + // { pDD.x, pDD.y } + 'a: loop { + // MERGE POINT: set of drop obligations must + // match on all incoming control-flow paths. + + // { pDD.x, pDD.y } + if test() { + // { pDD.x, pDD.y } + consume(pDD.x); + // { pDD.y } + break 'a; + } + // *not* merge point (only one path, the else branch, flows here) + + // { pDD.x, pDD.y } + + // never falls through; must merge with 'a loop. + } + + // RESUME POINT: break 'a above flows here + + // { pDD.y } + + // This is the point immediately preceding `'b: loop`; (1.) below. + + 'b: loop { + // MERGE POINT: set of drop obligations must match on all + // incoming control-flow paths. + // + // There are *three* such incoming paths: (1.) the statement + // preceding `'b: loop`, (2.) the `continue 'b;` below, and + // (3.) the end of the loop's block below. + + // { pDD.y } -Likewise, a `return` statement represents another control flow jump -where the set of drop obligations can be completely different from -elsewhere in the code (this ties into a related topic discussed in -"Scope end for owner can handle mismatched drop obligations"). + consume(pDD.y); -TODO: add concrete example of `return` with different drop obligation. + // { } + + if test() { + // { } + pDD.x = mk_d(); + // { pDD.x } + break 'b; + } + + // *not* merge point (only one path flows here) + + // { } + + if test() { + // { } + pDD.y = mk_d(); + + // This is (2.) referenced above. { pDD.y } + continue 'b; + } + // *not* merge point (only one path flows here) + + // { } + + pDD.y = mk_d(); + + // This is (3.) referenced above. { pDD.y } + } + + // RESUME POINT: break 'b above flows here + + // { pDD.x } +``` + +Likewise, a `return` statement represents another control flow jump. +In addition, the set of drop obligations for each `return` can be +completely different: even though every return conceptually flows to +the same place (namely the instruction following the call-site), each +`return` can have specialized code paths for its own drop obligations +(and then compiler optimizations can merge the common code paths). + +```rust +pub fn foo(b: || -> bool, c: || -> D, f: |D| -> i8) -> i8 { + + // DROP OBLIGATIONS + // ------------------------ + // { } + + let x = c(); + // { x } + let y = c(); + // { x, y } + + if b() { + // { x, y } + let ret = f(x); + // { y } + return ret; // emits code to drop `y` + } + // *not* merge point (only one path, the else branch, flows here) + + // { x, y } + if b() { + // { x, y } + let ret = f(y); + // { x } + return ret; // emits code to drop `x` + } + + // { x, y } + + return 0; // emits code to drop `x` and `y` +} +``` ### match expressions and enum variants that move @@ -460,13 +573,13 @@ Therefore, to defend against users being surprised by the early implicit drops induced by static drop semantics, this RFC proposes adding lints that tell the user about the points in the control-flow where implicit drops are injected. The specific proposal is to add two -lints, named `quiet_early_drop` and `unmarked_early_drop`, with the -default settings `#[allow(quiet_early_drop)]` and -`#[warn(unmarked_early_drop)]`. (The two lints are similar in name +lints, named `early_quiet_drop` and `early_loud_drop`, with the +default settings `#[allow(early_quiet_drop)]` and +`#[warn(early_loud_drop)]`. (The two lints are similar in name because they provide very similar functionality; the only difference is how aggressively each reports injected drop invocations.) -### The `unmarked_early_drop` lint +### The `early_loud_drop` lint Here is an example piece of code (very loosely adapted from the Rust `sync` crate): @@ -495,24 +608,24 @@ when reached via the `Variant2` branch of the match statement, has the `guard` still held under dynamic drop semantics, but the `guard` is *released* under static drop semantics. -The `unmarked_early_drop` lint is meant to catch cases such as this, +The `early_loud_drop` lint is meant to catch cases such as this, where the user has inadvertently written code where static drop semantics injects an implicit call to a side-effectful `drop` method. -Assuming that the `LockGuard` has a `Drop` impl but does not implement -the `QuietEarlyDrop` trait (see below `quiet_early_drop` lint below), -then the `#[warn(unmarked_early_drop)]` lint will report a warning for -the code above, telling the user that the `guard` is moved away on the +Assuming that `LockGuard`, or some subcomponent of it, implements the +`LoudDrop` trait, but does not implement the `QuietDrop` trait (see +below `early_quiet_drop` lint below), then the +`#[warn(early_loud_drop)]` lint will report a warning for the code +above, telling the user that the `guard` is moved away on the `Variant1` branch but not on the other branches. -In general the lint -cannot know what the actual intention of the user was. Therefore, the -lint suggests that the user either (1.) add an explicit drop call, for -clarity, or (2.) reinitialize `guard` on the `Variant1` arm, or (3.) -emulate a drop-flag by using `Option` instead of -`LockGuard` as the type of `guard`. +In general the lint cannot know what the actual intention of the user +was. Therefore, the lint suggests that the user either (1.) add an +explicit drop call, for clarity, or (2.) reinitialize `guard` on the +`Variant1` arm, or (3.) emulate a drop-flag by using +`Option` instead of `LockGuard` as the type of `guard`. -### The `quiet_early_drop` lint and `QuietEarlyDrop` trait +### The `early_quiet_drop` lint and `QuietDrop` trait To be effective, a lint must not issue a significant number of false positives: i.e., we do not want to tell the user about every site in @@ -532,69 +645,50 @@ comprehension. Therefore, rather than provide just a single lint for warning about all occurrences of injected early drops, this proposal suggests a -simple two-tier structure: by default, `Drop` implementations are -assumed to have significant side-effects, and thus qualify for warning -via the aforementioned `unmarked_early_drop` trait. However, when -defining a type, one can implement the `QuietEarlyDrop` trait, which -re-categorizes the type as having a `Drop` implementation that "pure" -(i.e. does not exhibit side-effects that the client of the crate is -likely to care about). +simple two-tier structure. Droppable types are categorized as +either "quiet" or "loud." A loud drop has significant side-effects +where the programmer is likely to care about ordering. +A quiet drop has no significant side-effects. + +There are two marker traits, `LoudDrop` and `QuietDrop`, that the +programmer can use to mark their types, in much the same manner as +described on RFC PR #127, "Opt-in builtin traits, take 2: default and +negative impls". By default, `Drop` implementations are assumed to be +quiet. (This is a semi-arbitrary choice.) An easy example of such a type whose `drop` method is likely to be considered pure is `Vec`, since the only side-effect of dropping a -`Vec` is deallocation of its backing buffer. More generally, -`Vec` should be `QuietEarlyDrop` for any `T` that is also -`QuietEarlyDrop`. - -If a type implements `QuietEarlyDrop`, then early implicit drops of -that type will no longer be reported by `#[warn(unmarked_early_drop)]` +`Vec` is deallocation of its backing buffer. (More generally, +`Vec` should be `QuietDrop` for any `T` that is also `QuietDrop`, +and `LoudDrop` for any `T` that is `LoudDrop`.) + +Then programmers can implement `LoudDrop` on a type like `LockGuard` +to declare that it has sie-effects when dropped, and can use +`QuietDrop` to make a type with a loud subcomponent quiet again, +e.g. if the containing type forms an abstraction that makes the +side-effect insignificant again. An example of the latter occurs in +`std::sync::Mutex`, where the lock is meant only to guard an +instance of the wrapped type `T`, and therefore it does no harm to +drop of the `Mutex` early. + +If a type implements `QuietDrop`, then early implicit drops of +that type will no longer be reported by `#[warn(early_loud_drop)]` (instead, such a type becomes the responsibility of the -`#[allow(quiet_early_drop)]` lint). Thus, the former lint will +`#[allow(early_quiet_drop)]` lint). Thus, the first lint will hopefully provide well-focused warnings with a low false-positive -rate, while the latter, being set to `allow` by default, will +rate, while the second, being set to `allow` by default, will not generate much noise. Meanwhile, to ensure that a particular fn item has no hidden early -drops, one can turn on `#[deny(quiet_early_drop)]` and -`#[deny(unmarked_early_drop)]`, and then all statically injected drops +drops at all, one can turn on `#[deny(early_quiet_drop)]` and +`#[deny(early_loud_drop)]`, and then all statically injected drops are reported (and the code rejected if any are present), regardless of -whether the types involved implement `QuietEarlyDrop` or not. - -### Scope end for owner can handle mismatched drop obligations - -Consider again the long examples from the "How static drop semantics -works" section above. Both examples took care to explicitly include a -bit at the end marked with the comment "(... some code that does not -change drop obligations ...)". This represents some potentially -side-effectful code that comes between a merge-point and the end of -the procedure (or more generally, the end of the lexical scope for -some local variables that own paths in the set of drop obligations). - -If you hit a merge-point with two sets of drop obligations like `{ -pDD.x, pDD.y }` and `{ pDD.x, z }`, and there is no side-effectful -computation between that merge-point and the end of the scope for -`pDD` and `z`, then there is no problem with the mismatches between -the set of drop obligations, and neither lint should report anything. - -The reasoning is as follows: under dynamic drop semantics, the drop -would have run at the end of the scope. But there are no side-effects -between the merge-point being analyzed and the end of the scope; -therefore there is no harm in running the `drop` method, no matter what -the side-effects of the `drop` are, instead at the end of the incoming -branch to that merge point. - -### match expressions and enum variants that copy (or do-not-bind) - -TODO: There is some special handling of these in the current prototype -implementation (mostly to avoid getting spurious lint warnings about -data that obviously cannot have an effectful drop). I need to -document what the motivation is for having special handling here and -describe what that special handling is. +whether the types involved implement `QuietDrop` or not. ### Type parameters, revisited We noted in the "How static drop semantics works" section that -type parameters are not particularly special with respect t +type parameters are not particularly special with respect to static drop semantics. However, with the lints there is potential for type parameters to be @@ -608,13 +702,6 @@ to `drop`. (See further discussion in the "Unresolved Questions.") -## Quality of output from the lints - -TODO: given all the fine-tuning modifications for the lints listed -here, I should put some data here on how well/poorly the lints do in -terms of false-positive rates depending on which modifications are -enabled/disabled. - ## Part 3: Removing the drop-flag; removing memory zeroing With the above two pieces in place, the remainder is trivial. Namely: @@ -626,16 +713,16 @@ implement `Drop`, and likewise memory zeroing can be removed. * The lint may be annoying to users whose programs are not affected by the early drops. (We mitigate this by providing ways for users to - opt-out of the lint `#[allow(unmarked_early_drops)]`, both in a + opt-out of the lint `#[allow(early_loud_drop)]`, both in a lexically scoped fashion, like other lints, and in a type-based - fashion via a `QuietEarlyDrop` trait.) + fashion via a `QuietDrop` trait.) * The early drops may surprise the users who are used to the dynamic drop semantics. (We mitigate this by providing warnings via the lint, a clear path for rewriting code in terms of `Option` to emulate a drop-flag, and a way for users to enable a stricter lint: - `#[warn(quiet_early_drops)]` that reports all early drops, including - those hidden via `QuietEarlyDrop`.) + `#[warn(early_quiet_drop)]` that reports all early drops, including + those hidden via `QuietDrop`.) * There may be benefits to implicit memory-zeroing that are not accounted for in this RFC, in which case we may end up only removing @@ -656,7 +743,8 @@ Rather than injecting implicit drops where necessary, we could just reject programs that have control-flow merge points with an inconsistent set of incoming drop-obligations. -This would be equivalent to doing `#[deny(unmarked_early_drops)]`. +This would be equivalent to doing `#[forbid(early_quiet_drops)]` +and `#[forbid(early_loud_drops)]`. Felix (the author) was originally planning to take this approach, but it became clear after a little experimentation that the annoyance @@ -687,23 +775,24 @@ in the short-term. ## Does the match-arm rule break expressiveness claim? -I made the claim in a number of places that a static drop semantics -should be *equal* in expressive power to the Rust language as we know -it today. +I made the claim in "Abandoning dynamic drop semantics" +that a static drop semantics should be *equal* in expressive power to +the Rust language as we know it today. However, when I made that claim, I did not think carefully -about the impliciations of the simple match-arm rule. +about the implications of the simple match-arm rule. Being forced to move out of the original owner in every arm -might imply that you cannot perform a mechanical transformation -on the program to reencode the prior behavior. -(I am not sure, it is too late at night right now for me to -be sure one way or another about this.) +might imply that you cannot perform a truly automatic mechanical +transformation on the program to reencode the prior behavior. +Still, I remain confident that one can find some encoding in terms +of `Option` for any current program. ## Associate drop flags with stack-local variables alone I mentioned in "Hidden bits are bad, part II" that some users have said they thought that the drop flag was only part of a local -variable's state, not part of every occurrence of a struct/enum. +variable's state, not part of every occurrence of a struct/enum, +regardless of where it is allocated. We could try to explore this option, but it seems potentially very complicated to me. E.g. would each droppable structs embedded within a @@ -752,24 +841,22 @@ complexity budget. ## Names (bikeshed welcome) -I know there must be better names for lints and the traits being added -here. It took me a while to come up with `unmarked_early_drop` to -categorized the types that implement `Drop` but do not have a -`QuietEarlyDrop` impl, and `quiet_early_drop` to categorize -(obviously) the types that do implement both traits. +There may be better names for lints and the traits being added +here. It took me a while to come up with the "loud" and "quiet" +mnemonics. -## Which library types should be `QuietEarlyDrop`. +## Which library types should be `QuietDrop`. Side-effectfulness is in the eye of the beholder. In particular, I wonder how to handle `rc`; should it be: ```rust -impl QuietEarlyDrop for Rc +impl QuietDrop for Rc ``` or should it be like other container types, like so: ```rust -impl QuietEarlyDrop for Rc +impl QuietDrop for Rc ``` One school of thought says that when you use `Rc`, you have @@ -880,7 +967,7 @@ write: ``` We could instead just treat this like another instance of a case where -there will be another early implciit drop (namely a drop of `s` at the +there will be another early implicit drop (namely a drop of `s` at the end of each arm where it has been accessed by reference) -- the difference is that we cannot suggest that the user add an explicit `drop` of `s` for such arms, since doing so would violate the @@ -892,35 +979,6 @@ drops will be unsound. This scenario leads me to think that we should strongly consider adopting the stronger form of the match-arm rule, for simplicity in the compiler itself.) -## Is the expressiveness claim just broken due to potential for borrows - -Consider the following code (TODO: check that this or some variant of -it actually passes borrowck today). - -```rust -let x; -let opt_ref = (if condition1 { - x = D1; - Some(&x.f) -} else { - None -}; -``` - -The above may work, but how does one port it to use `Option` to replace the drop-flag? - -In other words, does the below work? (TODO: check that below actually breaks in needsdrop branch.) - -```rust -let x = None; -let opt_ref = (if condition1 { - x = Some(D1); - Some(&x.as_ref().unwrap().f) -} else { - None -}; -``` - ## The most direct `Option` re-encoding of drop-flag yields dead_assignments When porting this old code: From 0923953257f941d5ae5389ab9118b6eaa61e5bf4 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 14:06:00 +0200 Subject: [PATCH 25/38] Consistently use "noisy drop" terminology (rather than "loud"). --- active/0000-remove-drop-flag-and-zeroing.md | 32 ++++++++++----------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index d8a24276227..8947857c836 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -573,13 +573,13 @@ Therefore, to defend against users being surprised by the early implicit drops induced by static drop semantics, this RFC proposes adding lints that tell the user about the points in the control-flow where implicit drops are injected. The specific proposal is to add two -lints, named `early_quiet_drop` and `early_loud_drop`, with the +lints, named `early_quiet_drop` and `early_noisy_drop`, with the default settings `#[allow(early_quiet_drop)]` and -`#[warn(early_loud_drop)]`. (The two lints are similar in name +`#[warn(early_noisy_drop)]`. (The two lints are similar in name because they provide very similar functionality; the only difference is how aggressively each reports injected drop invocations.) -### The `early_loud_drop` lint +### The `early_noisy_drop` lint Here is an example piece of code (very loosely adapted from the Rust `sync` crate): @@ -608,14 +608,14 @@ when reached via the `Variant2` branch of the match statement, has the `guard` still held under dynamic drop semantics, but the `guard` is *released* under static drop semantics. -The `early_loud_drop` lint is meant to catch cases such as this, +The `early_noisy_drop` lint is meant to catch cases such as this, where the user has inadvertently written code where static drop semantics injects an implicit call to a side-effectful `drop` method. Assuming that `LockGuard`, or some subcomponent of it, implements the -`LoudDrop` trait, but does not implement the `QuietDrop` trait (see +`NoisyDrop` trait, but does not implement the `QuietDrop` trait (see below `early_quiet_drop` lint below), then the -`#[warn(early_loud_drop)]` lint will report a warning for the code +`#[warn(early_noisy_drop)]` lint will report a warning for the code above, telling the user that the `guard` is moved away on the `Variant1` branch but not on the other branches. @@ -646,11 +646,11 @@ comprehension. Therefore, rather than provide just a single lint for warning about all occurrences of injected early drops, this proposal suggests a simple two-tier structure. Droppable types are categorized as -either "quiet" or "loud." A loud drop has significant side-effects +either "quiet" or "noisy." A noisy drop has significant side-effects where the programmer is likely to care about ordering. A quiet drop has no significant side-effects. -There are two marker traits, `LoudDrop` and `QuietDrop`, that the +There are two marker traits, `NoisyDrop` and `QuietDrop`, that the programmer can use to mark their types, in much the same manner as described on RFC PR #127, "Opt-in builtin traits, take 2: default and negative impls". By default, `Drop` implementations are assumed to be @@ -660,11 +660,11 @@ An easy example of such a type whose `drop` method is likely to be considered pure is `Vec`, since the only side-effect of dropping a `Vec` is deallocation of its backing buffer. (More generally, `Vec` should be `QuietDrop` for any `T` that is also `QuietDrop`, -and `LoudDrop` for any `T` that is `LoudDrop`.) +and `NoisyDrop` for any `T` that is `NoisyDrop`.) -Then programmers can implement `LoudDrop` on a type like `LockGuard` +Then programmers can implement `NoisyDrop` on a type like `LockGuard` to declare that it has sie-effects when dropped, and can use -`QuietDrop` to make a type with a loud subcomponent quiet again, +`QuietDrop` to make a type with a noisy subcomponent quiet again, e.g. if the containing type forms an abstraction that makes the side-effect insignificant again. An example of the latter occurs in `std::sync::Mutex`, where the lock is meant only to guard an @@ -672,7 +672,7 @@ instance of the wrapped type `T`, and therefore it does no harm to drop of the `Mutex` early. If a type implements `QuietDrop`, then early implicit drops of -that type will no longer be reported by `#[warn(early_loud_drop)]` +that type will no longer be reported by `#[warn(early_noisy_drop)]` (instead, such a type becomes the responsibility of the `#[allow(early_quiet_drop)]` lint). Thus, the first lint will hopefully provide well-focused warnings with a low false-positive @@ -681,7 +681,7 @@ not generate much noise. Meanwhile, to ensure that a particular fn item has no hidden early drops at all, one can turn on `#[deny(early_quiet_drop)]` and -`#[deny(early_loud_drop)]`, and then all statically injected drops +`#[deny(early_noisy_drop)]`, and then all statically injected drops are reported (and the code rejected if any are present), regardless of whether the types involved implement `QuietDrop` or not. @@ -713,7 +713,7 @@ implement `Drop`, and likewise memory zeroing can be removed. * The lint may be annoying to users whose programs are not affected by the early drops. (We mitigate this by providing ways for users to - opt-out of the lint `#[allow(early_loud_drop)]`, both in a + opt-out of the lint `#[allow(early_noisy_drop)]`, both in a lexically scoped fashion, like other lints, and in a type-based fashion via a `QuietDrop` trait.) @@ -744,7 +744,7 @@ reject programs that have control-flow merge points with an inconsistent set of incoming drop-obligations. This would be equivalent to doing `#[forbid(early_quiet_drops)]` -and `#[forbid(early_loud_drops)]`. +and `#[forbid(early_noisy_drops)]`. Felix (the author) was originally planning to take this approach, but it became clear after a little experimentation that the annoyance @@ -842,7 +842,7 @@ complexity budget. ## Names (bikeshed welcome) There may be better names for lints and the traits being added -here. It took me a while to come up with the "loud" and "quiet" +here. It took me a while to come up with the "noisy" and "quiet" mnemonics. ## Which library types should be `QuietDrop`. From f9f6c50bfb4bb9dc0e81423005da18e38f0f452b Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 14:08:33 +0200 Subject: [PATCH 26/38] Fix a typo. Add a hyperlink. --- active/0000-remove-drop-flag-and-zeroing.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 8947857c836..2c1546cb49a 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -18,7 +18,7 @@ Three step plan: Types that have side-effectful drop implement a marker trait, `NoisyDrop`, that drives a warn-by-default lint; another marker - trait, `QuietDrop`, allows traits to opt opt. An allow-by-default + trait, `QuietDrop`, allows types to opt opt. An allow-by-default lint provides a way for programmers to request notification of all auto-inserted early-drops. @@ -33,8 +33,8 @@ Currently, implementing `Drop` on a struct (or enum) injects a hidden bit, known as the "drop-flag", into the struct (and likewise, each of the the enum variants). The drop-flag, in tandem with Rust's implicit zeroing of dropped values, tracks whether a value has already been -moved to another owner or been dropped. (See the "How dynamic drop -semantics works" appendix for more details if you are unfamiliar +moved to another owner or been dropped. (See the ["How dynamic drop +semantics works"](#how-dynamic-drop-semantics-works) appendix for more details if you are unfamiliar with this part of Rust's current implementation.) ## Problems with dynamic drop semantics From 67201735c18ddce1c1acbe5e9ca190cbe1975e77 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 14:12:58 +0200 Subject: [PATCH 27/38] Added hyperlinks. --- active/0000-remove-drop-flag-and-zeroing.md | 24 +++++++++++---------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 2c1546cb49a..356f4f1c87f 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -52,7 +52,8 @@ Here are some problems with this situation: expect `struct Foo { x: u32, y: u32 }` to occupy 8 bytes, but if `Foo` implements `Drop`, the hidden drop flag will cause it to double in size (16 bytes). - See the "Program illustrating space impact of hidden drop flag" + See the ["Program illustrating space impact of hidden drop flag"] + (#program-illustrating-space-impact-of-hidden-drop-flag) appendix for a concrete illustration. * Hidden bits are bad, part II: Some users have expressed an @@ -78,8 +79,8 @@ i.e. before the end of their owner's lexical scope). A static drop semantics essentially works by inserting implicit calls to `mem::drop` at certain points in the control-flow to ensure that the set of values to drop is statically known at compile-time. -(See "How static drop semantics works" in the detailed design for more -discussion of how this is done.) +(See ["How static drop semantics works"](#how-static-drop-semantics-works) +in the detailed design for more discussion of how this is done.) There are two important things to note about a static drop semantics: @@ -496,8 +497,8 @@ moved in pieces into `a1` and `a2`, which are themselves then consumed by the calls to `dA`. While we *could* attempt to continue supporting this style of code -(see "variant-predicated drop-obligations" in the Alternatives -section), it seems simpler if we just disallow it. This RFC +(see ["variant-predicated drop-obligations"](#do-this-with-support-for-variant-predicated-drop-obligations) +in the Alternatives section), it seems simpler if we just disallow it. This RFC proposes the following so-called "match-arm rule": if any arm in a match consumes the input via `move`, then *every* arm in the match must consume the input *by the end of each arm's associated body*. @@ -614,10 +615,10 @@ semantics injects an implicit call to a side-effectful `drop` method. Assuming that `LockGuard`, or some subcomponent of it, implements the `NoisyDrop` trait, but does not implement the `QuietDrop` trait (see -below `early_quiet_drop` lint below), then the -`#[warn(early_noisy_drop)]` lint will report a warning for the code -above, telling the user that the `guard` is moved away on the -`Variant1` branch but not on the other branches. +below `early_quiet_drop` lint), then the `#[warn(early_noisy_drop)]` lint +will report a warning for the code above, telling the user that +the `guard` is moved away on the `Variant1` branch but not on the +other branches. In general the lint cannot know what the actual intention of the user was. Therefore, the lint suggests that the user either (1.) add an @@ -700,7 +701,8 @@ obligations, then that represents a hidden implicit drop that you may not have known was there, and it behooves you to make an explicit call to `drop`. -(See further discussion in the "Unresolved Questions.") +(See further discussion in the +["Unresolved Questions"](#unresolved-questions).) ## Part 3: Removing the drop-flag; removing memory zeroing @@ -750,7 +752,7 @@ Felix (the author) was originally planning to take this approach, but it became clear after a little experimentation that the annoyance implied here made this a non-starter. -## Do this, but add support for variant-predicated drop-obligations +## Do this with support for variant-predicated drop-obligations In "match expressions and enum variants" above, this RFC proposed the match-arm rule that if any arm in a match consumes the input via `move`, then From 498e53c914b7ecc79686536cce7da9b157b0a36d Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 14:27:42 +0200 Subject: [PATCH 28/38] Added table-of-contents. Fixed a miscategorized question. --- active/0000-remove-drop-flag-and-zeroing.md | 74 ++++++++++++++++----- 1 file changed, 56 insertions(+), 18 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 356f4f1c87f..73e818e8929 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -2,6 +2,43 @@ - RFC PR: (leave this empty) - Rust Issue: (leave this empty) +# Table of Contents +* [Summary](#summary) +* [Motivation](#motivation) +** [Abandoning dynamic drop semantics](#abandoning-dynamic-drop-semantics) +* [Detailed design](#detailed-design) +** [Part 1: How static drop semantics works](#part-1-how-static-drop-semantics-works) +*** [Drop obligations](#drop-obligations) +*** [Example of code with unchanged behavior under static drop semantics](#example-of-code-with-unchanged-behavior-under-static-drop-semantics) +*** [Example of code with changed behavior under static drop semantics](#example-of-code-with-changed-behavior-under-static-drop-semantics) +*** [Control-flow sensitivity](#control-flow-sensitivity) +*** [match expressions and enum variants that move](#match-expressions-and-enum-variants-that-move) +*** [Type parameters](#type-parameters) +** [Part 2: Early drop lints](#part-2-early-drop-lints) +*** [The `early_noisy_drop` lint](#the-early-noisy-drop-lint) +*** [The `early_quiet_drop` lint and `QuietDrop` trait](#the-early-quiet-drop-lint-and-quietdrop-trait) +*** [Type parameters, revisited](#type-parameters-revisited) +** [Part 3: Removing the drop-flag; removing memory zeroing](#part-3-removing-the-drop-flag-removing-memory-zeroing) +* [Drawbacks](#drawbacks) +* [Alternatives](#alternatives) +** [Do nothing](#do-nothing) +** [Require explicit drops rather than injecting them](#require-explicit-drops-rather-than-injecting-them) +** [Do this with support for variant-predicated drop-obligations](#do-this-with-support-for-variant-predicated drop-obligations) +** [Associate drop flags with stack-local variables alone](#associate-drop-flags-with-stack-local variables alone) +** [Separate individual and grouped instances of a type](#separate-individual-and-grouped-instances-of-a-type) +* [Unresolved questions](#unresolved-questions) +** [Names (bikeshed expected)](#names-bikeshed-expected) +** [Does the match-arm rule break expressiveness claim?](#does-the-match-arm-rule-break-expressiveness-claim) +** [Which library types should be `QuietDrop`](#which-library-types-should-be-quietdrop) +** [Should type parameters be treated specially](#should-type-parameters-be-treated-specially) +** [How should moving into wildcards be handled](#how-should-moving-into-wildcards-be-handled) +** [Should the match-arm rule be weakened to just a warning](#should-the-match-arm-rule-be-weakened-to-just-a-warning) +** [The most direct `Option` re-encoding of drop-flag yields dead_assignments](#the-most-direct-option-t-re-encoding-of-drop-flag-yields-dead-assignments) +* [Appendices](#appendices) +** [Program illustrating space impact of hidden drop flag](#program-illustrating-space-impact-of-hidden-drop-flag) +** [How dynamic drop semantics works](#how-dynamic-drop-semantics-works) +*** [Program illustrating semantic impact of hidden drop flag](#program-illustrating-semantic-impact-of-hidden-drop-flag) + # Summary Three step plan: @@ -775,20 +812,6 @@ Also, if we do figure out how to implement this, we could add this later backward compatibly. I do not want to attempt to implement it in the short-term. -## Does the match-arm rule break expressiveness claim? - -I made the claim in "Abandoning dynamic drop semantics" -that a static drop semantics should be *equal* in expressive power to -the Rust language as we know it today. - -However, when I made that claim, I did not think carefully -about the implications of the simple match-arm rule. -Being forced to move out of the original owner in every arm -might imply that you cannot perform a truly automatic mechanical -transformation on the program to reencode the prior behavior. -Still, I remain confident that one can find some encoding in terms -of `Option` for any current program. - ## Associate drop flags with stack-local variables alone I mentioned in "Hidden bits are bad, part II" that some users have @@ -841,13 +864,28 @@ complexity budget. # Unresolved questions -## Names (bikeshed welcome) +## Names (bikeshed expected) There may be better names for lints and the traits being added here. It took me a while to come up with the "noisy" and "quiet" mnemonics. -## Which library types should be `QuietDrop`. +## Does the match-arm rule break expressiveness claim? + +I made the claim in "Abandoning dynamic drop semantics" +that a static drop semantics should be *equal* in expressive power to +the Rust language as we know it today. + +However, when I made that claim, I did not think carefully +about the implications of the simple match-arm rule. +Being forced to move out of the original owner in every arm +might imply that you cannot perform a truly automatic mechanical +transformation on the program to reencode the prior behavior. +Still, I remain confident that one can find some encoding in terms +of `Option` for any current program. + + +## Which library types should be `QuietDrop` Side-effectfulness is in the eye of the beholder. In particular, I wonder how to handle `rc`; should it be: @@ -884,7 +922,7 @@ obligations.) -## How should moving into wildcards be handled? +## How should moving into wildcards be handled In an example like: @@ -1040,7 +1078,7 @@ if condition1 { # Appendices -## Program illustrating space impact of hidden drop flag +## Program illustrating space impact of hidden drop flag# ```rust From ecc64d08ea19a169e89f35469a72dcbe608d4c36 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 14:29:23 +0200 Subject: [PATCH 29/38] Fixed ToC formatting. --- active/0000-remove-drop-flag-and-zeroing.md | 62 ++++++++++----------- 1 file changed, 31 insertions(+), 31 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 73e818e8929..892c7901f82 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -3,41 +3,41 @@ - Rust Issue: (leave this empty) # Table of Contents -* [Summary](#summary) -* [Motivation](#motivation) -** [Abandoning dynamic drop semantics](#abandoning-dynamic-drop-semantics) -* [Detailed design](#detailed-design) -** [Part 1: How static drop semantics works](#part-1-how-static-drop-semantics-works) -*** [Drop obligations](#drop-obligations) -*** [Example of code with unchanged behavior under static drop semantics](#example-of-code-with-unchanged-behavior-under-static-drop-semantics) -*** [Example of code with changed behavior under static drop semantics](#example-of-code-with-changed-behavior-under-static-drop-semantics) -*** [Control-flow sensitivity](#control-flow-sensitivity) -*** [match expressions and enum variants that move](#match-expressions-and-enum-variants-that-move) -*** [Type parameters](#type-parameters) -** [Part 2: Early drop lints](#part-2-early-drop-lints) -*** [The `early_noisy_drop` lint](#the-early-noisy-drop-lint) -*** [The `early_quiet_drop` lint and `QuietDrop` trait](#the-early-quiet-drop-lint-and-quietdrop-trait) -*** [Type parameters, revisited](#type-parameters-revisited) -** [Part 3: Removing the drop-flag; removing memory zeroing](#part-3-removing-the-drop-flag-removing-memory-zeroing) +* [Summary](#summary) +* [Motivation](#motivation) + * [Abandoning dynamic drop semantics](#abandoning-dynamic-drop-semantics) +* [Detailed design](#detailed-design) + * [Part 1: How static drop semantics works](#part-1-how-static-drop-semantics-works) + * [Drop obligations](#drop-obligations) + * [Example of code with unchanged behavior under static drop semantics](#example-of-code-with-unchanged-behavior-under-static-drop-semantics) + * [Example of code with changed behavior under static drop semantics](#example-of-code-with-changed-behavior-under-static-drop-semantics) + * [Control-flow sensitivity](#control-flow-sensitivity) + * [match expressions and enum variants that move](#match-expressions-and-enum-variants-that-move) + * [Type parameters](#type-parameters) + * [Part 2: Early drop lints](#part-2-early-drop-lints) + * [The `early_noisy_drop` lint](#the-early-noisy-drop-lint) + * [The `early_quiet_drop` lint and `QuietDrop` trait](#the-early-quiet-drop-lint-and-quietdrop-trait) + * [Type parameters, revisited](#type-parameters-revisited) + * [Part 3: Removing the drop-flag; removing memory zeroing](#part-3-removing-the-drop-flag-removing-memory-zeroing) * [Drawbacks](#drawbacks) * [Alternatives](#alternatives) -** [Do nothing](#do-nothing) -** [Require explicit drops rather than injecting them](#require-explicit-drops-rather-than-injecting-them) -** [Do this with support for variant-predicated drop-obligations](#do-this-with-support-for-variant-predicated drop-obligations) -** [Associate drop flags with stack-local variables alone](#associate-drop-flags-with-stack-local variables alone) -** [Separate individual and grouped instances of a type](#separate-individual-and-grouped-instances-of-a-type) + * [Do nothing](#do-nothing) + * [Require explicit drops rather than injecting them](#require-explicit-drops-rather-than-injecting-them) + * [Do this with support for variant-predicated drop-obligations](#do-this-with-support-for-variant-predicated drop-obligations) + * [Associate drop flags with stack-local variables alone](#associate-drop-flags-with-stack-local variables alone) + * [Separate individual and grouped instances of a type](#separate-individual-and-grouped-instances-of-a-type) * [Unresolved questions](#unresolved-questions) -** [Names (bikeshed expected)](#names-bikeshed-expected) -** [Does the match-arm rule break expressiveness claim?](#does-the-match-arm-rule-break-expressiveness-claim) -** [Which library types should be `QuietDrop`](#which-library-types-should-be-quietdrop) -** [Should type parameters be treated specially](#should-type-parameters-be-treated-specially) -** [How should moving into wildcards be handled](#how-should-moving-into-wildcards-be-handled) -** [Should the match-arm rule be weakened to just a warning](#should-the-match-arm-rule-be-weakened-to-just-a-warning) -** [The most direct `Option` re-encoding of drop-flag yields dead_assignments](#the-most-direct-option-t-re-encoding-of-drop-flag-yields-dead-assignments) + * [Names (bikeshed expected)](#names-bikeshed-expected) + * [Does the match-arm rule break expressiveness claim?](#does-the-match-arm-rule-break-expressiveness-claim) + * [Which library types should be `QuietDrop`](#which-library-types-should-be-quietdrop) + * [Should type parameters be treated specially](#should-type-parameters-be-treated-specially) + * [How should moving into wildcards be handled](#how-should-moving-into-wildcards-be-handled) + * [Should the match-arm rule be weakened to just a warning](#should-the-match-arm-rule-be-weakened-to-just-a-warning) + * [The most direct `Option` re-encoding of drop-flag yields dead_assignments](#the-most-direct-option-t-re-encoding-of-drop-flag-yields-dead-assignments) * [Appendices](#appendices) -** [Program illustrating space impact of hidden drop flag](#program-illustrating-space-impact-of-hidden-drop-flag) -** [How dynamic drop semantics works](#how-dynamic-drop-semantics-works) -*** [Program illustrating semantic impact of hidden drop flag](#program-illustrating-semantic-impact-of-hidden-drop-flag) + * [Program illustrating space impact of hidden drop flag](#program-illustrating-space-impact-of-hidden-drop-flag) + * [How dynamic drop semantics works](#how-dynamic-drop-semantics-works) + * [Program illustrating semantic impact of hidden drop flag](#program-illustrating-semantic-impact-of-hidden-drop-flag) # Summary From e01c5201afad3c6b1e2d857d6c9a0185814740f3 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 14:30:51 +0200 Subject: [PATCH 30/38] Put Summary above ToC. --- active/0000-remove-drop-flag-and-zeroing.md | 50 +++++++++++---------- 1 file changed, 26 insertions(+), 24 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 892c7901f82..7aa93bf818e 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -2,8 +2,34 @@ - RFC PR: (leave this empty) - Rust Issue: (leave this empty) +# Summary + +Three step plan: + + 1. Revise language semantics for drop so that all branches move or drop + the same pieces of state ("drop obligations"). To satisfy this + constraint, the compiler has freedom to move the drop code for + some state to earlier points in the control flow ("early drops"). + + 2. Add lints to inform the programmer of situations when this new + drop-semantics could cause side-effects of RAII-style code + (e.g. releasing locks, flushing buffers) to occur sooner than + expected. + + Types that have side-effectful drop implement a marker trait, + `NoisyDrop`, that drives a warn-by-default lint; another marker + trait, `QuietDrop`, allows types to opt opt. An allow-by-default + lint provides a way for programmers to request notification of all + auto-inserted early-drops. + + 3. Remove the dynamic tracking of whether a value has been dropped or + not; in particular, (a) remove implicit addition of a drop-flag by + `Drop` impl, and (b) remove implicit zeroing of the memory that + occurs when values are dropped. + # Table of Contents * [Summary](#summary) +* [Table of Contents](#table-of-contents) * [Motivation](#motivation) * [Abandoning dynamic drop semantics](#abandoning-dynamic-drop-semantics) * [Detailed design](#detailed-design) @@ -39,30 +65,6 @@ * [How dynamic drop semantics works](#how-dynamic-drop-semantics-works) * [Program illustrating semantic impact of hidden drop flag](#program-illustrating-semantic-impact-of-hidden-drop-flag) -# Summary - -Three step plan: - - 1. Revise language semantics for drop so that all branches move or drop - the same pieces of state ("drop obligations"). To satisfy this - constraint, the compiler has freedom to move the drop code for - some state to earlier points in the control flow ("early drops"). - - 2. Add lints to inform the programmer of situations when this new - drop-semantics could cause side-effects of RAII-style code - (e.g. releasing locks, flushing buffers) to occur sooner than - expected. - - Types that have side-effectful drop implement a marker trait, - `NoisyDrop`, that drives a warn-by-default lint; another marker - trait, `QuietDrop`, allows types to opt opt. An allow-by-default - lint provides a way for programmers to request notification of all - auto-inserted early-drops. - - 3. Remove the dynamic tracking of whether a value has been dropped or - not; in particular, (a) remove implicit addition of a drop-flag by - `Drop` impl, and (b) remove implicit zeroing of the memory that - occurs when values are dropped. # Motivation From d9cae128c4a432c17c1fef4e231f908e98215fbe Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 14:33:46 +0200 Subject: [PATCH 31/38] Fix some hyperlinks. --- active/0000-remove-drop-flag-and-zeroing.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 7aa93bf818e..ae0e2fa21e1 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -41,8 +41,8 @@ Three step plan: * [match expressions and enum variants that move](#match-expressions-and-enum-variants-that-move) * [Type parameters](#type-parameters) * [Part 2: Early drop lints](#part-2-early-drop-lints) - * [The `early_noisy_drop` lint](#the-early-noisy-drop-lint) - * [The `early_quiet_drop` lint and `QuietDrop` trait](#the-early-quiet-drop-lint-and-quietdrop-trait) + * [The `early_noisy_drop` lint](#the-early_noisy_drop-lint) + * [The `early_quiet_drop` lint and `QuietDrop` trait](#the-early_quiet_drop-lint-and-quietdrop-trait) * [Type parameters, revisited](#type-parameters-revisited) * [Part 3: Removing the drop-flag; removing memory zeroing](#part-3-removing-the-drop-flag-removing-memory-zeroing) * [Drawbacks](#drawbacks) From 94043d3245e6a08204741c0e8c1f95f798dc494a Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 14:35:22 +0200 Subject: [PATCH 32/38] fix typo in hyperlink. --- active/0000-remove-drop-flag-and-zeroing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index ae0e2fa21e1..ef43e4148f9 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -49,7 +49,7 @@ Three step plan: * [Alternatives](#alternatives) * [Do nothing](#do-nothing) * [Require explicit drops rather than injecting them](#require-explicit-drops-rather-than-injecting-them) - * [Do this with support for variant-predicated drop-obligations](#do-this-with-support-for-variant-predicated drop-obligations) + * [Do this with support for variant-predicated drop-obligations](#do-this-with-support-for-variant-predicated-drop-obligations) * [Associate drop flags with stack-local variables alone](#associate-drop-flags-with-stack-local variables alone) * [Separate individual and grouped instances of a type](#separate-individual-and-grouped-instances-of-a-type) * [Unresolved questions](#unresolved-questions) From 7ae6da46c832d87147596a8648bc79129e4ed6cd Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 14:36:12 +0200 Subject: [PATCH 33/38] fix hyperlink. --- active/0000-remove-drop-flag-and-zeroing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index ef43e4148f9..c3f14760e67 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -50,7 +50,7 @@ Three step plan: * [Do nothing](#do-nothing) * [Require explicit drops rather than injecting them](#require-explicit-drops-rather-than-injecting-them) * [Do this with support for variant-predicated drop-obligations](#do-this-with-support-for-variant-predicated-drop-obligations) - * [Associate drop flags with stack-local variables alone](#associate-drop-flags-with-stack-local variables alone) + * [Associate drop flags with stack-local variables alone](#associate-drop-flags-with-stack-local-variables-alone) * [Separate individual and grouped instances of a type](#separate-individual-and-grouped-instances-of-a-type) * [Unresolved questions](#unresolved-questions) * [Names (bikeshed expected)](#names-bikeshed-expected) From 5f6f55934b35d244818dccec973ef20aaa30c2b6 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 14:37:49 +0200 Subject: [PATCH 34/38] fix hyperlink. --- active/0000-remove-drop-flag-and-zeroing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index c3f14760e67..5e4fffd4dca 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -59,7 +59,7 @@ Three step plan: * [Should type parameters be treated specially](#should-type-parameters-be-treated-specially) * [How should moving into wildcards be handled](#how-should-moving-into-wildcards-be-handled) * [Should the match-arm rule be weakened to just a warning](#should-the-match-arm-rule-be-weakened-to-just-a-warning) - * [The most direct `Option` re-encoding of drop-flag yields dead_assignments](#the-most-direct-option-t-re-encoding-of-drop-flag-yields-dead-assignments) + * [The most direct `Option` re-encoding of drop-flag yields dead_assignments](#the-most-direct-optiont-re-encoding-of-drop-flag-yields-dead_assignments) * [Appendices](#appendices) * [Program illustrating space impact of hidden drop flag](#program-illustrating-space-impact-of-hidden-drop-flag) * [How dynamic drop semantics works](#how-dynamic-drop-semantics-works) From 8f20122c87adca897e4517e87dc19bd599afa249 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 16:07:12 +0200 Subject: [PATCH 35/38] Add variant supporting dynamic semantics with fragment-drop-state stored out-of-band on stack. --- active/0000-remove-drop-flag-and-zeroing.md | 26 +++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 5e4fffd4dca..31f71c59de0 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -52,6 +52,7 @@ Three step plan: * [Do this with support for variant-predicated drop-obligations](#do-this-with-support-for-variant-predicated-drop-obligations) * [Associate drop flags with stack-local variables alone](#associate-drop-flags-with-stack-local-variables-alone) * [Separate individual and grouped instances of a type](#separate-individual-and-grouped-instances-of-a-type) + * [Store drop-flags for fragments of state on stack out-of-band](#store-drop-flags-for-fragments-of-state-on-stack-out-of-band) * [Unresolved questions](#unresolved-questions) * [Names (bikeshed expected)](#names-bikeshed-expected) * [Does the match-arm rule break expressiveness claim?](#does-the-match-arm-rule-break-expressiveness-claim) @@ -864,6 +865,31 @@ language/library change, and it is not clear whether it would be any better than static drop semantics in terms of Rust's language complexity budget. +## Store drop-flags for fragments of state on stack out-of-band + +(This is another variant on "Associate drop flags with stack-local variables alone", +but with appropriate support for tracking fragments of state.) + +Keep a dynamic drop semantics, but without the drop-flags. To +accomplish this, internally, the compiler does the same +drop-obligation analysis that is described above, to identify paths to +state where the drop-obligations differ at a merge point. On the +stack frame itself, keep an array of boolean drop-flags for the +function that tracks all such paths with differing drop obligations. +Then, at the end of a variable's scope, consult the drop-flag array as +necessary. + +Under this alternative, since we are continuing to use a dynamic drop +semantics, we do not add `NoisyDrop` nor `QuietDrop`. The main +difference of this alternative over what code does today is that code +would not be allowed to attempt to inspect the state of the drop flag. + +Whether or not to take this approach is largely dependent on whether +one thinks that it is too easy to be relying on a drop-flag +unintentionally; i.e., is it better to require code that wants a +drop-flag to use `Option`? Does that help code clarity? Or does +it just create unnecessary work? + # Unresolved questions ## Names (bikeshed expected) From 564814662940f2e5cd6f5cb1f4f0661b3c63a16e Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 16:59:05 +0200 Subject: [PATCH 36/38] Niko pointed out that we currently do support moving out of a `[Foo, ..k]` array. (And the borrowck just treats it imprecisely, as a potential move of any element and thus all become inaccessible.) --- active/0000-remove-drop-flag-and-zeroing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 31f71c59de0..60310af3777 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -100,7 +100,7 @@ Here are some problems with this situation: expectation that the drop-flag only be present for individual local variables, but that is not what Rust does currently: when `Foo` implements `Drop`, each instance of `Foo` carries a drop-flag, even - in contexts like a `Vec` or `[Foo, ..100]` where a program + in contexts like a `Vec` where a program cannot actually move individual values out of the collection. Thus, the amount of extra memory being used by drop-flags is not bounded by program stack growth; the memory wastage is strewn From 26c265f92e618c36177a7f1155a65684c9cf5536 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Mon, 25 Aug 2014 17:14:07 +0200 Subject: [PATCH 37/38] Edits suggested by niko. --- active/0000-remove-drop-flag-and-zeroing.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index 60310af3777..f626b62aca6 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -146,6 +146,9 @@ There are two important things to note about a static drop semantics: drop could lead to unexpected side-effects occurring earlier than expected. + (Note of course that a dynamic drop semantics may *also* be + surprising to some programmers.) + The bulk of the Detailed Design is dedicated to mitigating the second observation, to reduce the number of surprises for a programmer encountering an early-drop injected by `rustc`. @@ -584,6 +587,13 @@ above). ## Part 2: Early drop lints +The interaction of moves and "resource acquisition is initialization" +(RAII) patterns can have surprising consequences. Under the current +dynamic drop semantics, drops normally occur at the end of scope, but +not if a value has been moved. Under static drop semantics, drops of +moved values occur earlier. In many cases, it is unclear what the user +intended. + Some users may be surprised by the implicit drops that are injected by static drop semantics, especially if the drop code being executed has observable side-effects. @@ -600,8 +610,7 @@ Such side-effects include: change in timing semantics when writing concurrent algorithms). In particular, the injection of the implicit drops could silently -invalidate certain kinds of "resource acquisition is initialization" -(RAII) patterns. +invalidate certain kinds of RAII patterns. It is important to keep in mind that one can always recreate the effect of the former drop flag by wrapping one's type `T` in an @@ -704,7 +713,7 @@ considered pure is `Vec`, since the only side-effect of dropping a and `NoisyDrop` for any `T` that is `NoisyDrop`.) Then programmers can implement `NoisyDrop` on a type like `LockGuard` -to declare that it has sie-effects when dropped, and can use +to declare that it has side-effects when dropped, and can use `QuietDrop` to make a type with a noisy subcomponent quiet again, e.g. if the containing type forms an abstraction that makes the side-effect insignificant again. An example of the latter occurs in From a773ba113ba135ddb7f481c4829882733eaa5355 Mon Sep 17 00:00:00 2001 From: "Felix S. Klock II" Date: Wed, 27 Aug 2014 14:05:08 +0200 Subject: [PATCH 38/38] Removed the match-arm rule. Removed the match-arm rule, effectively adopting the proposal outlined in "Should the match-arm rule be weakened to just a warning." --- active/0000-remove-drop-flag-and-zeroing.md | 100 +++----------------- 1 file changed, 14 insertions(+), 86 deletions(-) diff --git a/active/0000-remove-drop-flag-and-zeroing.md b/active/0000-remove-drop-flag-and-zeroing.md index f626b62aca6..96a69150d23 100644 --- a/active/0000-remove-drop-flag-and-zeroing.md +++ b/active/0000-remove-drop-flag-and-zeroing.md @@ -55,11 +55,9 @@ Three step plan: * [Store drop-flags for fragments of state on stack out-of-band](#store-drop-flags-for-fragments-of-state-on-stack-out-of-band) * [Unresolved questions](#unresolved-questions) * [Names (bikeshed expected)](#names-bikeshed-expected) - * [Does the match-arm rule break expressiveness claim?](#does-the-match-arm-rule-break-expressiveness-claim) * [Which library types should be `QuietDrop`](#which-library-types-should-be-quietdrop) * [Should type parameters be treated specially](#should-type-parameters-be-treated-specially) * [How should moving into wildcards be handled](#how-should-moving-into-wildcards-be-handled) - * [Should the match-arm rule be weakened to just a warning](#should-the-match-arm-rule-be-weakened-to-just-a-warning) * [The most direct `Option` re-encoding of drop-flag yields dead_assignments](#the-most-direct-optiont-re-encoding-of-drop-flag-yields-dead_assignments) * [Appendices](#appendices) * [Program illustrating space impact of hidden drop flag](#program-illustrating-space-impact-of-hidden-drop-flag) @@ -539,43 +537,21 @@ while if the `One` arm matches, then the `s` is deconstructed and moved in pieces into `a1` and `a2`, which are themselves then consumed by the calls to `dA`. -While we *could* attempt to continue supporting this style of code -(see ["variant-predicated drop-obligations"](#do-this-with-support-for-variant-predicated-drop-obligations) -in the Alternatives section), it seems simpler if we just disallow it. This RFC -proposes the following so-called "match-arm rule": if any arm in a match consumes -the input via `move`, then *every* arm in the match must consume the -input *by the end of each arm's associated body*. +According to static drop semantics, if any arm in a match consumes the +input via `move`, then every arm must consume the input, possibly by +adding implicit early drops as necessary at the end of arms that do +not consume the input. In particular, the above code will remain +legal, but there will be an implicit early drop of `s` added at the +end of the first arm. -That last condition is crucial, because it enables patterns like -this to continue working: - -```rust - match s { - One(a1, a2) => { // a moving match here - dA(a1) + dA(a2) - } - Two(_, _) => { // a non-binding match here - helper_function(s) - } - }; - -``` - -Unfortunately, the same property does not hold -for a ref-binding match: we cannot write code like this: -```rust - match s { - One(a1, a2) => { // a moving match here - dA(a1) + dA(a2) - } - Two(ref r1, ref r2) => { // a ref-binding match here - let ret = helper_function(r1, r2); - mem::drop(s); // <-- oops, `s` is still borrowed. - ret - } - }; -``` +This is notable for two reasons. First, the programmer cannot +currently write such a drop explicitly today, since that would require +non-lexical borrows. Second, the borrow-checker must take care to not +allow borrowed references to live beyond the inserted early drops. +(An earlier version of this RFC proposed a so-called "match-arm rule" +that made the above code illegal, requiring that if any arm consumed +the input, then all arms must also consume the input.) ### Type parameters @@ -803,12 +779,7 @@ implied here made this a non-starter. ## Do this with support for variant-predicated drop-obligations -In "match expressions and enum variants" above, this RFC proposed the -match-arm rule that if any arm in a match consumes the input via `move`, then -every arm in the match must consume the input (by the end of its -body). - -There is an alternative, however. We could enrich the structure of +We could enrich the structure of drop-obligations to include paths that are predicated on enum variants, like so: `{(s is Two => s#0), (s is Two => s#1)}`. This represents the idea that (1.) all control flows where `s` is the `One` @@ -907,21 +878,6 @@ There may be better names for lints and the traits being added here. It took me a while to come up with the "noisy" and "quiet" mnemonics. -## Does the match-arm rule break expressiveness claim? - -I made the claim in "Abandoning dynamic drop semantics" -that a static drop semantics should be *equal* in expressive power to -the Rust language as we know it today. - -However, when I made that claim, I did not think carefully -about the implications of the simple match-arm rule. -Being forced to move out of the original owner in every arm -might imply that you cannot perform a truly automatic mechanical -transformation on the program to reencode the prior behavior. -Still, I remain confident that one can find some encoding in terms -of `Option` for any current program. - - ## Which library types should be `QuietDrop` Side-effectfulness is in the eye of the beholder. In particular, @@ -1028,34 +984,6 @@ match x { } ``` -## Should the match-arm rule be weakened to just a warning - -In principle we do not need to actually make it *illegal* to -write: -```rust - let ret = match s { - Two(ref r1, ref r2) => { - dR(r1) + dR(r2) - } - One(a1, a2) => { - dA(a1) + dA(a2) - } - }; -``` - -We could instead just treat this like another instance of a case where -there will be another early implicit drop (namely a drop of `s` at the -end of each arm where it has been accessed by reference) -- the -difference is that we cannot suggest that the user add an explicit -`drop` of `s` for such arms, since doing so would violate the -borrowing rules (since the references are still in scope). - -(But then again, if the borrowed references leak into the constructed -value that lives longer than the `match` itself, those implicit early -drops will be unsound. This scenario leads me to think that we should -strongly consider adopting the stronger form of the match-arm rule, -for simplicity in the compiler itself.) - ## The most direct `Option` re-encoding of drop-flag yields dead_assignments When porting this old code: