diff --git a/text/3680-use.md b/text/3680-use.md new file mode 100644 index 00000000000..a32f24abff4 --- /dev/null +++ b/text/3680-use.md @@ -0,0 +1,463 @@ +- Feature Name: `use` +- Start Date: 2024-07-20 +- RFC PR: [rust-lang/rfcs#3680](https://github.com/rust-lang/rfcs/pull/3680) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + +# Summary +[summary]: #summary + +Provide a feature to simplify performing lightweight clones (such as of +`Arc`/`Rc`), particularly cloning them into closures or async blocks, while +still keeping such cloning visible and explicit. + +# Motivation +[motivation]: #motivation + +A very common source of friction in asynchronous or multithreaded Rust +programming is having to clone various `Arc` reference-counted objects into +an async block or task. This is particularly common when spawning a closure as +a thread, or spawning an async block as a task. Common patterns for doing so +include: + +```rust +// Use new names throughout the block +let new_x = x.clone(); +let new_y = y.clone(); +spawn(async move { + func1(new_x).await; + func2(new_y).await; +}); + +// Introduce a scope to perform the clones in +{ + let x = x.clone(); + let y = y.clone(); + spawn(async move { + func1(x).await; + func2(y).await; + }); +} + +// Introduce a scope to perform the clones in, inside the call +spawn({ + let x = x.clone(); + let y = y.clone(); + async move { + func1(x).await; + func2(y).await; + } +}); +``` + +All of these patterns introduce noise every time the program wants to spawn a +thread or task, or otherwise clone an object into a closure or async block. +Feedback on Rust regularly brings up this friction, seeking a simpler solution. + +In addition, Rust developers trying to avoid heavyweight clones will sometimes +suggest eschewing invocations of `obj.clone()` in favor of writing +`Arc::clone(&obj)` explicitly, to mark the call explicitly as a lightweight +clone, at the cost of syntactic salt. This RFC proposes a syntax that can +*only* make a lightweight clone, while still using a simple postfix syntax. + +This RFC does *not* provide fully automatic/invisible cloning. Some users have +asked for that, wanting no visible indication at the point of the clone. +However, other users have asked that this *not* happen, and appreciate that +Rust does not have implicit clones or copy/move constructors. This RFC does not +change this, and does not provide any precedent or motivation for changing +this; any proposal to change this would need its own additional motivation and +detailed exploration of tradeoffs. + +This RFC proposes solutions to *minimize* the syntactic weight of +lightweight-cloning objects, particularly cloning objects into a closure or +async block, while still keeping an indication of this operation. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +When working with objects that support lightweight cloning, such as `Rc` or +`Arc`, you can get an additional clone (a new "use") of the object by invoking +`.use`: + +```rust +let obj: Arc = new_large_complex_object(); +some_function(obj.use); // Pass a separate use of the object to `some_function` +obj.method(); // The object is still owned afterwards +``` + +If you want to create a closure or async block that captures new uses of such +objects, you can put the `use` keyword on the closure or async block, similar +to the `move` keyword. + +```rust +let obj: Arc = new_large_complex_object(); +let map: Arc = new_mapping(); +std::thread::spawn(use || map.insert(42, func(obj))); +task::spawn(async use { op(map, obj).await }); +another_func(obj, map); +``` + +(Note that `use` and `move` are mutually exclusive.) + +`.use` supports chaining, so in particular it works when calling a method that +would otherwise consume `self`: + +```rust +obj.use.consume(); +obj.method(); +``` + +Calling `x.use` requires that the type of `x` implement the `Use` trait. This +trait identifies types whose clone implementation is lightweight, such as +reference-counted types. + +Various types in the standard library implement `Use`. You can implement this +trait for your own types, if they meet the requirements for being lightweight +to clone. (See the [reference-level explanation][reference-level-explanation] +for the requirements.) + +```rust +impl Use for MyType {} +``` + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +## The `Use` trait + +```rust +/// Trait for objects whose clone impl is lightweight (e.g. reference-counted) +/// +/// Cloning an object implementing this trait should in general: +/// - be O(1) (constant) time regardless of the amount of data managed by the object, +/// - not require a memory allocation, +/// - not require copying more than roughly 64 bytes (a typical cache line size), +/// - not block, +/// - not have any semantic side effects (e.g. allocating a file descriptor), and +/// - not have overhead larger than a couple of atomic operations. +/// +/// The `Use` trait does not provide a method; instead, it indicates that +/// `Clone::clone` is lightweight, and allows the use of the `.use` syntax. +trait Use: Clone {} +``` + +This trait should be implemented for anything in the standard library that +meets these criteria. Some notable types in the standard library that implement +`Use`: + +- `std::sync::Arc` +- `std::sync::Weak` +- `std::rc::Rc` +- `std::rc::Weak` +- `std::sync::mpsc::Sender` and `std::sync::mpsc::SyncSender` +- Tuples of types whose components all implement `Use` +- `Option` where `T: Use` +- `Result` where `T: Use` and `E: Use` + +Some notable types that implement `Clone` but should *not* implement `Use`: +arrays, `String`, `Box`, `Vec`, `HashMap`, and `BTreeMap`. + +We may want to add a clippy or rustc lint (e.g. `expensive_use`) for +implementations of `Use` on an excessively large type, or a type whose `clone` +implementation seems to be obviously breaking the intended constraints on the +`Use` trait. Such a lint would be best-effort only, and could always be marked +as `allow` by a crate, but could help to discourage such implementations. + +We may want to add a clippy or rustc lint for calls to `.clone()` that could +use `.use` instead. This would help the remaining calls to `.clone()` stand out +as "expensive" clones. (Such a lint would need to take MSRV into account before +making such a suggestion; clippy already has such a mechanism and rustc may +gain one in the future.) + +We may want to add a `derive(Use)`, which automatically adds `Use` bounds on +fields just as `derive(Clone)` adds `Clone` bounds on fields. + +## The implementation and optimization of `.use` + +An expression `x.use`, where `x` has type `T`, requires that `T: Use`. However, +`x.use` does not always invoke `Clone::clone(x)`; in some cases the compiler +can optimize away a use. + +If `x` is owned and statically known to be dead, the compiler will move `x` +rather than using it. This allows functions to write `x.use` without concern +for whether it's the last usage of `x` (e.g. when passing `x.use` to a series +of functions). Much like a trailing comma, this allows every usage to be +symmetric, making it easy to compare uses or add more uses afterwards. (This +optimization also means we should generally not lint on a final use of `x.use`, +such as we currently do with the clippy lint `redundant_clone`.) + +If `x` is not statically known to be dead, but *all* of the following +conditions are met, the compiler *may* elide an `x.use` and use `&x` instead: +- The compiler can statically see that `x` outlives the result of `x.use`, +- The compiler can statically see that the result of `x.use` is only accessed + via shared reference (including methods with `&self`, dereferences via + `Deref`, other invocations of `.use`, `use ||` closures, or `async use` + blocks). Effectively, these conditions mean that the user could theoretically + have refactored the code to use `&x` rather than `x.use`. + +An example of these elisions: + +```rust +fn f(obj: Arc) { + g(obj.use); // `g` takes `Arc` + h(use || obj.method()); // `Object::method` here takes `&self` +} + +fn main() { + let obj = Arc::new(Object::new()); + f(obj.use); + f(obj.use); + f(obj.use); + g(obj.use); +} +``` + +If every invocation of `.use` or `use ||` here resulted in a call to +`Clone::clone`, this program would call `Clone::clone` 10 times (and have 11 +`Arc`s to drop); however, the compiler can elide all the uses in `main` and +have `f` work with the original Arc, resulting in only 6 calls to +`Clone::clone` (and only 7 `Arc`s to drop). When an object is used repeatedly, +such as in a loop, this can result in many elisions over the course of the +program, and lower contention for atomics. + +If a user has an unusual `Use` type for which they wish to avoid these +potential elisions, they can call `.clone()` directly. + +At any time, we could potentially instrument the compiler to detect the number +of elided calls to `Clone::clone` in a crater run, to demonstrate the value of +this optimization. + +## `use ||` closures and `async use` blocks + +A closure can be written as `use |args| ...`, and an async block can be written +as `async use { ... }`, analogous to the use of `move`. (Note that `use` and +`move` are mutually exclusive.) + +For any object referenced within the closure or block that a `move` +closure/block would move: +- If the object is statically dead after the closure/block, ownership gets + moved into the closure/block. +- Otherwise, it gets `.use`ed, and the closure owns the new use of the object. +- If it isn't statically dead and doesn't implement `Use`, this produces an + error. + +# Drawbacks +[drawbacks]: #drawbacks + +This adds language surface area. + +While this still makes lightweight clones visible, it makes them *less* +visible. (See "Rationale and alternatives".) + +Users may misuse this by implementing `Use` for a type that doesn't meet the +requirements. Not all of the requirements can be checked by the compiler. While +this is still *safe*, it may result in types that violate user's expectations. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +We could do nothing, and require people to continue calling `.clone()` and +introducing new bindings for closures and async blocks. + +Rather than specifically supporting *lightweight* clones, we could add a syntax +for closures and async blocks to perform *any* clones (e.g. `async clone` / +`clone ||`). This would additionally allow expensive clones (such as +`String`/`Vec`). However, we've had many requests to distinguish between +expensive and lightweight clones, as well as ecosystem conventions attempting +to make such distinctions (e.g. past guidance to write `Arc::clone`/`Rc::clone` +explicitly). Having a syntax that only permits lightweight clones would allow +users to confidently use that syntax without worrying about an unexpectedly +expensive operation. We can then provide ways to perform the expensive clones +explicitly, such as the `use(x = x.clone())` syntax suggested in +[future possibilities][future-possibilities]. + +There are myriad names we could use for the trait, invocation syntax, and +closure / async block syntax. An ideal name for this needs to convey the +semantic of taking an additional reference to an object, without copying or +duplicating the object. Other names proposed for this mechanism include +`Claim`. + +We could use a different keyword, if one fits. For instance, we could use `ref` +rather than `use`. However, this 1) might over-fit to the concept of +reference-counting specifically, rather than lightweight clones, 2) might +confuse users who expect it to produce a `&` reference, and 3) for +closures/blocks, would not generalize to explicit capture lists. + +We could use a name that directly references referencing counting; for +instance, `AddRef`. However, that would be confusing for applications using +this with objects managed by something other than reference counting, such as +RCU or hazard pointers, or with (small) objects being copied. + +Rather than having the method `Clone::clone`, we could have a method +`Use::do_use` and have the compiler call that. We could prevent users from +overriding that method implementation, so that it always calls `Clone::clone`; +however, having a separate `do_use` method would allow users to call +`.do_use()` (when wanting to avoid the elision of `.use`) while knowing they +can't invoke an expensive clone operation. This does not seem worth the +additional complexity, since most users should invoke `.use` directly. + +Rather than using the special syntax `.use`, we could use an ordinary trait +method and invoke that method directly (e.g. `.claim()`). The syntax provided +for closures and async blocks could likewise always invoke that method. This +would not be compatible with adding smarter semantics such as eliding uses, +however. In addition, `.use` can be used in all editions of Rust (because `use` +is a reserved keyword), while a new trait with a method like `.claim()` would +require an import in existing editions and could only become part of the +prelude in a future edition. + +Rather than attaching this behavior to types (e.g. `Arc` and `Rc`), we could +attach this behavior to *bindings*. For instance, we could have `let +somekeyword x = Arc::new(...);` and make it easy to clone such bindings into +closures and async blocks. However, this would add additional noise to every +such binding, when users are likely to want this behavior for *every* binding +of a given type (e.g. every `Arc`). In addition, this would require adding such +an annotation to struct fields and similar. + +We could use an ordinary trait method *and* add a new set of semantics attached +to that trait method, such that the compiler is allowed to elide calls to the +trait method. This would avoid the introduction of new language syntax for +`.use`, at the cost of making something that looks like an ordinary method call +have semantics beyond those normally associated with a method call. + +Rather than having a single keyword in `use ||` or `async use`, we could +require naming every individual object being used (e.g. `use(x, y) ||`). +There's precedent for this kind of explicit capture syntax in other languages +(and having it as an available *option* is in the +[future possibilities][future-possibilities] section). However, requiring a +list of every object used adds overhead to closures and async blocks throughout +a program. This RFC proposes that the single keyword `use` suffices to indicate +that lightweight cloning will take place. + +Rather than having `Use` act as `Clone` and go from `&self` to `Self`, we could +translate through a trait like `ToOwned`. This would allow using `Use` when +owned values have a different type, such as `&MyType -> SmartPtr`. +However, this would also add complexity to the common case. + +We could omit the elision optimizations, and have `.use` *always* call +`Clone::clone` unconditionally. This would be slightly simpler, but would add +unnecessary overhead, and would encourage users to micro-optimize their code by +arranging to omit calls to `.use`. The elision optimizations encourage users to +always call `.use`. + +In the elision optimizations, we could potentially allow the last usage of the +result of `x.use` to take ownership of it, and the compiler could insert a +`.use` at that point. However, at that point the elision would not have +resulted in any fewer calls to `.use`, so this does not seem worth the extra +complexity. + +Rather than adding elision behavior for `.use`, we could add elision behavior +for specifically designated `Clone` implementations (e.g. with an attribute). +However, this would leave users unable to make an *un*-elided call to `clone`. +And if we added a mechanism to bypass this elision, the result would likely +have comparable complexity to the addition of a separate trait. Nonetheless, +there's potential for possible optimization here, and that optimization might +benefit expensive clones as well (e.g. eliding the clone of a `String` if the +`String` being cloned is statically known to be dead). We should explore +potential optimizations here. + +Rather than making the elision optimizations optional and allowing for future +improvements to them, we could mandate those optimizations and specify the +exact elision rules. This would, however, constrain our ability to improve +elision in the future, as well as requiring full implementation of those +elision rules before shipping the feature. This would not be the first Rust +mechanism whose implementation is subject to change, and leaving it flexible +allows the Rust compiler to improve in the future. + +We could specify that for `Copy` types, `use` always copies directly, ignoring +any `Clone` implementation. Would this have an advantage over calling `Clone` +and relying on that generally turning into a copy for `Copy` types? +(Interacting with that: in a future edition, would we want to make `.clone()` +of a `Copy` type ignore the `Clone` implementation and just copy?) + +We could make `x.use` inside of a closure cause the closure to get a new use of +`x` even if it otherwise could borrow, effectively "floating" the `use` outside +of the closure. This would have different semantics, such as if the closure +mutated the object before the `use`, so deciding this is a one-way door. + +# Prior art +[prior-art]: #prior-art + +Many languages have built-in reference counting, and automatically manage +reference counts when copying or passing around objects. `.use` provides a +simple and efficient way for Rust to manage reference counts. + +Some languages have copy constructors, allowing arbitrary code to run when +copying an object. Such languages can manage reference-counted smart pointers +implicitly. + +Rust already has other uses of postfix syntax, notably `.await` and `?`. In +particular, `.await` provides precedent for the use of `.keyword`. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +Should we allow *any* `x.clone()` call to be elided if a type is `Use` (or `Use ++ Copy`)? If we did this, that would not give us a way to explicitly avoid +elision (unless we added a separate mechanism for that), but it would also mean +that we could avoid tracking whether a clone call came from `use` or not when +applying the elision rules, which might make the implementation simpler. (This +RFC proposes not allowing explicit `x.clone()` calls to be elided, so that +users can make explicit `x.clone()` calls to avoid elision if desired.) + +Are there use cases for using something like `ToOwned` rather than always going +from `&self` to `Self`? Would any smart pointers need that? + +Should `Use` be a `#[marker]` trait? + +# Future possibilities +[future-possibilities]: #future-possibilities + +We could implement `Use` for small arrays (e.g. arrays smaller than 64 bytes). + +We could consider using lints or similar to help avoid unexpected expensive +copies of *large* arrays or other large `Copy` types, potentially using `Use` +as a marker for what is not expensive. More generally, we could explore the +space of types with various combinations of `Use` and `Copy`, and what we want +`Copy + Use` vs `Copy + !Use` to indicate. + +We could allow `.use` in struct construction without duplicating the name of a +field. For instance: + +```rust +let s = SomeStruct { field, field2.use, field3.use }; +/// This expands to: +let s = SomeStruct { + field: field, + field2: field2.use, + field3: field3.use, +}; +``` + +We could extend the `use` syntax on closures and async blocks to support +naming specific objects to use: + +```rust +use(x) || { ... } + +async use(x) { ... } +``` + +We could further extend the `use` syntax to support an explicit capture list of +named expressions: + +```rust +use(x = x.method(), y) || { ... } + +// `..` additionally captures everything that a plain `use` would +async use(x = &obj, move y, ..) +``` + +This could also align with other proposals for precise capture syntax (e.g. +`use(ref x)`). + +This potentially makes `use` closures/blocks almost completely supersede `move` +closures; we could consider in the future whether we want to deprecate them. + +We could consider providing a syntax to make invocations like `func(a.use, +b.use, c.use)` less verbose. In many cases, such a function could accept a +reference and call `.use` itself if needed, but we should evaluate whether +there are use cases not covered by that. + +When adding an `impl Use for Type`, if the compiler would otherwise error due +to a missing `Clone` bound, we may want the compiler to suggest a `Use` bound +instead. (The same problem applies to other traits; for instance, an `impl Eq +for Type` may want to suggest an `Eq` bound rather than a `PartialEq` bound.)