From b87601562b8e3e6d1613f21c514f4959a6150f34 Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Fri, 9 Apr 2021 13:17:47 -0700 Subject: [PATCH 01/17] RFC: I/O Safety --- text/0000-io-safety.md | 429 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 429 insertions(+) create mode 100644 text/0000-io-safety.md diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md new file mode 100644 index 00000000000..fcff865c332 --- /dev/null +++ b/text/0000-io-safety.md @@ -0,0 +1,429 @@ +- Feature Name: `io_safety` +- Start Date: 2021-05-24 +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + +# Summary +[summary]: #summary + +Close a hole in encapsulation boundaries in Rust by providing users of +`AsRawFd` and related traits guarantees about their raw resource handles, by +introducing a concept of *I/O safety* and a new `IoSafe` trait. Build on, and +provide an explanation for, the `from_raw_fd` function being unsafe. + +# Motivation +[motivation]: #motivation + +Rust's standard library almost provides *I/O safety*, a guarantee that if one +part of a program holds a raw handle privately, other parts cannot access it. +[`FromRawFd::from_raw_fd`] is unsafe, which prevents users from doing things +like `File::from_raw_fd(7)`, in safe Rust, and doing I/O on a file descriptor +which might be held privately elsewhere in the program. + +However, there's a loophole. Many library APIs use [`AsRawFd`]/[`IntoRawFd`] to +accept values to do I/O operations with: + +```rust +pub fn do_some_io(input: &FD) -> io::Result<()> { + some_syscall(input.as_raw_fd()) +} +``` + +`AsRawFd` doesn't restrict `as_raw_fd`'s return value, so `do_some_io` can end +up doing I/O on arbitrary `RawFd` values. One can even write `do_some_io(&7)`, +since [`RawFd`] itself implements `AsRawFd`. + +This can cause programs to [access the wrong resources], or even break +encapsulation boundaries by creating aliases to raw handles held privately +elsewhere, causing [spooky action at a distance]. + +And in specialized circumstances, violating I/O safety could even lead to +violating memory safety. For example, in theory it should be possible to make +a safe wrapper around an `mmap` of a file descriptor created by Linux's +[`memfd_create`] system call and pass `&[u8]`s to safe Rust, since it's an +anonymous open file which other processes wouldn't be able to access. However, +without I/O safety, and without permanently sealing the file, other code in +the program could accidentally call `write` or `ftruncate` on the file +descriptor, breaking the memory-safety invariants of `&[u8]`. + +This RFC introduces a path to gradually closing this loophole by introducing: + + - A new concept, I/O safety, to be documented in the standard library + documentation. + - A new trait, `std::io::IoSafe`. + - New documentation for + [`from_raw_fd`]/[`from_raw_handle`]/[`from_raw_socket`] explaining why + they're unsafe in terms of I/O safety, addressing a question that has + come up a [few] [times]. + +[few]: https://github.com/rust-lang/rust/issues/72175 +[times]: https://users.rust-lang.org/t/why-is-fromrawfd-unsafe/39670 +[access the wrong resources]: https://cwe.mitre.org/data/definitions/910.html +[spooky action at a distance]: https://en.wikipedia.org/wiki/Action_at_a_distance_(computer_programming) + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +## The I/O safety concept + +Rust's standard library has low-level types, [`RawFd`] on Unix-like platforms, +and [`RawHandle`]/[`RawSocket`] on Windows, which represent raw OS resource +handles. These don't provide any behavior on their own, and just represent +identifiers which can be passed to low-level OS APIs. + +These raw handles can be thought of as raw pointers, with similar hazards. +While it's safe to *obtain* a raw pointer, *dereferencing* a raw pointer could +invoke undefined behavior if it isn't a valid pointer or if it outlives the +lifetime of the memory it points to. Similarly, it's safe to *obtain* a raw +handle, via [`AsRawFd::as_raw_fd`] and similar, but using it to do I/O could +lead to corrupted output, lost or leaked input data, or violated encapsulation +boundaries, if it isn't a valid handle or it's used after the `close` of its +resource. And in both cases, the effects can be non-local, affecting otherwise +unrelated parts of a program. Protection from raw pointer hazards is called +memory safety, so protection from raw handle hazards is called *I/O safety*. + +Rust's standard library also has high-level types such as [`File`] and +[`TcpStream`] which are wrappers around these raw handles, providing high-level +interfaces to OS APIs. + +These high-level types also implement the traits [`FromRawFd`] on Unix-like +platforms, and [`FromRawHandle`]/[`FromRawSocket`] on Windows, which provide +functions which wrap a low-level value to produce a high-level value. These +functions are unsafe, since they're unable to guarantee I/O safety. The type +system doesn't constrain the handles passed in: + +```rust + use std::fs::File; + use std::os::unix::io::FromRawFd; + + // Create a file. + let file = File::open("data.txt")?; + + // Construct a `File` from an arbitrary integer value. This type checks, + // however 7 may not identify a live resource at runtime, or it may + // accidentally alias encapsulated raw handles elsewhere in the program. An + // `unsafe` block acknowledges that it's the caller's responsibility to + // avoid these hazards. + let forged = unsafe { File::from_raw_fd(7) }; + + // Obtain a copy of `file`'s inner raw handle. + let raw_fd = file.as_raw_fd(); + + // Close `file`. + drop(file); + + // Open some unrelated file. + let another = File::open("another.txt")?; + + // Further uses of `raw_fd`, which was `file`'s inner raw handle, would be + // outside the lifetime the OS associated with it. This could lead to it + // accidentally aliasing other otherwise encapsulated `File` instances, + // such as `another`. Consequently, an `unsafe` block acknowledges that + // it's the caller's responsibility to avoid these hazards. + let dangling = unsafe { File::from_raw_fd(raw_fd) }; +``` + +Callers must ensure that the value passed into `from_raw_fd` is explicitly +returned from the OS, and that `from_raw_fd`'s return value won't outlive the +lifetime the OS associates with the handle. + +I/O safety is new as an explicit concept, but it reflects common practices. +Rust's `std` will require no changes to stable interfaces, beyond the +introduction of a new trait and new impls for it. Initially, not all of the +Rust ecosystem will support I/O safety though; adoption will be gradual. + +## The `IoSafe` trait + +These high-level types also implement the traits [`AsRawFd`]/[`IntoRawFd`] on +Unix-like platforms and +[`AsRawHandle`]/[`AsRawSocket`]/[`IntoRawHandle`]/[`IntoRawSocket`] on Windows, +providing ways to obtain the low-level value contained in a high-level value. +APIs use these to accept any type containing a raw handle, such as in the +`do_some_io` example in the [motivation]. + +`AsRaw*` and `IntoRaw*` don't make any guarantees, so to add I/O safety, types +will implement a new trait, `IoSafe`: + +```rust +pub unsafe trait IoSafe {} +``` + +There are no required functions, so implementing it just takes one line, plus +comments: + +```rust +/// # Safety +/// +/// `MyType` wraps a `std::fs::File` which handles the low-level details, and +/// doesn't have a way to reassign or independently drop it. +unsafe impl IoSafe for MyType {} +``` + +It requires `unsafe`, to require the code to explicitly commit to upholding I/O +safety. With `IoSafe`, the `do_some_io` example should simply add a +`+ IoSafe` to provide I/O safety: + +```rust +pub fn do_some_io(input: &FD) -> io::Result<()> { + some_syscall(input.as_raw_fd()) +} +``` + +## Gradual adoption + +I/O safety and `IoSafe` wouldn't need to be adopted immediately, adoption +could be gradual: + + - First, `std` adds `IoSafe` with impls for all the relevant `std` types. + This is a backwards-compatible change. + + - After that, crates could implement `IoSafe` for their own types. These + changes would be small and semver-compatible, without special coordination. + + - Once the standard library and enough popular crates utilize `IoSafe`, + crates could start to add `+ IoSafe` bounds (or adding `unsafe`), at their + own pace. These would be semver-incompatible changes, though most users of + APIs adding `+ IoSafe` wouldn't need any changes. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +## The I/O safety concept + +In addition to the Rust language's memory safety, Rust's standard library also +guarantees I/O safety. An I/O operation is *valid* if the raw handles +([`RawFd`], [`RawHandle`], and [`RawSocket`]) it operates on are values +explicitly returned from the OS, and the operation occurs within the lifetime +the OS associates with them. Rust code has *I/O safety* if it's not possible +for that code to cause invalid I/O operations. + +While some OS's document their file descriptor allocation algorithms, a handle +value predicted with knowledge of these algorithms isn't considered "explicitly +returned from the OS". + +Functions accepting arbitrary raw I/O handle values ([`RawFd`], [`RawHandle`], +or [`RawSocket`]) should be `unsafe` if they can lead to any I/O being +performed on those handles through safe APIs. + +Functions accepting types implementing +[`AsRawFd`]/[`IntoRawFd`]/[`AsRawHandle`]/[`AsRawSocket`]/[`IntoRawHandle`]/[`IntoRawSocket`] +should add a `+ IoSafe` bound if they do I/O with the returned raw handle. + +## The `IoSafe` trait + +Types implementing `IoSafe` guarantee that they uphold I/O safety. They must +not make it possible to write a safe function which can perform invalid I/O +operations, and: + + - A type implementing `AsRaw* + IoSafe` means its `as_raw_*` function returns + a handle which is valid to use for the duration of the `&self` reference. + If such types have methods to close or reassign the handle without + dropping the whole object, they must document the conditions under which + existing raw handle values remain valid to use. + + - A type implementing `IntoRaw* + IoSafe` means its `into_raw_*` function + returns a handle which is valid to use at the point of the return from + the call. + +All standard library types implementing `AsRawFd` implement `IoSafe`, except +`RawFd`. + +Note that, despite the naming similarity, the `IoSafe` trait's requirements not +identical to the I/O safety requirements. The return value of `as_raw_*` is +valid only for the duration of the `&self` argument passed in. + +# Drawbacks +[drawbacks]: #drawbacks + +Crates with APIs that use file descriptors, such as [`nix`] and [`mio`], would +need to migrate to types implementing `AsRawFd + IoSafe`, use crates providing +equivalent mechanisms such as [`unsafe-io`], or change such functions to be +unsafe. + +Crates using `AsRawFd` or `IntoRawFd` to accept "any file-like type" or "any +socket-like type", such as [`socket2`]'s [`SockRef::from`], would need to +either add a `+ IoSafe` bound or make these functions unsafe. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +## Concerning "unsafe is for memory safety" + +Rust historically drew a line in the sand, stating that `unsafe` would only +be for memory safety. A famous example is [`std::mem::forget`], which was +once `unsafe`, and was [changed to safe]. The conclusion stating that unsafe +only be for memory safety observed that unsafe should not be for “footguns” +or for being “a general deterrent for "should be avoided" APIs”. + +Memory safety is elevated above other programming hazards because it isn't +just about avoiding unintended behavior, but about avoiding situations where +it's impossible to bound the set of things that a piece of code might do. + +I/O safety is also in this category, for two reasons. + + - I/O safety errors can lead to memory safety errors in the presence of + safe wrappers around `mmap` (on platforms with OS-specific APIs allowing + them to otherwise be safe). + + - I/O safety errors can also mean that a piece of code can read, write, or + delete data used by other parts of the program, without naming them or + being given a reference to them. It becomes impossible to bound the set + of things a crate can do without knowing the implementation details of all + other crates linked into the program. + +Raw handles are much like raw pointers into a separate address space; they can +dangle or be computed in bogus ways. I/O safety is similar to memory safety; +both prevent spooky-action-at-a-distance, and in both, ownership is the main +foundation for robust abstractions, so it's natural to use similar safety +concepts. + +[`std::mem::forget` being safe]: https://doc.rust-lang.org/std/mem/fn.forget.html +[changed to safe]: https://rust-lang.github.io/rfcs/1066-safe-mem-forget.html + +## I/O Handles as plain data + +The main alternative would be to say that raw handles are plain data, with no +concept of I/O safety and no inherent relationship to OS resource lifetimes. On +Unix-like platforms at least, this wouldn't ever lead to memory unsafety or +undefined behavior. + +However, most Rust code doesn't interact with raw handles directly. This is a +good thing, independently of this RFC, because resources ultimately do have +lifetimes, so most Rust code will always be better off using higher-level types +which manage these lifetimes automatically and which provide better ergonomics +in many other respects. As such, the plain-data approach would at best make raw +handles marginally more ergonomic for relatively uncommon use cases. This would +be a small benefit, and may even be a downside, if it ends up encouraging people +to write code that works with raw handles when they don't need to. + +The plain-data approach also wouldn't need any code changes in any crates. The +I/O safety approach will require changes to Rust code in crates such as +[`socket2`], [`nix`], and [`mio`] which have APIs involving [`AsRawFd`] and +[`RawFd`], though the changes can be made gradually across the ecosystem rather +than all at once. + +## New types for `RawFd`/`RawHandle`/`RawSocket` + +Some comments on [rust-lang/rust#76969] suggest introducing new wrappers +around the raw handles. Completely closing the safety loophole would also +require designing new traits, since `AsRaw*` doesn't have a way to limit the +lifetime of its return value. This RFC doesn't rule this out, but it would be a +bigger change. + +## I/O safety but not `IoSafe` + +The I/O safety concept doesn't depend on `IoSafe` being in `std`. Crates could +continue to use [`unsafe_io::OwnsRaw`], though that does involve adding a +dependency. + +## Define `IoSafe` in terms of the object, not the reference + +The [reference-level-explanation] explains `IoSafe + AsRawFd` as returning a +handle valid to use for "the duration of the `&self` reference". This makes it +similar to borrowing a reference to the handle, though it still uses a raw +type which doesn't enforce the borrowing rules. + +An alternative would be to define it in terms of the underlying object. Since +it returns raw types, arguably it would be better to make it work more like +`slice::as_ptr` and other functions which return raw pointers that aren't +connected to reference lifetimes. If the concept of borrowing is desired, new +types could be introduced, with better ergonomics, in a separate proposal. + +# Prior art +[prior-art]: #prior-art + +Most memory-safe programming languages have safe abstractions around raw +handles. Most often, they simply avoid exposing the raw handles altogether, +such as in [C#], [Java], and others. Making it `unsafe` to perform I/O through +a given raw handle would let safe Rust have the same guarantees as those +effectively provided by such languages. + +The `std::io::IoSafe` trait comes from [`unsafe_io::OwnsRaw`], and experience +with this trait, including in some production use cases, has shaped this RFC. + +[C#]: https://docs.microsoft.com/en-us/dotnet/api/system.io.file?view=net-5.0 +[Java]: https://docs.oracle.com/javase/7/docs/api/java/io/File.html?is-external=true + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +## Formalizing ownership + +This RFC doesn't define a formal model for raw handle ownership and lifetimes. +The rules for raw handles in this RFC are vague about their identity. What does +it mean for a resource lifetime to be associated with a handle if the handle is +just an integer type? Do all integer types with the same value share that +association? + +The Rust [reference] defines undefined behavior for memory in terms of +[LLVM's pointer aliasing rules]; I/O could conceivably need a similar concept +of handle aliasing rules. This doesn't seem necessary for present practical +needs, but it could be explored in the future. + +[reference]: https://doc.rust-lang.org/reference/behavior-considered-undefined.html + +# Future possibilities +[future-possibilities]: #future-possibilities + +Some possible future ideas that could build on this RFC include: + + - New wrapper types around `RawFd`/`RawHandle`/`RawSocket`, to improve the + ergonomics of some common use cases. Such types may also provide portability + features as well, abstracting over some of the `Fd`/`Handle`/`Socket` + differences between platforms. + + - Higher-level abstractions built on `IoSafe`. Features like + [`from_filelike`] and others in [`unsafe-io`] eliminate the need for + `unsafe` in user code in some common use cases. [`posish`] uses this to + provide safe interfaces for POSIX-like functionality without having `unsafe` + in user code, such as in [this wrapper around `posix_fadvise`]. + + - Clippy lints warning about common I/O-unsafe patterns. + + - A formal model of ownership for raw handles. One could even imagine + extending Miri to catch "use after close" and "use of bogus computed handle" + bugs. + + - A fine-grained capability-based security model for Rust, built on the fact + that, with this new guarantee, the high-level wrappers around raw handles + are unforgeable in safe Rust. + +[`from_filelike`]: https://docs.rs/unsafe-io/0.6.2/unsafe_io/trait.FromUnsafeFile.html#method.from_filelike +[this wrapper around `posix_fadvise`]: https://docs.rs/posish/0.6.1/posish/fs/fn.fadvise.html + +# Thanks +[thanks]: #thanks + +Thanks to Ralf Jung ([@RalfJung]) for leading me to my current understanding +of this topic, for encouraging and reviewing drafts of this RFC, and for +patiently answering my many questions! + +[@RalfJung]: https://github.com/RalfJung +[`File`]: https://doc.rust-lang.org/stable/std/fs/struct.File.html +[`TcpStream`]: https://doc.rust-lang.org/stable/std/net/struct.TcpStream.html +[`FromRawFd`]: https://doc.rust-lang.org/stable/std/os/unix/io/trait.FromRawFd.html +[`FromRawHandle`]: https://doc.rust-lang.org/stable/std/os/windows/io/trait.FromRawHandle.html +[`FromRawSocket`]: https://doc.rust-lang.org/stable/std/os/windows/io/trait.FromRawSocket.html +[`AsRawFd`]: https://doc.rust-lang.org/stable/std/os/unix/io/trait.AsRawFd.html +[`AsRawHandle`]: https://doc.rust-lang.org/stable/std/os/windows/io/trait.AsRawHandle.html +[`AsRawSocket`]: https://doc.rust-lang.org/stable/std/os/windows/io/trait.AsRawSocket.html +[`IntoRawFd`]: https://doc.rust-lang.org/stable/std/os/unix/io/trait.IntoRawFd.html +[`IntoRawHandle`]: https://doc.rust-lang.org/stable/std/os/windows/io/trait.IntoRawHandle.html +[`IntoRawSocket`]: https://doc.rust-lang.org/stable/std/os/windows/io/trait.IntoRawSocket.html +[`RawFd`]: https://doc.rust-lang.org/stable/std/os/unix/io/type.RawFd.html +[`RawHandle`]: https://doc.rust-lang.org/stable/std/os/windows/io/type.RawHandle.html +[`RawSocket`]: https://doc.rust-lang.org/stable/std/os/windows/io/type.RawSocket.html +[`FromRawFd::from_raw_fd`]: https://doc.rust-lang.org/stable/std/os/unix/io/trait.FromRawFd.html#tymethod.from_raw_fd +[`from_raw_fd`]: https://doc.rust-lang.org/stable/std/os/unix/io/trait.FromRawFd.html#tymethod.from_raw_fd +[`from_raw_handle`]: https://doc.rust-lang.org/stable/std/os/windows/io/trait.FromRawHandle.html#tymethod.from_raw_handle +[`from_raw_socket`]: https://doc.rust-lang.org/stable/std/os/windows/io/trait.FromRawSocket.html#tymethod.from_raw_socket +[`SockRef::from`]: https://docs.rs/socket2/0.4.0/socket2/struct.SockRef.html#method.from +[`unsafe_io::OwnsRaw`]: https://docs.rs/unsafe-io/0.6.2/unsafe_io/trait.OwnsRaw.html +[LLVM's pointer aliasing rules]: http://llvm.org/docs/LangRef.html#pointer-aliasing-rules +[`nix`]: https://crates.io/crates/nix +[`mio`]: https://crates.io/crates/mio +[`socket2`]: https://crates.io/crates/socket2 +[`unsafe-io`]: https://crates.io/crates/unsafe-io +[`posish`]: https://crates.io/crates/posish +[rust-lang/rust#76969]: https://github.com/rust-lang/rust/pull/76969 +[`memfd_create`]: https://man7.org/linux/man-pages/man2/memfd_create.2.html From 5fc62f4b199ffc9298dafcc56dfefe9f7dae88b6 Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Mon, 24 May 2021 18:03:35 -0700 Subject: [PATCH 02/17] Set the PR number. --- text/0000-io-safety.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index fcff865c332..e6b4dca25e1 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -1,6 +1,6 @@ - Feature Name: `io_safety` - Start Date: 2021-05-24 -- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- RFC PR: [rust-lang/rfcs#3128](https://github.com/rust-lang/rfcs/pull/3128) - Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) # Summary From fda84ab67b2c403228fd50ac409779e5b7ce6146 Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Wed, 26 May 2021 09:20:23 -0700 Subject: [PATCH 03/17] Update text/0000-io-safety.md Co-authored-by: Josh Triplett --- text/0000-io-safety.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index e6b4dca25e1..abc9657d781 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -228,7 +228,7 @@ operations, and: All standard library types implementing `AsRawFd` implement `IoSafe`, except `RawFd`. -Note that, despite the naming similarity, the `IoSafe` trait's requirements not +Note that, despite the naming similarity, the `IoSafe` trait's requirements are not identical to the I/O safety requirements. The return value of `as_raw_*` is valid only for the duration of the `&self` argument passed in. From 87db7d124dafd3fdbcb407247ac80bc9cf4b2f2a Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Thu, 27 May 2021 09:22:31 -0700 Subject: [PATCH 04/17] Add an example of a class which can dynamically drop its resource. --- text/0000-io-safety.md | 45 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index abc9657d781..80448c07d75 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -169,6 +169,51 @@ pub fn do_some_io(input: &FD) -> io::Result<()> { } ``` +Some types have the ability to dynamically drop their resources, and +these types require special consideration when implementing `IoSafe`. For +example, a class representing a dynamically reassignable output source might +have code like this: + +```rust +struct VirtualStdout { + current: RefCell +} + +impl VirtualStdout { + /// Assign a new output destination. + /// + /// This function ends the lifetime of the resource that `as_raw_fd` + /// returns a handle to. + pub fn set_output(&self, new: std::fs::File) { + *self.current.borrow_mut() = new; + } +} + +impl AsRawFd for VirtualStdout { + fn as_raw_fd(&self) -> RawFd { + self.current.borrow().as_raw_fd() + } +} +``` + +If a user of this type were to hold a `RawFd` value over a call to `set_file`, +the `RawFd` value would become dangling, even though its within the lifetime of +the `&self` reference passed to `as_raw_fd`: + +```rust + fn foo(output: &VirtualStdout) -> io::Result<()> { + let raw_fd = output.as_raw_fd(); + output.set_file(File::open("/some/other/file")?); + use(raw_fd)?; // Use of dangling file descriptor! + Ok(()) + } +``` + +The `IoSafe` trait requires types capable of dynamically dropping their +resources within the lifetime of the `&self` passed to `as_raw_fd` must +document the conditions under which this can occur, as the documentation +comment above does. + ## Gradual adoption I/O safety and `IoSafe` wouldn't need to be adopted immediately, adoption From 8b273b600a0e11fd9e246ff260f374880c64edef Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Sat, 29 May 2021 18:38:40 -0700 Subject: [PATCH 05/17] Add a new experimental API as a possible alternative. --- text/0000-io-safety.md | 37 +++++++++++++++++++++++++++++-------- 1 file changed, 29 insertions(+), 8 deletions(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index 80448c07d75..942c6a6001d 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -347,14 +347,6 @@ I/O safety approach will require changes to Rust code in crates such as [`RawFd`], though the changes can be made gradually across the ecosystem rather than all at once. -## New types for `RawFd`/`RawHandle`/`RawSocket` - -Some comments on [rust-lang/rust#76969] suggest introducing new wrappers -around the raw handles. Completely closing the safety loophole would also -require designing new traits, since `AsRaw*` doesn't have a way to limit the -lifetime of its return value. This RFC doesn't rule this out, but it would be a -bigger change. - ## I/O safety but not `IoSafe` The I/O safety concept doesn't depend on `IoSafe` being in `std`. Crates could @@ -374,6 +366,35 @@ it returns raw types, arguably it would be better to make it work more like connected to reference lifetimes. If the concept of borrowing is desired, new types could be introduced, with better ergonomics, in a separate proposal. +## New types and traits + +New types and traits could provide a much cleaner API, along the lines of: + +```rust +pub struct BorrowedFd<'owned> { ... } +pub struct OwnedFd { ... } +pub struct OptionFd { ... } + +pub trait AsBorrowedFd { ... } +pub trait IntoOwnedFd { ... } +pub trait FromOwnedFd { ... } +``` + +An initial prototype of this here: + + + +The details are mostly obvious, though one notable aspect of this design is +the use of `repr(transparent)` to define types that can participate in FFI +directly, leading to FFI usage patterns that don't interact with raw types +at all. An example of this is here: + + + +This provides a cleaner API than `*Raw*` + `IoSafe`. The main obvious downside +is that a lot of code will likely need to continue to support `*Raw*` for a +long time, so this would increase the amount of code they have to maintain. + # Prior art [prior-art]: #prior-art From 79160e807cf069b7cf230eb7a948da8824fc2028 Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Mon, 31 May 2021 13:24:29 -0700 Subject: [PATCH 06/17] Remove `OptionFd`. --- text/0000-io-safety.md | 1 - 1 file changed, 1 deletion(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index 942c6a6001d..39f75f37430 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -373,7 +373,6 @@ New types and traits could provide a much cleaner API, along the lines of: ```rust pub struct BorrowedFd<'owned> { ... } pub struct OwnedFd { ... } -pub struct OptionFd { ... } pub trait AsBorrowedFd { ... } pub trait IntoOwnedFd { ... } From 2e82b837a254d0502df6842460690765412ab696 Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Tue, 1 Jun 2021 08:21:26 -0700 Subject: [PATCH 07/17] Rename `AsBorrowed*`, `IntoOwned*`, `FromOwned*` to `As*`, `Into*`, `From*`. --- text/0000-io-safety.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index 39f75f37430..1ea6ffc04dd 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -374,9 +374,9 @@ New types and traits could provide a much cleaner API, along the lines of: pub struct BorrowedFd<'owned> { ... } pub struct OwnedFd { ... } -pub trait AsBorrowedFd { ... } -pub trait IntoOwnedFd { ... } -pub trait FromOwnedFd { ... } +pub trait AsFd { ... } +pub trait IntoFd { ... } +pub trait FromFd { ... } ``` An initial prototype of this here: From ac872ec4229bd5e54021b70c7fd55e93c0139269 Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Fri, 4 Jun 2021 14:36:46 -0700 Subject: [PATCH 08/17] Remove `IoSafe`, add `OwnedFd`, `BorrowedFd`, and friends. --- text/0000-io-safety.md | 258 +++++++++++++++-------------------------- 1 file changed, 96 insertions(+), 162 deletions(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index 1ea6ffc04dd..7a297c60ef7 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -8,8 +8,7 @@ Close a hole in encapsulation boundaries in Rust by providing users of `AsRawFd` and related traits guarantees about their raw resource handles, by -introducing a concept of *I/O safety* and a new `IoSafe` trait. Build on, and -provide an explanation for, the `from_raw_fd` function being unsafe. +introducing a concept of *I/O safety* and a new set of types and traits. # Motivation [motivation]: #motivation @@ -50,7 +49,7 @@ This RFC introduces a path to gradually closing this loophole by introducing: - A new concept, I/O safety, to be documented in the standard library documentation. - - A new trait, `std::io::IoSafe`. + - A new set of types and traits. - New documentation for [`from_raw_fd`]/[`from_raw_handle`]/[`from_raw_socket`] explaining why they're unsafe in terms of I/O safety, addressing a question that has @@ -129,106 +128,71 @@ lifetime the OS associates with the handle. I/O safety is new as an explicit concept, but it reflects common practices. Rust's `std` will require no changes to stable interfaces, beyond the -introduction of a new trait and new impls for it. Initially, not all of the -Rust ecosystem will support I/O safety though; adoption will be gradual. +introduction of some new types and traits and new impls for them. Initially, +not all of the Rust ecosystem will support I/O safety though; adoption will +be gradual. -## The `IoSafe` trait +## `OwnedFd` and `BorrowedFd<'owned>` -These high-level types also implement the traits [`AsRawFd`]/[`IntoRawFd`] on -Unix-like platforms and -[`AsRawHandle`]/[`AsRawSocket`]/[`IntoRawHandle`]/[`IntoRawSocket`] on Windows, -providing ways to obtain the low-level value contained in a high-level value. -APIs use these to accept any type containing a raw handle, such as in the -`do_some_io` example in the [motivation]. +These two types are conceptual replacements for `RawFd`, and represent owned +and borrowed handle values. `OwnedFd` owns a file descriptor, including closing +it when it's dropped. `BorrowedFd`'s lifetime parameter ties it to the lifetime +of something that owns a file descriptor. These types enforce all of their I/O +safety invariants automatically. -`AsRaw*` and `IntoRaw*` don't make any guarantees, so to add I/O safety, types -will implement a new trait, `IoSafe`: +For Windows, similar types, but in `Handle` and `Socket` forms. -```rust -pub unsafe trait IoSafe {} -``` - -There are no required functions, so implementing it just takes one line, plus -comments: +## `AsFd`, `IntoFd`, and `FromFd` -```rust -/// # Safety -/// -/// `MyType` wraps a `std::fs::File` which handles the low-level details, and -/// doesn't have a way to reassign or independently drop it. -unsafe impl IoSafe for MyType {} -``` +These three traits are conceptual replacements for `AsRawFd`, `IntoRawFd`, and +`FromRawFd` for most use cases. They work in terms of `OwnedFd` and +`BorrowedFd`, so they automatically enforce their I/O safety invariants. -It requires `unsafe`, to require the code to explicitly commit to upholding I/O -safety. With `IoSafe`, the `do_some_io` example should simply add a -`+ IoSafe` to provide I/O safety: +Using these traits, the `do_some_io` example in the [motivation] can avoid +the original problems. Since `AsFd` is only implemented for types which +properly own their file descriptors, this version of `do_some_io` doesn't +have to worry about being passed bogus or dangling file descriptors: ```rust -pub fn do_some_io(input: &FD) -> io::Result<()> { - some_syscall(input.as_raw_fd()) +pub fn do_some_io(input: &FD) -> io::Result<()> { + some_syscall(input.as_fd()) } ``` -Some types have the ability to dynamically drop their resources, and -these types require special consideration when implementing `IoSafe`. For -example, a class representing a dynamically reassignable output source might -have code like this: - -```rust -struct VirtualStdout { - current: RefCell -} - -impl VirtualStdout { - /// Assign a new output destination. - /// - /// This function ends the lifetime of the resource that `as_raw_fd` - /// returns a handle to. - pub fn set_output(&self, new: std::fs::File) { - *self.current.borrow_mut() = new; - } -} +For Windows, similar traits, but in `Handle` and `Socket` forms. -impl AsRawFd for VirtualStdout { - fn as_raw_fd(&self) -> RawFd { - self.current.borrow().as_raw_fd() - } -} -``` +## Portability for simple use cases -If a user of this type were to hold a `RawFd` value over a call to `set_file`, -the `RawFd` value would become dangling, even though its within the lifetime of -the `&self` reference passed to `as_raw_fd`: +Portability in this space isn't easy, since Windows has two different handle +types while Unix has one. However, some use cases can treat `AsFd` and +`AsHandle` similarly, while some other uses can treat `AsFd` and `AsSocket` +similarly. In these two cases, trivial `Filelike` and `Socketlike` abstractions +allow code which works in this way to be generic over Unix and Windows. -```rust - fn foo(output: &VirtualStdout) -> io::Result<()> { - let raw_fd = output.as_raw_fd(); - output.set_file(File::open("/some/other/file")?); - use(raw_fd)?; // Use of dangling file descriptor! - Ok(()) - } -``` +On Unix, `AsFilelike` and `AsSocketlike` have blanket implementations for +any type that implements `AsFd`. On Windows, `AsFilelike` has a blanket +implementation for any type that implements `AsHandle`, and `AsSocketlike` +has a blanket implementation for any type that implements `AsSocket`. -The `IoSafe` trait requires types capable of dynamically dropping their -resources within the lifetime of the `&self` passed to `as_raw_fd` must -document the conditions under which this can occur, as the documentation -comment above does. +Similar portability abstractions apply to the `From*` and `Into*` traits. ## Gradual adoption -I/O safety and `IoSafe` wouldn't need to be adopted immediately, adoption -could be gradual: +I/O safety and the new types and traits wouldn't need to be adopted +immediately; adoption could be gradual: - - First, `std` adds `IoSafe` with impls for all the relevant `std` types. - This is a backwards-compatible change. + - First, `std` adds the new types and traits with impls for all the relevant + `std` types. This is a backwards-compatible change. - - After that, crates could implement `IoSafe` for their own types. These - changes would be small and semver-compatible, without special coordination. + - After that, crates could begin to use the new types and implement the new + traits for their own types. These changes would be small and semver-compatible, + without special coordination. - - Once the standard library and enough popular crates utilize `IoSafe`, - crates could start to add `+ IoSafe` bounds (or adding `unsafe`), at their - own pace. These would be semver-incompatible changes, though most users of - APIs adding `+ IoSafe` wouldn't need any changes. + - Once the standard library and enough popular crates implement the new + traits, crates could start to switch to using the new traits as bounds when + accepting generic arguments, at their own pace. These would be + semver-incompatible changes, though most users of APIs switching to these + new traits wouldn't need any changes. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation @@ -250,44 +214,52 @@ Functions accepting arbitrary raw I/O handle values ([`RawFd`], [`RawHandle`], or [`RawSocket`]) should be `unsafe` if they can lead to any I/O being performed on those handles through safe APIs. -Functions accepting types implementing -[`AsRawFd`]/[`IntoRawFd`]/[`AsRawHandle`]/[`AsRawSocket`]/[`IntoRawHandle`]/[`IntoRawSocket`] -should add a `+ IoSafe` bound if they do I/O with the returned raw handle. +## `OwnedFd` and `BorrowedFd<'owned>` + +`OwnedFd` and `BorrowedFd` are both `repr(transparent)` with a `RawFd` value +on the inside, and both can use niche optimizations so that `Option` +and `Option>` are the same size, and can be used in FFI +declarations for functions like `open`, `read`, `write`, `close`, and so on. +When used this way, they ensure I/O safety all the way out to the FFI boundary. + +These types also implement the existing `AsRawFd`, `IntoRawFd`, and `FromRawFd` +traits, so they can interoperate with existing code that works with `RawFd` +types. + +## `AsFd`, `IntoFd`, and `FromFd` -## The `IoSafe` trait +These types provide `as_fd`, `into_fd`, and `from_fd` functions similar to +their `Raw` counterparts, but with the benefit of a safe interface, it's safe +to provide a few simple conveniences which make the API much more flexible: -Types implementing `IoSafe` guarantee that they uphold I/O safety. They must -not make it possible to write a safe function which can perform invalid I/O -operations, and: + - A `from_into_fd` function which takes a `IntoFd` and converts it into a + `FromFd`, allowing users to perform this common sequence in a single + step, and without having to use `unsafe`. - - A type implementing `AsRaw* + IoSafe` means its `as_raw_*` function returns - a handle which is valid to use for the duration of the `&self` reference. - If such types have methods to close or reassign the handle without - dropping the whole object, they must document the conditions under which - existing raw handle values remain valid to use. + - A `as_filelike_view::()` function returns a `View`, which contains a + temporary `ManuallyDrop` instance of T constructed from the contained + file descriptor, allowing users to "view" a raw file descriptor as a + `File`, `TcpStream`, and so on. - - A type implementing `IntoRaw* + IoSafe` means its `into_raw_*` function - returns a handle which is valid to use at the point of the return from - the call. +## Prototype implementation -All standard library types implementing `AsRawFd` implement `IoSafe`, except -`RawFd`. +All of the above is prototyped here: -Note that, despite the naming similarity, the `IoSafe` trait's requirements are not -identical to the I/O safety requirements. The return value of `as_raw_*` is -valid only for the duration of the `&self` argument passed in. + + +The README.md has links to documentation, examples, and a survey of existing +crates providing similar features. # Drawbacks [drawbacks]: #drawbacks Crates with APIs that use file descriptors, such as [`nix`] and [`mio`], would -need to migrate to types implementing `AsRawFd + IoSafe`, use crates providing -equivalent mechanisms such as [`unsafe-io`], or change such functions to be +need to migrate to types implementing `AsFd`, or change such functions to be unsafe. Crates using `AsRawFd` or `IntoRawFd` to accept "any file-like type" or "any socket-like type", such as [`socket2`]'s [`SockRef::from`], would need to -either add a `+ IoSafe` bound or make these functions unsafe. +either switch to `AsFd` or `IntoFd`, or make these functions unsafe. # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives @@ -347,52 +319,19 @@ I/O safety approach will require changes to Rust code in crates such as [`RawFd`], though the changes can be made gradually across the ecosystem rather than all at once. -## I/O safety but not `IoSafe` - -The I/O safety concept doesn't depend on `IoSafe` being in `std`. Crates could -continue to use [`unsafe_io::OwnsRaw`], though that does involve adding a -dependency. - -## Define `IoSafe` in terms of the object, not the reference - -The [reference-level-explanation] explains `IoSafe + AsRawFd` as returning a -handle valid to use for "the duration of the `&self` reference". This makes it -similar to borrowing a reference to the handle, though it still uses a raw -type which doesn't enforce the borrowing rules. - -An alternative would be to define it in terms of the underlying object. Since -it returns raw types, arguably it would be better to make it work more like -`slice::as_ptr` and other functions which return raw pointers that aren't -connected to reference lifetimes. If the concept of borrowing is desired, new -types could be introduced, with better ergonomics, in a separate proposal. - -## New types and traits - -New types and traits could provide a much cleaner API, along the lines of: - -```rust -pub struct BorrowedFd<'owned> { ... } -pub struct OwnedFd { ... } - -pub trait AsFd { ... } -pub trait IntoFd { ... } -pub trait FromFd { ... } -``` +## The `IoSafe` trait (and `OwnsRaw` before it) -An initial prototype of this here: - - +Earlier versions of this RFC proposed an `IoSafe` trait, which was meant as a +minimally intrusive fix. Feedback from the RFC process led to the development +of a new set of types and traits. This has a much larger API surface area, +which will take more work to design and review. And it and will require more +extensive changes in the crates ecosystem over time. However, early indications +are that the new types and traits are easier to understand, and easier and +safer to use, and so are a better foundation for the long term. -The details are mostly obvious, though one notable aspect of this design is -the use of `repr(transparent)` to define types that can participate in FFI -directly, leading to FFI usage patterns that don't interact with raw types -at all. An example of this is here: - - - -This provides a cleaner API than `*Raw*` + `IoSafe`. The main obvious downside -is that a lot of code will likely need to continue to support `*Raw*` for a -long time, so this would increase the amount of code they have to maintain. +Earlier versions of `IoSafe` were called `OwnsRaw`. It was difficult to find a +name for this trait which described exactly what it does, and arguably this is +one of the signs that it wasn't the right trait. # Prior art [prior-art]: #prior-art @@ -403,9 +342,15 @@ such as in [C#], [Java], and others. Making it `unsafe` to perform I/O through a given raw handle would let safe Rust have the same guarantees as those effectively provided by such languages. -The `std::io::IoSafe` trait comes from [`unsafe_io::OwnsRaw`], and experience -with this trait, including in some production use cases, has shaped this RFC. +There are several crates on crates.io providing owning and borrowing file +descriptor wrappers. The [io-experiment README.md's Prior Art section] +describes these and details how io-experiment's similarities and differences +with these existing crates in detail. At a high level, these existing crates +share the same basic concepts that io-experiment uses. All are built around +Rust's lifetime and ownership concepts, and confirm that these concepts +are a good fit for this problem. +[io-experiment README.md's Prior Art section]: https://github.com/sunfishcode/io-experiment#prior-art [C#]: https://docs.microsoft.com/en-us/dotnet/api/system.io.file?view=net-5.0 [Java]: https://docs.oracle.com/javase/7/docs/api/java/io/File.html?is-external=true @@ -432,17 +377,6 @@ needs, but it could be explored in the future. Some possible future ideas that could build on this RFC include: - - New wrapper types around `RawFd`/`RawHandle`/`RawSocket`, to improve the - ergonomics of some common use cases. Such types may also provide portability - features as well, abstracting over some of the `Fd`/`Handle`/`Socket` - differences between platforms. - - - Higher-level abstractions built on `IoSafe`. Features like - [`from_filelike`] and others in [`unsafe-io`] eliminate the need for - `unsafe` in user code in some common use cases. [`posish`] uses this to - provide safe interfaces for POSIX-like functionality without having `unsafe` - in user code, such as in [this wrapper around `posix_fadvise`]. - - Clippy lints warning about common I/O-unsafe patterns. - A formal model of ownership for raw handles. One could even imagine From 9d83018c59abb5fc6349d2e874061942d3dad11d Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Sat, 5 Jun 2021 08:57:22 -0700 Subject: [PATCH 09/17] Clarify `BorrowedFd`'s lifetime parameter. --- text/0000-io-safety.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index 7a297c60ef7..00fe571b831 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -136,9 +136,9 @@ be gradual. These two types are conceptual replacements for `RawFd`, and represent owned and borrowed handle values. `OwnedFd` owns a file descriptor, including closing -it when it's dropped. `BorrowedFd`'s lifetime parameter ties it to the lifetime -of something that owns a file descriptor. These types enforce all of their I/O -safety invariants automatically. +it when it's dropped. `BorrowedFd`'s lifetime parameter says for how long +access to this file descriptor has been borrowed. These types enforce all of +their I/O safety invariants automatically. For Windows, similar types, but in `Handle` and `Socket` forms. From a94b6dc616109f1166a5e5ba075a1eb4135d1088 Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Sat, 12 Jun 2021 11:54:27 -0700 Subject: [PATCH 10/17] Rename 'owned to 'fd/'handle/'filelike/'socketlike. --- text/0000-io-safety.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index 00fe571b831..79f1329cac4 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -132,7 +132,7 @@ introduction of some new types and traits and new impls for them. Initially, not all of the Rust ecosystem will support I/O safety though; adoption will be gradual. -## `OwnedFd` and `BorrowedFd<'owned>` +## `OwnedFd` and `BorrowedFd<'fd>` These two types are conceptual replacements for `RawFd`, and represent owned and borrowed handle values. `OwnedFd` owns a file descriptor, including closing @@ -214,7 +214,7 @@ Functions accepting arbitrary raw I/O handle values ([`RawFd`], [`RawHandle`], or [`RawSocket`]) should be `unsafe` if they can lead to any I/O being performed on those handles through safe APIs. -## `OwnedFd` and `BorrowedFd<'owned>` +## `OwnedFd` and `BorrowedFd<'fd>` `OwnedFd` and `BorrowedFd` are both `repr(transparent)` with a `RawFd` value on the inside, and both can use niche optimizations so that `Option` From 429c646d16fe2ac7d1a32c3ec6fe60dd35b4f24c Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Wed, 30 Jun 2021 14:36:45 -0700 Subject: [PATCH 11/17] Rename io-experiment to io-lifetimes. --- text/0000-io-safety.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index 79f1329cac4..a775d716752 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -245,7 +245,7 @@ to provide a few simple conveniences which make the API much more flexible: All of the above is prototyped here: - + The README.md has links to documentation, examples, and a survey of existing crates providing similar features. @@ -343,14 +343,14 @@ a given raw handle would let safe Rust have the same guarantees as those effectively provided by such languages. There are several crates on crates.io providing owning and borrowing file -descriptor wrappers. The [io-experiment README.md's Prior Art section] -describes these and details how io-experiment's similarities and differences +descriptor wrappers. The [io-lifetimes README.md's Prior Art section] +describes these and details how io-lifetimes' similarities and differences with these existing crates in detail. At a high level, these existing crates -share the same basic concepts that io-experiment uses. All are built around +share the same basic concepts that io-lifetimes uses. All are built around Rust's lifetime and ownership concepts, and confirm that these concepts are a good fit for this problem. -[io-experiment README.md's Prior Art section]: https://github.com/sunfishcode/io-experiment#prior-art +[io-lifetimes README.md's Prior Art section]: https://github.com/sunfishcode/io-lifetimes#prior-art [C#]: https://docs.microsoft.com/en-us/dotnet/api/system.io.file?view=net-5.0 [Java]: https://docs.oracle.com/javase/7/docs/api/java/io/File.html?is-external=true From 6854c4d55d22e74e48f12c50cf6fe7870c407c50 Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Wed, 30 Jun 2021 14:36:01 -0700 Subject: [PATCH 12/17] Move `View` and `from_into_fd` into the Future Possibilities section. --- text/0000-io-safety.md | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index a775d716752..d95dfb1aaf8 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -229,17 +229,7 @@ types. ## `AsFd`, `IntoFd`, and `FromFd` These types provide `as_fd`, `into_fd`, and `from_fd` functions similar to -their `Raw` counterparts, but with the benefit of a safe interface, it's safe -to provide a few simple conveniences which make the API much more flexible: - - - A `from_into_fd` function which takes a `IntoFd` and converts it into a - `FromFd`, allowing users to perform this common sequence in a single - step, and without having to use `unsafe`. - - - A `as_filelike_view::()` function returns a `View`, which contains a - temporary `ManuallyDrop` instance of T constructed from the contained - file descriptor, allowing users to "view" a raw file descriptor as a - `File`, `TcpStream`, and so on. +their `Raw` counterparts. ## Prototype implementation @@ -387,8 +377,15 @@ Some possible future ideas that could build on this RFC include: that, with this new guarantee, the high-level wrappers around raw handles are unforgeable in safe Rust. -[`from_filelike`]: https://docs.rs/unsafe-io/0.6.2/unsafe_io/trait.FromUnsafeFile.html#method.from_filelike -[this wrapper around `posix_fadvise`]: https://docs.rs/posish/0.6.1/posish/fs/fn.fadvise.html + - There are a few convenience features which can be implemented on top + of the basic `As`/`Into`/`From` traits: + - A `from_into_fd` function which takes a `IntoFd` and converts it into a + `FromFd`, allowing users to perform this common sequence in a single + step. + - A `as_filelike_view::()` function returns a `View`, which contains a + temporary instance of T constructed from the contained file descriptor, + allowing users to "view" a raw file descriptor as a `File`, `TcpStream`, + and so on. # Thanks [thanks]: #thanks From bf1ef7fcb7e2656e740d3eafe9eafbf22d322003 Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Wed, 30 Jun 2021 14:55:05 -0700 Subject: [PATCH 13/17] Move the portability traits into Future Possibilities too. --- text/0000-io-safety.md | 24 +++++++++--------------- 1 file changed, 9 insertions(+), 15 deletions(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index d95dfb1aaf8..ffa982a385d 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -161,21 +161,6 @@ pub fn do_some_io(input: &FD) -> io::Result<()> { For Windows, similar traits, but in `Handle` and `Socket` forms. -## Portability for simple use cases - -Portability in this space isn't easy, since Windows has two different handle -types while Unix has one. However, some use cases can treat `AsFd` and -`AsHandle` similarly, while some other uses can treat `AsFd` and `AsSocket` -similarly. In these two cases, trivial `Filelike` and `Socketlike` abstractions -allow code which works in this way to be generic over Unix and Windows. - -On Unix, `AsFilelike` and `AsSocketlike` have blanket implementations for -any type that implements `AsFd`. On Windows, `AsFilelike` has a blanket -implementation for any type that implements `AsHandle`, and `AsSocketlike` -has a blanket implementation for any type that implements `AsSocket`. - -Similar portability abstractions apply to the `From*` and `Into*` traits. - ## Gradual adoption I/O safety and the new types and traits wouldn't need to be adopted @@ -387,6 +372,15 @@ Some possible future ideas that could build on this RFC include: allowing users to "view" a raw file descriptor as a `File`, `TcpStream`, and so on. + - Portability for simple use cases. Portability in this space isn't easy, + since Windows has two different handle types while Unix has one. However, + some use cases can treat `AsFd` and `AsHandle` similarly, while some other + uses can treat `AsFd` and `AsSocket` similarly. In these two cases, trivial + `Filelike` and `Socketlike` abstractions could allow code which works in + this way to be generic over Unix and Windows. + + Similar portability abstractions could apply to the `From*` and `Into*` traits. + # Thanks [thanks]: #thanks From aa509de0d3d33c79f59b894e481edd679b283bdd Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Thu, 1 Jul 2021 08:01:34 -0700 Subject: [PATCH 14/17] Clarify that `AsFd` implementations may also *borrow* their file descriptors. --- text/0000-io-safety.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index ffa982a385d..eb73946c4d5 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -150,8 +150,8 @@ These three traits are conceptual replacements for `AsRawFd`, `IntoRawFd`, and Using these traits, the `do_some_io` example in the [motivation] can avoid the original problems. Since `AsFd` is only implemented for types which -properly own their file descriptors, this version of `do_some_io` doesn't -have to worry about being passed bogus or dangling file descriptors: +properly own or borrow their file descriptors, this version of `do_some_io` +doesn't have to worry about being passed bogus or dangling file descriptors: ```rust pub fn do_some_io(input: &FD) -> io::Result<()> { From 60b6181cdfd7bd46af8ee9099b1df3b1913fa422 Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Mon, 5 Jul 2021 11:59:48 -0700 Subject: [PATCH 15/17] Mention Android's file descriptor ownership APIs in Prior Art. --- text/0000-io-safety.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index eb73946c4d5..3d7d0c8b9c6 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -325,6 +325,14 @@ share the same basic concepts that io-lifetimes uses. All are built around Rust's lifetime and ownership concepts, and confirm that these concepts are a good fit for this problem. +Android has special APIs for detecting improper `close`s; see +rust-lang/rust#74860 for details. The motivation for these APIs also applies +to I/O safety here. Android's special APIs use dynamic checks, which enable +them to enforce rules across source language boundaries. The I/O safety +types and traits proposed here are only aiming to enforce rules within Rust +code, so they're able to use Rust's type system to enforce rules at +compile time rather than run time. + [io-lifetimes README.md's Prior Art section]: https://github.com/sunfishcode/io-lifetimes#prior-art [C#]: https://docs.microsoft.com/en-us/dotnet/api/system.io.file?view=net-5.0 [Java]: https://docs.oracle.com/javase/7/docs/api/java/io/File.html?is-external=true From a63e42fe8017cdd2e597a60e4aea5e1f8f25b2d1 Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Mon, 5 Jul 2021 13:41:21 -0700 Subject: [PATCH 16/17] Add a table describing `OwnedFd`, `BorrowedFd`, and `RawFd` by analogy. --- text/0000-io-safety.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index 3d7d0c8b9c6..d208d353cf2 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -142,6 +142,21 @@ their I/O safety invariants automatically. For Windows, similar types, but in `Handle` and `Socket` forms. +These types play a role for I/O which is analogous to what existing types +in Rust play for memory: + +| Type | Analogous to | +| ---------------- | ------------ | +| `OwnedFd` | `Box<_>` | +| `BorrowedFd<'a>` | `&'a _` | +| `RawFd` | `*const _` | + +One difference is that I/O types don't make a distinction between mutable +and immutable. OS resources can be shared in a variety of ways outside of +Rust's control, so I/O can be thought of as using [interior mutability]. + +[interior mutability]: https://doc.rust-lang.org/reference/interior-mutability.html + ## `AsFd`, `IntoFd`, and `FromFd` These three traits are conceptual replacements for `AsRawFd`, `IntoRawFd`, and From 1878c8cbbefa80d3e24d9a29b92befd1541b5a58 Mon Sep 17 00:00:00 2001 From: Dan Gohman Date: Thu, 8 Jul 2021 07:50:23 -0700 Subject: [PATCH 17/17] Use `Into` and `From` instead of `IntoFd` and `FromFd`. --- text/0000-io-safety.md | 39 +++++++++++++++++++++------------------ 1 file changed, 21 insertions(+), 18 deletions(-) diff --git a/text/0000-io-safety.md b/text/0000-io-safety.md index d208d353cf2..3849027551d 100644 --- a/text/0000-io-safety.md +++ b/text/0000-io-safety.md @@ -157,16 +157,17 @@ Rust's control, so I/O can be thought of as using [interior mutability]. [interior mutability]: https://doc.rust-lang.org/reference/interior-mutability.html -## `AsFd`, `IntoFd`, and `FromFd` +## `AsFd`, `Into`, and `From` -These three traits are conceptual replacements for `AsRawFd`, `IntoRawFd`, and -`FromRawFd` for most use cases. They work in terms of `OwnedFd` and -`BorrowedFd`, so they automatically enforce their I/O safety invariants. +These three are conceptual replacements for `AsRawFd::as_raw_fd`, +`IntoRawFd::into_raw_fd`, and `FromRawFd::from_raw_fd`, respectively, +for most use cases. They work in terms of `OwnedFd` and `BorrowedFd`, so +they automatically enforce their I/O safety invariants. -Using these traits, the `do_some_io` example in the [motivation] can avoid -the original problems. Since `AsFd` is only implemented for types which -properly own or borrow their file descriptors, this version of `do_some_io` -doesn't have to worry about being passed bogus or dangling file descriptors: +Using these, the `do_some_io` example in the [motivation] can avoid the +original problems. Since `AsFd` is only implemented for types which properly +own or borrow their file descriptors, this version of `do_some_io` doesn't +have to worry about being passed bogus or dangling file descriptors: ```rust pub fn do_some_io(input: &FD) -> io::Result<()> { @@ -226,10 +227,11 @@ These types also implement the existing `AsRawFd`, `IntoRawFd`, and `FromRawFd` traits, so they can interoperate with existing code that works with `RawFd` types. -## `AsFd`, `IntoFd`, and `FromFd` +## `AsFd`, `Into`, and `From` -These types provide `as_fd`, `into_fd`, and `from_fd` functions similar to -their `Raw` counterparts. +These types provide `as_fd`, `into`, and `from` functions similar to +`AsRawFd::as_raw_fd`, `IntoRawFd::into_raw_fd`, and `FromRawFd::from_raw_fd`, +respectively. ## Prototype implementation @@ -249,7 +251,7 @@ unsafe. Crates using `AsRawFd` or `IntoRawFd` to accept "any file-like type" or "any socket-like type", such as [`socket2`]'s [`SockRef::from`], would need to -either switch to `AsFd` or `IntoFd`, or make these functions unsafe. +either switch to `AsFd` or `Into`, or make these functions unsafe. # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives @@ -385,11 +387,11 @@ Some possible future ideas that could build on this RFC include: that, with this new guarantee, the high-level wrappers around raw handles are unforgeable in safe Rust. - - There are a few convenience features which can be implemented on top - of the basic `As`/`Into`/`From` traits: - - A `from_into_fd` function which takes a `IntoFd` and converts it into a - `FromFd`, allowing users to perform this common sequence in a single - step. + - There are a few convenience features which can be implemented for types + that implement `AsFd`, `Into`, and/or `From`: + - A `from_into_fd` function which takes a `Into` and converts it + into a `From`, allowing users to perform this common sequence + in a single step. - A `as_filelike_view::()` function returns a `View`, which contains a temporary instance of T constructed from the contained file descriptor, allowing users to "view" a raw file descriptor as a `File`, `TcpStream`, @@ -402,7 +404,8 @@ Some possible future ideas that could build on this RFC include: `Filelike` and `Socketlike` abstractions could allow code which works in this way to be generic over Unix and Windows. - Similar portability abstractions could apply to the `From*` and `Into*` traits. + Similar portability abstractions could apply to `From` and + `Into`. # Thanks [thanks]: #thanks