Add the Close trait

[czipperz]

Rendered

[Nemo157]

For some prior-art in the Rust ecosystem Tokio has AsyncWrite::shutdown and Futures 0.3 has AsyncWrite::poll_close that have a similar function. One way that these differ from the proposed trait is that they are explicitly only associated with the writer side of a potentially duplex IO stream. This becomes interesting when you consider something like TcpStream where you may want to close your outgoing data but continue receiving data until the other side also closes their stream.

[CryZe]

I think this could also be generalized to a general TryDrop similar to TryFrom and TryInto.

[czipperz]

@CryZe As it stands, Close is a generic way to try dropping an object. I thought about renaming it to TryDrop, but decided on Close because it is more consistent with other programming languages' features. However, we could have a method close on File and other resources that simply try_drops them.

[czipperz]

As an aside, I was unsure of whether Close should be default implemented for all types. TryDrop would also seem to be in favor of defining a default implementation for any type implementing Drop by simply returning Ok(()), similar to how TryFrom and TryInto work. But this implementation is fundamentally wrong -- it ignores these errors.

[czipperz]

Close also has the signature fn close(self) -> Result<(), Error>. But Drop takes &mut self. That is, it isn't a direct generalization from Drop to Close because the signatures are different. I'm proposing a library solution here, not a language solution.

A language solution could take &mut self and call the try_drop with ?, cascading errors. This would, however, require a large breaking change to the language and I am not in favor of this.

[WiSaGaN]

What is the benefit of adding a new trait rather than just adding as an inherent method?

[Ixrec]

@WiSaGaN The RFC gives the examples of close() methods on BufReader<T> and BufWriter<T> that would not be possible to write if they couldn't be generic over the closeable T they're buffering.

I do think the RFC would benefit from calling that out early in the motivation; I had the same question until I got to those examples.

[czipperz]

This is at the end of the first paragraph of motivation. Do you have any other concerns @WiSaGaN ?

This will also allow us to close wrapper types, such as BufReader or BufWriter when they wrap any type implementing Close.

[vi]

Does it need to be self? Is &mut self not enough to close a resource?

[pnkfelix]

@vi if you want to statically ensure no one can use the resource after it’s closed, then you need to take self, not &mut self

[matklad]

cc rust-lang/rust#32677

[burdges]

If I understand, there are many designs suitable for different settings, but this one fits with std::io, right?

We'd manually propagate errors with this by calling .close() ?, right? I suppose you considered automatically propagating errors with roughly

pub trait Drop
    type Error = !;
    try_drop(?? self) -> Result<(),Error> { self.drop(); Ok(()) }
    drop(&mut self);
}

Actual drop glue would call Drop::try_drop(..) ? resulting in type errors if you lack the correct conversion impls though, requiring a truly massive breaking change, which sounds unacceptable. It goes against Rust's explicitness goal too I guess.

I'd think transactional data structures require multiple &mut self functions that confirm or above transactions, but those sound custom and unlikely to fit anything std provides. I doubt ending session types could be homogenized enough to fit into std but..

/// Try ending a session type
pub trait TryEnd {
    type Error;
    fn close(self) -> Result<(), (Error,Option<Self>)>;
}
impl<T: TryEnd> Drop for T {
    drop(&mut x) { panic!() }
}

I think this RFC's design makes sense, but the documentation should explain that Close exists primarily for std::io, and that it may or may not fit other uses cases, so users should not attempt to shoehorn every type into Close.

[czipperz]

@burdges

pub trait Drop
   type Error = !;
   try_drop(?? self) -> Result<(),Error> { self.drop(); Ok(()) }
   drop(&mut self);
}

This won't work for the following reasons:

1. drop cannot be manually called. Thus try_drop isn't writable.
2. try_drop shouldn't be manually called in this instance for the same reason
3. How do we recursively try_drop while we can't manually call try_drop? For instance, how does a BufReader implement this?

[scooter-dangle]

For someone like myself who might not be familiar with them, can you give some examples of what these failures might be? Are there any that File::sync_all wouldn’t surface?

[czipperz]

For specifically File, that is correct. sync_all allows us to handle these errors. Maybe a better example is something like a TCPStream where you have to send a finish packet to signal to the other side to close the connection. (At least that is my understanding from googling tcp for a few minutes)

[LunaBorowska]

About alternatives, I'm wondering about adding a close method to std::io::Write trait.

fn close(mut self) -> io::Result<()> {
    self.flush() // Or `Ok(())` I suppose, this is arguable
}

The RFC argues this is a bad idea as "it would remove backwards compatibility", but I don't see it myself being a case. A default implementation would mean that the already existing structures would work, sure, maybe they wouldn't close the file, but it would be good enough in many cases. One interpretation I could think of is that code explicitly implementing close would not work in Rust versions before close introduction, but so would be the case whatever we would go with. Could this be clarified?

It also argues that "It also prevents us from closing reading resources", which frankly seems like non-issue, what does close failing for a structure implementing Read and not Write would even mean - there were no changes, so surely an error could be ignored - everything what was needed was done with a file, bytes were retrieved without errors. In theory, close can always fail, but I'm not sure what reasonable implementation would return an error other than EINTR (which is not really an error) on close if a file was never written to. A Drop implementation should be sufficient here.

As for other traits where closing can error, I think it makes sense for those traits to provide their own close methods. Having Rust defined Close trait isn't really advantageous compared to traits providing their own close method.

Keep in mind that having Close trait would mean changing <T: Write> to <T: Write + Close> would break backwards compatibility (unless we would have a default implementation for Close), as not every Write implementation will implement Close (if say, due to being written before Close existed).

One issue with Write having close method is that Box<dyn Write> would get a bad close implementation due to close method not being object safe. I have no suggestions on how to deal with this issue. It's possible to have an implementation detail like fn close_box(self: Box<Self>) -> io::Result<()> { (**self).close() } specifically to let close on Box<dyn Write> work, but that's non-ideal. Alternatively, it's possible to make close take a &mut self parameter, but that would cause practical issues with implementing the close method.

[czipperz]

@xfix Thanks for the feedback!

I was arguing that it would be backwards incompatible if we didn't have a default implementation. That is what Tokio and futures-preview does (see the comment by Nemo above). That is, they require an implementation of shutdown or poll_close.

I think it makes sense just to flush and auto drop. I'll add this in.

dyn Write

To be perfectly honest, it's unclear to me what we should do about this case. Because when we close a resource we should take it by value and dispose of it. But at the same time, dynamic types can't work with this. The approach taken by both Tokio and futures-preview is to take &mut self. I don't particularly like this implementation because it puts the stream into an invalid state and it is unclear what to do besides drop it at that point.

[LunaBorowska]

Actually, Box<dyn Write> should be non-issue, now that Box<dyn FnOnce()> works in Rust 1.35, we could do the same thing for already existing Box<T> where T: ?Sized + Write implementation making use of unsized_locals feature (considering we already do for FnOnce(), doing that for Write as well should be non-issue).

Other dynamic dispatch types are non-issue, because they cannot pass the ownership, Box<dyn Write> was the only one I was concerned about. Using close on &mut T will work, but I think there could be a Clippy lint for that, similar to already existing drop_ref lint - it's non-ideal that it will work, but cannot really do anything about that. But hey, if you pass &mut T to a function expecting a Write implementation, maybe you don't want the file to be actually closed.

[LunaBorowska]

Question, let's say close method gets added to Write, what should be the implementation for &mut T where T: ?Sized + Write? Specifically, should it be flushing or not. I can imagine a popular use-case of close will be to properly clean up a value instead of implicitly dropping in generic functions accepting Write implementations, and for &mut T, you may not want a drop because the value is still alive, in which case flushing is probably unnecessary.

[Nemo157]

IMO if Write::close exists (and is defined over &mut Self) then impl Write for &mut impl Write + ?Sized should be proxying it through correctly, if you want to pass a borrowed Write implementation to a function and not have it close it (or not flush it) you can wrap the &mut Write in a type that will proxy through all other functions and ignore calls to close (or flush).

[LunaBorowska]

@Nemo157 fn close(self) takes ownership, so &mut T implementation cannot redirect to the inner close implementation.

In my opinion, doing nothing (returning Ok(()) is reasonable here). I'm assuming people may want to update their generic APIs to explicitly call close instead of dropping the value, so it makes sense to avoid flushing.

[Nemo157]

@xfix it doesn't have to take self by value, e.g. TcpStream::shutdown takes &self so you can call that and acquire a Write instance that is already closed.

I'm assuming people may want to update their generic APIs to explicitly call close instead of dropping the value, so it makes sense to avoid flushing.

That depends a lot on the specific API, if it's just a helper API then it shouldn't be calling close (or flush). If it is an API that expects to take full ownership and manage the lifetime of the data stream, including closing it once complete, then not flushing it could cause issues, e.g. if it sends a final small message, closes the writer, then waits for an ack on a reader; if closing the writer doesn't flush out the final message (i.e. the writer were a &mut BufWriter) this would sit waiting on the ack forever.

[tanriol]

@czipperz Please note that both Tokio's shutdown and Futures's poll_close have one significant difference in semantics from the suggested Close trait. Both are poll-style async, so if they return NotReady you're supposed to call them again after they're ready to make progress, possibly more than once.

This is actually a known problem that completion-based IO operations may have no way to cancel them synchronously and thus Drop does not work with them... Close, as defined now, will not work either. This does not feel good, especially taking into account that these are just the kind of operations where one can expect an error during shutdown.

[LunaBorowska]

Closing asynchronous I/O is out of scope for this RFC in my opinion. It's so different to regular closing that even if we somehow made Close work for both synchronous and asynchronous code, the bounds for functions using Close trait would be so different that there would be no reason for doing so.

I also feel similarly about Close trait being general, and I strongly would prefer a close method in io::Write. I don't see users ever just going <T: Close>, it would be <T: Write + Close<Error = io::Error>> most of the time. If any other trait would like to have a closing abstraction, it should provide its own - there is no real way to make Close practically general. If most implementations of a trait don't require closing, having a Close trait isn't an improvement, as it would require an user to put a Close bound while implementations of a trait wouldn't implement Close, and often enough this bound wouldn't be put, because most implementations wouldn't require closing. Additionally, having a single Close trait would prevent having multiple Close implementations with different error types.

When having a close method, the implementations could look like this.

• File - self.sync_all()
• Cursor<Box<[u8]>>, Cursor<Vec<u8>>, Cursor<&mut Vec<u8>>, Cursor<&mut [u8]> - default (no-op)
• Sink - default (no-op)
• Stdout, Stderr, StdoutLock, StderrLock - default (flush) (stdout and stderr can be closed, but it's a bad idea, probably unsafe, and would lead to bugs)
• TcpStream, UnixStream - self.shutdown(Shutdown::Both)
• ChildStdin - default (flush)
• Vec<u8> - default (no-op)
• File - self.flush()?; self.sync_data() (I think self.sync_all() would be unnecessary here)
• &mut W - default (flush)
• &mut [u8] - default (no-op)
• Box<W: Write> - close inner (using unsized_locals feature)
• BufWriter<W>/LineWriter<W> - flush buffer to inner and then close inner
• &File, &TcpStream, &UnixStream - no idea? just flush? Keep in mind that while those can shutdown or sync data, &mut File uses a &mut W implementation (which cannot close due to not having ownership) and it would be weird if &File did sync while &mut File didn't.

As mentioned, poll_close polls, which is why it cannot really require ownership. Meanwhile close due to not polling can require ownership.

[mzabaluev]

@xfix

• File - self.sync_all()

That's not what the OS close typically does, and it may be unnecessary in many cases. The generic close for I/O objects should only do the minimal work to finalize the object's pending operations and resources for which failures may need to be observed.

[mzabaluev]

Following the design of a session-ending trait thrown in by @burdges, Close/TryDrop, too, should return any information that can be used for recovery in the Err variant. For BufWriter this could be:

impl<W: Write + Close> Close for BufWriter<W> {
    type CloseError = buf_writer::CloseError<W>;
    fn close(self) -> Result<(), Self::CloseError> {
        let flush_res = self.flush();
        if let Err(e) = flush_res {
            return buf_writer::CloseError::incomplete_flush(e, self);
        }
        self.inner.close()?;  // The error variant returned here features W::CloseError
    }
}

[pdolezal]

I recently hit the need of some kind of Close as well and I got reference to this PR. Perhaps I could refresh it here a bit with the current discussion on Rust Internals which I opened a few days ago (not knowing about this PR yet).

[czipperz]

I'm not using Rust anymore so if someone else wants this PR to go through feel free to fork my PR.

[3point2]

I came across this PR while working on BurntSushi/ripgrep#1766. Just want to add that during my research I found https://crates.io/crates/io-close, which implements a Close trait. It works by taking ownership, flushing, using unsafe {} to close the underlying fd/handle/socket, and returning any error.

[yaahc]

Hey, we discussed this RFC in last week's libs team meeting. There are some additional notes in the agenda but the gist of the discussion is that we're unsure about the specific motivating use cases, and we're unsure if this should be a trait. As an alternative we were wondering if an inherent File::close method would be sufficient.

Could you update the RFC to include some example use cases that motivate / justify this being a Trait? Assuming, of course, that you feel it needs to be a trait. If you're happy with the File::close suggestion then don't worry about the motivating examples for the Trait version.

[pdolezal]

Hey, we discussed this RFC in last week's libs team meeting. There are some additional notes in the agenda but the gist of the discussion is that we're unsure about the specific motivating use cases, and we're unsure if this should be a trait. As an alternative we were wondering if an inherent File::close method would be sufficient.

I can point out to the IRLO discussion where the example, which led to the discussion, was mentioned. Briefly, I wanted to implement a bitwise compression algorithm that is supposed to terminate the output, so it has to emit the termination when the output is going to finish. I wanted to implement it as a Write decorator and I wanted yet to have the ability to indicate that emitting the terminator failed, so that the user of the decorator could react rather than, e.g., getting a corrupted file silently. I don't see any obvious way to make such a generic decorator without having a generic way to learn that a final operation of some sort failed.

I mentioned in the discussion that a more generic trait, let's call it TryDrop, might be available instead. Closing a file seems like an important, but still special case. I can imagine streams of different elements than raw bytes to be pushed into a (buffered) sink, which may, e.g., return an error (that might include yet unprocessed elements) on a premature drop attempt.

[czipperz]

I no longer use Rust so am going to close this issue. If anyone is interested in moving forward with this proposal feel free to fork it.