-
Notifications
You must be signed in to change notification settings - Fork 301
1.52.1 - Blog post explaining the fingerprint issue. #836
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 7 commits
099e2b5
ba3ca92
5609ca6
30691e5
36ce95f
5fad60a
7d67f88
2190a79
f5369ed
e1c66ae
e304ef0
c250c4e
7ac6272
cda2b34
2c47d8a
227384b
de97a34
4b940c3
410c4d0
1dcc8be
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,181 @@ | ||
--- | ||
layout: post | ||
title: "Rust 1.52.1" | ||
author: Felix Klock, Mark Rousskov | ||
team: the compiler team <https://www.rust-lang.org/governance/teams/compiler> | ||
release: true | ||
--- | ||
|
||
The Rust team has prepared a new release, 1.52.1, working around a bug | ||
introduced in 1.52.0. We recommend all Rust users, including those currently | ||
Mark-Simulacrum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
using stable versions prior to 1.52.0 upgrade to 1.52.1. | ||
Mark-Simulacrum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
If you have a previous version of Rust installed via rustup, getting Rust | ||
1.52.1 is as easy as: | ||
|
||
```console | ||
rustup update stable | ||
``` | ||
|
||
If you don't have it already, you can [get `rustup`][install] | ||
from the appropriate page on our website. | ||
|
||
[install]: https://www.rust-lang.org/install.html | ||
|
||
# What's in 1.52.1 stable | ||
|
||
This point release contains a single change: it disables incremental | ||
compilation. Read on for more details as to why, and the next steps the Rust | ||
Mark-Simulacrum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
project is pursuing on this issue. | ||
|
||
# Summary | ||
|
||
The Rust teams are always excited to report on new features offered with each release. Sometimes, however, an important change that is not yet "fully baked" gets accidentally included in a release, and we need to issue a point release. | ||
|
||
There was an instance of this in the most recent release, 1.52.0, which added a new bit of internal-consistency checking, called "incremental compilation hash verification" (abbreviated `verify-ich`). This check is also called an "unstable fingerprint" check, because the diagnostic it currently prints look [like this](https://github.com/rust-lang/rust/issues/84336): | ||
Mark-Simulacrum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
``` | ||
thread 'rustc' panicked at 'assertion failed: `(left == right)` | ||
left: `Some(Fingerprint(4565771098143344972, 7869445775526300234))`, | ||
right: `Some(Fingerprint(14934403843752251060, 623484215826468126))`: found unstable fingerprints for <massive text describing rustc internals elided> | ||
|
||
error: internal compiler error: unexpected panic | ||
|
||
note: the compiler unexpectedly panicked. this is a bug. | ||
``` | ||
|
||
This internal-consistency check, as stated in the diagnostic, yields an "Internal Compiler Error" (or ICE). In other words, it represents a bug in the internals of the Rust compiler itself. In *this* case, though, the ICE is revealing a bug that 1.) predates the 1.52.0 release and 2.) could result in miscompilation if it had not been caught by `verify-ich`. | ||
|
||
In other words: If you are seeing the above Internal Compiler Error, you may be tempted to respond by reverting to the 1.51 release. It is important to note that a downgrade is *not* the best response to this problem. | ||
|
||
This post is going to: | ||
|
||
1. Explain [what the check does][part1], at a high level, | ||
2. Explain [how the check is presenting itself][part2] in the Rust 1.52.0 release, | ||
3. Tell you [what you should do][part3] if you see an unstable fingerprint on your project, | ||
4. Describe our plans for [how the Rust project will address][part4] the problems discussed here. | ||
|
||
[part1]: #what-are-fingerprints-why-are-we-checking-them | ||
[part2]: #how-does-this-show-up | ||
[part3]: #what-should-a-rust-programmer-do-in-response | ||
[part4]: #what-is-the-rust-project-going-to-do-to-fix-this | ||
|
||
## What are fingerprints? Why are we checking them? | ||
|
||
The Rust compiler has support for "incremental compilation", which has been described in a [2016 blog post][]. When incremental compilation is turned on, the compiler breaks the input source into pieces, and tracks how those input pieces influence the final build product. Then, when the inputs change, it detects this and reuses artifacts from previous builds, striving to expend effort solely on building the parts that need to respond to the changes to the input source code. | ||
|
||
[2016 blog post]: https://blog.rust-lang.org/2016/09/08/incremental.html | ||
|
||
Fingerprints are part of our architecture for detecting when inputs change. More specifically, a fingerprint (along with some other state to establish context) is a 128-bit value intended to uniquely identify internal values used within the compiler. Some compiler-internal results are stored on disk ("cached") between runs. Fingerprints are used to validate that a newly computed result is unchanged from the cached result. (More details about this are available in the [relevant chapter of the rustc dev guide][rustc-dev-guide-fingerprints].) | ||
|
||
[rustc-dev-guide-fingerprints]: https://rustc-dev-guide.rust-lang.org/queries/incremental-compilation-in-detail.html#checking-query-results-for-changes-hashstable-and-fingerprints | ||
|
||
The `verify-ich` check is a safeguard asserting internal inconsistency of the fingerprints. | ||
The compiler stores fingerprints for both cached and uncached values. | ||
Every time we compute an uncached value, we double-check that its newly computed fingerprint against the finger print stored in the cache. | ||
Mark-Simulacrum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
There are multiple ways that a fingerprint mismatch could arise, but they all represent bugs within the Rust compiler itself. | ||
|
||
## History of deployment | ||
|
||
We [initially added][pr-45867] `verify-ich` as a tool to use when developing rustc itself, back in 2017; it was solely provided via an unstable `-Z` flag, only available to nightly and development builds. | ||
Mark-Simulacrum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
More recently, in March, we encountered a [miscompilation][issue-82920] that led us to [turn on `verify-ich` by default][pr-83007]. The Rust compiler team decided it was better to catch fingerprint problems and abort compilation, rather than allow for potential miscompilations (and subsequent misbehavior) to sneak into Rust programmer's binaries. | ||
|
||
[pr-45867]: https://github.com/rust-lang/rust/pull/45867 | ||
[issue-82920]: https://github.com/rust-lang/rust/issues/82920 | ||
[pr-83007]: https://github.com/rust-lang/rust/pull/83007 | ||
|
||
When we first turned on `verify-ich` by default, there was a steady stream of | ||
issues filed by users of the nightly (and beta) toolchains, and steady progress | ||
has been made on identifying fixes, a number of which have already landed. This | ||
last week, we noted incorrectly that the error would be shipping to stable | ||
next cycle in 1.53.0, and we started [making plans][issue-84970] to improve the | ||
user-experience, so that the diagnostic issued by the check would do a better | ||
job of telling the programmer what to do in response. | ||
|
||
[issue-84970]: https://github.com/rust-lang/rust/issues/84970 | ||
|
||
It turns out `verify-ich` was turned on in version 1.52.0, which was [released recently][]. | ||
|
||
[released recently]: /2021/05/06/Rust-1.52.0.html | ||
|
||
This release, 1.52.1, works around the breakage caused by the newly added | ||
Mark-Simulacrum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
verification by temporarily changing the defaults in the Rust compiler to disable | ||
incremental unless the user knowingly opts in. | ||
|
||
## How does this show up | ||
|
||
Essentially, for some crates, certain sequences of edit-compile cycles will cause `rustc` to hit the "unstable fingerprints" ICE. I showed one example at the start of this blog post. | ||
|
||
Another recent example looks [like this](https://github.com/rust-lang/rust/issues/85039): | ||
|
||
``` | ||
thread 'rustc' panicked at 'found unstable fingerprints for predicates_of(<massive text describing rustc internals elided>)', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/compiler/rustc_query_system/src/query/plumbing.rs:593:5 | ||
``` | ||
|
||
They all arise from inconsistencies when comparing the incremental-compilation cache stored on disk against the values computed during a current `rustc` invocation, which means they all arise from using incremental compilation. | ||
|
||
There are three ways that you may have incremental compilation turned on: You may have set the [environment variable][env-vars] `CARGO_INCREMENTAL=1`, or you may have enabled the `build.incremental` [setting in your Cargo.toml][cargo-toml], or you may be building with the `dev` or `test` [profiles][], which default to having incremental compilation enabled. | ||
Mark-Simulacrum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
[env-vars]: https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-reads | ||
[cargo-toml]: https://doc.rust-lang.org/cargo/reference/config.html#buildincremental | ||
[profiles]: https://doc.rust-lang.org/cargo/reference/profiles.html | ||
|
||
Incremental is disabled by default for the release profile, which should mean | ||
that unless you have enabled it yourself, these errors do not affect release | ||
builds, as they are only present in incremental builds. | ||
|
||
## What should a Rust programmer do in response | ||
|
||
The Internal Compiler Error asks you to report a bug, and if you can do so, we still want that information. We *want* to know about the cases that are failing. | ||
|
||
But regardless of whether or not you file a bug, the problem here can be resolved by either: | ||
Mark-Simulacrum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
1. upgrading to 1.52.1, if you have not yet done so (which will disable | ||
incremental for you). | ||
Mark-Simulacrum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
2. deleting your incremental compilation cache (e.g. by running `cargo clean`), or | ||
3. force incremental compilation to be disabled, by setting `CARGO_INCREMENTAL=0` in your environment or `build.incremental` to `false` in the `config.toml`. | ||
Mark-Simulacrum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
We recommend that users of 1.52.0 upgrade to 1.52.1, which disables incremental | ||
compilation. | ||
|
||
We do *not* recommend that users of 1.52.0 downgrade to an earlier version of Rust in response to this problem. As noted above, there is at least one instance of a silent [miscompilation][issue-82920] caused by incremental compilation that was not caught until we added the fingerprint checking. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And we don't know how far back these bugs are present. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (I think the text in the summary does a good job of explaining that aspect of things. I don't think we need to reiterate the point here.) |
||
|
||
If a user is willing to deal with the incremental verification ICE's, and wishes | ||
to opt back into the 1.52.0 behavior, they may set `RUSTC_FORCE_INCREMENTAL` to | ||
pnkfelix marked this conversation as resolved.
Show resolved
Hide resolved
|
||
`1` in their environment. The Rust compiler will then respect the | ||
`-Cincremental` option passed by Cargo, and things will work as before. Note | ||
that this flag does not enable incremental if it has not already been separately | ||
enabled (whether by Cargo or otherwise). | ||
|
||
## What is the Rust project going to do to fix this | ||
Mark-Simulacrum marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
### Short-term plan | ||
|
||
We have issued 1.52.1 today which: | ||
|
||
* Disables incremental compilation in the Rust compiler (unless asked for by a | ||
new environment variable, `RUSTC_FORCE_INCREMENTAL=1`). | ||
* Improves diagnostic output for the new verification if incremental compilation is enabled, indicating how to work around the bugs. | ||
|
||
This is intended to be a mitigation that helps the majority of Rust users have | ||
an upgrade path to a safe Rust compiler which does not have the risk of | ||
miscompiling their code, but also provide the option for users willing to deal | ||
with the errors to do so. | ||
|
||
We expect to continue to actively invest in fixing the bugs, and depending on | ||
our confidence in the fixes, may issue a 1.52.2 point release which backports | ||
those fixes to the stable channel. Users wishing to help us test can use the | ||
nightly channel, and report bugs to rust-lang/rust with any ICEs they are | ||
seeing. We do not expect at this time to disable incremental by default on the | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this raises the question "will incremental be disabled on beta?" It might be good to preemptively address that question. |
||
nightly channel. | ||
|
||
### Long-term plan | ||
|
||
The long-term plan is to fix the bugs! Incremental compilation is the only realistic way for the Rust compiler to be able to provide a fast edit-compile-run cycle for all of its programmers, and so we need to address [all of the issues][issue-list] that have been identified thus far via `verify-ich`. (There are 32 such issues as of this writing, though many are duplicates.) | ||
|
||
We are actively investing in this, and a number of bugs have already been | ||
identified and fixed. Depending on the state of the fixes, future stable | ||
releases (1.53 and onwards) will likely re-enable incremental compilation. | ||
|
||
[issue-list]: https://github.com/rust-lang/rust/issues?q=is%3Aissue+is%3Aopen+unstable+fingerprints |
Uh oh!
There was an error while loading. Please reload this page.