Properly track safety invariants #3846

Job	Run time
pure-rust-build	1m 59s
test	21m 3s
test-32bit (armv7-linux-androideabi)	1m 10s
cargo-deny (advisories)	31s
installation (win-msvc)	5m 8s
cargo-deny (bans licenses sources)	38s
lint	3m 43s
installation (win-gnu)	6m 6s
installation (win32-msvc)	5m 46s
test-fast (windows-latest)	13m 46s
installation (win32-gnu)	6m 1s
WebAssembly (wasm32-unknown-unknown)	56s
test-fast (macos-latest)	4m 46s
WebAssembly (wasm32-wasi)	41s
test-fast (ubuntu-latest)	3m 0s
	1h 15m 14s

Provide feedback