--quiet propagation is unavailable to custom test harnesses

[kpreid]

Currently, Cargo will forward the --quiet option from its command line to test binaries. However, it does not do this when harness = false.

cargo/src/cargo/ops/cargo_test.rs

Lines 383 to 385 in 4a7e881

	if unit.target.harness() && gctx.shell().verbosity() == Verbosity::Quiet {
	cmd.arg("--quiet");
	}

This means that test targets with custom test harnesses cannot replicate the automatic coordination the rustc test harness gets. Given the stated plans to promote custom test harnesses over adding more features to libtest, I believe this behavior needs to be revised.

I think it is likely that the correct solution here is to add some larger API for more structured, extensible communication between cargo and a custom test harness, rather than changing what individual args are sent. Still, I’m filing this issue to make sure this is considered and to document the quirk for anyone else who is writing a libtest-compatible harness and debugging why their output stays verbose.

@rustbot label +Command-test +A-cli

[epage]

Over time, we expect to tighten the relationship between cargo test and libtest further (e.g. using the json output). Cargo will need some way of knowing a custom test harnesses capabilities.

Some options include

• Some field on the build-target table to specify a version of the cargo/libtest protocol
• Name the custom test harness (<target>.test = "dep:libtest2") and allow the custom test harness' package to specify the cargo/libtest protocol it implements

Either way, we'll need a way to set this across the entire package, instead of hand-enumerating every target (#6945).