fix: set OPENSSL_DIR=/usr by default during cargo test#42
Merged
Conversation
The test harness invokes cargo test without setting OPENSSL_DIR. Any CRUST project depending on the `openssl-sys` crate (e.g. c_blind_rsa_signatures) fails to build for environmental reasons unrelated to translation quality, producing spurious build_ok=false in result.json. This silently affected scoring: agents whose result.json was generated in an environment with OPENSSL_DIR pre-set passed, while later refresh runs in cleaner environments saw "build failures" for the same code. Set OPENSSL_DIR to /usr by default (overridable via the env var) in both cargo test invocations and the runtests Python invocation.
3 tasks
benedikt-schesch
added a commit
that referenced
this pull request
Jun 20, 2026
Bumps results pointer from 8df0744 to 4ff73b7. Includes:
* 6bc629aa Refresh result.json across all Test-Corpus agents using the
parser from #40 (per-vector failures, accurate counts).
* 75005f8e Refresh result.json for gemini-3.1-pro-preview and gpt-5.4
(oneshot agents that were missed in the all-agent pass).
* 4ff73b73 Add codex-gpt54 CRUST + CRUST-blind results (95 projects
each, full translation pass with in-harness Bedrock retry).
Also refreshes codex-gpt54 Test-Corpus after retrying 6
cases that hit transient Bedrock errors during the original
run, and re-tests c_blind_rsa_signatures for claude variants
now that OPENSSL_DIR is set by default (#42).
Co-authored-by: Benedikt Schesch <scheschb@amazon.co.uk>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The test harness invokes
cargo testwithout settingOPENSSL_DIR. Any CRUST project depending on theopenssl-syscrate (notablyc_blind_rsa_signatures) fails to build with:This produces a spurious
build_ok: falseinresult.jsoneven though the translation is correct. It silently affected scoring: agents whoseresult.jsonwas generated in an environment withOPENSSL_DIRpre-set passed, while later refresh runs in cleaner environments saw "build failures" for the same source.This PR sets
OPENSSL_DIR=/usrby default (still overridable via the env var) in:cargo testinvocation intest_one_crustcargo test --verbosere-run for diagnosticspython3 -m runtests.rustinvocation inrun_runtestsTest plan
cargo build --releasecleanharvest-tools test CRUST/c_blind_rsa_signatures --updatefor claude (was build_ok=false): nowtests_ok: 3, tests_failed: 0, build_ok: true✅