Skip to content

Reproducible workspace - #163

Open
PatrickHaecker wants to merge 2 commits into
JuliaLang:mainfrom
PatrickHaecker:reproducible_workspace
Open

PatrickHaecker wants to merge 2 commits into
JuliaLang:mainfrom
PatrickHaecker:reproducible_workspace

Conversation

@PatrickHaecker

@PatrickHaecker PatrickHaecker commented Jul 3, 2026 •

Copy link
Copy Markdown
Contributor

compile_products previously copied the entire project tree into a
temporary directory so Pkg.instantiate could write a manifest even when
the source was read-only. That copy is harmful because it kills reproducibility:

  • An app which is part of a workspace loses its Manifest by the copy.
  • Therefore, the instantiate will be a resolve (already bad because
    any version changes can happen within the compatible set). But as
    the source entries from the top-level project are missing, too, it will
    in general also resolve to totally different branches from different
    URLs (very bad!).

This PR :

  • Resolve the manifest in place. If no manifest applies and the directory is
    read-only, error out. Such an environment is not usable by Julia in the first
    place: plain using against an explicit project without a manifest fails with
    "Package X is required but does not seem to be installed", regardless of
    permissions. The copy was the only reason those builds ever succeeded, and what
    it produced was a resolution the user could not reproduce with --project=.
  • Inject HostCPUFeatures' freeze_cpu_target preference through a throwaway
    load-path overlay environment rather than writing a LocalPreferences.toml
    into the project. A leading empty JULIA_LOAD_PATH entry expands to Julia's
    default load path, so JuliaC no longer hardcodes it.
  • Drop the now-unused TOML and Preferences imports.
  • Add WorkspaceProject test fixtures and cover the in-place workspace-member
    build.

Please consider whether a version bump to 0.4.0 makes sense:

  • a build against a read-only project without a manifest now errors instead of
    succeeding
  • a build now creates Manifest.toml in the project instead of in
    a throwaway copy.

Both are now consistent to what Julia itself does, so they can also be seen
as a bugfix.

This PR is ideally combined with Pkg.jl#4713

This PR builds on #117 just because I was too lazy to create yet another branch.
We should really get the number of open PRs containing bugfixes down.

#117 has been merged and I rebased #163 on main. Thanks, @gbaraldi!

Claude Opus 4.8 helped in creating this PR.

`compile_products` previously copied the entire project tree into a
temporary directory so `Pkg.instantiate` could write a manifest even when
the source was read-only. That copy is harmful because it kills reproducibility:

- An app which is part of a workspace loses its Manifest by the copy.
- Therefore, the `instantiate` will be a `resolve` (already bad because
any version changes can happen within the compatible set). But as
the source entries from the top-level project are missing, too, it will
in general also resolve to totally different branches from different
URLs (very bad!).

This PR :
- Resolve the manifest in place. If no manifest applies and the directory is
  read-only, error out instead of silently resolving into a throwaway,
  non-reproducible manifest.
- Inject HostCPUFeatures' `freeze_cpu_target` preference through a throwaway
  load-path overlay environment rather than writing a `LocalPreferences.toml`
  into the project. A leading empty JULIA_LOAD_PATH entry expands to Julia's
  default load path, so JuliaC no longer hardcodes it.
- Drop the now-unused `TOML` and `Preferences` imports.
- Add WorkspaceProject test fixtures and cover the in-place workspace-member
  build.

  This PR is ideally combined with [Pkg.jl#4713](JuliaLang/Pkg.jl#4713)

  Claude Opus 4.8 helped in creating this PR.
@PatrickHaecker

Copy link
Copy Markdown
Contributor Author

I let Opus 5 do a review. I needed to correct more invalid findings than there were useful findings, but there were useful findings and they should now be fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant