Repository navigation
Cross compilation support - #11179
andreabedini wants to merge 54 commits into
Conversation
|
Looks like very impressive and important work @andreabedini, it's great that you and your team have taken on this work. I'm looking forward to digging into these changes together. Would it be possible to write a cabal proposal to specify the user-facing changes to this patch? It would be good to get some feedback from other user's interested in cross-compilation and to write down the specification which this patch implements. Once a specification is established it will be easier to assess if the patch implements the agreed specification and there are suitable tests in place. From your description, it seems that someone could test this patch for themselves by using a wasm or javascript compiler and using `cabal-install" "normally"? Finally, hope you have a great holiday :) |
|
This has been open for almost half a year with no technical commentary whatsoever. |
Thanks for the reminder Julian, I left a comment above with some questions and outlining a possible way forward. No-one followed up on that yet. Did you have some thoughts? Once the direction and user-facing aspects are agreed then we can dive into a technical review of the code. |
|
I'm glad to see movement here. I'm excited for this feature! I'm not a cabal maintainer so take this with a pinch of salt, but it looks to me like there are smaller feature additions and bug fixes on this branch. I think these might benefit from being split into their own PRs and being reviewed independently. I think that would lead to a nicer reviewing experience for everyone, and would allow for some progress to be made while the cabal proposal is discussed. |
|
Andrea is on vacation until 13th January. I've pinged him on this matter. |
|
Sorry for the long radio silence.
Thank you and likewise: I would be happy to walk you through the PR. Please reach out if you are still available.
With the due respect, at the time the stable-haskell team decided to not write a proposal because we did not considered a good use of our (my) time. That said, I appreciate the value of a specification so I will try to find some time to document the overall goals and the technical approach. I did spend a lot of effort in making every change understandable but I am afraid I must have missed the forest for the trees :)
Basically the core of patch adds the concept of stage ("build" or "host" for lack of better names). Anything that has to run locally (Setup.hs, build-tools-depends and their dependencies) will belong to the "build" stage and will use the "build" compiler ( So yes, you can try
I absolutely agree. Might I ask which commits or changes you want to see separate? With that information I can assess how much effort it would take me to split them out, and maybe create a separate PR. |
| newtype Staged a = Staged | ||
| { getStage :: Stage -> a | ||
| } |
There was a problem hiding this comment.
I would be tempted to go with:
data Staged a =
Staged { build :: a, host :: a }
I think that tends to be simpler, even if you don't expose those record constructors.
There was a problem hiding this comment.
Yes, I had considered that; then I got tempted by the idea of having a hierarchy of stages (e.g. separating the build tools of the build tools, etc). Nevertheless, at this stage, this would be a good simplification.
I am not super familiar with the cabal code base so I'm not immediately sure. 4f0b69a jumped out to me as something that should be split out. But that's mostly cause I recently touched this code. I would like to see (1) a description of what the issue is (in the form of a ticket ideally), (2) a test, (3) a commit description. I think it's quite valuable to perhaps do this for all of the Basically, if it can be split out, it should be is my view. Although take this all with a grain of salt as I'm not a maintainer |
I made one pass through the commits related to the dependency solver to find the ones that could be separated, though I haven't read any of the code in depth or looked at the overall design. 97f7aff d975d43 5ec63e5 db76bd1 086ccc2 a70bb7d c303cc0 4d51ab9 ca8828f a8050fc |
8e9a1e3 to
fdbfbbe
Compare
There was a problem hiding this comment.
Pull request overview
Adds cross-compilation support to cabal-install by introducing explicit build/host stages and stage-aware toolchain/solver/planning plumbing, plus related refactors (including constraint scoping changes and removal of --independent-goals). Also extends Cabal syntax for “extra source” per-file options and updates tests/CI scripts accordingly.
Changes:
- Introduce
Stage/Stagedabstractions and thread them through the solver, planning, target selection, and build execution. - Add toolchain configuration for host/build stages and update command implementations to select the appropriate stage toolchain.
- Add
ExtraSource(with per-file options) and update build logic + testsuite coverage for extra sources.
Reviewed changes
Copilot reviewed 131 out of 133 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| cabal-testsuite/PackageTests/ExtraSources/src/MyLib.hs | Adds a testsuite library module for extra-sources coverage. |
| cabal-testsuite/PackageTests/ExtraSources/setup.test.hs | Adds a testsuite driver for configure/build. |
| cabal-testsuite/PackageTests/ExtraSources/setup.out | Adds expected output for the new testsuite case. |
| cabal-testsuite/PackageTests/ExtraSources/extra-sources.cabal | New fixture cabal file exercising c-sources options syntax. |
| cabal-testsuite/PackageTests/ExtraSources/cbits/test.c | C source fixture validating -D propagation. |
| cabal-install/tests/UnitTests/Distribution/Solver/Modular/QuickCheck.hs | Updates solver QC tests for stage + removal of independent-goals. |
| cabal-install/tests/UnitTests/Distribution/Solver/Modular/DSL/TestCaseUtils.hs | Updates test DSL utilities for new stage/scoping model. |
| cabal-install/tests/UnitTests/Distribution/Solver/Modular/DSL.hs | Adjusts DSL resolver plumbing to staged inputs. |
| cabal-install/tests/UnitTests/Distribution/Client/TreeDiffInstances.hs | Adds TreeDiff instances for new types (Stage/toolchain/qualifiers). |
| cabal-install/tests/UnitTests/Distribution/Client/Targets.hs | Updates imports/types for user constraint qualifier rename. |
| cabal-install/tests/UnitTests/Distribution/Client/ProjectConfig.hs | Adds Arbitrary instances for toolchain config split-out. |
| cabal-install/tests/UnitTests/Distribution/Client/InstallPlan.hs | Updates install plan generators to new InstallPlan.new'/logging. |
| cabal-install/tests/UnitTests/Distribution/Client/ArbitraryInstances.hs | Adds Arbitrary for new constraint qualifier and Stage. |
| cabal-install/src/Distribution/Client/Types/ReadyPackage.hs | Refactors ReadyPackage to reuse new GenericReadyPackage module. |
| cabal-install/src/Distribution/Client/Types/PackageSpecifier.hs | Updates constraint scoping constructor usage. |
| cabal-install/src/Distribution/Client/Types/GenericReadyPackage.hs | Introduces a reusable GenericReadyPackage newtype. |
| cabal-install/src/Distribution/Client/Types.hs | Re-exports GenericReadyPackage from the central Types module. |
| cabal-install/src/Distribution/Client/Toolchain.hs | Adds host/build toolchain configuration and helpers. |
| cabal-install/src/Distribution/Client/Targets.hs | Extends user constraints with stage qualifiers + new patterns. |
| cabal-install/src/Distribution/Client/TargetSelector.hs | Uses ExtraSourceClass to collect C/JS sources correctly. |
| cabal-install/src/Distribution/Client/SourceFiles.hs | Switches source traversal to ExtraSource accessors. |
| cabal-install/src/Distribution/Client/SetupWrapper.hs | Improves setup runner logging and toolchain setup path. |
| cabal-install/src/Distribution/Client/Setup.hs | Adds build-stage compiler flags and removes independent-goals flags. |
| cabal-install/src/Distribution/Client/ScriptUtils.hs | Makes script builds stage-aware and uses toolchain selection. |
| cabal-install/src/Distribution/Client/RebuildMonad.hs | Adds self-executable monitoring to rebuild cache invalidation. |
| cabal-install/src/Distribution/Client/ProjectPlanning/Stage.hs | Introduces WithStage wrapper for stage-tagged nodes. |
| cabal-install/src/Distribution/Client/ProjectConfig/Types.hs | Splits toolchain config into ProjectConfigToolchain. |
| cabal-install/src/Distribution/Client/ProjectConfig/Parsec.hs | Updates “modifies compiler” logic for toolchain nesting. |
| cabal-install/src/Distribution/Client/ProjectConfig/Lens.hs | Adds lenses for toolchain fields and removes independent-goals lens. |
| cabal-install/src/Distribution/Client/ProjectConfig/Legacy.hs | Maps legacy config parsing to new toolchain config structure. |
| cabal-install/src/Distribution/Client/ProjectConfig/FieldGrammar.hs | Adds grammar for toolchain sub-record and build-toolchain fields. |
| cabal-install/src/Distribution/Client/ProjectConfig.hs | Removes independent-goals from solver settings resolution. |
| cabal-install/src/Distribution/Client/ProjectBuilding/UnpackedPackage.hs | Uses per-stage toolchain for configure/build/register phases. |
| cabal-install/src/Distribution/Client/ProjectBuilding/Types.hs | Re-keys build status/outcomes by stage-aware plan keys. |
| cabal-install/src/Distribution/Client/PackageHash.hs | Splits direct deps into lib-deps vs exe-deps; adjusts inputs. |
| cabal-install/src/Distribution/Client/JobControl.hs | Simplifies -jsem behavior; always uses semaphore job control. |
| cabal-install/src/Distribution/Client/Install.hs | Updates solver invocation to staged inputs and removes independent-goals. |
| cabal-install/src/Distribution/Client/Get.hs | Updates install policy invocation signature. |
| cabal-install/src/Distribution/Client/Freeze.hs | Updates solver invocation + pruning logic for new solver IDs. |
| cabal-install/src/Distribution/Client/Fetch.hs | Updates solver invocation and resolveWithoutDependencies signature. |
| cabal-install/src/Distribution/Client/Errors.hs | Makes UnknownExecutable stage-aware via WithStage. |
| cabal-install/src/Distribution/Client/DistDirLayout.hs | Adds stage to dist dir params and alters build dir layout scheme. |
| cabal-install/src/Distribution/Client/Configure.hs | Updates solver invocation to staged inputs. |
| cabal-install/src/Distribution/Client/Config.hs | Adds SavedConfig support for build-stage compiler flags. |
| cabal-install/src/Distribution/Client/CmdTest.hs | Runs prune-to-targets in LogProgress. |
| cabal-install/src/Distribution/Client/CmdTarget.hs | Updates target map types to stage-aware keys. |
| cabal-install/src/Distribution/Client/CmdRun.hs | Updates selection and error types for stage-aware unit IDs. |
| cabal-install/src/Distribution/Client/CmdRepl.hs | Updates repl logic for stage-aware toolchains/targets/keys. |
| cabal-install/src/Distribution/Client/CmdPath.hs | Switches to toolchain configuration; warns about host assumption. |
| cabal-install/src/Distribution/Client/CmdListBin.hs | Uses stage toolchain platform and LogProgress pruning. |
| cabal-install/src/Distribution/Client/CmdInstall.hs | Updates install flow for staged targets and new DB stack logic. |
| cabal-install/src/Distribution/Client/CmdHaddockProject.hs | Refactors haddock-project to staged toolchains (index generation currently stubbed). |
| cabal-install/src/Distribution/Client/CmdHaddock.hs | Refactors haddock command to adjust toolchain program DB. |
| cabal-install/src/Distribution/Client/CmdGenBounds.hs | Updates bounds generation for stage-aware keys/deps. |
| cabal-install/src/Distribution/Client/CmdFreeze.hs | Updates constraint qualifier import/types. |
| cabal-install/src/Distribution/Client/CmdExec.hs | Updates exec environment generation to use staged toolchains. |
| cabal-install/src/Distribution/Client/CmdErrorMessages.hs | Adjusts rendering for NonEmpty broken-deps collection. |
| cabal-install/src/Distribution/Client/CmdBuild.hs | Runs pruning/new-plan creation via LogProgress. |
| cabal-install/src/Distribution/Client/CmdBench.hs | Runs prune-to-targets in LogProgress. |
| cabal-install/parser-tests/Tests/ParserTests.hs | Updates expected ProjectConfigShared parsing for toolchain split. |
| cabal-install/cabal-install.cabal | Bumps version and adds new modules to library stanza. |
| cabal-install-solver/src/Distribution/Solver/Types/Toolchain.hs | Adds solver-level Toolchain type keyed by stage. |
| cabal-install-solver/src/Distribution/Solver/Types/Stage.hs | Introduces Stage and Staged utilities + parsing/pretty. |
| cabal-install-solver/src/Distribution/Solver/Types/SolverPackage.hs | Adds stage and qualified package path fields to SolverPackage. |
| cabal-install-solver/src/Distribution/Solver/Types/SolverId.hs | Adds stage to solver IDs and Pretty instance. |
| cabal-install-solver/src/Distribution/Solver/Types/Settings.hs | Removes IndependentGoals setting. |
| cabal-install-solver/src/Distribution/Solver/Types/ResolverPackage.hs | Adds helpers for stage-aware solver IDs and QPNs. |
| cabal-install-solver/src/Distribution/Solver/Types/PackagePath.hs | Replaces Namespace with Stage in package paths; adds Pretty/Binary/Structured. |
| cabal-install-solver/src/Distribution/Solver/Types/PackageConstraint.hs | Adds stage to constraint scoping and new exe-qualifier scope. |
| cabal-install-solver/src/Distribution/Solver/Types/InstSolverPackage.hs | Adds stage + QPN to installed solver package representation. |
| cabal-install-solver/src/Distribution/Solver/Types/DependencyResolver.hs | Changes resolver signature to staged compiler/platform/pkg-config/iidx inputs. |
| cabal-install-solver/src/Distribution/Solver/Types/ConstraintSource.hs | Adds a new internal constraint source tag. |
| cabal-install-solver/src/Distribution/Solver/Types/ComponentDeps.hs | Adds null and adjusts Prelude hiding for new symbol. |
| cabal-install-solver/src/Distribution/Solver/Modular/Tree.hs | Refactors tree constructors with clearer structure/comments. |
| cabal-install-solver/src/Distribution/Solver/Modular/Solver.hs | Threads staged inputs; removes independent-goals in solver config. |
| cabal-install-solver/src/Distribution/Solver/Modular/Preference.hs | Updates I structure and qualification logic. |
| cabal-install-solver/src/Distribution/Solver/Modular/Package.hs | Adds stage into instance (I) and refactors show/loc representation. |
| cabal-install-solver/src/Distribution/Solver/Modular/Message.hs | Updates option rendering to match new I representation. |
| cabal-install-solver/src/Distribution/Solver/Modular/Linking.hs | Simplifies dependency qualification API usage. |
| cabal-install-solver/src/Distribution/Solver/Modular/Index.hs | Refactors index metadata and removes qualify-options machinery. |
| cabal-install-solver/src/Distribution/Solver/Modular/Explore.hs | Updates qualifyDeps usage for new API. |
| cabal-install-solver/src/Distribution/Solver/Modular/Cycles.hs | Adds callstack constraints to cycle detection helpers. |
| cabal-install-solver/src/Distribution/Solver/Modular/ConfiguredConversion.hs | Converts solver results using stage-aware installed/source indices. |
| cabal-install-solver/src/Distribution/Solver/Modular.hs | Updates resolver glue code for staged toolchains and staged pkg-config. |
| cabal-install-solver/cabal-install-solver.cabal | Exposes new Stage/Toolchain modules. |
| Cabal/src/Distribution/Verbosity.hs | Changes default verbosity flags to include VNoWrap. |
| Cabal/src/Distribution/Utils/LogProgress.hs | Adds pure runLogProgress' and eitherToLogProgress. |
| Cabal/src/Distribution/Types/LocalBuildInfo.hs | Adds HasCallStack to neededTargetsInBuildOrder'. |
| Cabal/src/Distribution/Simple/SrcDist.hs | Switches C-like source lists to ExtraSource files. |
| Cabal/src/Distribution/Simple/Program/ResponseFile.hs | Removes verbose debug dumping; makes verbosity arg unused. |
| Cabal/src/Distribution/Simple/Program/Db.hs | Adds programDbSignature helper for monitoring changes. |
| Cabal/src/Distribution/Simple/GHCJS.hs | Updates to build extra sources via ExtraSource and opts support. |
| Cabal/src/Distribution/Simple/GHC/Build/ExtraSources.hs | Adds ExtraSourceBuilder type alias + builds autogen CMM sources. |
| Cabal/src/Distribution/Simple/GHC.hs | Hardens abi-tag parsing and adjusts toolchain configuration API. |
| Cabal/src/Distribution/Simple/Configure.hs | Adds callstack constraints; disables looping debug pretty-print. |
| Cabal/src/Distribution/Simple/BuildTarget.hs | Switches component info to ExtraSource file accessors. |
| Cabal/src/Distribution/Simple/Build.hs | Wraps generated extra sources into ExtraSourcePkg. |
| Cabal/src/Distribution/PackageDescription/Check/Target.hs | Validates build-info paths via extraSourceFile. |
| Cabal/src/Distribution/Backpack/ConfiguredComponent.hs | Updates comments to reflect mixin terminology. |
| Cabal/src/Distribution/Backpack/Configure.hs | Adds callstack constraints and NonEmpty broken-deps formatting. |
| Cabal/src/Distribution/Backpack/ComponentsGraph.hs | Adds callstack constraints. |
| Cabal/Cabal.cabal | Bumps Cabal version. |
| Cabal-tree-diff/src/Data/TreeDiff/Instances/Cabal.hs | Adds TreeDiff instances for ExtraSource types. |
| Cabal-tests/tests/UnitTests/Distribution/Utils/Structured.hs | Updates structure hash expectations due to type changes. |
| Cabal-tests/tests/NoThunks.hs | Adds NoThunks instances for ExtraSource types. |
| Cabal-syntax/src/Distribution/Utils/Path.hs | Derives Data for Pkg and Build markers. |
| Cabal-syntax/src/Distribution/Types/ExtraSource.hs | Introduces ExtraSource and associated parsing/pretty/class APIs. |
| Cabal-syntax/src/Distribution/Types/BuildInfo/Lens.hs | Switches build info source lenses to ExtraSource types. |
| Cabal-syntax/src/Distribution/Types/BuildInfo.hs | Adds ExtraSource-typed fields and autogenCmmSources. |
| Cabal-syntax/src/Distribution/PackageDescription/FieldGrammar.hs | Extends grammar to parse/print ExtraSource lists. |
| Cabal-syntax/src/Distribution/PackageDescription.hs | Re-exports ExtraSource. |
| Cabal-syntax/src/Distribution/Compat/Graph.hs | Changes broken-edge reporting to NonEmpty and adds callstack checks. |
| Cabal-syntax/Cabal-syntax.cabal | Exposes the new ExtraSource module. |
| Cabal-described/src/Distribution/Described.hs | Extends described grammar with ExtraSource formats. |
| .github/workflows/release.yaml | Tweaks permissions and disables tests in release workflow invocation. |
| .github/scripts/test.sh | Adjusts how the testsuite is invoked (drops explicit -w). |
| .github/scripts/build.sh | Adds a new release build script to produce artifacts/tarballs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| let toolchains' = Staged $ \case | ||
| Host -> (getStage toolchains' Host){toolchainProgramDb = progs} | ||
| Build -> getStage toolchains' Build | ||
|
|
| buildToolchain <- do | ||
| (toolchainCompiler, toolchainPlatform, toolchainProgramDb) <- | ||
| configCompilerExSafe | ||
| verbosity | ||
| (flagToMaybe configBuildHcFlavor) | ||
| (flagToMaybe configBuildHcPath) | ||
| (flagToMaybe configBuildHcPkg) | ||
| programDb | ||
| let toolchainPackageDBs = interpretPackageDBStack Nothing $ interpretPackageDbFlags False $ configPackageDBs | ||
| return Toolchain{..} |
| pretty (ExtraSourcePkg path opts) = | ||
| pretty path <<>> PP.parens (PP.hsep (map PP.text opts)) | ||
|
|
||
| instance Pretty (ExtraSource Build) where | ||
| pretty (ExtraSourceBuild path opts) = | ||
| pretty path <<>> PP.parens (PP.hsep (map PP.text opts)) |
| instance Pretty ConstraintScope where | ||
| pretty (ConstraintScope mstage qualifier) = | ||
| maybe mempty pretty mstage <+> pretty qualifier | ||
|
|
| @@ -334,28 +333,31 @@ haddockProjectAction flags _extraArgs globalFlags = do | |||
| -- `documentation: True` in the global config). | |||
| info verbosity (intercalate "\n" missingHaddocks) | |||
|
|
|||
| let flags' = | |||
| flags | |||
| { haddockProjectDir = Flag outputDir | |||
| , haddockProjectInterfaces = | |||
| Flag | |||
| [ ( interfacePath | |||
| , Just url | |||
| , Just url | |||
| , visibility | |||
| ) | |||
| | (url, interfacePath, visibility) <- packageInfos' | |||
| ] | |||
| , haddockProjectUseUnicode = NoFlag | |||
| } | |||
| createHaddockIndex | |||
| verbosity | |||
| (pkgConfigCompilerProgs sharedConfig') | |||
| (pkgConfigCompiler sharedConfig') | |||
| (pkgConfigPlatform sharedConfig') | |||
| Nothing | |||
| flags' | |||
| warn verbosity "createHaddockIndex not implemented" | |||
| {-# LANGUAGE RecordWildCards #-} | ||
| {-# OPTIONS_GHC -Wno-unused-imports #-} | ||
| {-# OPTIONS_GHC -Wno-unused-local-binds #-} | ||
| {-# OPTIONS_GHC -Wno-unused-matches #-} |
| info verbosity $ | ||
| "Using internal setup method with build-type " | ||
| ++ show bt | ||
| ++ " and args:\n " | ||
| ++ show args | ||
| ++ unwords args |
| uses: ./.github/workflows/reusable-release.yml | ||
| with: | ||
| branches: '["${{ github.ref }}"]' | ||
| test: false |
|
Sorry for the noise. I have been working to separate out some out-of-scope changes but at the same time I must have accidentally added other CI related changes. There's too many changes 😂 |
|
@andreabedini Thanks anyway for the work, it's really appreciated and we certainly owe you some great amount of your favourite beverage. ;) |
…build dependencies
…antiatedWith from ElaboratedConfiguredPackage to ElaboratedComponent Instantiation only makes sense for components.
add stages list
fdbfbbe to
f14de46
Compare
f14de46 to
c0de6fa
Compare
Replace `Namespace` in `PackagePath` with `Stage`, so every node in the solver decision tree carries a `Host`/`Build` tag. This allows the solver to maintain separate installed-package indexes, compiler infos, and pkg-config databases for each build stage. - `PackagePath` now holds a `Stage` instead of a `Namespace` (`DefaultNamespace`/`Independent`), eliminating the old namespace concept - `DependencyResolver` and `resolveDependencies` take `Staged` versions of `(CompilerInfo, Platform)`, `InstalledPackageIndex`, and `PkgConfigDb` - Index conversion (`convPIs`, `convIPI'`) iterates over both stages, tagging each installed package with its origin stage - Validation (`Validate.hs`) extracts the stage from each `PackagePath` to select the right extension/language/pkg-config checker - `runSolver`/`solve` signatures updated accordingly
…g jsem This is a user problem. User should not enable jsem on a compiler that does not support it. This change also avoid us to pass the compiler all the way down. A better approach to restore this functionality would be to defer the application of the parallel strategy.
Split the function into multiple ones.
If a node has dangling edges, then the list of missing neighbours cannot be empty.
Exceptions are not nice but this is an obvious invariant. Graph should provide a better API to make this unnecessary.
Not really a fix. I do not know why this happens.
It is duplicate information since we write the program invocation right after.
This is a partial revert of f47840d GHC toggles -dyanmic-too for TH and now we're missing shared interface files.
…r usage Adds a test package with a Custom Setup.hs that injects the build compiler's GHC version as a CPP define (SETUP_GHC_VERSION) into the executable's cppOptions. The executable prints both the setup-injected version and its own __GLASGOW_HASKELL__, allowing the test to assert the build compiler was actually used.
c0de6fa to
de8db60
Compare
…ode keys
The install-plan machinery in Distribution.Client.InstallPlan was written to
be polymorphic in its two node types (`GenericPlanPackage ipkg srcpkg`), yet
it hardcoded the *key* of those nodes to `UnitId` everywhere: the `IsUnit`
synonym pinned `Key a ~ UnitId`, `Processing` held `Set UnitId`, and
`BuildOutcomes` was a `Map UnitId`. That is an over-specialisation — nothing
in the plan logic actually depends on the key being a `UnitId`; it only needs
the two node types to share a common key that is an `IsNode` key.
This replaces the `UnitId`-specific `IsUnit` constraint with
type IsGraph ipkg srcpkg = (IsNode ipkg, IsNode srcpkg, Key ipkg ~ Key srcpkg)
and threads the shared key type (`Key ipkg`) through the operations that
previously mentioned `UnitId` directly, so `Processing` and `BuildOutcomes`
become parameterised over the key. The few functions that print or report on
keys gain `Pretty`/`Show (Key ipkg)` constraints accordingly. Helpers become
more honest about what they need — e.g. `depends :: IsNode a => a -> [Key a]`
rather than `depends :: IsUnit a => a -> [UnitId]`.
This is a behaviour-preserving refactor. The sole instantiation on master,
`type PlanPackage = GenericPlanPackage InstalledPackageInfo (ConfiguredPackage
UnresolvedPkgLoc)`, remains `UnitId`-keyed, so there is no functional change.
The motivation is cross-compilation (haskell#11179), where install plans are keyed by
a stage-qualified key rather than a bare `UnitId`; extracting this
generalisation on its own removes the `UnitId` assumption ahead of that work
and keeps the eventual core change smaller.
Pull cabal-install/Cabal/Cabal-syntax/cabal-install-solver 3.17 from the stable-haskell/Cabal fork (the cross "stage" system, haskell/cabal#11179) via source-repository-package in nix-tools/cabal.project, so make-install-plan gets `with-build-compiler` and build:/host: stage-qualified constraints. Port nix-tools' sources to the Cabal-syntax 3.17 API (dropped the constraint type param from CondTree/CondBranch; CondNode lost its constraints field; *Sources became [ExtraSource Pkg]; Verbosity split into Verbosity/VerbosityFlags; plan values are Stage-tagged): - lib-cabal2nix/Cabal2Nix.hs - make-install-plan/{Freeze,MakeInstallPlan,ProjectPlanOutput}.hs (ProjectPlanOutput strips the build:/host: stage qualifier from every emitted dependency id so plan.json ids stay bare) - setup-ghcjs/Setup.hs Reconcile the installed-package-id-os-override patch to 3.17 (preserving the fork's OSX->Long unit-id format), and add a patch porting Cabal-syntax-json (the cabal2json provider) to the 3.17 CondTree/ExtraSource/Verbosity API.
…ode keys
The install-plan machinery in Distribution.Client.InstallPlan was written to
be polymorphic in its two node types (`GenericPlanPackage ipkg srcpkg`), yet
it hardcoded the *key* of those nodes to `UnitId` everywhere: the `IsUnit`
synonym pinned `Key a ~ UnitId`, `Processing` held `Set UnitId`, and
`BuildOutcomes` was a `Map UnitId`. That is an over-specialisation — nothing
in the plan logic actually depends on the key being a `UnitId`; it only needs
the two node types to share a common key that is an `IsNode` key.
This replaces the `UnitId`-specific `IsUnit` constraint with
type IsGraph ipkg srcpkg = (IsNode ipkg, IsNode srcpkg, Key ipkg ~ Key srcpkg)
and threads the shared key type (`Key ipkg`) through the operations that
previously mentioned `UnitId` directly, so `Processing` and `BuildOutcomes`
become parameterised over the key. The few functions that print or report on
keys gain `Pretty`/`Show (Key ipkg)` constraints accordingly. Helpers become
more honest about what they need — e.g. `depends :: IsNode a => a -> [Key a]`
rather than `depends :: IsUnit a => a -> [UnitId]`.
This is a behaviour-preserving refactor. The sole instantiation on master,
`type PlanPackage = GenericPlanPackage InstalledPackageInfo (ConfiguredPackage
UnresolvedPkgLoc)`, remains `UnitId`-keyed, so there is no functional change.
The motivation is cross-compilation (haskell#11179), where install plans are keyed by
a stage-qualified key rather than a bare `UnitId`; extracting this
generalisation on its own removes the `UnitId` assumption ahead of that work
and keeps the eventual core change smaller.
…ode keys
The install-plan machinery in Distribution.Client.InstallPlan was written to
be polymorphic in its two node types (`GenericPlanPackage ipkg srcpkg`), yet
it hardcoded the *key* of those nodes to `UnitId` everywhere: the `IsUnit`
synonym pinned `Key a ~ UnitId`, `Processing` held `Set UnitId`, and
`BuildOutcomes` was a `Map UnitId`. That is an over-specialisation — nothing
in the plan logic actually depends on the key being a `UnitId`; it only needs
the two node types to share a common key that is an `IsNode` key.
This replaces the `UnitId`-specific `IsUnit` constraint with
type IsGraph ipkg srcpkg = (IsNode ipkg, IsNode srcpkg, Key ipkg ~ Key srcpkg)
and threads the shared key type (`Key ipkg`) through the operations that
previously mentioned `UnitId` directly, so `Processing` and `BuildOutcomes`
become parameterised over the key. The few functions that print or report on
keys gain `Pretty`/`Show (Key ipkg)` constraints accordingly. Helpers become
more honest about what they need — e.g. `depends :: IsNode a => a -> [Key a]`
rather than `depends :: IsUnit a => a -> [UnitId]`.
This is a behaviour-preserving refactor. The sole instantiation on master,
`type PlanPackage = GenericPlanPackage InstalledPackageInfo (ConfiguredPackage
UnresolvedPkgLoc)`, remains `UnitId`-keyed, so there is no functional change.
The motivation is cross-compilation (haskell#11179), where install plans are keyed by
a stage-qualified key rather than a bare `UnitId`; extracting this
generalisation on its own removes the `UnitId` assumption ahead of that work
and keeps the eventual core change smaller.
…ode keys
The install-plan machinery in Distribution.Client.InstallPlan was written to
be polymorphic in its two node types (`GenericPlanPackage ipkg srcpkg`), yet
it hardcoded the *key* of those nodes to `UnitId` everywhere: the `IsUnit`
synonym pinned `Key a ~ UnitId`, `Processing` held `Set UnitId`, and
`BuildOutcomes` was a `Map UnitId`. That is an over-specialisation — nothing
in the plan logic actually depends on the key being a `UnitId`; it only needs
the two node types to share a common key that is an `IsNode` key.
This replaces the `UnitId`-specific `IsUnit` constraint with
type IsGraph ipkg srcpkg = (IsNode ipkg, IsNode srcpkg, Key ipkg ~ Key srcpkg)
and threads the shared key type (`Key ipkg`) through the operations that
previously mentioned `UnitId` directly, so `Processing` and `BuildOutcomes`
become parameterised over the key. The few functions that print or report on
keys gain `Pretty`/`Show (Key ipkg)` constraints accordingly. Helpers become
more honest about what they need — e.g. `depends :: IsNode a => a -> [Key a]`
rather than `depends :: IsUnit a => a -> [UnitId]`.
This is a behaviour-preserving refactor. The sole instantiation on master,
`type PlanPackage = GenericPlanPackage InstalledPackageInfo (ConfiguredPackage
UnresolvedPkgLoc)`, remains `UnitId`-keyed, so there is no functional change.
The motivation is cross-compilation (haskell#11179), where install plans are keyed by
a stage-qualified key rather than a bare `UnitId`; extracting this
generalisation on its own removes the `UnitId` assumption ahead of that work
and keeps the eventual core change smaller.
…ode keys
The install-plan machinery in Distribution.Client.InstallPlan was written to
be polymorphic in its two node types (`GenericPlanPackage ipkg srcpkg`), yet
it hardcoded the *key* of those nodes to `UnitId` everywhere: the `IsUnit`
synonym pinned `Key a ~ UnitId`, `Processing` held `Set UnitId`, and
`BuildOutcomes` was a `Map UnitId`. That is an over-specialisation — nothing
in the plan logic actually depends on the key being a `UnitId`; it only needs
the two node types to share a common key that is an `IsNode` key.
This replaces the `UnitId`-specific `IsUnit` constraint with
type IsGraph ipkg srcpkg = (IsNode ipkg, IsNode srcpkg, Key ipkg ~ Key srcpkg)
and threads the shared key type (`Key ipkg`) through the operations that
previously mentioned `UnitId` directly, so `Processing` and `BuildOutcomes`
become parameterised over the key. The few functions that print or report on
keys gain `Pretty`/`Show (Key ipkg)` constraints accordingly. Helpers become
more honest about what they need — e.g. `depends :: IsNode a => a -> [Key a]`
rather than `depends :: IsUnit a => a -> [UnitId]`.
This is a behaviour-preserving refactor. The sole instantiation on master,
`type PlanPackage = GenericPlanPackage InstalledPackageInfo (ConfiguredPackage
UnresolvedPkgLoc)`, remains `UnitId`-keyed, so there is no functional change.
The motivation is cross-compilation (haskell#11179), where install plans are keyed by
a stage-qualified key rather than a bare `UnitId`; extracting this
generalisation on its own removes the `UnitId` assumption ahead of that work
and keeps the eventual core change smaller.
|
Replaced by #12356 |
Hello,
this PR adds support for cross-compilation to cabal-install.
In this work cross-compilation is viewed exclusively as a cabal-install feature, and it is implemented without any substantial change to Cabal. I introduced the concept of stages (which I call build and host) with their respective toolchain (compiler and program, package and pkgconfig databases); and re-used the solver concept of scopes to separate executable dependencies from library dependencies. Executable dependencies (like setup scripts and build-tool dependencies) are solved in the build scope, while library dependencies are solved in the host scope. Roughly speaking, the build scope is for binaries you will run on the build machine while the host scope is for binaries you might want to run elsewhere (e.g. on a different platform).
As you can imagine, this involved a fair amount of work and I am well aware that this PR is too big to be considered as a whole. Furthermore I arbitrarily dropped (obscure) feature and (legacy) workarounds to simplify my work. One example being the "independent goals" solver setting: I do not think anybody is using it (or even should use it) and it was interfering with my rework of the solver scopes. That said, I do not think anything prevents bringing these things back, if only lack of time and effort.
While I am proud of how I was able to make everything fit together nicely, the PR is not without issues (
Stagedis over-engineered, build stage should be entirely optional,IsGraphis just plain awkward, etc). I welcome (and encourage!) a thorough technical review of the implementation.This work was prompted by IOG's efforts to build GHC with just cabal-install; and for this reason this branch contains a few commits that are unrelated to cross-compilation. I have spent considerable amount of time making sure each commit corresponds to a single small change1; so it should be easy to identify and separate unrelated, unneeded or unwelcome changes.
Also, shame on me, there are no tests; indeed my only testing was building GHC itself. While some tests for the solver should be easy to add, I do not have a clear testing strategy in mind. Changelog and documentation are not done either.
I have no expectations for this to be merged as-in or even with only minor changes. You are welcome to mark this as draft or to consider it a "implementation included" RFC. I will indeed be offline for the next two weeks, so go wild but I will answer any question or comment only later.
Here is an itemised list of changes, produced with the help of Claude.
Core Cross-Compilation Features
Stage System:
Toolchain Management:
Solver Enhancements:
Major Refactoring
Project Planning (ProjectPlanning.hs):
Install Plan:
Build System Integration:
Removed/Simplified Features
Legacy Removals:
Simplified Logic:
Infrastructure Improvements
Error Handling & Logging:
Code Quality:
New Configuration Options
Build Options:
Project Configuration:
File Structure Changes
New Modules:
Major Updates:
Cabal Library Enhancements
Feature Additions:
Build Improvements:
Dependency & Build Logic:
Check list:
significance: significantin the changelog file.Footnotes
and sometimes failed, see the three "all of it" commits. ↩