CEP XXXX: Build provenance metadata

[jaimergp]

Checklist for submitter

• I am submitting a new CEP: Build provenance metadata.
  • I am using the CEP template by creating a copy cep-0000.md named cep-XXXX.md in the root level.
• I am submitting modifications to CEP XX.
• Something else: (add your description here).

Checklist for CEP approvals

• The vote period has ended and the vote has passed the necessary quorum and approval thresholds.
• A new CEP number has been minted. Usually, this is ${greatest-number-in-main} + 1.
• The cep-XXXX.md file has been renamed accordingly.
• The # CEP XXXX - header has been edited accordingly.
• The CEP status in the table has been changed to approved.
• The last modification date in the table has been updated accordingly.
• The pre-commit checks are passing.

• Closes Provenance metadata #62.
• Standardises what conda-forge has been doing for a while, and I think defaults too? More details in Burn in flow_run_id, remote_url, sha into package meta data conda-forge/conda-smithy#1577.

[JeanChristopheMorinPerso]

I'm excited to see this being formalized!

[JeanChristopheMorinPerso]

Suggested change

	For local workflows such as those specified by CFEP 03, `remote_url` MAY be omitted, but authors strongly recommend providing the adequate value manually if necessary.
	For local workflows such as those specified by [CFEP-03](https://github.com/conda-forge/cfep/blob/main/cfep-03.md), `remote_url` MAY be omitted, but authors strongly recommend providing the adequate value manually if necessary.

Also, if remote_url is omitted, should sha also be omitted?

[jaimergp]

Hm, I had assumed users interested in provenance are already using some type of version control, but maybe we can't force that either.

About the dash in CFEP 03, see

ceps/.pre-commit-config.yaml

Line 35 in 971cb23

name: CEPs must be referred to with 'CEP N' (no dash)

.

[dbast]

please keep sha as mandatory ... some packages may not have the remote_url to not expose private repositories to the public, but the sha is this helpful for internal audits / attestations to check that e.g. the sha actually exists / was reviewed as part of an PR / triggered a CI run etc.

[jaimergp]

I am writing this to describe the current conventions, not to impose on how packages should be built. I think that should be decided by packaging organizations separately. I plan to submit a CFEP for conda-forge where we do "require these fields in CI pipelines, as recommended by CEP XYZ".

Someone using conda-build to share an artifact with their research lab internally may not need to care about whether the recipe is version controlled or what a git hash is.

[JeanChristopheMorinPerso]

The name flow_run_id originates from Anaconda's current build system. I wish that we could use something more generic and meaningful. But I guess the boat has sailed already? Or do you foresee a future where CF would be willing to change this to something else (on our side at Anaconda, this is very easy to do).

[jaimergp]

We can change it easily too, but then we will have to maintain two ways of accessing this metadata because the already stamped artifacts won't be rebuilt. I think flow_run_id is sufficiently generic. I always read it as "(work)flow run ID".

[dbast]

flow_run_id is constantly used for defaults and conda-forge and the naming was accepted on both sites in the past ... have a look into this PR ... you can see it that PR that every CI system (Azure, Travis, Github,...) name their variable containing the ID differently ... flow_run_id was/is meant to be agnostic and the value prefix tells what automation/ci/flow/workflow system was used.

[chenghlee]

Is this an intermediate step towards generating SLSA provenance attestations? If so, should we just straight in that direction, or would trying to implement SLSA delay the gains we want to get now?

[dbast]

thanks @jaimergp for the initiative here.

some background:

• this got enabled in conda-build in Jan 2022 via Add extra meta to about.json conda-build#4303
• this got enabled for defaults packages beginning of April 2022
• this got enabled in conda-forge in Nov 2023 via Burn in flow_run_id, remote_url, sha into package meta data conda-forge/conda-smithy#1577

@chenghlee yes, it was meant as intermediate step to enable (multiple different) actual attestations via an attestation worker using the data to lookup things:

• process attestation: does the mentioned sha actually exist and was it part of a PR review process?
• automation attestation: the sha after merge to main triggered a CI build matching the flow_run_id and the sha256 of the package is also present in the build log -> proof that a package is automatically build without manual interventions.
• provenance attestation: if the previous attestations are successfully it becomes implicitly clear that the provenance data is correct.

though, if and how those attestations can be stored via e.g. Sigstore in context of SLSA is a different thing.

tl;dr enables attestations that would be hard otherwise without any provenance data.

[jaimergp]

Is this an intermediate step towards generating SLSA provenance attestations? If so, should we just straight in that direction, or would trying to implement SLSA delay the gains we want to get now?

I'm not personally aiming for that, only wanted to standardise what otherwise was an undocumented convention. I think that SLSA provenance can be iterated on later, and this can just reflect the current state. This way we can refer at non-SLSA provenance like "CEP XYZ metadata".

[baszalmstra]

Is there an example on how this value can be used? I dont think conda-metadata-app uses this right?

[jaimergp]

We only link to the commit hash, which on the GH UI should provide enough information for a user to navigate to the CI workflow (and then manually check the workflow run ID or something). I guess I was just lazy to postprocess strings like github_1234565435 or azure_79979955940; we'd require more information to build the full URL.

[jaimergp]

Also since the workflow logs tend to expire, they are not as useful long term. But we could also add other known keys like flow_run_url for build farms to populate.

[baszalmstra]

After properly reading this CEP, we immediately updated this on https://prefix.dev!

In the next few days when you click a variant on https://prefix.dev you will see something like:

This is using the provenance metadata from this CEP. If however, the package has signed CEP 27 compliant attestation you will see:

[jaimergp]

Dear @conda/steering-council,

The vote for this CEP has started. It will be open for two weeks, until March 2nd, 2026, 23:59 Anywhere on Earth. This time period has been chosen to make it eligible for time-out rules. As an Enhancement Proposal vote, it requires 60% affirmative votes to pass.

To vote, please mark the relevant checkbox under your username:

• Uwe Korn (@xhochy)
  • yes
  • no
  • abstain
• Christopher J. 'CJ' Wright (@CJ-Wright)
  • yes
  • no
  • abstain
• Marius van Niekerk (@mariusvniekerk)
  • yes
  • no
  • abstain
• Cheng H. Lee (@chenghlee)
  • yes
  • no
  • abstain
• Filipe Fernandes (@ocefpaf)
  • yes
  • no
  • abstain
• Marcelo Duarte Trevisani (@marcelotrevisani)
  • yes
  • no
  • abstain
• Michael Sarahan (@msarahan)
  • yes
  • no
  • abstain
• Marcel Bargull (@mbargull)
  • yes
  • no
  • abstain
• John Kirkham (@jakirkham)
  • yes
  • no
  • abstain
• Jannis Leidel (@jezdez)
  • yes
  • no
  • abstain
• Wolf Vollprecht (@wolfv)
  • yes
  • no
  • abstain
• Jaime Rodríguez-Guerra (@jaimergp)
  • yes
  • no
  • abstain
• Bas Zalmstra (@baszalmstra)
  • yes
  • no
  • abstain
• Hind Montassif (@Hind-M)
  • yes
  • no
  • abstain
• Pavel Zwerschke (@pavelzw)
  • yes
  • no
  • abstain

[jaimergp]

@CJ-Wright @mariusvniekerk @chenghlee @marcelotrevisani @msarahan @mbargull @jakirkham, gentle reminder to cast your vote on this CEP.