Filter specific versions of a package

[hanzo]

Is your feature request related to a problem? Please describe.
It would be useful to prevent specific versions of a packages from being downloaded, e.g. if a security vulnerability is discovered to be present in certain versions. From what I can tell, it used to be possible to filter specific versions of a package using a filter file, but this mechanism is now deprecated. The currently supported solution of using a download mode file seems to operate at the package level without any ability to configure behavior for specific versions of the package.

Describe the solution you'd like
Ideally the download mode file would allow configuration to be applied to a range of versions for a particular package. I haven't thought too hard about the config structure but something along these lines:

download "golang.org/x/crypto" {
    versions = ">=v0.25.0,<v0.31.0"
    mode = "none"
}

The effect of this config block would be to delete all versions of the crypto package between v0.25.0-v0.31.0 from persistent storage and anywhere that they're cached, and to return a 404 when go get is called for those particular versions.

Describe alternatives you've considered
I don't know of any good alternatives. I could manually delete files from the underlying data stores and purge the cache, but this isn't a scaleable or safe solution.

Additional context
My organization has some internal packages that have thousands of versions (periodically generated code from IDL schema), so for our use case it's necessary that the solution supports a range of versions instead of needing to specify exact version numbers.

[matt0x6F]

Hey @hanzo, awesome idea. I have some questions about your proposal:

The effect of this config block would be to delete all versions of the crypto package between v0.25.0-v0.31.0 from persistent storage and anywhere that they're cached, and to return a 404 when go get is called for those particular versions.

Does your implementation require that we manage the storage of these artifacts or is good enough that we just respond at the API layer as if the artifact does not exist? I ask because Athens does not currently manage the underlying stored artifacts in anyway; more simply: it's the difference between an entirely new subsystem versus a feature of the API and configuration.

download "golang.org/x/crypto" {
    versions = ">=v0.25.0,<v0.31.0"
    mode = "none"
}

Is there a DSL you had in mind? This kind of reminds me of HCL.

The versions value I could see getting increasingly complex and possibly conflicting over time. Is there a way that version bounds could be expressed that makes maintaining the list as time goes on easier on the reader?

I didn't see where you expressed what mode would do in this case. The Go Compiler and its use of SHAs for verification are relatively strict; we can't offer them a higher or lower version - we can only that package at the version does not exist given the current implementation of the download protocol.

[matt0x6F]

@nrwiersma showed me retractions. I feel like we could probably model the syntax after retract and take its functionality a step further.

Retractions inform the compiler, but don't make an artifact unavailable. We could give that capability to single tenant organizations through configuration. Instead of seeing a retraction message, the artifact is not served. The logic would be somewhat easy to follow:

• Retractions at the module level give warnings and inform update paths
• Retractions at the Athens proxy level make the artifact unavailable

[hanzo]

@matt0x6F thanks for your responses! Sorry for the delay, I was on vacation but I'm back now and can respond more quickly going forward.

Does your implementation require that we manage the storage of these artifacts or is good enough that we just respond at the API layer as if the artifact does not exist? I ask because Athens does not currently manage the underlying stored artifacts in anyway; more simply: it's the difference between an entirely new subsystem versus a feature of the API and configuration.

My goal is to improve the performance of our Athens deployment when serving requests like go get and go list for a package with thousands of versions (these requests can take a few minutes currently). I'm assuming that even if Athens started responding as if the blocked versions didn't exist, if we don't remove the artifacts from storage it won't actually improve the performance, correct? Since Athens will still have to fetch the list of every version from storage (including blocked versions) to be able to serve a go list request.

Regarding retractions, that looks very promising! Are you envisioning that Athens would provide its own custom configuration that uses the same syntax as the retract directive to specify version ranges which should not be served for a given package, or a simpler package-level (or global?) config setting that basically says "if any versions are retracted in the go.mod, do not serve those versions" ?