OSV.dev can't match deleted versions of packages #2407
Comments
maaaaz
changed the title
|
This looks like an osv.dev API bug, as both |
|
I can confirm that is not matching https://api.osv.dev/v1/vulns/MAL-2022-7426 so this is an OSV.dev API bug not an OSV-Scanner one. I'll move this over. |
|
Looking at https://api.osv.dev/v1/vulns/MAL-2022-7426, I can see what the problem is, and it's somewhat systemic to malicious packages records: because typically such packages get removed from the package registry, there are no versions to enumerate. OSV.dev's API today is reliant upon all known vulnerable (in this case, "malicious") versions being enumerated and present in the This deficiency is being worked on in #2401 |
|
@andrewpollock would it be straightforward and worthwhile to introduce a (hopefully) hotpath for advisories that are marked as impacting all versions, since shouldn't that be a case of matching the ecosystem + name? I'd say that technically it's an optimization which as a bonus would enable these advisories to be matched against without requiring a more fulsome api change |
|
Hello and thanks for taking care of that issue.
I totally agree, version is irrelevant for known malicious packages. |
|
There are cases where version do matter for malicious packages, e.g. in cases where a normal package repository was taken over by a malicious actor, and they made a new release containing malicious code. All previous versions are still valid non-malicious packages in that case (or sometimes not, if the registry is not immutable, the attacker might be able to swap out old versions as well). |
|
I don't know if and how this issue is linked to this one: github/advisory-database#4612 |
They're related only in that they both relate to malware (as opposed to security vulnerabilities) |
|
This issue has not had any activity for 60 days and will be automatically closed in two weeks See https://github.com/google/osv.dev/blob/master/CONTRIBUTING.md for how to contribute a PR if you're interested in helping out. |
github-actions
bot
added
the
stale
|
Looking at this again, I think a solution if a package is removed (therefore cannot be enumerated) is to assume all versions are vulnerable? If it was the case I commented above, and the original maintainer regains control, I would guess that the entire package would not be removed from the repository, but only the latest version will be yanked. We would need a way to determine if us not being able to enumerate is not a temporary/transient error, or because a package is removed completely. |
github-actions
bot
removed
the
stale
I agree. When a non-legitimate-package-spoofing malicious package is found, removed or not, it should be simply marked as malicious by the package name. |
|
This issue has not had any activity for 60 days and will be automatically closed in two weeks See https://github.com/google/osv.dev/blob/master/CONTRIBUTING.md for how to contribute a PR if you're interested in helping out. |
github-actions
bot
added
the
stale
|
Hey Guys, just checking on this again, looks like this has been open for a long, let me know if there is already a PR or WIP, I can also contribute to this feature if someone can point me to the responsible code block (If OSV API is also open Source). |
|
We are still currently working out the approach we want to take to resolve this, as there are a few different ways to solve this issue. Definitely still WIP here, but hopefully we'll have a solution for this soon. The API is open source, you can see the code under the |
oliverchang
changed the title
[Malicious package](https://github.com/ossf/malicious-packages/tree/main/osv/malicious) publishes OSV records for crates.io, npm, NuGet, PyPI, and RubyGems. Queries for NuGet, PyPI, and RubyGems (npm and crates.io use semantic versioning, so the matching process is different) only match vulnerabilities against specific `affected versions`. However, malicious package records may only provide `affected ranges` instead of individual versions in some cases (e.g. https://api.osv.dev/v1/vulns/MAL-2022-7426). OSV also can't enumerate affected versions for a malicious package as those versions have been deleted. This causes issues like #2407 Switching the API query version matching from `_query_by_generic_version()` to [`_query_by_comparing_versions()`](https://github.com/google/osv.dev/blob/4d981692feb5e088d12177d97a44fe30701b3854/gcp/api/server.py#L1186) can address this issue. The `_query_by_comparing_versions()` function matches [both affected versions and affected ranges](https://github.com/google/osv.dev/blob/4d981692feb5e088d12177d97a44fe30701b3854/gcp/api/server.py#L1381), but might slow down the performance for a bit. Adding this PR after the end-of-year release to give more time to verify performance on the test instance before rolling out to prod.
Hello there,
Thanks for this amazing work but I am reporting here a crucial bug: known malicious packages are not detected when scanned.
How to reproduce:
pymocksin an examplepip freezeoutput (second line here below):Nothing is told about this
pymockspackage.I tried with different expressions:
pymocks==0.0.1,pymocksetc. but it never got detected.As this package is globally malicious, its detection should not need a version string: the sole presence of the package name in a lockfile should be enough to detect it !
Cheers!
The text was updated successfully, but these errors were encountered: