Feature: output in SARIF format

[htadashi]

Is your feature request related to a problem? Please describe.

No.

Describe the solution you'd like

It would be nice to have the option of getting pip-audit's output in SARIF format. This would help create GitHub actions (#198) that naturally integrate with GitHub security advisories page.

[woodruffw]

Thanks for the request!

Looks like this is the reference SARIF implementation for Python: https://github.com/microsoft/sarif-python-om

[artemrys]

@woodruffw I would like to take this one, especially since we have an official GitHub Action already implemented.

[woodruffw]

@artemrys Please do! Feel free to ping me if you have any technical questions about the codebase.

[artemrys]

Started working on this feature, created a sample SARIF file, and uploaded it to GitHub, attaching the screenshot of how it looks.

Some links for future reference:

• tool created by Microsoft to validate SARIF files here
• found an example of a tool that performs similar actions to what pip-audit do and we can see how they are assembling the SARIF file here

I'll continue working on exploring more options.

The content of the SARIF file used to generate this alert:

{
    "$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json",
    "version": "2.1.0",
    "runs": [
        {
            "tool": {
                "driver": {
                    "name": "pip-audit",
                    "version": "1.2.3",
                    "informationUri": "https://github.com/pypa/pip-audit",
                    "rules": [
                        {
                            "id": "001",
                            "name": "vulnarable-dependency",
                            "shortDescription": {
                                "text": "vulnarable-dependency short"
                            },
                            "fullDescription": {
                                "text": "vulnarable-dependency short"
                            }
                        }
                    ]
                }
            },
            "results": [
                {
                    "ruleId": "001",
                    "message": {
                        "text": "Result text. This result does not have a rule associated."
                    },
                    "locations": [
                        {
                            "physicalLocation": {
                                "artifactLocation": {
                                    "uri": "requirements.txt"
                                },
                                "region": {
                                    "startLine": 1,
                                    "startColumn": 1,
                                    "endColumn": 11
                                }
                            }
                        }
                    ]
                }
            ]
        }
    ]
}

[artemrys]

I think we might be blocked here because GitHub requires a physicalLocation field to be populated in a SARIF file.

The physicalLocation requires the file location, line, and column where the code scanning result was found. If I understand correctly, pip-audit currently does not provide this information in the VulnerabilityResult class.

Especially, it hurts with a vulnerability in a transitive dependency. If the requirements file contains onlypytest==7.1.3 line and pytest v7.1.3 has a dependency on py which has a vulnerability (PYSEC-2022-42969), then showing the exact place in the "code" with a vulnerability might be tricky.

[woodruffw]

The physicalLocation requires the file location, line, and column where the code scanning result was found. If I understand correctly, pip-audit currently does not provide this information in the VulnerabilityResult class.

Yeah, we don't currently provide this in VulnerabilityResult, so we'd need to. Importantly, we can't provide it for every auditing "source": environments, for example, don't come from a file input so they don't have (file, line, col) information.

I'm not sure if this is the right approach, but for transitive dependencies we could probably flag them based on the file location of the direct dependencies that include them. We might need to think a bit more about the UX there to clarify that that's what we're doing, but I think that's the best we can do in terms of surfacing them.

[woodruffw]

To summarize: I'd be okay with us adding a source_locations: Optional[List[SourceLocation]] to each Dependency, which VulnerabiliyResult would then map with.

SourceLocation would then be:

class SourceLocation:
    path: Path
    line: int
    col: int

...and source_locations needs to be a list, because of subdependencies: subdependency S might be introduced by deps A and B, so we want to record the source locations of both A and B.

[jsoref]

If you generate a sarif file and it's slightly incomplete, but some items have a physical location and you also generate an artifact, then people can have most of your cake and eat it too.

That's been my approach to sarif integration w/ github. Don't let the perfect be the enemy of the good.

[woodruffw]

I don't think anybody is letting the perfect be the enemy of the good here; if you'd like to see this feature added in a state that doesn't include source locations, please go ahead and work on it!

(I'm happy to merge SARIF support in whatever state we can, so long as (1) we conform to the schema's requirements, and (2) test it as comprehensively as the other output formats.)

[denishpillay]

Any updates on this feature request? More and more security visualization tools use SARIF now. Would be great to have it for pip-audit