🎉 advance NoseyParker to support version 0.22.0

[manuel-sommer]

#11535

[dryrunsecurity]

DryRun Security Summary

The GitHub Pull Request enhances the Nosey Parker parser's documentation, test coverage, and functionality by adding support for version 0.22.0, improving secret detection capabilities, and ensuring robust parsing of security scan reports.

Expand for full summary

Summary:

The changes in this GitHub Pull Request focus on improving the documentation and test coverage for the Nosey Parker parser, which is a security tool used to scan repositories for secrets and sensitive information. The key highlights from a security perspective are:

1. The documentation updates include support for the latest version (0.22.0) of the Nosey Parker tool, in addition to the previously supported version (0.16.0). This ensures that users have access to the most up-to-date information and can effectively use the parser with the latest version of the Nosey Parker tool.

2. The documentation provides important details about the Nosey Parker parser, such as the severity of the identified issues, the deduplication algorithm, and the differences between full history scans and targeted branch scans. This information is crucial for users to understand the capabilities and limitations of the parser, and how to effectively integrate it into their security workflows.

3. The new unit tests cover the parsing of Nosey Parker scan reports for version 0.22.0, with and without Git history information. These tests help ensure the robustness and reliability of the NoseyParkerParser implementation, which is essential for maintaining the security of the application.

4. The changes to the NoseyParkerParser class itself add support for the newer version 0.22.0 of the Nosey Parker tool, while maintaining the existing support for version 0.16.0. The parser implementation handles various aspects of secret detection, including deduplication, severity assignment, and mitigation recommendations.

Overall, the changes in this Pull Request demonstrate a comprehensive approach to improving the documentation, test coverage, and functionality of the Nosey Parker parser, which is a crucial component for integrating security tools into the application's development and deployment processes.

Files Changed:

1. docs/content/en/connecting_your_tools/parsers/file/noseyparker.md: This documentation file has been updated to include support for version 0.22.0 of the Nosey Parker tool, in addition to the previously supported version 0.16.0. The documentation provides important details about the parser's functionality and usage.

2. unittests/scans/noseyparker/noseyparker_0_22_0_without_githistory.jsonl: This file appears to be a sample Nosey Parker scan report for version 0.22.0, without Git history information, which is used for testing purposes.

3. unittests/tools/test_noseyparker_parser.py: The unit tests for the NoseyParkerParser class have been updated to include new test cases for version 0.22.0 of the Nosey Parker tool, with and without Git history information. These tests help ensure the parser's ability to handle different Nosey Parker scan report formats.

4. dojo/tools/noseyparker/parser.py: The NoseyParkerParser class has been updated to add support for version 0.22.0 of the Nosey Parker tool, while maintaining the existing support for version 0.16.0. The parser's implementation handles various aspects of secret detection, including deduplication, severity assignment, and mitigation recommendations.

Code Analysis

We ran 9 analyzers against 5 files and 0 analyzers had findings. 9 analyzers had no findings.

View PR in the DryRun Dashboard.

[mtesauro]

Approved

[Himan10]

Hey @manuel-sommer, I've checked your PR and recently took a clone of your branch as well, so it does support our use-cases which I've mentioned in the issue. Also, please I'll request to not merge this PR for a while, I am doing some tinkering with it right now as per our other use-cases, will let you know quick soon about if we need to include any additional code snippet in your PR or else we can merge it then.

[manuel-sommer]

We need another three approvals, so it will remain open a couple of days anyway.

[Maffooch]

@Himan10 would the end of January work for you?

[Himan10]

@Maffooch, sure that'd work. Thanks

[Himan10]

Hey @manuel-sommer, I've identified some issues with the changes you've pushed to this PR. In Noseyparker, we have a flag to scan repositories without including their Git history (the flag --git-history=none is used for this). The structure of the reports generated by Noseyparker with --git-history=none is different from those generated without this flag. When running a normal scan with Noseyparker (which includes the Git history), the scan generally includes commit metadata. However, this commit metadata is not present in scans where --git-history=none is used.

Previously, we were unable to import Noseyparker scans, where the Git history of the repository was excluded, into DefectDojo. I've made some changes to your PR to support importing both the versions of the Noseyparker report, one with Git history and one without. I've tested this code locally with the latest version of DefectDojo and v2.33.0 (release mode).

Could you please review the changes and let me know your thoughts?

UPDATE: I've raised a new PR #11615 that contains the latest changes and support for both --git-history scans as well. I was not able to push changes to your PR and was facing some issues with it as well, so I went through with the new PR. Please check and let me know.

[mtesauro]

@manuel-sommer Your choice on how you want to proceed here - update this PR based on #11615 or close this and move over to that PR.

[manuel-sommer]

Hi @mtesauro
I added the functionality to differentiate between git history and without git history reported findings.

[Himan10]

Hey @manuel-sommer as per our discussion yesterday. I'll do the code review of your changes and you add changes to your PR regarding the support of both the scans i.e., with and without git history.

[manuel-sommer]

I already did including also unittests.

[Himan10]

@manuel-sommer you've only added the git/without git scans for noseyparker v22 only?

[manuel-sommer]

Could you provide a unittestfile for v16?
If yes, I can add that as well.

[Himan10]

Sure, I'll provide you the unit test cases for v16 shortly. Ig then we can merge this PR