[2022-04-08]: Secret scanning: add org level dry run for custom patterns (#26545)

* add org level dry run

Author: vgrl

assets/images/help/repository/secret-scanning-dry-run-custom-pattern-select-repo.png

@@ -69,10 +69,7 @@ Before defining a custom pattern, you must ensure that {% data variables.product
 {% data reusables.advanced-security.secret-scanning-new-custom-pattern %}
 {% data reusables.advanced-security.secret-scanning-add-custom-pattern-details %}{% ifversion fpt or ghec or ghes > 3.4 or ghae-issue-5499 %}
 1. When you're ready to test your new custom pattern, to identify matches in the repository without creating alerts, click **Save and dry run**.
-1. When the dry run finishes, you'll see a sample of results (up to 1000) from the repository. Review the results and identify any false positive results.
-   ![Screenshot showing results from dry run](/assets/images/help/repository/secret-scanning-publish-pattern.png)
-1. Edit the new custom pattern to fix any problems with the results, then click **Save and dry run** to test your changes.
-{% indented_data_reference reusables.secret-scanning.beta-dry-runs spaces=3 %}
+{% data reusables.advanced-security.secret-scanning-dry-run-results %}
 {% endif %}
 {% data reusables.advanced-security.secret-scanning-create-custom-pattern %}
 
@@ -111,18 +108,27 @@ aAAAe9
 
 Before defining a custom pattern, you must ensure that you enable {% data variables.product.prodname_secret_scanning %} for the repositories that you want to scan in your organization. To enable {% data variables.product.prodname_secret_scanning %} on all repositories in your organization, see "[Managing security and analysis settings for your organization](/organizations/keeping-your-organization-secure/managing-security-and-analysis-settings-for-your-organization)."
 
+{% ifversion ghes < 3.5 or ghae %}
 {% note %}
 
 **Note:** As there is no dry-run functionality, we recommend that you test your custom patterns in a repository before defining them for your entire organization. That way, you can avoid creating excess false-positive {% data variables.product.prodname_secret_scanning %} alerts.
 
 {% endnote %}
+{% endif %}
 
 {% data reusables.profile.access_org %}
 {% data reusables.profile.org_settings %}
 {% data reusables.organizations.security-and-analysis %}
 {% data reusables.repositories.navigate-to-ghas-settings %}
 {% data reusables.advanced-security.secret-scanning-new-custom-pattern %}
 {% data reusables.advanced-security.secret-scanning-add-custom-pattern-details %}
+{%- if secret-scanning-org-dry-runs %}
+1. When you're ready to test your new custom pattern, to identify matches in select repositories without creating alerts, click **Save and dry run**.
+1. Search for and select the repositories where you want to perform the dry run. You can select up to 10 repositories.
+   ![Screenshot showing repositories selected for the dry run](/assets/images/help/repository/secret-scanning-dry-run-custom-pattern-select-repo.png)
+1. When you're ready to test your new custom pattern, click **Dry run**.
+{% data reusables.advanced-security.secret-scanning-dry-run-results %}
+{%- endif %}
 {% data reusables.advanced-security.secret-scanning-create-custom-pattern %}
 
 After your pattern is created, {% data variables.product.prodname_secret_scanning %} scans for any secrets in repositories in your organization, including their entire Git history on all branches. Organization owners and repository administrators will be alerted to any secrets found and can review the alert in the repository where the secret is found. For more information on viewing {% data variables.product.prodname_secret_scanning %} alerts, see "[Managing alerts from {% data variables.product.prodname_secret_scanning %}](/code-security/secret-security/managing-alerts-from-secret-scanning)."

content/code-security/secret-scanning/defining-custom-patterns-for-secret-scanning.md

@@ -0,0 +1,6 @@
+# Issue #6367
+# Documentation for the "org level dry runs (Public Beta)" for custom patterns under secret scanning
+versions:
+  ghec: '*'
+  ghes: '>3.4'
+  ghae: 'issue-6367'

data/features/secret-scanning-org-dry-runs.yml

@@ -0,0 +1,4 @@
+1. When the dry run finishes, you'll see a sample of results (up to 1000) from the repository. Review the results and identify any false positive results.
+   ![Screenshot showing results from dry run](/assets/images/help/repository/secret-scanning-publish-pattern.png)
+1. Edit the new custom pattern to fix any problems with the results, then, to test your changes, click **Save and dry run**.
+{% indented_data_reference reusables.secret-scanning.beta-dry-runs spaces=3 %}