Merge pull request #14 from kmcquade/fix/GH-10-add-recursive-scan-option

Added recursive scanning option - and fixed exclude-actions

Author: kmcquade

CHANGELOG.md

@@ -1,5 +1,9 @@
 # CHANGELOG
 
+## 0.0.6 (2020-05-01)
+* Fix `exclude-actions` in the exclusions file - it was not being respected before.
+* Add a recursive scanning option.
+
 ## 0.0.4 (2020-05-01)
 * Provide option to skip opening HTML report (`--skip-open-report`)
 * Provide report indicator on whether it is assumable by compute services

README.md

@@ -141,7 +141,7 @@ This will generate a file in your current directory titled `exclusions.yml`.
 Now when you run the `scan` command, you can use the exclusions file like this:
 
 ```bash
-cloudsplaining scan --exclusions-file exclusions.yml --file examples/files/example.json --output examples/files/
+cloudsplaining scan --exclusions-file exclusions.yml --input examples/files/example.json --output examples/files/
 ```
 
 For more information on the structure of the exclusions file, see [Filtering False Positives](#filtering-false-positives)
@@ -153,7 +153,7 @@ Now that we've downloaded the account authorization file, we can scan *all* of t
 Run the following command:
 
 ```bash
-cloudsplaining scan --exclusions-file exclusions.yml --file examples/files/example.json --output examples/files/
+cloudsplaining scan --exclusions-file exclusions.yml --input examples/files/example.json --output examples/files/
 ```
 
 It will create an HTML report like [this](https://opensource.salesforce.com/cloudsplaining/):
@@ -243,7 +243,7 @@ exclude-actions:
 Now when you run the `scan` command, you can use the exclusions file like this:
 
 ```bash
-cloudsplaining scan --exclusions-file exclusions.yml --file examples/files/example.json --output examples/files/
+cloudsplaining scan --exclusions-file exclusions.yml --input examples/files/example.json --output examples/files/
 ```
 
 
@@ -252,7 +252,7 @@ cloudsplaining scan --exclusions-file exclusions.yml --file examples/files/examp
 You can also scan a single policy file to identify risks instead of an entire account.
 
 ```bash
-cloudsplaining scan-policy-file --file examples/policies/explicit-actions.json
+cloudsplaining scan-policy-file --input examples/policies/explicit-actions.json
 ```
 
 The output will include a finding description and a list of the IAM actions that do not leverage resource constraints.
@@ -283,13 +283,13 @@ cloudsplaining download --profile someprofile
 cloudsplaining download --profile all
 
 # Scan Authorization details
-cloudsplaining scan --file default.json
+cloudsplaining scan --input default.json
 # Scan Authorization details with custom exclusions
-cloudsplaining scan --file default.json --exclusions-file exclusions.yml
+cloudsplaining scan --input default.json --exclusions-file exclusions.yml
 
 # Scan Policy Files
-cloudsplaining scan-policy-file --file examples/policies/wildcards.json
-cloudsplaining scan-policy-file --file examples/policies/wildcards.json  --exclusions-file examples/example-exclusions.yml
+cloudsplaining scan-policy-file --input examples/policies/wildcards.json
+cloudsplaining scan-policy-file --input examples/policies/wildcards.json  --exclusions-file examples/example-exclusions.yml
 ```
 
 ## FAQ

cloudsplaining/bin/cloudsplaining

@@ -7,7 +7,7 @@
 """
     Cloudsplaining is an AWS IAM Assessment tool that identifies violations of least privilege and generates a risk-prioritized HTML report with a triage worksheet.
 """
-__version__ = "0.0.5"
+__version__ = "0.0.6"
 import click
 from cloudsplaining import command
 

cloudsplaining/command/create_exclusions_file.py

@@ -46,4 +46,4 @@ def create_exclusions_file(output_file):
     )
     print("\tcloudsplaining download")
     print("You can use this with the scan command as shown below: ")
-    print("\tcloudsplaining scan --exclusions-file exclusions.yml --file default.json")
+    print("\tcloudsplaining scan --exclusions-file exclusions.yml --input default.json")

cloudsplaining/command/expand_policy.py

@@ -20,16 +20,17 @@
     short_help="Expand the * Actions in IAM policy files to improve readability"
 )
 @click.option(
-    "--file",
+    "--input",
     type=click.Path(exists=True),
     required=True,
     help="Path to the JSON policy file.",
 )
 @click_log.simple_verbosity_option(logger)
-def expand_policy(file):
+def expand_policy(input):  # pylint: disable=redefined-builtin
     """
     Expand the * Actions in IAM policy files to improve readability
     """
+    file = input
     with open(file) as json_file:
         logger.debug(f"Opening {file}")
         data = json.load(json_file)

cloudsplaining/command/scan.py

@@ -31,7 +31,7 @@
     "IAM security posture. "
 )
 @click.option(
-    "--file",
+    "--input",
     type=click.Path(exists=True),
     required=True,
     help="Path of IAM account authorization details file",
@@ -66,7 +66,7 @@
     "This helps when running the report in automation.",
 )
 @click_log.simple_verbosity_option()
-def scan(file, exclusions_file, output, all_access_levels, skip_open_report):
+def scan(input, exclusions_file, output, all_access_levels, skip_open_report):  # pylint: disable=redefined-builtin
     """
     Given the path to account authorization details files and the exclusions config file, scan all inline and
     managed policies in the account to identify actions that do not leverage resource constraints.
@@ -78,8 +78,22 @@ def scan(file, exclusions_file, output, all_access_levels, skip_open_report):
             except yaml.YAMLError as exc:
                 logger.critical(exc)
         check_exclusions_schema(exclusions_cfg)
+    if os.path.isfile(input):
+        scan_account_authorization_file(input, exclusions_cfg, output, all_access_levels, skip_open_report)
+    if os.path.isdir(input):
+        logger.info("The path given is a directory. Scanning for account authorization files and generating report.")
+        input_files = get_authorization_files_in_directory(input)
+        for file in input_files:
+            logger.info(f"Scanning file: {file}")
+            scan_account_authorization_file(file, exclusions_cfg, output, all_access_levels, skip_open_report)
 
-    with open(file) as f:
+
+def scan_account_authorization_file(input_file, exclusions_cfg, output, all_access_levels, skip_open_report):
+    """
+    Given the path to account authorization details files and the exclusions config file, scan all inline and
+    managed policies in the account to identify actions that do not leverage resource constraints.
+    """
+    with open(input_file) as f:
         contents = f.read()
         account_authorization_details_cfg = json.loads(contents)
         check_authorization_details_schema(account_authorization_details_cfg)
@@ -104,7 +118,7 @@ def scan(file, exclusions_file, output, all_access_levels, skip_open_report):
             exclusions_cfg, modify_only=True
         )
 
-    account_name = Path(file).stem
+    account_name = Path(input_file).stem
 
     # Lazy method to get an account ID
     account_id = None
@@ -139,3 +153,21 @@ def scan(file, exclusions_file, output, all_access_levels, skip_open_report):
         exclusions_cfg,
         skip_open_report=skip_open_report,
     )
+
+
+def get_authorization_files_in_directory(directory):
+    """Get a list of download-account-authorization-files in a directory"""
+    file_list = [f for f in os.listdir(directory) if os.path.isfile(os.path.join(directory, f))]
+    file_list_with_full_path = []
+    for file in file_list:
+        if file.endswith(".json"):
+            file_list_with_full_path.append(os.path.abspath(os.path.join(directory, file)))
+    new_file_list = []
+    for file in file_list_with_full_path:
+        with open(file) as f:
+            contents = f.read()
+        account_authorization_details_cfg = json.loads(contents)
+        valid_schema = check_authorization_details_schema(account_authorization_details_cfg)
+        if valid_schema:
+            new_file_list.append(file)
+    return new_file_list

cloudsplaining/command/scan_policy_file.py

@@ -29,7 +29,7 @@
     short_help="Scan a single policy file to identify identify missing resource constraints."
 )
 @click.option(
-    "--file",
+    "--input",
     type=click.Path(exists=True),
     required=True,
     help="Path of to the IAM policy file.",
@@ -50,9 +50,9 @@
     " (Resource Exposure, Privilege Escalation, Data Exfiltration). This can help with prioritization.",
 )
 @click_log.simple_verbosity_option(logger)
-def scan_policy_file(file, exclusions_file, high_priority_only):
+def scan_policy_file(input, exclusions_file, high_priority_only):  # pylint: disable=redefined-builtin
     """Scan a single policy file to identify missing resource constraints."""
-
+    file = input
     # Get the exclusions configuration
     with open(exclusions_file, "r") as yaml_file:
         try:
@@ -129,7 +129,7 @@ def scan_policy(policy_json, policy_name, exclusions_cfg=DEFAULT_EXCLUSIONS_CONF
         results_placeholder = []
         for action in actions_missing_resource_constraints:
             if excluded_actions:
-                if not is_name_excluded(action.lower, excluded_actions):
+                if not is_name_excluded(action.lower(), excluded_actions):
                     results_placeholder.append(action)
             else:
                 results_placeholder.append(action)

cloudsplaining/output/findings.py

@@ -6,8 +6,9 @@
 # or https://opensource.org/licenses/BSD-3-Clause
 import logging
 from policy_sentry.util.arns import get_account_from_arn, get_resource_string
-from cloudsplaining.shared.utils import capitalize_first_character
 from cloudsplaining.shared.constants import READ_ONLY_DATA_LEAK_ACTIONS
+from cloudsplaining.shared.exclusions import is_name_excluded
+from cloudsplaining.shared.utils import capitalize_first_character
 
 logger = logging.getLogger(__name__)
 
@@ -23,9 +24,13 @@ def __init__(self):
     def add(self, finding):
         """Add a finding to the list."""
         if isinstance(finding, list):
-            self.findings.extend(finding)
+            for a_finding in finding:
+                # Only add it if there are any actions after processing exclusions
+                if a_finding.actions:
+                    self.findings.append(finding)
         elif isinstance(finding, Finding):
-            self.findings.append(finding)
+            if finding.actions:
+                self.findings.append(finding)
 
     @property
     def json(self):
@@ -49,13 +54,29 @@ def __init__(
         actions,
         policy_document,
         assume_role_policy_document=None,
+        always_exclude_actions=None,
     ):
         self.policy_name = policy_name
         self.arn = arn
-        self.actions = actions
+        # self.actions = actions
+        self.always_exclude_actions = always_exclude_actions
+        self.actions = self._actions(actions)
         self.policy_document = policy_document
         self.assume_role_policy_document = assume_role_policy_document
 
+    def _actions(self, actions):
+        results = []
+        if self.always_exclude_actions:
+            for action in actions:
+                if is_name_excluded(action.lower(), self.always_exclude_actions):
+                    # logger.info("Excluded action: %s" % action)
+                    pass
+                else:
+                    results.append(action)
+            return results
+        else:
+            return actions
+
     @property
     def managed_by(self):
         """Determine whether the policy is AWS-Managed or Customer-managed based on a Policy ARN pattern."""
@@ -110,8 +131,17 @@ def role_assumable_by_compute_services(self):
     @property
     def permissions_management_actions_without_constraints(self):
         """Return a list of actions that could cause resource exposure via actions at the 'Permissions management'
-        access level, if applicable. """
-        return self.policy_document.permissions_management_without_constraints
+        access level, if applicable."""
+        results = []
+        if self.always_exclude_actions:
+            for action in self.policy_document.permissions_management_without_constraints:
+                if is_name_excluded(action.lower(), self.always_exclude_actions):
+                    pass
+                else:
+                    results.append(action)
+            return results
+        else:
+            return self.policy_document.permissions_management_without_constraints
 
     @property
     def privilege_escalation(self):
@@ -148,7 +178,7 @@ def json(self):
             "DataExfiltrationActions": self.data_leak_actions,
             "PermissionsManagementActions": self.permissions_management_actions_without_constraints,
             # Separate the "Write" and "Tagging" actions in the machine-readable output only
-            "WriteActions": self.policy_document.write_actions_without_constraints,
-            "TaggingActions": self.policy_document.tagging_actions_without_constraints,
+            # "WriteActions": self.policy_document.write_actions_without_constraints,
+            # "TaggingActions": self.policy_document.tagging_actions_without_constraints,
         }
         return result

cloudsplaining/output/templates/guidance/2-triage-guidance.md

@@ -35,7 +35,7 @@ To recap: you've followed these steps to generate this report:
   <ul><li><code>cloudsplaining create-exclusions-file --output-file exclusions.yml</code></li></ul>
 <li>Generated the report</li>
   <ul>
-    <li><code>cloudsplaining scan --file default-account-details.json --exclusions-file exclusions.yml</code></li>
+    <li><code>cloudsplaining scan --input default-account-details.json --exclusions-file exclusions.yml</code></li>
     <li>This generates three files: (1) The single-file HTML report, (2) The triage CSV worksheet, and (3) The raw JSON data file</li>
   </ul>
 </ul>
@@ -149,7 +149,7 @@ Add whatever values you want to the above depending on your organization's conte
 * Now, run the scan to generate a *new* Cloudsplaining report  that considers your exclusions criteria. This way, you are working with a report version that consists of True Positives only.
 
 ```bash
-cloudsplaining scan --file default.json --exclusions-file exclusions.yml
+cloudsplaining scan --input default.json --exclusions-file exclusions.yml
 ```
 
 You can now proceed to the Remediation stage.

cloudsplaining/output/templates/guidance/4-validation.md

@@ -13,7 +13,7 @@ After you've rewritten your IAM policy, we suggest two options for validating th
 
 You can validate that your remediated policy passes Cloudsplaining by running the following command:
 
-```cloudsplaining scan-policy-file --file policy.json --exclusions-file exclusions.yml```
+```cloudsplaining scan-policy-file --input policy.json --exclusions-file exclusions.yml```
 
 When there are no more results, it passes!