Merge pull request #42 from rckasa/elblogs

Enable Load Balancer Logging

Author: cydneystude

README.md

@@ -6,7 +6,8 @@ With Assisted Log Enabler for AWS, logging is turned on automatically for the va
 * AWS CloudTrail (Single Account Only)
 * Amazon Elastic Kubernetes Service (EKS) Audit and Authenticator Logs (Single Account and Multi-Account using AWS Organizations)
 * Amazon Route 53 Resolver Query Logs (Single Account and Multi-Account using AWS Organizations)
-* NEW! Amazon S3 Server Access Logs (Single Account and Multi-Account using AWS Organizations)
+* Amazon S3 Server Access Logs (Single Account and Multi-Account using AWS Organizations)
+* NEW! Elastic Load Balancing Access Logs (Single Account and Multi-Account using AWS Organizations)
 
 Link to related AWS Open Source Blog Post: [Introducing Assisted Log Enabler for AWS](https://aws.amazon.com/blogs/opensource/introducing-assisted-log-enabler-for-aws/)
 
@@ -22,10 +23,14 @@ The following is a simple diagram on how Assisted Log Enabler for AWS works in a
 
 ![Alt text](diagrams/assisted_log_enabler.png)
 
-NEW! The following is a simple diagram on how Assisted Log Enabler for AWS works with turning on Amazon S3 Server Access Logging in a single account:
+The following is a simple diagram on how Assisted Log Enabler for AWS works with turning on Amazon S3 Server Access Logging in a single account:
 
 ![Alt text](diagrams/assisted_log_enabler_s3.png)
 
+The following is a simple diagram on how Assisted Log Enabler for AWS works with turning on Elastic Load Balancing Access Logging in a single account:
+
+![Alt text](diagrams/assisted_log_enabler_lb.png)
+
 ## Prerequisites
 ### Permissions
 The following permissions are needed within AWS IAM for Assisted Log Enabler for AWS to run. Please see each section for a breakdown per AWS Service and functionality:
@@ -100,7 +105,7 @@ The following permissions are needed within AWS IAM for Assisted Log Enabler for
 "route53resolver:AssociateResolverQueryLogConfig",
 "iam:CreateServiceLinkRole" # This is used to create the AWSServiceRoleForRoute53 Resolver, which is used for creating the Amazon Route 53 Query Logging Configurations.
 
-# NEW! For adding Amazon S3 Server Access Logs:
+# For adding Amazon S3 Server Access Logs:
 "s3:PutBucketLogging",
 "s3:GetBucketLogging",
 "s3:ListBucket",
@@ -111,6 +116,17 @@ The following permissions are needed within AWS IAM for Assisted Log Enabler for
 "s3:PutBucketPublicAccessBlock",
 "s3:PutBucketLifecycleConfiguration"
 
+# NEW! For adding Elastic Load Balancing Access Logs:
+"elb:DescribeLoadBalancers",
+"elb:DescribeLoadBalancerAttributes",
+"elb:ModifyLoadBalancerAttributes",
+"elbv2:DescribeLoadBalancers",
+"elbv2:DescribeLoadBalancerAttributes",
+"elbv2:ModifyLoadBalancerAttributes",
+"elasticloadbalancing:DescribeLoadBalancers",
+"elasticloadbalancing:DescribeLoadBalancerAttributes",
+"elasticloadbalancing:ModifyLoadBalancerAttributes"
+
 # For cleanup of Amazon Route 53 Resolver Query Logs created by Assisted Log Enabler for AWS:
 "route53resolver:ListResolverQueryLogConfigs",
 "route53resolver:ListTagsForResource",
@@ -135,10 +151,12 @@ The following are the details of what happens within the Assisted Log Enabler fo
 * If no trail is configured, one is created and configured to log to the bucket created. (Single Account only as of this release)
 * If Amazon EKS Clusters exist, audit & authenticator logs are turned on.
 * Amazon Route 53 Query Logging is turned on for VPCs that do not have it turned on already.
-* NEW! Amazon S3 Server Access Logs are created for buckets that do not have it turned on already.
+* Amazon S3 Server Access Logs are created for buckets that do not have it turned on already.
    * This does not include for S3 buckets created by Assisted Log Enabler for AWS
    * Amazon S3 Server Access Logs require buckets that reside in the same account & region, so additional buckets for Amazon S3 Server Access logs are created for this.
-
+ * NEW! Elastic Load Balancing Access Logs are created for Application, Network and Classic Load Balancers that do not have it turned on already.
+   * Elastic Load Balancing Access Logs require buckets that reside in the region, so additional buckets for Elastic Load Balancing Access logs are created for this.
+   * The following table contains the account IDs to use in place of elb-account-id in the bucket policy: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html
 
 ## Running the Code
 The code in its current form can be ran inside the following:
@@ -177,10 +195,10 @@ No valid option selected. Please run with -h to display valid options.
 ```
 python3 assisted_log_enabler.py -h
 usage: assisted_log_enabler.py [-h] [--mode MODE] [--all] [--eks] [--vpcflow]
-                               [--r53querylogs] [--s3logs] [--cloudtrail]
+                               [--r53querylogs] [--s3logs] [--lblogs] [--cloudtrail]
                                [--single_r53querylogs] [--single_cloudtrail]
                                [--single_vpcflow] [--single_all]
-                               [--single_s3logs] [--single_account]
+                               [--single_s3logs] [--single_lblogs] [--single_account]
                                [--multi_account]
 
 Assisted Log Enabler - Find resources that are not logging, and turn them on.
@@ -203,6 +221,7 @@ Single & Multi Account Options:
   --vpcflow             Turns on Amazon VPC Flow Logs.
   --r53querylogs        Turns on Amazon Route 53 Resolver Query Logs.
   --s3logs              Turns on Amazon Bucket Logs.
+  --lblogs              Turns on Elastic Load Balancing Logs.
   --cloudtrail          Turns on AWS CloudTrail.
 
 Cleanup Options:
@@ -220,6 +239,8 @@ Cleanup Options:
                         Enabler for AWS.
   --single_s3logs       Removes Amazon Bucket Log resources created by
                         Assisted Log Enabler for AWS.
+  --single_lblogs       Removes Elastic Load Balancing Log resources created by
+                        Assisted Log Enabler for AWS.
 
 Dry Run Options:
   Use these flags to run Assisted Log Enabler for AWS in Dry Run mode.
@@ -256,8 +277,10 @@ python3 assisted_log_enabler.py --mode single_account --vpcflow
 python3 assisted_log_enabler.py --mode single_account --r53querylogs
 # For AWS CloudTrail:
 python3 assisted_log_enabler.py --mode single_account --cloudtrail
-# NEW! For Amazon S3 Server Access Logs:
+# For Amazon S3 Server Access Logs:
 python3 assisted_log_enabler.py --mode single_account --s3logs
+# NEW! For Elastic Load Balancing Access Logs:
+python3 assisted_log_enabler.py --mode single_account --lblogs
 ```
 
 ### Step-by-Step Instructions (for running in AWS CloudShell, multi account mode)
@@ -315,8 +338,10 @@ python3 assisted_log_enabler.py --mode multi_account --eks
 python3 assisted_log_enabler.py --mode multi_account --vpcflow
 # For Amazon Route 53 Resolver Query Logs:
 python3 assisted_log_enabler.py --mode multi_account --r53querylogs
-# NEW! For Amazon S3 Server Access Logs:
+For Amazon S3 Server Access Logs:
 python3 assisted_log_enabler.py --mode multi_account --s3logs
+# NEW! For Elastic Load Balancing Access Logs:
+python3 assisted_log_enabler.py --mode multi_account --lblogs
 
 ```
 
@@ -341,7 +366,7 @@ Sample output within the log file:
 2021-02-23 05:31:54,984 - INFO - Turning on audit and authenticator logging for EKS clusters in region af-south-1.
 ```
 
-## NEW! Dry Run Mode
+## Dry Run Mode
 Dry Run modes for single and multi-account are both available. These modes allow you to check for resources in your environment that do not have logging turned on, but does not activate the logging for said resources.
 
 To run Assisted Log Enabler for AWS in Dry Run mode, you can use the commands below:
@@ -366,8 +391,10 @@ python3 assisted_log_enabler.py --mode cleanup --single_r53querylogs
 python3 assisted_log_enabler.py --mode cleanup --single_vpcflow
 # To remove AWS CloudTrail trails created by Assisted Log Enabler for AWS (single account):
 python3 assisted_log_enabler.py --mode cleanup --single_cloudtrail
-# NEW! To remove Amazon S3 Server Access logging created by Assisted Log Enabler for AWS (single account):
+# To remove Amazon S3 Server Access logging created by Assisted Log Enabler for AWS (single account):
 python3 assisted_log_enabler.py --mode cleanup --single_s3logs
+# NEW! To remove Elastic Load Balancing Access logging created by Assisted Log Enabler for AWS (single account):
+python3 assisted_log_enabler.py --mode cleanup --single_lblogs
 ```
 
 ## Shared Responsibility Model
@@ -388,6 +415,7 @@ For answers to cost-related questions involved with this solution, refer to the
 * Amazon VPC Flow Logs Pricing: [Link](https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html#flow-logs-pricing)
 * Amazon Route 53 Pricing (look for the Route 53 Resolver Query Logs section): [Link](https://aws.amazon.com/route53/pricing/)
 * Amazon EKS Control Plane Logging: [Link](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html)
+* Elastic Load Balancing Logging: [Link](https://aws.amazon.com/elasticloadbalancing/pricing/)
 
 
 ## Feedback

assisted_log_enabler.py

@@ -82,6 +82,7 @@ def assisted_log_enabler():
     function_parser_group.add_argument('--vpcflow', action='store_true', help=' Turns on Amazon VPC Flow Logs.')
     function_parser_group.add_argument('--r53querylogs', action='store_true', help=' Turns on Amazon Route 53 Resolver Query Logs.')
     function_parser_group.add_argument('--s3logs', action='store_true', help=' Turns on Amazon Bucket Logs.')
+    function_parser_group.add_argument('--lblogs', action='store_true', help=' Turns on Amazon Load Balancer Logs.')
     function_parser_group.add_argument('--cloudtrail', action='store_true', help=' Turns on AWS CloudTrail. Only available in Single Account version.')
 
     cleanup_parser_group = parser.add_argument_group('Cleanup Options', 'Use these flags to choose which resources you want to turn logging off for.')
@@ -90,6 +91,7 @@ def assisted_log_enabler():
     cleanup_parser_group.add_argument('--single_vpcflow', action='store_true', help=' Removes Amazon VPC Flow Log resources created by Assisted Log Enabler for AWS.')
     cleanup_parser_group.add_argument('--single_all', action='store_true', help=' Turns off all of the log types within the Assisted Log Enabler for AWS.')
     cleanup_parser_group.add_argument('--single_s3logs', action='store_true', help=' Removes Amazon Bucket Log resources created by Assisted Log Enabler for AWS.')
+    cleanup_parser_group.add_argument('--single_lblogs', action='store_true', help=' Removes Amazon Load Balancer Log resources created by Assisted Log Enabler for AWS.')
 
     dryrun_parser_group = parser.add_argument_group('Dry Run Options', 'Use these flags to run Assisted Log Enabler for AWS in Dry Run mode.')
     dryrun_parser_group.add_argument('--single_account', action='store_true', help=' Runs Assisted Log Enabler for AWS in Dry Run mode for a single AWS account.')
@@ -109,6 +111,8 @@ def assisted_log_enabler():
             ALE_single_account.run_r53_query_logs()
         elif args.s3logs:
             ALE_single_account.run_s3_logs()
+        elif args.lblogs:
+            ALE_single_account.run_lb_logs()
         elif args.cloudtrail:
             ALE_single_account.run_cloudtrail()
         elif args.all:
@@ -124,6 +128,8 @@ def assisted_log_enabler():
             ALE_multi_account.run_r53_query_logs()
         elif args.s3logs:
             ALE_multi_account.run_s3_logs()
+        elif args.lblogs:
+            ALE_multi_account.run_lb_logs()
         elif args.all:
             ALE_multi_account.lambda_handler(event, context)
         else:
@@ -133,6 +139,8 @@ def assisted_log_enabler():
             ALE_cleanup_single.run_r53_cleanup()
         elif args.single_s3logs:
             ALE_cleanup_single.run_s3_cleanup()
+        elif args.single_lblogs:
+            ALE_cleanup_single.run_lb_cleanup()
         elif args.single_cloudtrail:
             ALE_cleanup_single.run_cloudtrail_cleanup()
         elif args.single_vpcflow:

diagrams/assisted_log_enabler_lb.png

@@ -59,6 +59,16 @@ Resources:
               - s3:PutBucketAcl
               - s3:PutBucketPublicAccessBlock
               - s3:PutBucketLifecycleConfiguration
+              - elb:DescribeLoadBalancers
+              - elb:DescribeLoadBalancerAttributes
+              - elb:ModifyLoadBalancerAttributes
+              - elbv2:DescribeLoadBalancers
+              - elbv2:DescribeLoadBalancerAttributes
+              - elbv2:ModifyLoadBalancerAttributes
+              - elasticloadbalancing:DescribeLoadBalancers
+              - elasticloadbalancing:DescribeLoadBalancerAttributes
+              - elasticloadbalancing:ModifyLoadBalancerAttributes
+              - eks:ListClusters
             Resource: '*'
             Condition:
               StringEquals:

permissions/ALE_child_account_role.yaml

@@ -34,7 +34,17 @@
                 "s3:GetBucketAcl",
                 "s3:PutBucketAcl",
                 "s3:PutBucketPublicAccessBlock",
-                "s3:PutBucketLifecycleConfiguration"
+                "s3:PutBucketLifecycleConfiguration",
+                "elb:DescribeLoadBalancers",
+                "elb:DescribeLoadBalancerAttributes",
+                "elb:ModifyLoadBalancerAttributes",
+                "elbv2:DescribeLoadBalancers",
+                "elbv2:DescribeLoadBalancerAttributes",
+                "elbv2:ModifyLoadBalancerAttributes"
+                "elasticloadbalancing:DescribeLoadBalancers",
+                "elasticloadbalancing:DescribeLoadBalancerAttributes",
+                "elasticloadbalancing:ModifyLoadBalancerAttributes",
+                "eks:ListClusters"
             ],
             "Resource": "*",
             "Condition": {
@@ -56,4 +66,4 @@
             }
         }
     ]
-}
+}

permissions/ALE_permissions_example_multi_account.json

@@ -35,7 +35,17 @@
                 "s3:GetBucketAcl",
                 "s3:PutBucketAcl",
                 "s3:PutBucketPublicAccessBlock",
-                "s3:PutBucketLifecycleConfiguration"
+                "s3:PutBucketLifecycleConfiguration",
+                "elb:DescribeLoadBalancers",
+                "elb:DescribeLoadBalancerAttributes",
+                "elb:ModifyLoadBalancerAttributes",
+                "elbv2:DescribeLoadBalancers",
+                "elbv2:DescribeLoadBalancerAttributes",
+                "elbv2:ModifyLoadBalancerAttributes"
+                "elasticloadbalancing:DescribeLoadBalancers",
+                "elasticloadbalancing:DescribeLoadBalancerAttributes",
+                "elasticloadbalancing:ModifyLoadBalancerAttributes",
+                "eks:ListClusters"
             ],
             "Resource": "*",
             "Condition": {
@@ -57,4 +67,4 @@
             }
         }
     ]
-}
+}

permissions/ALE_permissions_example_single_account.json

@@ -225,6 +225,82 @@ def s3_cleanup():
         except Exception as exception_handle:
             logging.error(exception_handle)
 
+# 5. Remove the Load Balancer Logging Resources created by Assisted Log Enabler
+def lb_cleanup():
+    """Function to clean up Load Balancer Logs"""
+    logging.info("Cleaning up Load Balancer Logs created by Assisted Log Enabler for AWS.")
+    for aws_region in region_list:
+        elbv1client = boto3.client('elb', region_name=aws_region)
+        elbv2client = boto3.client('elbv2', region_name=aws_region)
+        ELBList1: list = []
+        ELBList2: list = []
+        ELBv1LogList: list = []
+        ELBv2LogList: list = []
+        removal_list: list = []
+        try:
+            logging.info("---- LINE BREAK BETWEEN REGIONS ----")
+            logging.info("Cleaning up Bucket Logs created by Assisted Log Enabler for AWS in region " + aws_region + ".")
+            logging.info("DescribeLoadBalancers API Call")
+            ELBList1 = elbv1client.describe_load_balancers()
+            for lb in ELBList1['LoadBalancerDescriptions']:
+                logging.info("DescribeLoadBalancerAttibute API Call")
+                lblog=elbv1client.describe_load_balancer_attributes(LoadBalancerName=lb['LoadBalancerName'])
+                logging.info("Parsing out for ELB Access Logging")
+                if lblog['LoadBalancerAttributes']['AccessLog']['Enabled'] == True:
+                    if 'aws-lb-log-collection-' in str(lblog['LoadBalancerAttributes']['AccessLog']['S3BucketName']):
+                        ELBv1LogList.append([lb['LoadBalancerName'],'classic'])
+            logging.info("DescribeLoadBalancers v2 API Call")
+            ELBList2 = elbv2client.describe_load_balancers()
+            for lb in ELBList2['LoadBalancers']:
+                logging.info("DescribeLoadBalancerAttibute v2 API Call")
+                lblog=elbv2client.describe_load_balancer_attributes(LoadBalancerArn=lb['LoadBalancerArn'])
+                logging.info("Parsing out for ELBv2 Access Logging")
+                for lbtemp in lblog['Attributes']:
+                    if lbtemp['Key'] == 'access_logs.s3.enabled':
+                        if lbtemp['Value'] == 'true':
+                            for lbtemp2 in lblog['Attributes']:
+                                if lbtemp2['Key'] == 'access_logs.s3.bucket':
+                                    if 'aws-lb-log-collection-' in str(lbtemp2['Value']):
+                                        ELBv2LogList.append([lb['LoadBalancerName'],lb['LoadBalancerArn']])
+            removal_list=ELBv1LogList+ELBv2LogList   
+            if removal_list != []:
+                logging.info("List Load Balancers with Logging enabled by by Assisted Log Enabler for AWS in " + aws_region)
+                print(removal_list)
+                for elb in removal_list:
+                    logging.info(elb[0] + " has Load Balancer logging on. It will be turned on within this function.")
+                if ELBv1LogList != []:
+                    for elb in ELBv1LogList:
+                        logging.info("Removing logs for Load Balancer " + elb[0])
+                        logging.info("ModifyLoadBalancerAttributes API Call")
+                        remove_lb_log = elbv1client.modify_load_balancer_attributes(
+                            LoadBalancerName=elb[0],
+                            LoadBalancerAttributes={
+                                'AccessLog': {
+                                    'Enabled': False }
+                            }
+                        )
+                        logging.info("Logging Disabled for Load Balancer " + elb[0])
+                if ELBv2LogList != []:
+                    for elb in ELBv2LogList:
+                        logging.info("Removing logs for Load Balancer " + elb[0])
+                        logging.info("ModifyLoadBalancerAttributes v2 API Call")
+                        remove_lb_log = elbv2client.modify_load_balancer_attributes(
+                            LoadBalancerArn=elb[1],
+                            Attributes=[
+                                {
+                                    'Key': 'access_logs.s3.enabled',
+                                    'Value': 'false'
+                                }
+                            ]
+                        )
+                        logging.info("Logging Disabled for Load Balancer " + elb[0])
+                logging.info("Removed Load Balancers Logging created by Assisted Log Enabler for AWS.")
+                time.sleep(1)
+            else:
+                logging.info("There are no Load Balancers Logs set by Log Enabler in " + aws_region)
+        except Exception as exception_handle:
+            logging.error(exception_handle)
+
 def run_vpcflow_cleanup():
     """Function to run the vpcflow_cleanup function"""
     vpcflow_cleanup()
@@ -247,9 +323,18 @@ def run_s3_cleanup():
     s3_cleanup()
     logging.info("This is the end of the script. Please feel free to validate that logging resources have been cleaned up.")
 
+def run_lb_cleanup():
+    """Function to run the lb_cleanup function"""
+    lb_cleanup()
+    logging.info("This is the end of the script. Please feel free to validate that logging resources have been cleaned up.")
+
 def lambda_handler(event, context):
     """Function that runs all of the previously defined functions"""
     r53_cleanup()
+    vpcflow_cleanup()
+    cloudtrail_cleanup()
+    s3_cleanup()
+    lb_cleanup()
     logging.info("This is the end of the script. Please feel free to validate that logging resources have been cleaned up.")