Cloudops

Collection of different projects and learning experience around cloud technologies I'm experimenting with. This is a work in progress, and will be adding other projects soon.

• cloudops/public-blog/: Python Flask blog (https://www.devopsinthe.cloud) project that is deployed on AWS EKS. The entire infrastructure is created using the AWS CDK.
• /aws-cdk/nj-driver-license/: AWS CDK Project and Python application. This projects uses Selenium to scrape New Jersey's DMV site and look for available appointments for out of state driver license change. The code will run on a schedule of every 10 minutes and use AWS SNS to send a notification to the user once and available appointment is found.
  • The CDK project creates the infrastructure for the application code to run. Here's we create a VPC, SNS topic, ECS Cluster, and Scheduled ECS Fargate task definition. This will provide a serverless environment that requires no intervention to operate.
• cloudops/aws-scripts/: Contains a collection of scripts to automate several aspects of operating in AWS.
  • /ebs_snapshot_inventory_and_del: This script will scan an AWS Account looking for EC2 EBS Snapshots. It will create a DynamoDB table and store the snapshot information (snapshot_id, snapshot_age, volume_size, snapshot_state and date_taken). It will create a report for snapshots that are older than certain amount of time, for the customer to decide if snapshots older that selected age should be deleted.
  • /s3-incomplete-mpu: Get S3 buckets in an AWS account and apply a incomplete multi-part uploads (MPU) Lifecycle rule to cleanup orphaned incomplete MPUs after 7 days. It extends the ideas covered in this blog. The README goes in detail on how the logic works.
    • /s3-incomplete-mpu/s3-mpu-cdk: CDK project that automates the above idea by creating an event-driven and serverless architecture to detect when new buckets are created to apply the rules automatically, with no human intervention.
  • /get_tagged_resource: This script uses AWS Python SDK boto3 to scan all regions for a specific account and pull all resources are tagged or were previously tagged using the Resource Groups and Tagging API.
  • /service_quotas: This script, given two or more accounts will pull the quotas for a given service and compare the difference between the accounts. The idea is to be able to quickly compare the difference between two or more accounts when, for example, you are promoting your application between stages (Dev, Test, QA and Prod) to ensure the limits match and the application doesn't brake to a mismatch in AWS service limits.
• cloudops/notify_new_eks_ami: Script to automate the process of notifying when a new version of the EKS Managed Nodegroup AMI is released using Boto3 and Amazon EventBridge.