Skip to content

Expand AWS provider: 7 services, 24 ARN→typed conversions, Step Functions, Batch, AppMesh#7161

Closed
tas50 wants to merge 9 commits intomainfrom
expand-sqs-sns-ecr-efs-lambda-resources
Closed

Expand AWS provider: 7 services, 24 ARN→typed conversions, Step Functions, Batch, AppMesh#7161
tas50 wants to merge 9 commits intomainfrom
expand-sqs-sns-ecr-efs-lambda-resources

Conversation

@tas50
Copy link
Copy Markdown
Member

@tas50 tas50 commented Apr 12, 2026

Summary

Commit 1: SQS, SNS, ECR, EFS, Lambda expansion

  • SQS: tags, contentBasedDeduplication, redriveAllowPolicy
  • SNS: fifoTopic, contentBasedDeduplication, dataProtectionPolicy, tracingConfig, deliveryPolicy, displayName
  • ECR: Repository tags, typed aws.ecr.image.scanFinding with scanStatus/scanFindings/scanFindingSeverityCounts, aws.ecr.scanningConfiguration with rules
  • EFS: performanceMode, throughputMode, sizeInBytes, lifecycleState, typed lifecycleConfiguration, typed replicationConfiguration with destinations, fileSystemProtection
  • Lambda: versions() → typed aws.lambda.function.version; versions() → typed aws.lambda.layer.version

Commit 2: KMS and CloudTrail expansion

  • KMS: rotationPeriodInDays, nextRotationDate, onDemandRotationStartDate, encryptionAlgorithms, signingAlgorithms, keyAgreementAlgorithms, expirationModel, validTo. Grant key() typed ref (deprecates keyArn)
  • CloudTrail: snsTopic(), cloudWatchLogsRole() typed refs. latestDeliveredAt, latestNotifiedAt, latestCloudWatchLogsDeliveredAt, latestDigestDeliveredAt (deprecate *Time names). latestDeliveryAttemptedAt, latestDeliveryAttemptSucceededAt, latestNotificationAttemptedAt, latestNotificationAttemptSucceededAt. Event data store federationStatus, federationRole() typed ref

Commit 3: Review fixes

  • convert.JsonToDict() for SQS redriveAllowPolicy and SNS dataProtectionPolicy
  • ECR public image detection via cachePublic instead of registryId == ""
  • Spell-check: RSAES, RSASSA

Commit 4: ARN→typed conversions + Batch/AppMesh/StepFunctions

24 ARN→typed reference conversions (deprecate old, keep for compat):

Service Old Field New Typed Field
ECS container taskDefinitionArn taskDefinition()
ECS container taskArn task()
ECS service clusterArn cluster()
ECS taskSet clusterArn cluster()
ECS taskSet serviceArn service()
Backup vault encryptionKeyArn encryptionKey()
Backup recoveryPoint iamRoleArn iamRole()
Backup recoveryPoint encryptionKeyArn encryptionKey()
Backup copyAction destinationBackupVaultArn destinationVault()
CloudWatch sub filter roleArn iamRole()
Route53 queryLogging cloudWatchLogsLogGroupArn logGroup()
Config delivery channel snsTopicARN snsTopic()
Neptune instance monitoringRoleArn monitoringRole()
DRS repl config ebsEncryptionKeyArn ebsEncryptionKey()
API Gateway stage webAclArn webAcl()
Lambda ESM functionArn function()
ELB listener loadBalancerArn loadBalancer()
CodeDeploy group serviceRoleArn serviceRole()
AppStream fleet iamRoleArn iamRole()
AppStream imageBuilder iamRoleArn iamRole()
Bedrock customModel baseModelArn baseModel()
Bedrock provisioned foundationModelArn foundationModel()
WorkspacesWeb logging kinesisStreamArn kinesisStream()

Batch expansion — dict fields → typed sub-resources:

  • aws.batch.jobDefinition.containerProperties — image, vcpus, memory, command, jobRole/executionRole typed refs, environment, privileged, readonlyRootFilesystem, resourceRequirements
  • aws.batch.jobDefinition.retryStrategy — attempts, evaluateOnExit
  • aws.batch.jobDefinition.timeout — attemptDurationSeconds

AppMesh expansion — new resources:

  • aws.appmesh.virtualRouter — listeners, routes, tags
  • aws.appmesh.route — spec, tags
  • virtualNode.backends changed from count (int) to actual backend data ([]dict)

Step Functions — new service from scratch:

  • aws.stepfunctions.stateMachine — arn, name, type, status, definition, iamRole() typed ref, loggingConfiguration, tracingEnabled, tags
  • aws.stepfunctions.stateMachine.loggingConfiguration — level, includeExecutionData, destinations

New API calls

Service API Purpose
SQS ListQueueTags Queue resource tags
SNS GetDataProtectionPolicy PII policies
ECR ListTagsForResource, DescribeImageScanFindings, GetRegistryScanningConfiguration Tags, scan findings, scan config
EFS DescribeLifecycleConfiguration, DescribeReplicationConfigurations Lifecycle, replication
Lambda ListVersionsByFunction, ListLayerVersions Version history
AppMesh ListVirtualRouters, DescribeVirtualRouter, ListRoutes, DescribeRoute Virtual routers and routes
Step Functions ListStateMachines, DescribeStateMachine, ListTagsForResource New service

KMS, CloudTrail, and all ARN→typed conversions require zero new API calls.

Test plan

  • make providers/build/aws
  • SQS: mql run aws -c "aws.sqs.queues { tags contentBasedDeduplication }"
  • SNS: mql run aws -c "aws.sns.topics { fifoTopic dataProtectionPolicy }"
  • ECR: mql run aws -c "aws.ecr.privateRepositories { tags images { scanStatus } }"
  • EFS: mql run aws -c "aws.efs.filesystems { performanceMode lifecycleConfiguration replicationConfiguration }"
  • Lambda: mql run aws -c "aws.lambda.functions { versions { version state } }"
  • KMS: mql run aws -c "aws.kms.keys { rotationPeriodInDays encryptionAlgorithms }"
  • CloudTrail: mql run aws -c "aws.cloudtrail.trails { snsTopic cloudWatchLogsRole latestDeliveredAt }"
  • Batch: mql run aws -c "aws.batch.jobDefinitions { containerProperties { image vcpus jobRole } }"
  • AppMesh: mql run aws -c "aws.appmesh.meshes { virtualRouters { routes } }"
  • Step Functions: mql run aws -c "aws.stepfunctions.stateMachines { name type iamRole loggingConfiguration }"
  • ECS typed refs: mql run aws -c "aws.ecs.containers { taskDefinition task }"
  • Backup typed refs: mql run aws -c "aws.backup.vaults { encryptionKey }"

🤖 Generated with Claude Code

Add security-relevant fields and typed sub-resources across five AWS
services to improve visibility into queue policies, container
vulnerabilities, file system replication, and function versioning.

SQS: tags, contentBasedDeduplication, redriveAllowPolicy
SNS: fifoTopic, contentBasedDeduplication, dataProtectionPolicy,
     tracingConfig, deliveryPolicy, displayName
ECR: repository tags, image scan findings (typed), scan status,
     severity counts, registry scanning configuration (typed)
EFS: performanceMode, throughputMode, sizeInBytes, lifecycleState,
     lifecycle configuration (typed), replication configuration
     (typed with destinations), file system protection
Lambda: function versions (typed), layer versions (typed)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New security-focused fields for SQS, SNS, ECR, EFS, and Lambda are well-implemented overall, but two dict-returning methods skip type conversion and the public-image detection for ECR scan findings is fragile.

KMS key: Add rotation details (rotationPeriodInDays, nextRotationDate,
onDemandRotationStartDate) from cached GetKeyRotationStatus. Add
algorithm fields (encryptionAlgorithms, signingAlgorithms,
keyAgreementAlgorithms) and key material expiration (expirationModel,
validTo) from cached DescribeKey. Add key() typed reference on grants,
deprecate keyArn.

CloudTrail trail: Add snsTopic() typed aws.sns.topic reference,
deprecate snsTopicARN. Add cloudWatchLogsRole() typed aws.iam.role
reference, deprecate cloudWatchLogsRoleArn. Add delivery and
notification attempt fields (latestDeliveryAttemptedAt,
latestDeliveryAttemptSucceededAt, latestNotificationAttemptedAt,
latestNotificationAttemptSucceededAt). Add new *At naming convention
for existing time fields (latestDeliveredAt, latestNotifiedAt,
latestCloudWatchLogsDeliveredAt, latestDigestDeliveredAt), deprecate
old *Time names.

CloudTrail event data store: Add federationStatus and federationRole()
typed aws.iam.role reference from cached GetEventDataStore.

Zero new API calls — all fields from already-cached responses.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@tas50 tas50 changed the title Expand SQS, SNS, ECR, EFS, and Lambda with security-focused resources Expand SQS, SNS, ECR, EFS, Lambda, KMS, and CloudTrail with security-focused resources Apr 12, 2026
@mondoo-code-review mondoo-code-review bot dismissed their stale review April 12, 2026 04:34

Superseded by new review

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New KMS and CloudTrail fields look correct; three pre-existing issues in SQS, SNS, and ECR remain unaddressed.

@github-actions

This comment has been minimized.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 12, 2026

Test Results

6 502 tests  ±0   6 498 ✅ ±0   2m 8s ⏱️ -1s
  459 suites ±0       4 💤 ±0 
   38 files   ±0       0 ❌ ±0 

Results for commit 7857e4d. ± Comparison against base commit 78fcbcf.

♻️ This comment has been updated with latest results.

…elling

- SQS redriveAllowPolicy: use convert.JsonToDict instead of raw unmarshal
- SNS dataProtectionPolicy: use convert.JsonToDict instead of raw unmarshal
- ECR scan findings: cache public flag on image Internal struct instead of
  checking registryId=="" which is unreliable for public images
- Add RSAES and RSASSA to spell-check expect.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three previous review findings have been properly addressed.

ARN→typed reference conversions (24 fields across 15 services):
- ECS: taskDefinition(), task() on container; cluster() on service/taskSet; service() on taskSet
- Backup: encryptionKey() on vault/recoveryPoint; iamRole() on recoveryPoint; destinationVault() on copyAction
- CloudWatch: iamRole() on subscription filter
- Route53: logGroup() on queryLoggingConfig
- Config: snsTopic() on delivery channel
- Neptune: monitoringRole() on instance
- DRS: ebsEncryptionKey() on replicationConfiguration
- API Gateway: webAcl() on stage
- Lambda: function() on eventSourceMapping
- ELB: loadBalancer() on listener
- CodeDeploy: serviceRole() on deploymentGroup
- AppStream: iamRole() on fleet and imageBuilder
- Bedrock: baseModel() on customModel; foundationModel() on provisionedModelThroughput
- WorkspacesWeb: kinesisStream() on userAccessLoggingSetting

All old ARN fields deprecated with comments, kept for backwards compat.

Batch expansion:
- containerProperties: typed resource with image, vcpus, memory, command,
  jobRole/executionRole typed refs, environment, privileged, readonlyRootFilesystem
- retryStrategy: typed resource with attempts and evaluateOnExit
- timeout: typed resource with attemptDurationSeconds

AppMesh expansion:
- New aws.appmesh.virtualRouter resource with listeners and routes
- New aws.appmesh.route resource with spec and tags
- virtualNode.backends changed from count to []dict with actual backend data
- New APIs: ListVirtualRouters, DescribeVirtualRouter, ListRoutes, DescribeRoute

Step Functions (new service):
- aws.stepfunctions with stateMachines() discovery
- aws.stepfunctions.stateMachine with definition, iamRole() typed ref,
  loggingConfiguration, tracingEnabled, tags
- aws.stepfunctions.stateMachine.loggingConfiguration sub-resource
- APIs: ListStateMachines, DescribeStateMachine, ListTagsForResource

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@tas50 tas50 changed the title Expand SQS, SNS, ECR, EFS, Lambda, KMS, and CloudTrail with security-focused resources Expand AWS provider: 7 services, 24 ARN→typed conversions, Step Functions, Batch, AppMesh Apr 12, 2026
@github-actions

This comment has been minimized.

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Queries using aws.appmesh.virtualNode.backends will break due to type change from integer to array.

- AppMesh virtualNode.backends: restore as int (count), add new
  backendServices() []dict for actual backend data. Deprecate backends().
- Add stepfunctions, vcpus to spell-check expect.txt
- Remove Vcpus from expect.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Breaking type change on backends properly resolved by deprecating it and adding backendServices; silent error discards in Step Functions and Batch remain.

ElastiCache cluster: add tags() via ListTagsForResource API
Timestream LiveAnalytics database: add tags() via ListTagsForResource API

MQ broker and Glue catalog encryption were already implemented.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@mondoo-code-review mondoo-code-review bot dismissed their stale review April 12, 2026 05:29

Superseded by new review

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tags added to ElastiCache and Timestream resources follow existing patterns correctly; two prior warnings in unrelated files remain unaddressed.

tas50 and others added 3 commits April 11, 2026 22:43
Both aws.rds.dbinstance and aws.rds.dbcluster now expose
performanceInsightsRetentionPeriod (int) from the already-fetched
DescribeDBInstances/DescribeDBClusters responses. No new API calls.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Batch containerProperties: propagate errors from 5 JsonToDict/DictSlice
calls instead of silently discarding them.
Batch retryStrategy: same for evaluateOnExit conversion.
Step Functions loggingConfiguration: propagate error from destination
conversion instead of appending nil.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add fallback format "2006-01-02T15:04:05.000-0700" to parseAwsTimestamp
for AWS APIs (e.g., Lambda layers) that return timestamps with non-RFC3339
timezone offsets like "+0000" instead of "+00:00".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@tas50
Copy link
Copy Markdown
Member Author

tas50 commented Apr 12, 2026

/review

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Large schema expansion adding typed resource references and new services; batch job definition fields changed from dict to typed resources which may break existing queries.

Comment on lines 11479 to 11481
// Tags for the job definition
tags map[string]string
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 warningcontainerProperties(), retryStrategy(), and timeout() changed from returning dict to typed resources. This is a breaking change for any existing MQL queries that use dict-style access on these fields (e.g., aws.batch.jobDefinition { containerProperties["image"] }). If this is intentional, consider noting it in release notes. If not, the old dict return types should be preserved alongside the new typed resources (similar to how ARN→typed conversions elsewhere keep the old string field with a deprecation comment).

@tas50 tas50 closed this Apr 12, 2026
@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant