Fix AWS account create retry for transient service activation errors - #144
Merged
bshutterkion merged 2 commits intoMar 30, 2026
Merged
Conversation
Newly created AWS accounts may not have all services (e.g. CloudFormation) fully activated when the provider attempts to move them from the cache to a project. The convert call now retries on transient AWS errors including OptInRequired, SubscriptionRequiredException, InvalidClientTokenId, ServiceUnavailable, ThrottlingException, and InternalFailure. Retry attempts increased from 3 to 6 (~3 min total) to accommodate propagation delays.
Account creation now has two phases: first wait for the account number
to be assigned, then poll /v3/account-cache/{id}/status to verify the
account is actually accessible in AWS before attempting the move. This
prevents failures caused by AWS service activation delays (e.g.
CloudFormation not yet available) on newly created accounts.
The cache-to-project conversion also now retries all errors until the
Terraform create timeout expires rather than failing after 3 fixed
attempts.
egramens
approved these changes
Mar 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/v3/account-cache/{id}until the account number is assigned/v3/account-cache/{id}/statusuntil the account is actually accessible in AWS (verifies the service role can be assumed and APIs called)AccountCacheStatusResponsemodel for the status endpointContext
A customer creating a new AWS account and project simultaneously via Terraform was hitting:
The account was created successfully in the cache, but the move to the project failed because CloudFormation wasn't yet available in the brand-new account. Manual moves from the cache worked fine since enough time had passed.
The root cause is that AWS assigns an account number before all services are fully activated. The previous code only waited for the account number, then immediately attempted the convert. Now we also verify the account is accessible via the existing
/v3/account-cache/{id}/statusendpoint before proceeding.Test plan
OptInRequirederror was occurring