Description
Feature Request / Improvement
The AWS parameter botocore_session
has been flagged as deprecated as of #922, and is due to be removed at Milestone 0.8.
I'd like to request that this parameter is not deprecated, and I'd be happy to add a PR to bring the credential name in-line with the rest of the updated client configuration. botocore_session
is helpful to make available to override in order to support automatically refreshable credentials for long-running jobs.
For example in my project I have the following boto3 utility code:
from boto3 import Session
from botocore.credentials import (
AssumeRoleCredentialFetcher,
Credentials,
DeferredRefreshableCredentials,
)
from botocore.session import Session as BotoSession
def get_refreshable_botocore_session(
source_credentials: Credentials | None,
assume_role_arn: str,
role_session_name: str | None = None,
) -> BotoSession:
"""Get a refreshable botocore session for assuming a role."""
if source_credentials is not None:
boto3_session = Session(
aws_access_key_id=source_credentials.access_key,
aws_secret_access_key=source_credentials.secret_key,
aws_session_token=source_credentials.token,
)
else:
boto3_session = Session()
extra_args = {}
if role_session_name:
extra_args["RoleSessionName"] = role_session_name
fetcher = AssumeRoleCredentialFetcher(
client_creator=boto3_session.client,
source_credentials=source_credentials,
role_arn=assume_role_arn,
extra_args={},
)
refreshable_credentials = DeferredRefreshableCredentials(
method="assume-role",
refresh_using=fetcher.fetch_credentials,
)
botocore_session = BotoSession()
botocore_session._credentials = refreshable_credentials # noqa: SLF001
return botocore_session
Which can be used as follows:
credentials = Credentials(
access_key=client_access_key_id,
secret_key=client_secret_access_key,
token=client_session_token,
)
botocore_session = get_refreshable_botocore_session(
source_credentials=credentials,
assume_role_arn=self.config["client_iam_role_arn"],
)
catalog_properties["botocore_session"] = botocore_session
load_catalog(**catalog_properties)
This allows the user to elapse over the IAM role-chaining limitation of 1 hour, very useful for reading extremely large tables.
I'd also like to contribute some of this code upstream at some point to support refreshable botocore sessions in both the AWS Glue/DynamoDB clients, as well as the underlying S3 file system code.