Skip to content

Do not deprecate Botocore Session in upcoming release (0.8)ย #1104

Closed
@BTheunissen

Description

@BTheunissen

Feature Request / Improvement

The AWS parameter botocore_session has been flagged as deprecated as of #922, and is due to be removed at Milestone 0.8.

I'd like to request that this parameter is not deprecated, and I'd be happy to add a PR to bring the credential name in-line with the rest of the updated client configuration. botocore_session is helpful to make available to override in order to support automatically refreshable credentials for long-running jobs.

For example in my project I have the following boto3 utility code:

from boto3 import Session
from botocore.credentials import (
    AssumeRoleCredentialFetcher,
    Credentials,
    DeferredRefreshableCredentials,
)
from botocore.session import Session as BotoSession

def get_refreshable_botocore_session(
    source_credentials: Credentials | None,
    assume_role_arn: str,
    role_session_name: str | None = None,
) -> BotoSession:
    """Get a refreshable botocore session for assuming a role."""
    if source_credentials is not None:
        boto3_session = Session(
            aws_access_key_id=source_credentials.access_key,
            aws_secret_access_key=source_credentials.secret_key,
            aws_session_token=source_credentials.token,
        )
    else:
        boto3_session = Session()

    extra_args = {}
    if role_session_name:
        extra_args["RoleSessionName"] = role_session_name
    fetcher = AssumeRoleCredentialFetcher(
        client_creator=boto3_session.client,
        source_credentials=source_credentials,
        role_arn=assume_role_arn,
        extra_args={},
    )
    refreshable_credentials = DeferredRefreshableCredentials(
        method="assume-role",
        refresh_using=fetcher.fetch_credentials,
    )
    botocore_session = BotoSession()
    botocore_session._credentials = refreshable_credentials  # noqa: SLF001
    return botocore_session

Which can be used as follows:

credentials = Credentials(
    access_key=client_access_key_id,
    secret_key=client_secret_access_key,
    token=client_session_token,
)
botocore_session = get_refreshable_botocore_session(
    source_credentials=credentials,
    assume_role_arn=self.config["client_iam_role_arn"],
)
catalog_properties["botocore_session"] = botocore_session
load_catalog(**catalog_properties)

This allows the user to elapse over the IAM role-chaining limitation of 1 hour, very useful for reading extremely large tables.

I'd also like to contribute some of this code upstream at some point to support refreshable botocore sessions in both the AWS Glue/DynamoDB clients, as well as the underlying S3 file system code.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions