Releases: aws/sagemaker-spark-container
3.5-cpu-py312-v1.0
Release Notes
This image is built against Apache Spark 3.5.0, EMR 7.0.0 libs, and updated to use Python 3.12.11 and is intended for use in SageMaker Processing Jobs.
Images Published
314815235551.dkr.ecr.us-east-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
153931337802.dkr.ecr.us-west-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
446299261295.dkr.ecr.ca-central-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
750251592176.dkr.ecr.me-south-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
173754725891.dkr.ecr.us-east-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
440695851116.dkr.ecr.ap-southeast-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
860869212795.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
836651553127.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
732049463269.dkr.ecr.ap-east-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
571004829621.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
411782140378.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
906073651304.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
667973535471.dkr.ecr.us-west-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
759080221371.dkr.ecr.ap-southeast-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
105495057255.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
136845547031.dkr.ecr.eu-west-3.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
330188676905.dkr.ecr.eu-north-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
737130764395.dkr.ecr.sa-east-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
309385258863.dkr.ecr.af-south-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
753923664805.dkr.ecr.eu-south-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
102471314380.dkr.ecr.ap-northeast-3.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
395420993607.dkr.ecr.me-central-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
142351485170.dkr.ecr.eu-central-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
833944533722.dkr.ecr.eu-south-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
873151114052.dkr.ecr.ap-south-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
819679513684.dkr.ecr.ap-southeast-4.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
471112967968..dkr.ecr.ap-southeast-7.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
800295151634.dkr.ecr.ap-southeast-3.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
211125459255.dkr.ecr.mx-central-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
408426139102.dkr.ecr.il-central-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
000907499111.dkr.ecr.ca-west-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
271483468897.dkr.ecr.us-gov-west-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
260923028637.dkr.ecr.us-gov-east-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.0
844356804704.dkr.ecr.cn-northwest-1.amazonaws.com.cn/sagemaker-spark-processing:3.5-cpu-py312-v1.0
671472414489.dkr.ecr.cn-north-1.amazonaws.com.cn/sagemaker-spark-processing:3.5-cpu-py312-v1.0
Image Hash:
sha256:072f9595a2cc469769f0d92a6aed85954770ddc8637ae3131f2585b735e86e82
All Tags:
sagemaker-spark-processing:3.5-cpu-py312-v1.0
sagemaker-spark-processing:3.5-cpu-py312-v1
sagemaker-spark-processing:3.5-cpu
Relevant PRs:
https://github.com/aws/sagemaker-spark-container/pull/161
3.5-cpu-py39-v1.0
Release Notes
This image is built against Apache Spark 3.5.0 and EMR 7.0.0 libs and intended for use in SageMaker Processing Jobs.
Images Published
173754725891.dkr.ecr.us-east-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
153931337802.dkr.ecr.us-west-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
314815235551.dkr.ecr.us-east-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
667973535471.dkr.ecr.us-west-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
411782140378.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
860869212795.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
102471314380.dkr.ecr.ap-northeast-3.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
732049463269.dkr.ecr.ap-east-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
105495057255.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
873151114052.dkr.ecr.ap-south-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
759080221371.dkr.ecr.ap-southeast-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
440695851116.dkr.ecr.ap-southeast-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
800295151634.dkr.ecr.ap-southeast-3.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
819679513684.dkr.ecr.ap-southeast-4.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
395420993607.dkr.ecr.me-central-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
750251592176.dkr.ecr.me-south-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
737130764395.dkr.ecr.sa-east-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
446299261295.dkr.ecr.ca-central-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
906073651304.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
142351485170.dkr.ecr.eu-central-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
330188676905.dkr.ecr.eu-north-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
833944533722.dkr.ecr.eu-south-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
571004829621.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
836651553127.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
136845547031.dkr.ecr.eu-west-3.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
309385258863.dkr.ecr.af-south-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
753923664805.dkr.ecr.eu-south-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
844356804704.dkr.ecr.cn-northwest-1.amazonaws.com.cn/sagemaker-spark-processing:3.5-cpu-py39-v1.0
671472414489.dkr.ecr.cn-north-1.amazonaws.com.cn/sagemaker-spark-processing:3.5-cpu-py39-v1.0
260923028637.dkr.ecr.us-gov-east-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
271483468897.dkr.ecr.us-gov-west-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
Image Hash:
sha256:4a11a29846875ad9aec9865c3e85a8c926d3e0db7dfba341737a73a324b17a71
All Tags:
sagemaker-spark-processing:3.5-cpu-py39-v1.0
sagemaker-spark-processing:3.5-cpu-py39-v1
sagemaker-spark-processing:3.5-cpu
Relevant PRs:
https://github.com/aws/sagemaker-spark-container/pull/148
https://github.com/aws/sagemaker-spark-container/pull/150
3.4-cpu-py39-v1.1
Release Notes
This image is built against Apache Spark 3.4.1 and EMR 6.15.0 libs and intended for use in SageMaker Processing Jobs.
Images Published
173754725891.dkr.ecr.us-east-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
153931337802.dkr.ecr.us-west-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
314815235551.dkr.ecr.us-east-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
667973535471.dkr.ecr.us-west-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
411782140378.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
860869212795.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
102471314380.dkr.ecr.ap-northeast-3.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
732049463269.dkr.ecr.ap-east-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
105495057255.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
873151114052.dkr.ecr.ap-south-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
759080221371.dkr.ecr.ap-southeast-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
440695851116.dkr.ecr.ap-southeast-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
800295151634.dkr.ecr.ap-southeast-3.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
819679513684.dkr.ecr.ap-southeast-4.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
395420993607.dkr.ecr.me-central-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
750251592176.dkr.ecr.me-south-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
737130764395.dkr.ecr.sa-east-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
446299261295.dkr.ecr.ca-central-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
906073651304.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
142351485170.dkr.ecr.eu-central-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
330188676905.dkr.ecr.eu-north-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
833944533722.dkr.ecr.eu-south-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
571004829621.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
836651553127.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
136845547031.dkr.ecr.eu-west-3.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
309385258863.dkr.ecr.af-south-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
753923664805.dkr.ecr.eu-south-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
844356804704.dkr.ecr.cn-northwest-1.amazonaws.com.cn/sagemaker-spark-processing:3.4-cpu-py39-v1.1
671472414489.dkr.ecr.cn-north-1.amazonaws.com.cn/sagemaker-spark-processing:3.4-cpu-py39-v1.1
260923028637.dkr.ecr.us-gov-east-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
271483468897.dkr.ecr.us-gov-west-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.1
Image Hash:
sha256:39368c3056a9a7ae9a260431b20a5df7c2620d4f650f5a826a2bcae192f61fb4
All Tags:
sagemaker-spark-processing:3.4-cpu-py39-v1.1
sagemaker-spark-processing:3.4-cpu-py39-v1
sagemaker-spark-processing:3.4-cpu
Relevant Pull Requests:
https://github.com/aws/sagemaker-spark-container/pull/141
https://github.com/aws/sagemaker-spark-container/pull/142
https://github.com/aws/sagemaker-spark-container/pull/143
https://github.com/aws/sagemaker-spark-container/pull/146
3.4-cpu-py39-v1.0
Release spark 3.4
173754725891.dkr.ecr.us-east-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
153931337802.dkr.ecr.us-west-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
314815235551.dkr.ecr.us-east-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
667973535471.dkr.ecr.us-west-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
411782140378.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
860869212795.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
732049463269.dkr.ecr.ap-east-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
105495057255.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
759080221371.dkr.ecr.ap-southeast-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
440695851116.dkr.ecr.ap-southeast-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
750251592176.dkr.ecr.me-south-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
737130764395.dkr.ecr.sa-east-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
446299261295.dkr.ecr.ca-central-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
906073651304.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
330188676905.dkr.ecr.eu-north-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
571004829621.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
836651553127.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
136845547031.dkr.ecr.eu-west-3.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
309385258863.dkr.ecr.af-south-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
753923664805.dkr.ecr.eu-south-1.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
844356804704.dkr.ecr.cn-northwest-1.amazonaws.com.cn/sagemaker-spark-processing:3.4-cpu-py39-v1.0
671472414489.dkr.ecr.cn-north-1.amazonaws.com.cn/sagemaker-spark-processing:3.4-cpu-py39-v1.0
800295151634.dkr.ecr.ap-southeast-3.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
102471314380.dkr.ecr.ap-northeast-3.amazonaws.com/sagemaker-spark-processing:3.4-cpu-py39-v1.0
3.3-cpu-py39-v1.2
[Beta] Add advanced spark configuration
This release contains changes to configure advanced spark configurations. The configurations can be enabled by providing environment variable as part of the job run. This has been tested only on small set of use cases and is not recommended for all customers. Once we get enough evidence for performance improvement for variety of workloads we will make it as default configuration.
173754725891.dkr.ecr.us-east-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
153931337802.dkr.ecr.us-west-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
314815235551.dkr.ecr.us-east-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
667973535471.dkr.ecr.us-west-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
411782140378.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
860869212795.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
732049463269.dkr.ecr.ap-east-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
105495057255.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
759080221371.dkr.ecr.ap-southeast-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
440695851116.dkr.ecr.ap-southeast-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
750251592176.dkr.ecr.me-south-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
737130764395.dkr.ecr.sa-east-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
446299261295.dkr.ecr.ca-central-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
906073651304.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
330188676905.dkr.ecr.eu-north-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
571004829621.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
836651553127.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
136845547031.dkr.ecr.eu-west-3.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
309385258863.dkr.ecr.af-south-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
753923664805.dkr.ecr.eu-south-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
844356804704.dkr.ecr.cn-northwest-1.amazonaws.com.cn/sagemaker-spark-processing:3.3-cpu-py39-v1.2
671472414489.dkr.ecr.cn-north-1.amazonaws.com.cn/sagemaker-spark-processing:3.3-cpu-py39-v1.2
800295151634.dkr.ecr.ap-southeast-3.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
102471314380.dkr.ecr.ap-northeast-3.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.2
3.3-cpu-py39-v1.1
- Pre-Install the FeatureStore Spark connector in the image
- Support running the container on SageMaker training platform
173754725891.dkr.ecr.us-east-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
153931337802.dkr.ecr.us-west-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
314815235551.dkr.ecr.us-east-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
667973535471.dkr.ecr.us-west-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
411782140378.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
860869212795.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
732049463269.dkr.ecr.ap-east-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
105495057255.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
759080221371.dkr.ecr.ap-southeast-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
440695851116.dkr.ecr.ap-southeast-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
750251592176.dkr.ecr.me-south-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
737130764395.dkr.ecr.sa-east-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
446299261295.dkr.ecr.ca-central-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
906073651304.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
330188676905.dkr.ecr.eu-north-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
571004829621.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
836651553127.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
136845547031.dkr.ecr.eu-west-3.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
309385258863.dkr.ecr.af-south-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
753923664805.dkr.ecr.eu-south-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
844356804704.dkr.ecr.cn-northwest-1.amazonaws.com.cn/sagemaker-spark-processing:3.3-cpu-py39-v1.1
671472414489.dkr.ecr.cn-north-1.amazonaws.com.cn/sagemaker-spark-processing:3.3-cpu-py39-v1.1
800295151634.dkr.ecr.ap-southeast-3.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
102471314380.dkr.ecr.ap-northeast-3.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.1
3.3-cpu-py39-v1.0
- releasing the patch version for 3.3-cpu-py39-v1.0 which upgrades spark version to 3.3 and EMR version to 6.10
173754725891.dkr.ecr.us-east-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
153931337802.dkr.ecr.us-west-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
314815235551.dkr.ecr.us-east-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
667973535471.dkr.ecr.us-west-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
411782140378.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
860869212795.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
732049463269.dkr.ecr.ap-east-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
105495057255.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
759080221371.dkr.ecr.ap-southeast-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
440695851116.dkr.ecr.ap-southeast-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
750251592176.dkr.ecr.me-south-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
737130764395.dkr.ecr.sa-east-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
446299261295.dkr.ecr.ca-central-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
906073651304.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
330188676905.dkr.ecr.eu-north-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
571004829621.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
836651553127.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
136845547031.dkr.ecr.eu-west-3.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
309385258863.dkr.ecr.af-south-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
753923664805.dkr.ecr.eu-south-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
271483468897.dkr.ecr.us-gov-west-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
844356804704.dkr.ecr.cn-northwest-1.amazonaws.com.cn/sagemaker-spark-processing:3.3-cpu-py39-v1.0
671472414489.dkr.ecr.cn-north-1.amazonaws.com.cn/sagemaker-spark-processing:3.3-cpu-py39-v1.0
800295151634.dkr.ecr.ap-southeast-3.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
102471314380.dkr.ecr.ap-northeast-3.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
395420993607.dkr.ecr.me-central-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
260923028637.dkr.ecr.us-gov-east-1.amazonaws.com/sagemaker-spark-processing:3.3-cpu-py39-v1.0
3.2-cpu-py39-v1.0
173754725891.dkr.ecr.us-east-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
153931337802.dkr.ecr.us-west-2.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
314815235551.dkr.ecr.us-east-2.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
667973535471.dkr.ecr.us-west-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
411782140378.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
860869212795.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
732049463269.dkr.ecr.ap-east-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
105495057255.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
759080221371.dkr.ecr.ap-southeast-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
440695851116.dkr.ecr.ap-southeast-2.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
750251592176.dkr.ecr.me-south-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
737130764395.dkr.ecr.sa-east-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
446299261295.dkr.ecr.ca-central-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
906073651304.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
330188676905.dkr.ecr.eu-north-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
571004829621.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
836651553127.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
136845547031.dkr.ecr.eu-west-3.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
309385258863.dkr.ecr.af-south-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
753923664805.dkr.ecr.eu-south-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
271483468897.dkr.ecr.us-gov-west-1.amazonaws.com/sagemaker-spark-processing:3.2-cpu-py39-v1.0
844356804704.dkr.ecr.cn-northwest-1.amazonaws.com.cn/sagemaker-spark-processing:3.2-cpu-py39-v1.0
671472414489.dkr.ecr.cn-north-1.amazonaws.com.cn/sagemaker-spark-processing:3.2-cpu-py39-v1.0
3.1-cpu-py37-v1.2
- This release fixes the python3 cli
Images Published
173754725891.dkr.ecr.us-east-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
153931337802.dkr.ecr.us-west-2.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
314815235551.dkr.ecr.us-east-2.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
667973535471.dkr.ecr.us-west-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
411782140378.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
860869212795.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
732049463269.dkr.ecr.ap-east-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
105495057255.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
759080221371.dkr.ecr.ap-southeast-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
440695851116.dkr.ecr.ap-southeast-2.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
750251592176.dkr.ecr.me-south-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
737130764395.dkr.ecr.sa-east-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
446299261295.dkr.ecr.ca-central-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
906073651304.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
330188676905.dkr.ecr.eu-north-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
571004829621.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
836651553127.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
136845547031.dkr.ecr.eu-west-3.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
309385258863.dkr.ecr.af-south-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
753923664805.dkr.ecr.eu-south-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
271483468897.dkr.ecr.us-gov-west-1.amazonaws.com/sagemaker-spark-processing:3.1-cpu-py39-v1.2
844356804704.dkr.ecr.cn-northwest-1.amazonaws.com.cn/sagemaker-spark-processing:3.1-cpu-py39-v1.2
671472414489.dkr.ecr.cn-north-1.amazonaws.com.cn/sagemaker-spark-processing:3.1-cpu-py39-v1.2
Image Hash:
sha256:397c0b7506124083a9f750ad7edc83ada6fff5388e64c0fda2e1c45e7473db7d
All Tags:
3.1-cpu-py39-v1.2
3.1-cpu-py39-v1
3.1-cpu
Python Packages
| pip package | version |
|---|---|
| attrs | 20.3.0 |
| bcrypt | 3.2.0 |
| black | 22.3.0 |
| boto3 | 1.21.33 |
| botocore | 1.24.46 |
| certifi | 2021.10.8 |
| cffi | 1.15.0 |
| charset-normalizer | 2.0.12 |
| click | 8.1.2 |
| coverage | 6.3.2 |
| cryptography | 36.0.2 |
| dill | 0.3.4 |
| distlib | 0.3.4 |
| distro | 1.7.0 |
| docker | 5.0.3 |
| docker-compose | 1.29.2 |
| dockerpty | 0.4.1 |
| docopt | 0.6.2 |
| dparse | 0.5.1 |
| execnet | 1.9.0 |
| filelock | 3.6.0 |
| flake8 | 4.0.1 |
| flake8-docstrings | 1.5.0 |
| google-pasta | 0.2.0 |
| idna | 3.3 |
| importlib-metadata | 4.11.3 |
| iniconfig | 1.1.1 |
| jmespath | 1.0.0 |
| jsonschema | 3.2.0 |
| mccabe | 0.6.1 |
| multiprocess | 0.70.12.2 |
| mypy | 0.942 |
| mypy-extensions | 0.4.3 |
| numpy | 1.22.2 |
| packaging | 21.3 |
| pandas | 1.4.2 |
| paramiko | 2.10.3 |
| pathos | 0.2.8 |
| pathspec | 0.9.0 |
| pathtools | 0.1.2 |
| pipenv | 2022.4.8 |
| platformdirs | 2.5.2 |
| pluggy | 1.0.0 |
| pox | 0.3.0 |
| ppft | 1.6.6.4 |
| protobuf | 3.20.1 |
| protobuf3-to-dict | 0.1.5 |
| psutil | 5.9.0 |
| py | 1.11.0 |
| pyasn1 | 0.4.8 |
| pycodestyle | 2.8.0 |
| pycparser | 2.21 |
| pydocstyle | 6.1.1 |
| pyflakes | 2.4.0 |
| PyNaCl | 1.5.0 |
| pyparsing | 3.0.8 |
| pyrsistent | 0.18.1 |
| pytest | 7.1.1 |
| pytest-cov | 2.10.0 |
| pytest-forked | 1.4.0 |
| pytest-parallel | 0.1.1 |
| pytest-rerunfailures | 10.2 |
| pytest-xdist | 2.5.0 |
| python-dateutil | 2.8.2 |
| python-dotenv | 0.20.0 |
| pytz | 2022.1 |
| PyYAML | 5.4.1 |
| requests | 2.27.1 |
| rsa | 4.3 |
| s3transfer | 0.5.2 |
| safety | 1.10.3 |
| sagemaker | 2.83.0 |
| six | 1.16.0 |
| smdebug-rulesconfig | 1.0.1 |
| snowballstemmer | 2.2.0 |
| tblib | 1.7.0 |
| tenacity | 8.0.1 |
| texttable | 1.6.4 |
| toml | 0.10.2 |
| tomli | 2.0.1 |
| types-requests | 2.27.16 |
| types-urllib3 | 1.26.13 |
| types-waitress | 2.0.6 |
| typing_extensions | 4.1.1 |
| urllib3 | 1.26.9 |
| virtualenv | 20.14.1 |
| virtualenv-clone | 0.5.7 |
| waitress | 2.1.1 |
| watchdog | 0.10.3 |
| websocket-client | 0.59.0 |
| zipp | 3.8.0 |
3.0-cpu-py39-v1.0
- This release aligns with EMR 6.2.1 release
- This release upgrade spark 3.0 to python 3.9
Images Published
173754725891.dkr.ecr.us-east-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
153931337802.dkr.ecr.us-west-2.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
314815235551.dkr.ecr.us-east-2.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
667973535471.dkr.ecr.us-west-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
411782140378.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
860869212795.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
732049463269.dkr.ecr.ap-east-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
105495057255.dkr.ecr.ap-south-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
759080221371.dkr.ecr.ap-southeast-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
440695851116.dkr.ecr.ap-southeast-2.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
750251592176.dkr.ecr.me-south-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
737130764395.dkr.ecr.sa-east-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
446299261295.dkr.ecr.ca-central-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
906073651304.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
330188676905.dkr.ecr.eu-north-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
571004829621.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
836651553127.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
136845547031.dkr.ecr.eu-west-3.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
309385258863.dkr.ecr.af-south-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
753923664805.dkr.ecr.eu-south-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
271483468897.dkr.ecr.us-gov-west-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu-py39-v1.0
844356804704.dkr.ecr.cn-northwest-1.amazonaws.com.cn/sagemaker-spark-processing:3.0-cpu-py39-v1.0
671472414489.dkr.ecr.cn-north-1.amazonaws.com.cn/sagemaker-spark-processing:3.0-cpu-py39-v1.0
Image Hash:
sha256:c4727649cab0ba418cc849c39776f2f3536556cddb130a126b75d9d7ee899cd1
All Tags:
3.0-cpu-py39-v1.0
3.0-cpu-py39-v1
3.0-cpu
Python Packages
| pip package | version |
|---|---|
| attrs | 20.3.0 |
| bcrypt | 3.2.0 |
| black | 22.3.0 |
| boto3 | 1.21.33 |
| botocore | 1.24.46 |
| certifi | 2021.10.8 |
| cffi | 1.15.0 |
| charset-normalizer | 2.0.12 |
| click | 8.1.2 |
| coverage | 6.3.2 |
| cryptography | 36.0.2 |
| dill | 0.3.4 |
| distlib | 0.3.4 |
| distro | 1.7.0 |
| docker | 5.0.3 |
| docker-compose | 1.29.2 |
| dockerpty | 0.4.1 |
| docopt | 0.6.2 |
| dparse | 0.5.1 |
| execnet | 1.9.0 |
| filelock | 3.6.0 |
| flake8 | 4.0.1 |
| flake8-docstrings | 1.5.0 |
| google-pasta | 0.2.0 |
| idna | 3.3 |
| importlib-metadata | 4.11.3 |
| iniconfig | 1.1.1 |
| jmespath | 1.0.0 |
| jsonschema | 3.2.0 |
| mccabe | 0.6.1 |
| multiprocess | 0.70.12.2 |
| mypy | 0.942 |
| mypy-extensions | 0.4.3 |
| numpy | 1.22.2 |
| packaging | 21.3 |
| pandas | 1.4.2 |
| paramiko | 2.10.3 |
| pathos | 0.2.8 |
| pathspec | 0.9.0 |
| pathtools | 0.1.2 |
| pipenv | 2022.4.8 |
| platformdirs | 2.5.2 |
| pluggy | 1.0.0 |
| pox | 0.3.0 |
| ppft | 1.6.6.4 |
| protobuf | 3.20.1 |
| protobuf3-to-dict | 0.1.5 |
| psutil | 5.9.0 |
| py | 1.11.0 |
| pyasn1 | 0.4.8 |
| pycodestyle | 2.8.0 |
| pycparser | 2.21 |
| pydocstyle | 6.1.1 |
| pyflakes | 2.4.0 |
| PyNaCl | 1.5.0 |
| pyparsing | 3.0.8 |
| pyrsistent | 0.18.1 |
| pytest | 7.1.1 |
| pytest-cov | 2.10.0 |
| pytest-forked | 1.4.0 |
| pytest-parallel | 0.1.1 |
| pytest-rerunfailures | 10.2 |
| pytest-xdist | 2.5.0 |
| python-dateutil | 2.8.2 |
| python-dotenv | 0.20.0 |
| pytz | 2022.1 |
| PyYAML | 5.4.1 |
| requests | 2.27.1 |
| rsa | 4.3 |
| s3transfer | 0.5.2 |
| safety | 1.10.3 |
| sagemaker | 2.83.0 |
| six | 1.16.0 |
| smdebug-rulesconfig | 1.0.1 |
| snowballstemmer | 2.2.0 |
| tblib | 1.7.0 |
| tenacity | 8.0.1 |
| texttable | 1.6.4 |
| toml | 0.10.2 |
| tomli | 2.0.1 |
| types-requests | 2.27.16 |
| types-urllib3 | 1.26.13 |
| types-waitress | 2.0.6 |
| typing_extensions | 4.1.1 |
| urllib3 | 1.26.9 |
| virtualenv | 20.14.1 |
| virtualenv-clone | 0.5.7 |
| waitress | 2.1.1 |
| watchdog | 0.10.3 |
| websocket-client | 0.59.0 |
| zipp | 3.8.0 |