Skip to content

Latest commit

聽

History

History
59 lines (44 loc) 路 1.38 KB

File metadata and controls

59 lines (44 loc) 路 1.38 KB

Amazon S3

Traits

  • Same-region/cross-region replication (possible)
  • 99.99999999999% durability

Usecases

  • Data-lake
  • Analytics
  • Data-archiving
  • Static website-hosting
  • ...

Integration

  • Athena - Queries S3

Query with S3 Select

Query Functionality on top of S3.

  • SelectObjectContents - SDK
    • InputType
      • SQL
  • DataFormats
    • CSV
    • JSON
    • Parquet
  • Compression
    • GZIP
    • BZIP2 for CSV/JSON
    • GZIP Columnar for Parquet
  • Encoding
    • UTF-8

Storage Classes

  • Standard - Frequently accessed data
  • Standard IA - Long-lived, infrequently accessed (but still ms response when needed)
  • Intelligent Tiering - Changing or unknown pattern
  • One-Zone IA - Non-critical, infrequently accessed data (usually that can be recreated)
  • Glacier - Long-term data-archiving - minutes to hours of retrieval time
  • Glacier Deep Archive - Rarely accessed data - 12h+ retrieval time
  • RRS - Frequently accessed non-critical data (usually that can be recreated)

Life-cycle Policy

  • Lifecycle rules to graduate files to colder storage classes e.g.: from Standard to Glacier
  • Transition always goes one-way only, from Standard -> to colder classes
  • Encrypted objects remain encrypted
  • Archived objects can't use life-cycle policy

Replication

  • Copies objects asynchronously across S3 buckets
  • CRR - Cross Region Replication
  • SRR - Same Region Replication