Skip to content

Automated archiving, tooling, aggressive storage cleanup #377

Description

  • create the archive storage location on RDF-A
  • Capability to archive any level or layer between two dates to RDF-A
  • Capability to restore from archive any level between two dates
  • Capability to delete based on level or layer, content data and latest or not - "delete l2pre for Dec 2025" or "delete all but latest layer for jan 3rd"
  • L1D (all reference frames) deleted from BOX and server after 6  months. Not archived
  • everything else archived to RDF-A automatically
  • L1B & C deleted from servers (already in archive) after 6m but can easily be restored
  • l2pre archived automatically and deleted when mhairi says so
  • l2 synced to box and archive, then can be deleted from servers
  • everything else (prob except spice) archived & deleted from servers after 1 yr

Expect calibration to use storage ~35GB per month

  • 1GB per day per run being calibrated if 64/8. (Assumes L2Pre in GSE only)
  • 1.8GB per day per run being calibrated if 128/128

Spice is grows 70GB PA

Metadata

Metadata

Labels

featureAdding a new functionality, small or large

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions