Releases: armadaproject/armada
Releases · armadaproject/armada
Release list
v0.22.1
Armada v0.22.1
For more info, head over to the docs page at https://armadaproject.io
Armada CLI
armadactl controls the Armada batch job queueing system and is used for interacting with the system.
The CLI can be downloaded for a specific OS & Architecture from the Assets section below.
NOTE: The OSX binary is packaged as an universal binary and should work on both Intel and Apple Silicon based Macs.
Docker images
Armada Bundle
docker pull gresearch/armada:0.22.1docker pull gresearch/armada:latest
Armada Lookout Bundle
docker pull gresearch/armada-lookout-bundle:0.22.1docker pull gresearch/armada-lookout-bundle:latest
Armada Full Bundle
docker pull gresearch/armada-full-bundle:0.22.1docker pull gresearch/armada-full-bundle:latest
Armada Server
docker pull gresearch/armada-server:0.22.1docker pull gresearch/armada-server:latest
Armada Executor
docker pull gresearch/armada-executor:0.22.1docker pull gresearch/armada-executor:latest
Armada Lookout
docker pull gresearch/armada-lookout:0.22.1docker pull gresearch/armada-lookout:latest
Armada Lookout Ingester
docker pull gresearch/armada-lookout-ingester:0.22.1docker pull gresearch/armada-lookout-ingester:latest
Armada Event Ingester
docker pull gresearch/armada-event-ingester:0.22.1docker pull gresearch/armada-event-ingester:latest
Armada Scheduler
docker pull gresearch/armada-scheduler:0.22.1docker pull gresearch/armada-scheduler:latest
Armada Scheduler Ingester
docker pull gresearch/armada-scheduler-ingester:0.22.1docker pull gresearch/armada-scheduler-ingester:latest
Armada Binoculars
docker pull gresearch/armada-binoculars:0.22.1docker pull gresearch/armada-binoculars:latest
armadactl
docker pull gresearch/armadactl:0.22.1docker pull gresearch/armadactl:latest
Changelog
Full Changelog: v0.22.0...v0.22.1
What's Changed
- Hot cold local dev by @dslear in #4899
- Fix cross-partition duplicate job rows in Lookout ingester (experimental hot/cold) by @mauriceyap in #4996
- Bump mage to 1.17.2 everywhere, guard dev:up against old mage versions, and fix localdev doc drift by @dejanzele in #4992
- Drop legacy submit_message & groups columns from jobs by @tgucks in #4984
- Fix RangeError when null dates passed to DateTimePicker by @mauriceyap in #4997
- Bump next from 15.5.10 to 15.5.18 in /website by @dependabot[bot] in #5003
Full Changelog: v0.21.7...v0.22.1
Contributors
mauriceyap, dejanzele, and 3 other contributors
Assets 14
v0.22.0
Armada v0.22.0
For more info, head over to the docs page at https://armadaproject.io
NOTICE
This release completes the move of job metadata from the jobs -> job_metadata table in the scheduler database.
This is not a breaking change but as part of the upgrade a forceful migration of data will happen moving groups and submit_message data from the jobs -> jobs_metadata table.
This will block all writes to the scheduler db during the migration, so if you have a large dataset this could cause a production outage during this migration.
- For timings on approximate outage time is shown in this PR #4984
To avoid outage, it is recommended to upgrade to a version between v0.21.1 and v0.21.7 and follow this migration:
- Update scheduler configuration to set jobMetadataMigrationPhase to dualWrite
- Update scheduler ingester configuration to set jobMetadataMigrationPhase to cutover
- Wait for all rows in your database to be updated
- This will only happen due to natural job churn, so you'll need to wait for all jobs in the system prior to the config updates to finish
- If waiting on all jobs to finish is impractical, the outage time is determined by the number of rows left to migrate and should be relatively quick even for relatively large datasets (see #4984 for timings)
- Upgrade to v0.22.0+
More detail:
#4906
https://github.com/armadaproject/armada/releases#release-v0.21.1
WARNING
jobMetadataMigrationPhase should not be set to cutover in the scheduler ingester before the jobMetadataMigrationPhase is set to dualWrite in the scheduler.
- Doing so will mean that the metadata will be written to the job_metadata table but the scheduler will only be looking at the jobs table
jobMetadataMigrationPhase should not be set to cutover in the scheduler before all jobs in the database have an entry in the job_metadata table
- Doing so will mean it will fail to load the podspec to send to the executor and therefore be unable to run those jobs
Armada CLI
armadactl controls the Armada batch job queueing system and is used for interacting with the system.
The CLI can be downloaded for a specific OS & Architecture from the Assets section below.
NOTE: The OSX binary is packaged as an universal binary and should work on both Intel and Apple Silicon based Macs.
Docker images
Armada Bundle
docker pull gresearch/armada:0.22.0docker pull gresearch/armada:latest
Armada Lookout Bundle
docker pull gresearch/armada-lookout-bundle:0.22.0docker pull gresearch/armada-lookout-bundle:latest
Armada Full Bundle
docker pull gresearch/armada-full-bundle:0.22.0docker pull gresearch/armada-full-bundle:latest
Armada Server
docker pull gresearch/armada-server:0.22.0docker pull gresearch/armada-server:latest
Armada Executor
docker pull gresearch/armada-executor:0.22.0docker pull gresearch/armada-executor:latest
Armada Lookout
docker pull gresearch/armada-lookout:0.22.0docker pull gresearch/armada-lookout:latest
Armada Lookout Ingester
docker pull gresearch/armada-lookout-ingester:0.22.0docker pull gresearch/armada-lookout-ingester:latest
Armada Event Ingester
docker pull gresearch/armada-event-ingester:0.22.0docker pull gresearch/armada-event-ingester:latest
Armada Scheduler
docker pull gresearch/armada-scheduler:0.22.0docker pull gresearch/armada-scheduler:latest
Armada Scheduler Ingester
docker pull gresearch/armada-scheduler-ingester:0.22.0docker pull gresearch/armada-scheduler-ingester:latest
Armada Binoculars
docker pull gresearch/armada-binoculars:0.22.0docker pull gresearch/armada-binoculars:latest
armadactl
docker pull gresearch/armadactl:0.22.0docker pull gresearch/armadactl:latest
Changelog
Other work
- 7e5aec7: Drop legacy submit_message & groups columns from jobs (#4984) (@tgucks)
- 9eff451: Fix RangeError when null dates passed to DateTimePicker (#4997) (@mauriceyap)
- f6ab3a9: Fix cross-partition duplicate job rows in Lookout ingester (experimental hot/cold) (#4996) (@mauriceyap)
- d0a0344: Hot cold local dev (#4899) (@dslear)
Full Changelog: v0.21.7...v0.22.0
Contributors
mauriceyap, tgucks, and dslear
Assets 14
v0.21.7
Armada v0.21.7
For more info, head over to the docs page at https://armadaproject.io
Armada CLI
armadactl controls the Armada batch job queueing system and is used for interacting with the system.
The CLI can be downloaded for a specific OS & Architecture from the Assets section below.
NOTE: The OSX binary is packaged as an universal binary and should work on both Intel and Apple Silicon based Macs.
Docker images
Armada Bundle
docker pull gresearch/armada:0.21.7docker pull gresearch/armada:latest
Armada Lookout Bundle
docker pull gresearch/armada-lookout-bundle:0.21.7docker pull gresearch/armada-lookout-bundle:latest
Armada Full Bundle
docker pull gresearch/armada-full-bundle:0.21.7docker pull gresearch/armada-full-bundle:latest
Armada Server
docker pull gresearch/armada-server:0.21.7docker pull gresearch/armada-server:latest
Armada Executor
docker pull gresearch/armada-executor:0.21.7docker pull gresearch/armada-executor:latest
Armada Lookout
docker pull gresearch/armada-lookout:0.21.7docker pull gresearch/armada-lookout:latest
Armada Lookout Ingester
docker pull gresearch/armada-lookout-ingester:0.21.7docker pull gresearch/armada-lookout-ingester:latest
Armada Event Ingester
docker pull gresearch/armada-event-ingester:0.21.7docker pull gresearch/armada-event-ingester:latest
Armada Scheduler
docker pull gresearch/armada-scheduler:0.21.7docker pull gresearch/armada-scheduler:latest
Armada Scheduler Ingester
docker pull gresearch/armada-scheduler-ingester:0.21.7docker pull gresearch/armada-scheduler-ingester:latest
Armada Binoculars
docker pull gresearch/armada-binoculars:0.21.7docker pull gresearch/armada-binoculars:latest
armadactl
docker pull gresearch/armadactl:0.21.7docker pull gresearch/armadactl:latest
Changelog
Features
Other work
- 86c958e: Add OTEL config and wrappers (#4985) (@nikola-jokic)
- b249c96: Add developer doc for job lifecycle events (#4935) (@dejanzele)
- d26b6e1: Add flags to disable scheduling preemption by type (#4989) (@tgucks)
- a2ff450: Add tgucks to Armada maintainers list (#4991) (@JamesMurkin)
- 94d4c93: Create observability package introducing OTEL (#4975) (@nikola-jokic)
- 8c85bf5: Fix crash on Jobs page from non-array job-table preferences (#4986) (@mauriceyap)
- d378362: Fix unhandled clipboard rejection in CopyIconButton (#4988) (@mauriceyap)
- 7fc1267: Fix unhandled rejection on the Job Sets page; migrate JobSetsContainer to a functional component (#4987) (@mauriceyap)
- 17026e1: Improve the main scheduler loop (#4993) (@JamesMurkin)
- 096c12c: Local Development: Change VS Code to use Goreman (#4929) (@d2burkhalter)
- 1eface2: Source scheduler failure metric labels from error categorization, remove TrackedErrorRegexes (#4980) (@dejanzele)
Full Changelog: v0.21.6...v0.21.7
Contributors
JamesMurkin, mauriceyap, and 4 other contributors
Assets 14
v0.21.6
Armada v0.21.6
For more info, head over to the docs page at https://armadaproject.io
Armada CLI
armadactl controls the Armada batch job queueing system and is used for interacting with the system.
The CLI can be downloaded for a specific OS & Architecture from the Assets section below.
NOTE: The OSX binary is packaged as an universal binary and should work on both Intel and Apple Silicon based Macs.
Docker images
Armada Bundle
docker pull gresearch/armada:0.21.6docker pull gresearch/armada:latest
Armada Lookout Bundle
docker pull gresearch/armada-lookout-bundle:0.21.6docker pull gresearch/armada-lookout-bundle:latest
Armada Full Bundle
docker pull gresearch/armada-full-bundle:0.21.6docker pull gresearch/armada-full-bundle:latest
Armada Server
docker pull gresearch/armada-server:0.21.6docker pull gresearch/armada-server:latest
Armada Executor
docker pull gresearch/armada-executor:0.21.6docker pull gresearch/armada-executor:latest
Armada Lookout
docker pull gresearch/armada-lookout:0.21.6docker pull gresearch/armada-lookout:latest
Armada Lookout Ingester
docker pull gresearch/armada-lookout-ingester:0.21.6docker pull gresearch/armada-lookout-ingester:latest
Armada Event Ingester
docker pull gresearch/armada-event-ingester:0.21.6docker pull gresearch/armada-event-ingester:latest
Armada Scheduler
docker pull gresearch/armada-scheduler:0.21.6docker pull gresearch/armada-scheduler:latest
Armada Scheduler Ingester
docker pull gresearch/armada-scheduler-ingester:0.21.6docker pull gresearch/armada-scheduler-ingester:latest
Armada Binoculars
docker pull gresearch/armada-binoculars:0.21.6docker pull gresearch/armada-binoculars:latest
armadactl
docker pull gresearch/armadactl:0.21.6docker pull gresearch/armadactl:latest
Changelog
Bug fixes
- f9cb164: fix(airflow): cancel jobs only when task in terminal state (#490) (#4979) (@masipauskas)
Other work
- 3ad5f90: Add a log message for when a issue is registered in executor (#4982) (@JamesMurkin)
- 8132e2b: Add name field to pod checks allowing easier debugging (#4954) (@nikola-jokic)
- 85b582d: Categorize internal failures with a static failure category (#4972) (@dejanzele)
- c248860: Docs openssf bestpractices badge (#4974) (@sarhiri)
- 3cacc6b: Fix bug when determining if GlobalRateLimit is exceeded (#4977) (@JamesMurkin)
- c389d7c: Improve the efficiency of updating job prices (#4978) (@JamesMurkin)
- 9509c0e: Introduce deadline for init containers (#4960) (@nikola-jokic)
- 4096832: Update BidPriceSnapshot.Id to be a uuid rather than string (#4981) (@JamesMurkin)
Full Changelog: v0.21.5...v0.21.6
Contributors
masipauskas, JamesMurkin, and 3 other contributors
Assets 14
v0.21.5
Armada v0.21.5
For more info, head over to the docs page at https://armadaproject.io
Armada CLI
armadactl controls the Armada batch job queueing system and is used for interacting with the system.
The CLI can be downloaded for a specific OS & Architecture from the Assets section below.
NOTE: The OSX binary is packaged as an universal binary and should work on both Intel and Apple Silicon based Macs.
Docker images
Armada Bundle
docker pull gresearch/armada:0.21.5docker pull gresearch/armada:latest
Armada Lookout Bundle
docker pull gresearch/armada-lookout-bundle:0.21.5docker pull gresearch/armada-lookout-bundle:latest
Armada Full Bundle
docker pull gresearch/armada-full-bundle:0.21.5docker pull gresearch/armada-full-bundle:latest
Armada Server
docker pull gresearch/armada-server:0.21.5docker pull gresearch/armada-server:latest
Armada Executor
docker pull gresearch/armada-executor:0.21.5docker pull gresearch/armada-executor:latest
Armada Lookout
docker pull gresearch/armada-lookout:0.21.5docker pull gresearch/armada-lookout:latest
Armada Lookout Ingester
docker pull gresearch/armada-lookout-ingester:0.21.5docker pull gresearch/armada-lookout-ingester:latest
Armada Event Ingester
docker pull gresearch/armada-event-ingester:0.21.5docker pull gresearch/armada-event-ingester:latest
Armada Scheduler
docker pull gresearch/armada-scheduler:0.21.5docker pull gresearch/armada-scheduler:latest
Armada Scheduler Ingester
docker pull gresearch/armada-scheduler-ingester:0.21.5docker pull gresearch/armada-scheduler-ingester:latest
Armada Binoculars
docker pull gresearch/armada-binoculars:0.21.5docker pull gresearch/armada-binoculars:latest
armadactl
docker pull gresearch/armadactl:0.21.5docker pull gresearch/armadactl:latest
Changelog
Other work
- 08afc85: Add a scheduling cycle number field to async runner logs (#4971) (@JamesMurkin)
- 1ea4492: Clean up localdev consolidation leftovers (#4967) (@dejanzele)
- 1f353de: Fix a bug in async runner causing job state to get corrupted (#4970) (@JamesMurkin)
Full Changelog: v0.21.4...v0.21.5
Contributors
JamesMurkin and dejanzele
Assets 14
v0.21.4
Armada v0.21.4
For more info, head over to the docs page at https://armadaproject.io
Armada CLI
armadactl controls the Armada batch job queueing system and is used for interacting with the system.
The CLI can be downloaded for a specific OS & Architecture from the Assets section below.
NOTE: The OSX binary is packaged as an universal binary and should work on both Intel and Apple Silicon based Macs.
Docker images
Armada Bundle
docker pull gresearch/armada:0.21.4docker pull gresearch/armada:latest
Armada Lookout Bundle
docker pull gresearch/armada-lookout-bundle:0.21.4docker pull gresearch/armada-lookout-bundle:latest
Armada Full Bundle
docker pull gresearch/armada-full-bundle:0.21.4docker pull gresearch/armada-full-bundle:latest
Armada Server
docker pull gresearch/armada-server:0.21.4docker pull gresearch/armada-server:latest
Armada Executor
docker pull gresearch/armada-executor:0.21.4docker pull gresearch/armada-executor:latest
Armada Lookout
docker pull gresearch/armada-lookout:0.21.4docker pull gresearch/armada-lookout:latest
Armada Lookout Ingester
docker pull gresearch/armada-lookout-ingester:0.21.4docker pull gresearch/armada-lookout-ingester:latest
Armada Event Ingester
docker pull gresearch/armada-event-ingester:0.21.4docker pull gresearch/armada-event-ingester:latest
Armada Scheduler
docker pull gresearch/armada-scheduler:0.21.4docker pull gresearch/armada-scheduler:latest
Armada Scheduler Ingester
docker pull gresearch/armada-scheduler-ingester:0.21.4docker pull gresearch/armada-scheduler-ingester:latest
Armada Binoculars
docker pull gresearch/armada-binoculars:0.21.4docker pull gresearch/armada-binoculars:latest
armadactl
docker pull gresearch/armadactl:0.21.4docker pull gresearch/armadactl:latest
Changelog
Features
- b3bab4a: feat(airflow): Fix minor operator bug-fixes to work with airflow-3.1x (#4962) (@masipauskas)
- 008eb56: feat(airflow): support operator extra links (@masipauskas)
Other work
- 9c8126f: Add UnscheduleJob and RemoveJob to sctx/qctx (#4955) (@JamesMurkin)
- 7aa860a: Add bid price cache staleness metric (#483) (#4961) (@MustafaI)
- 8e42625: Display cancelling user in job sidebar header (#4965) (@mauriceyap)
- b9a764b: Extract leader election (#4915) (@nikola-jokic)
- 00261fc: Fix preempted job metrics (#4966) (@JamesMurkin)
- dd6d435: Free disk space in integration test jobs to avoid exhausting small CI runners (#4953) (@dejanzele)
- 8d4f60d: Hoist executor fetch & reconciliation out of per-pool loop (#4934) (@tgucks)
- d98e032: Improve logging in async runner (#4968) (@JamesMurkin)
- 7e2e2e8: Localdev: collapse setup into a single
mage dev:upcommand (#4933) (@dejanzele) - f6f899f: Making scheduling cycle optionally run asynchronously (#4952) (@JamesMurkin)
- 8d050b6: Populate bid pricing details on job creation (#4949) (@JamesMurkin)
- adbc8d0: Remove resourceUnits from SchedulingAlgo.Schedule (#4956) (@JamesMurkin)
- cb981f2: Remove unused Airflow extra-link plugin (#4948) (@dejanzele)
- 8ac05af: Simplify async mutex usage (#4964) (@JamesMurkin)
- 4e2daf3: Support Apache Airflow 2.10, 3.1 and 3.2 in the Airflow operator (#4939) (@dejanzele)
- aaff2fe: Treat cordoned nodes as available in the submit checker (#4947) (@dejanzele)
- ec32a12: Upgrade doc generation and validation tox entrypoints (@masipauskas)
- 3a379ce: Upgrade python client and airflow operator to work with python3.10, 3.12, 3.14 (#484) (#4951) (@masipauskas)
- 33bedcb: Upgrade python client to publish new version with upgraded protobuf libraries (#4950) (@masipauskas)
- b3d4fc6: Use more efficient jobdb index when expiring jobs (#4957) (@JamesMurkin)
- 13b59ef: Write to
job_metadatatable prior to migration (#4912) (@tgucks) - db00e34: ci: split python format/docs/lint into dedicated 3.12 jobs (#4959) (@masipauskas)
Full Changelog: v0.21.3...v0.21.4
Contributors
masipauskas, JamesMurkin, and 5 other contributors
Assets 14
v0.21.3
Armada v0.21.3
For more info, head over to the docs page at https://armadaproject.io
Armada CLI
armadactl controls the Armada batch job queueing system and is used for interacting with the system.
The CLI can be downloaded for a specific OS & Architecture from the Assets section below.
NOTE: The OSX binary is packaged as an universal binary and should work on both Intel and Apple Silicon based Macs.
Docker images
Armada Bundle
docker pull gresearch/armada:0.21.3docker pull gresearch/armada:latest
Armada Lookout Bundle
docker pull gresearch/armada-lookout-bundle:0.21.3docker pull gresearch/armada-lookout-bundle:latest
Armada Full Bundle
docker pull gresearch/armada-full-bundle:0.21.3docker pull gresearch/armada-full-bundle:latest
Armada Server
docker pull gresearch/armada-server:0.21.3docker pull gresearch/armada-server:latest
Armada Executor
docker pull gresearch/armada-executor:0.21.3docker pull gresearch/armada-executor:latest
Armada Lookout
docker pull gresearch/armada-lookout:0.21.3docker pull gresearch/armada-lookout:latest
Armada Lookout Ingester
docker pull gresearch/armada-lookout-ingester:0.21.3docker pull gresearch/armada-lookout-ingester:latest
Armada Event Ingester
docker pull gresearch/armada-event-ingester:0.21.3docker pull gresearch/armada-event-ingester:latest
Armada Scheduler
docker pull gresearch/armada-scheduler:0.21.3docker pull gresearch/armada-scheduler:latest
Armada Scheduler Ingester
docker pull gresearch/armada-scheduler-ingester:0.21.3docker pull gresearch/armada-scheduler-ingester:latest
Armada Binoculars
docker pull gresearch/armada-binoculars:0.21.3docker pull gresearch/armada-binoculars:latest
armadactl
docker pull gresearch/armadactl:0.21.3docker pull gresearch/armadactl:latest
Changelog
Bug fixes
- 1e5e1c7: Scheduler fix: only advance event cursor adter Pulsar publish succeeds (#4918) (@mauriceyap)
- 9a81877: fix: job run scheduler termination column (@williamvega)
Other work
- 6de919a: Add DryRunTxn to JobDb (#4923) (@JamesMurkin)
- b258848: Display Armada release version on Lookout settings page (#4917) (@mauriceyap)
- e3d4493: Fix CI integration test setup flakiness (#4942) (@mauriceyap)
- 61f6ce1: Fix flaky TestEventServer_GetJobSetEvents context budget (#4925) (@dejanzele)
- 2db6e6b: Fix lookout pruner metrics never being recorded (#4941) (@mauriceyap)
- f3ad700: LookoutUI: add analytics for custom view actions (#4931) (@ianhockett)
- b3ca9ce: LookoutUI: enrich custom view filter analytics with filterDetails (#4944) (@YHines004)
- 34b7159: Remove unused params/methods (#4930) (@Sovietaced)
- 1b7db0c: Use eventingesters redis client (#4910) (@nikola-jokic)
Full Changelog: v0.21.2...v0.21.3
Contributors
Sovietaced, JamesMurkin, and 6 other contributors
Assets 14
v0.21.2
Armada v0.21.2
For more info, head over to the docs page at https://armadaproject.io
Armada CLI
armadactl controls the Armada batch job queueing system and is used for interacting with the system.
The CLI can be downloaded for a specific OS & Architecture from the Assets section below.
NOTE: The OSX binary is packaged as an universal binary and should work on both Intel and Apple Silicon based Macs.
Docker images
Armada Bundle
docker pull gresearch/armada:0.21.2docker pull gresearch/armada:latest
Armada Lookout Bundle
docker pull gresearch/armada-lookout-bundle:0.21.2docker pull gresearch/armada-lookout-bundle:latest
Armada Full Bundle
docker pull gresearch/armada-full-bundle:0.21.2docker pull gresearch/armada-full-bundle:latest
Armada Server
docker pull gresearch/armada-server:0.21.2docker pull gresearch/armada-server:latest
Armada Executor
docker pull gresearch/armada-executor:0.21.2docker pull gresearch/armada-executor:latest
Armada Lookout
docker pull gresearch/armada-lookout:0.21.2docker pull gresearch/armada-lookout:latest
Armada Lookout Ingester
docker pull gresearch/armada-lookout-ingester:0.21.2docker pull gresearch/armada-lookout-ingester:latest
Armada Event Ingester
docker pull gresearch/armada-event-ingester:0.21.2docker pull gresearch/armada-event-ingester:latest
Armada Scheduler
docker pull gresearch/armada-scheduler:0.21.2docker pull gresearch/armada-scheduler:latest
Armada Scheduler Ingester
docker pull gresearch/armada-scheduler-ingester:0.21.2docker pull gresearch/armada-scheduler-ingester:latest
Armada Binoculars
docker pull gresearch/armada-binoculars:0.21.2docker pull gresearch/armada-binoculars:latest
armadactl
docker pull gresearch/armadactl:0.21.2docker pull gresearch/armadactl:latest
Changelog
Other work
- 0f24f0d: Disable only-new-issues from lint check (#4927) (@JamesMurkin)
- d014b33: Fix insertion_test not checking errors (#4926) (@JamesMurkin)
- 1bbf52c: Fix lint issues in Broadside code (#4928) (@mauriceyap)
Full Changelog: v0.21.1...v0.21.2
Contributors
JamesMurkin and mauriceyap
Assets 14
v0.21.1
Note
This release starts the move of job metadata from the jobs -> job_metadata table in the scheduler database.
The default configuration will not be a breaking change, but it is worth being aware it introduces possible incompatible configuration that users should take care when setting.
jobMetadataMigrationPhase should not be set to cutover in the scheduler ingester before the jobMetadataMigrationPhase is set to dualWrite in the scheduler.
- Doing so will mean that the metadata will be written to the job_metadata table but the scheduler will only be looking at the jobs table
Suggested migration path:
- Rollout v0.21.1
- Update scheduler configuration to set jobMetadataMigrationPhase to dualWrite
- Update scheduler ingester configuration to set
jobMetadataMigrationPhaseto dualWrite - Wait for all rows in your database to be updated
- Update scheduler ingester configuration to set
jobMetadataMigrationPhaseto cutover - Update scheduler configuration to set jobMetadataMigrationPhase to cutover
For now no action needs to be taken manually and subsequent releases will perform these steps for you.
Be aware invalid configuration (as described above) is possible if you do the migration yourself.
What's Changed
- Support optional schema creation in database migrators by @mauriceyap in #4911
- chore: add .lscache files to gitignore by @mauriceyap in #4919
- update the distinct active state for the Job and Job Sets tab by @YHines004 in #4887
- Broadside: support partitioning by submitted time feature by @mauriceyap in #4774
- Lookout Ingester: propagate context cancellation from Store() to prevent "zombie" jobs by @mauriceyap in #4921
- Split submit_message and groups into job_metadata table by @tgucks in #4906
- LookoutUI: add analytics for columns selected by @d2burkhalter in #4908
- Show 'Last updated: X ago' alongside the refresh button by @YHines004 in #4901
- Lookout pruner: reconcile zombie jobs by @mauriceyap in #4922
Full Changelog: v0.20.44...v0.21.1
Contributors
mauriceyap, d2burkhalter, and 2 other contributors
Assets 2
v0.20.44
Armada v0.20.44
For more info, head over to the docs page at https://armadaproject.io
Armada CLI
armadactl controls the Armada batch job queueing system and is used for interacting with the system.
The CLI can be downloaded for a specific OS & Architecture from the Assets section below.
NOTE: The OSX binary is packaged as an universal binary and should work on both Intel and Apple Silicon based Macs.
Docker images
Armada Bundle
docker pull gresearch/armada:0.20.44docker pull gresearch/armada:latest
Armada Lookout Bundle
docker pull gresearch/armada-lookout-bundle:0.20.44docker pull gresearch/armada-lookout-bundle:latest
Armada Full Bundle
docker pull gresearch/armada-full-bundle:0.20.44docker pull gresearch/armada-full-bundle:latest
Armada Server
docker pull gresearch/armada-server:0.20.44docker pull gresearch/armada-server:latest
Armada Executor
docker pull gresearch/armada-executor:0.20.44docker pull gresearch/armada-executor:latest
Armada Lookout
docker pull gresearch/armada-lookout:0.20.44docker pull gresearch/armada-lookout:latest
Armada Lookout Ingester
docker pull gresearch/armada-lookout-ingester:0.20.44docker pull gresearch/armada-lookout-ingester:latest
Armada Event Ingester
docker pull gresearch/armada-event-ingester:0.20.44docker pull gresearch/armada-event-ingester:latest
Armada Scheduler
docker pull gresearch/armada-scheduler:0.20.44docker pull gresearch/armada-scheduler:latest
Armada Scheduler Ingester
docker pull gresearch/armada-scheduler-ingester:0.20.44docker pull gresearch/armada-scheduler-ingester:latest
Armada Binoculars
docker pull gresearch/armada-binoculars:0.20.44docker pull gresearch/armada-binoculars:latest
armadactl
docker pull gresearch/armadactl:0.20.44docker pull gresearch/armadactl:latest
Changelog
Bug fixes
- 228c5df: fix: scheduler over-packs nodes when lower-priority incumbents are non-preemptible (#4879) (@dejanzele)
Other work
- d881895: Fix stale metrics from non-leader scheduler replicas (#4904) (@mauriceyap)
- 8d97ac4: Improve postgres statistics on queue/jobset column - scheduler db (#4914) (@JamesMurkin)
- e4866b0: Lookout: wrap ErrNotFound in GetJobSpec not-found error (#4773) (@mauriceyap)
Full Changelog: v0.20.43...v0.20.44
Contributors
JamesMurkin, mauriceyap, and dejanzele