Releases: VachaLab/qq
Releases · VachaLab/qq
v0.11.0
qq v0.11.0
qq respawn
- Failed or killed jobs can now be easily "respawned" using
qq respawn. When respawning a job, qq will remove the working directory of the failed job, clear all runtime files, and resubmit the job with the same parameters as before.
Specifying multiple job IDs
qq info,qq kill,qq sync,qq respawn,qq wipe, andqq gonow accept multiple job IDs as arguments.- All of these commands now also internally resolve the jobs in parallel, making them much faster when dealing with a large number of jobs.
Clearing runtime files in a specified directory
qq clearnow supports clearing runtime files in a directory other than the current one via the-d/--dirflag.
Better support for non-bash interpreters
- Interpreters now support additional command-line arguments.
Resubmitting with fallback hosts
- Resubmission of loop and continuous jobs now supports multiple fallback hosts. Previously, jobs were resubmitted from a single machine (the input machine or the working node, depending on the batch system). A list of hosts can now be specified via
--resubmit-fromor in the config file; they are tried in order until one succeeds, making resubmission more resilient to individual machine failures. See the manual for more information.
Other changes
- Updated the installation scripts to more clearly report issues that occurred during the install.
Internal changes
- qq now uses Python 3.13 for better generics support.
- Fixed type errors in qq scripts.
- Refactored BatchMeta.
- Jobs can now be submitted from a remote machine (only via Python API).
- Submitting jobs using
Submitter.submitis now thread-safe. - When reading a configuration file fails, an exception is no longer raised; instead, an error is reported and a default configuration is used.
CFGis now a frozen dataclass.- Default archive directory and archive format are now configurable.
- Changed the internal representation of the interpreter specified to execute the script.
v0.10.1
v0.10.0
qq v0.10.0
New features
- Added support for submitting jobs to different batch servers on Metacentrum-family clusters. See the manual for more information.
- Added support for specifying a custom interpreter when submitting a job (e.g. Python, Julia). See the manual for more information.
Python API changes
- Breaking Python API change: Renamed methods from camelCase to snake_case.
Small changes and bug fixes
- Paths are now resolved to absolute paths without following symlinks, ensuring compatibility across machines with different mount points (e.g. Robox and Sokar).
- Improved hostname resolution to allow accessing worker nodes of the Sokar cluster from machines outside the
ncbr.muni.czdomain. - Fixed log lines in
qqoutfiles being truncated. - Single-node qq jobs should no longer fail on Metacentrum when a batch server is temporarily unreachable during job initialization.
- On LUMI,
qq nodesnow shows the physical number of CPU cores available on each node, not the number of threads. - qq no longer includes completed array tasks within uncompleted array jobs in the output of
qq jobsandqq stat, unless the-a/--alloption is used. - Bug fix: Fixed incorrect conversion of default walltime of Slurm partitions.
v0.10.0-dev.1
qq v0.10.0-dev.1
Pre-release for version 0.10. You should not use this version yet. See CHANGELOG.md in the v0.10 branch for details about what has changed.
v0.9.0
qq v0.9.0
- Added
continuousjobs: a light-weight alternative to loop jobs. Continuous jobs automatically submit their continuation but do not track their cycle nor do they perform archival operations. See the manual for more information.
v0.8.0
qq v0.8.0
- Added the
--transfer-modeand--archive-modeoptions, which allow automatically transferring (and archiving, respectively) files from the working directory for other jobs than those successfully finished. See the manual for more information. - As a consequence of the above change, the behavior of
qq go,qq sync, andqq wipehas been slightly adjusted. - Breaking change: In
qq submit, list options (e.g.,--include,--exclude,--depend,--props) are now sourced exclusively from either the command line or the submitted script, if specified. Values from both sources are no longer merged. The previous behavior was inconsistent and could cause confusion and bugs, such as duplicated resources in loop jobs. - Bug fix: Fixed an issue where autocomplete for the script name in
qq submitdid not work if a previous option's value contained=. - Bug fix: Fixed parsing of
qqdirectives in submitted scripts containing numeric values.