feat: add dorado/aligner module - #11222
Conversation
Wraps `dorado aligner` (minimap2) to align unaligned ONT BAMs (e.g. from dorado/basecaller) while preserving modification tags (MM/ML). CPU-only — no GPU required. - main.nf: DORADO_ALIGNER process, process_high label, vendor container - meta.yml: EDAM ontologies for BAM/FASTA/FAI/TSV inputs and outputs - tests: stub + real test against GIAB HG002 unaligned BAM and hg38 slice (snapshot reduced to stable fields to avoid BAM header path drift) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…skip conda All six nf-tests were failing because the fixture they referenced, `HG002_PAW70337_giab_10reads.unaligned.bam`, was never merged into nf-core/test-datasets (it is still pending in nf-core/test-datasets#1969), so `checkIfExists: true` threw before the process ever ran. Tests now use `test2.sorted.bam`, which is already merged and maps to the same chr22 subset reference. `test.sorted.phased.bam` cannot be used: each of its records carries two RG tags (one from minimap2 `-R`, one from the original dorado basecall) while its header declares only one. dorado preserves both, so the output BAM trips htsjdk's READ_GROUP_NOT_FOUND check inside `getReadsMD5()`. Other fixes found while verifying against dorado 1.4.0: - The `summary` output could never be produced. `--emit-summary` writes a fixed-name `sequencing_summary.txt`, not `*_summary.tsv`. The glob is corrected and the script renames the file to `<prefix>.sequencing_summary.txt` so outputs stay per-sample. The test now sets `--emit-summary` and asserts the channel is populated. - Removed the dead `versions.yml` heredoc from the stub: this module reports versions through the `versions` topic, so nothing consumed that file. - Removed a no-op `sed 's/^//'` from the version eval. - The real test now actually computes the reads-only MD5 its comment promised, via nft-bam's `getReadsMD5()`, and snapshots it under a `versions` key so the `test_snap_versions` lint check passes. - Corrected the `.fai` input ontology, which pointed at the EDAM term for BED. Added `modules/nf-core/dorado` to the conda skip list, matching how parabricks is handled: dorado is not on bioconda (ONTPL licence) so the conda profile can never resolve it. docker and singularity are unaffected. Remaining known failure is `container_links`: nf-core lint only accepts `quay.io` or `community.wave.seqera.io/library` prefixes, and dorado is published only on Docker Hub. Flagged for maintainer input. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E7RSvpouxbMBdimYSwJ5ZP
…tection Two remaining lint failures, both now fixed: `container_links` failed because `nf-core lint` only accepts prefixes listed under `container-registry:` in `.nf-core.yml`. Added `docker.io/nanoporetech` next to the existing `nvcr.io/nvidia/clara` entry that lets the parabricks modules use NVIDIA's registry. `docker_tag` and `singularity_tag` failed with "No container was extracted". The cause is obscure: lint resolves the container by running `nextflow inspect`, and Nextflow was rejecting this file with "DSL1 is no longer supported". The trigger was the word "from" in the `//` comment on the input BAM line — inside an `input:` block it makes Nextflow's DSL-version heuristic read the file as DSL1. Rewording the comment to "produced by" resolves it. Verified by bisection: changing only that word flips `nextflow inspect` between failing and returning the container. Also switched the snapshot from nft-bam's `getReadsMD5()` to the record count. dorado writes records in a thread-dependent order with `SO:unknown`, so the reads md5 differs between runs on identical input (8dba95c6… vs c436519161…). The count is stable; verified over three consecutive runs. nf-core/tools 4.1.0 lint: 56 passed, 0 warnings, 0 failed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E7RSvpouxbMBdimYSwJ5ZP
|
This PR had all six nf-tests plus lint failing. All of it is now fixed and it is Why every test failed: the fixture it referenced, Rather than block on that, the tests now use Why lint failed, in two parts:
Other fixes found while verifying against dorado 1.4.0:
nf-core/tools 4.1.0 lint: 56 passed, 0 warnings, 0 failed. Both nf-tests |
dorado is not distributed on bioconda or conda-forge (ONTPL licence), so `environment.yml` is a placeholder carrying only samtools. The conda profile run would therefore execute `dorado basecaller` against an environment that does not contain dorado. Adds the `modules/nf-core/dorado` prefix to the `conda` skip list, the same mechanism parabricks and cellranger use. Matches the entry already present in nf-core#11222 (dorado/aligner) so each PR stands alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013fqrTt7TV2RSSUHaA4wP4v
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013fqrTt7TV2RSSUHaA4wP4v
|
Posting CI evidence, because I think this needs a decision on the container rather than another re-run.
On a re-run one shard went green and the other failed with the disk error, so it looks like flake but is really size. Measured from the registry:
Singularity needs roughly 2–3× the image size on disk to convert OCI layers to a SIF, which is what exhausts the runner. Worth noting for this module specifically: Options as I see them, and I am happy to do the work on whichever you prefer:
@SPPearce @famosab @edmundmiller any preference? I do not want to change the container reference on a PR under review without agreement on where the image should live. |
Description
Adds
dorado/aligner— a wrapper around Oxford Nanopore'sdorado aligner, which uses minimap2 under the hood to align unaligned ONT BAMs (e.g. produced bydorado/basecaller) while preserving modified base tags (MM/ML) and other BAM auxiliary tags.CPU-only (no GPU required) —
dorado alignerwraps minimap2 which is CPU-native. GPU is only needed for basecalling.Why a separate module from
dorado/basecaller?Follows nf-core convention of one tool-subcommand per module. This is also the approach recommended by @Kevin-Brockers / @dialvarezs in #11122 (dorado/basecaller review). Keeping basecall and align separate lets users:
minimap2/alignif preferredTest data
Test paths depend on nf-core/test-datasets#1969 (unaligned HG002 GIAB 10-read BAM). Tests were verified locally against the same files before PR; CI will go green once #1969 merges.
Verified real-test output:
Snapshot strategy
Real-test snapshot intentionally captures only filename + dorado version (not BAM MD5), because
dorado alignerembeds absolute paths in@SQ UR:and@PG CL:BAM header lines — these vary between test environments. Stub-test snapshot uses fullprocess.out(stable because all files are empty touch'd).PR checklist
topic: versionsprocess_high(CPU/RAM scales with input)docker.io/nanoporetech/dorado:shac8f356489fa8b44b31beba841b84d2879de2088e(ONTPL license — not on bioconda; vendor container pattern matches nf-core/parabricks)nf-core modules test dorado/aligner --profile singularity✅nf-core modules lint dorado/aligner✅ (2 expected warnings re: vendor Docker Hub image, same as dorado/basecaller)Conda and Docker profile tests not yet run (dorado not on bioconda; Docker not available on my dev host — expecting CI to validate).
🤖 Generated with Claude Code