Consensus crash

[Gianluca-Alessio-Mariani-1512]

Description of the bug

I have a new crash following the #66 update, that I think is completely unrelated to the fix for that issue.
I am running the pipeline with
aligner="bwa-mem2"
tools="mutect2,filtering,consensus,rna_filtering"
dna=true
rna=true
along other options to use a locally saved GATK igenomes version of hg38 genome (config attached), and when the pipeline gets to the consensus step, it crashes giving me the error below.

If I am not mistaken the consensus part should just filter out the variants called by less callers than a specific threshold set internally by the pipeline.
In my case obviously there is only one caller, mutect2, and probably if i remove the consensus from the tools list, it would work, but it is strange that the pipeline doesn't check how many callers are used and if only one, then it prevents the consensus step from even starting.

Command used and terminal output

Monitor the execution with Seqera Platform using this URL: https://seqera.ieo.it/orgs/IEO/workspaces/DIMA/watch/5bciEqQXpbudil
executor >  slurm (82)
[-        ] NFC…EPARE_GENOME:BWAMEM1_INDEX -
[25/bce5e9] NFC…_sapiens_assembly38.fasta) | 1 of 1, cached: 1 ✔
[-        ] NFC…E_GENOME:DRAGMAP_HASHTABLE -
[-        ] NFC…4_CREATESEQUENCEDICTIONARY -
[-        ] NFC…PARE_GENOME:SAMTOOLS_FAIDX -
[-        ] NFC…PREPARE_GENOME:TABIX_DBSNP -
[-        ] NFC…ME:TABIX_GERMLINE_RESOURCE -
[-        ] NFC…RE_GENOME:TABIX_KNOWN_SNPS -
[-        ] NFC…_GENOME:TABIX_KNOWN_INDELS -
[-        ] NFC…S:PREPARE_GENOME:TABIX_PON -
[b4/b61fa2] NFC…_sapiens_assembly38.fasta) | 1 of 1, cached: 1 ✔
[8c/26d248] NFC…egions_noseconds.hg38.bed) | 1 of 1, cached: 1 ✔
[d9/2d630b] NFC…LIT (chrY_9055175-9057608) | 21 of 21, cached: 21 ✔
[5d/157bde] NFC…ng_regions_noseconds.hg38) | 1 of 1, cached: 1 ✔
[-        ] NFC…NPUT:SAMTOOLS_VIEW_MAP_MAP -
[-        ] NFC…:SAMTOOLS_VIEW_UNMAP_UNMAP -
[-        ] NFC…UT:SAMTOOLS_VIEW_UNMAP_MAP -
[-        ] NFC…UT:SAMTOOLS_VIEW_MAP_UNMAP -
[-        ] NFC…INPUT:SAMTOOLS_MERGE_UNMAP -
[-        ] NFC…_INPUT:COLLATE_FASTQ_UNMAP -
[68/e1ca24] NFC…_186721838654315-T-DNA-L1) | 143 of 150, cached: 143
[2c/5d3ec6] NFC…_186721838654315-N-DNA-L1) | 150 of 150, cached: 150 ✔
[66/1d24ae] NFC… (p_150932597190170-T-DNA) | 247 of 404, cached: 247
[ab/c3ff0c] NFC… (p_177567410717426-T-RNA) | 169 of 200, cached: 169
[9d/03c82f] NFC… (p_174661203620763-T-RNA) | 168 of 169, cached: 167
[d7/ad4c37] NFC… (p_174965100071390-T-RNA) | 167 of 168, cached: 167
[-        ] NFC…TS_SAMTOOLS:SAMTOOLS_STATS | 0 of 1
[-        ] NFC…SAMTOOLS:SAMTOOLS_FLAGSTAT | 0 of 1
[-        ] NFC…SAMTOOLS:SAMTOOLS_IDXSTATS | 0 of 1
[45/877021] NFC… (p_131885320096831-T-DNA) | 53 of 101, cached: 53
[48/3a98b0] NFC… (p_131885320096831-T-RNA) | 50 of 53, cached: 49
[9b/24a282] NFC… (p_131885320096831-T-RNA) | 53 of 53, cached: 52 ✔
[74/fea4cc] NFC…S (p_11106986109050-T-RNA) | 162 of 441, cached: 162
[09/60a18b] NFC… (p_111533365357834-T-RNA) | 6 of 6, cached: 6 ✔
[30/b01fa8] NFC… (p_111533365357834-T-RNA) | 6 of 6, cached: 6 ✔
[74/ad09cb] NFC… (p_105948407101155-T-DNA) | 347 of 798, cached: 342
[4a/89d43a] NFC… (p_108382212477642-T-RNA) | 15 of 15, cached: 15 ✔
[25/da9ef9] NFC… (p_103357847226467-T-DNA) | 285 of 315, cached: 285
[77/459bf3] NFC… (p_108382212477642-T-RNA) | 13 of 13, cached: 13 ✔
[c2/da0ba1] NFC… (p_108382212477642-T-RNA) | 13 of 13, cached: 13 ✔
[b1/57dfb5] NFC… (p_108382212477642-T-RNA) | 13 of 13, cached: 13 ✔
[0c/fc7b93] NFC… (p_108382212477642-T-RNA) | 13 of 13, cached: 13 ✔
[5d/525cb4] NFC…s_p_103087989026304-N-DNA) | 41 of 42, cached: 41
[89/826fac] NFC…s_p_108382212477642-N-DNA) | 1 of 1, cached: 1
[4c/44ac04] NFC…s_p_108382212477642-N-DNA) | 1 of 1, cached: 1
[1d/8dc96a] NFC…s_p_108382212477642-N-DNA) | 1 of 1, cached: 1 ✔
[a2/7da089] NFC… (p_108382212477642-N-DNA) | 42 of 42, cached: 42 ✔
[b4/78edd8] NFC… (p_108382212477642-T-RNA) | 42 of 42, cached: 42 ✔
[77/028fad] NFC… (p_108382212477642-N-DNA) | 2 of 2, cached: 2 ✔
[10/6127de] NFC… (p_108382212477642-T-RNA) | 2 of 2, cached: 2 ✔
[0b/eccdfe] NFC…s_p_108382212477642-N-DNA) | 2 of 2, cached: 2 ✔
[f9/a26275] NFC…s_p_108382212477642-N-DNA) | 1 of 1, cached: 1 ✔
[39/cc60d2] NFC…s_p_108382212477642-N-DNA) | 1 of 1, cached: 1 ✔
[25/33e8d7] NFC…s_p_108382212477642-N-DNA) | 1 of 1, cached: 1 ✔
[a6/57c667] NFC…s_p_108382212477642-N-DNA) | 1 of 1, cached: 1 ✔
[dc/d8404d] NFC…s_p_108382212477642-N-DNA) | 1 of 1, cached: 1 ✔
[fd/b63595] NFC…s_p_108382212477642-N-DNA) | 1 of 1, cached: 1
[cc/490ee8] NFC…s_p_108382212477642-N-DNA) | 1 of 1, cached: 1
[23/88f8d2] NFC…s_p_108382212477642-N-DNA) | 1 of 1, cached: 1 ✔
[f6/471aa1] NFC…s_p_108382212477642-N-DNA) | 0 of 1
Plus 13 more processes waiting for tasks…
Execution cancelled -- Finishing pending tasks before exit
-[nf-core/rnadnavar] Pipeline completed with errors-
WARN: Killing running tasks (73)
ERROR ~ Error executing process > 'NFCORE_RNADNAVAR:RNADNAVAR:BAM_PROCESSING:VCF_CONSENSUS:RUN_CONSENSUS (p_108382212477642-T-RNA_vs_p_108382212477642-N-DNA)'

Caused by:
  Process `NFCORE_RNADNAVAR:RNADNAVAR:BAM_PROCESSING:VCF_CONSENSUS:RUN_CONSENSUS (p_108382212477642-T-RNA_vs_p_108382212477642-N-DNA)` terminated with an error exit status (1)


Command executed:

  run_consensus.R --input_dir=inputs/ --out_prefix=p_108382212477642-T-RNA_vs_p_108382212477642-N-DNA.consensus --cpu=12 --id=p_108382212477642-T-RNA_vs_p_108382212477642-N-DNA
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_RNADNAVAR:RNADNAVAR:BAM_PROCESSING:VCF_CONSENSUS:RUN_CONSENSUS":
      R: $(echo $(R --version 2>&1) | head -n 1)
  END_VERSIONS

Command exit status:
  1

Command output:
             used  (Mb) gc trigger  (Mb) max used  (Mb)
  Ncells  8764248 468.1   13839095 739.1 11414870 609.7
  Vcells 16142897 123.2   25842636 197.2 21363794 163.0
             used  (Mb) gc trigger  (Mb) max used  (Mb)
  Ncells  8764630 468.1   13839095 739.1 11414870 609.7
  Vcells 16145785 123.2   25842636 197.2 21363794 163.0

Command error:
  - Parsing headers
  - Converting to genomic ranges
    - inputs//p_108382212477642-T-RNA_vs_p_108382212477642-N-DNA.mutect2.maf
  - Finding overlaps
  Error in eval(bysub, parent.frame(), parent.frame()) : 
    object 'DNAchange' not found
  Calls: [ -> [.data.table -> eval -> eval
  Execution halted

Work dir:
  /beegfs/scratch/ieo7224/__softwares/rnadnavar/pipeline/dev/work/f6/471aa1b42bc269010e0f1c82c3ed4e

Container:
  /hpcnfs/data/DIMA/singularity_cache/quay.io-nf-core-rnadnavar_renv_consensus-1.0.img

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting

 -- Check '.nextflow.log' file for details
Join mismatch for the following entries: 
- key=[patient:p_124629082149991, sample:p_124629082149991-N-DNA, status:0, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_124629082149991-N-DNA] values= 
- key=[patient:p_124964680653624, sample:p_124964680653624-N-DNA, status:0, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_124964680653624-N-DNA] values= 
- key=[patient:p_107131954473644, sample:p_107131954473644-T-DNA, status:1, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_107131954473644-T-DNA] values= 
- key=[patient:p_108382212477642, sample:p_108382212477642-T-DNA, status:1, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_108382212477642-T-DNA] values= 
- key=[patient:p_118374324618226, sample:p_118374324618226-T-DNA, status:1, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_118374324618226-T-DNA] values= 
- key=[patient:p_127735874401492, sample:p_127735874401492-N-DNA, status:0, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_127735874401492-N-DNA] values= 
- key=[patient:p_114197840097816, sample:p_114197840097816-T-DNA, status:1, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_114197840097816-T-DNA] values= 
- key=[patient:p_104160464307343, sample:p_104160464307343-T-RNA, status:2, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_104160464307343-T-RNA] values= 
- key=[patient:p_131624348662920, sample:p_131624348662920-N-DNA, status:0, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_131624348662920-N-DNA] values= 
- key=[patient:p_131885320096831, sample:p_131885320096831-N-DNA, status:0, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_131885320096831-N-DNA] values=
(more omitted)


WARN: Seqera Platform request field `workflow.errorMessage` exceeds expected size | offending value: `- Parsing headers
- Converting to genomic ranges
  - inputs//p_108382212477642-T-RNA_vs_p_1083822124(truncated)`, size: 287 (max: 255)
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting

 -- Check '.nextflow.log' file for details
WARN: Failed to render execution report -- see the log file for details
WARN: Failed to render execution timeline -- see the log file for details
WARN: Seqera Platform request field `workflow.errorMessage` exceeds expected size | offending value: `- Parsing headers
- Converting to genomic ranges
  - inputs//p_108382212477642-T-RNA_vs_p_1083822124(truncated)`, size: 287 (max: 255)

Relevant files

immotion_rnadnavar.config.gz

System information

No response

[Gianluca-Alessio-Mariani-1512]

Additional note: I tried running the pipeline with the same exact config with the only difference of removing consensus from the tools parameter and I confirm that it works in this case.

It is still running, but it passed the point of the crash and it is running the next processes without problems.

I will confirm this officially when the pipeline will finish completely, but for now it seems to be working.

[Gianluca-Alessio-Mariani-1512]

Yes I officially confirm that without the consensus tool the pipeline does not crash.
I tried this twice on two different sets of inputs (both in the form of 50 samples with DNA tum+norm + RNA each) and in both cases the pipeline completes without problems.

At this point I assume the problem manifests only if consensus is selected in the tools parameter but there is only one variant caller chosen.

Maybe I could try also with consensus + 2 variant callers and see if it does not crash in that case, but I guess you may have already discovered the problem and this is way too overkill.

[RaqManzano]

Are you able to provide some extract of the input files for the consensus step and the complete .nextflow.log please?