Skip to content

Replace --annotations_filter with --filter #109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nvnieuwk merged 8 commits into from
Feb 21, 2025
Merged

Replace --annotations_filter with --filter #109

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a8fe163
move the filter outside of the annotations
nvnieuwk Feb 11, 2025
265de1a
no longer use permissive overlaps in vcfanno
nvnieuwk Feb 12, 2025
785fe18
fix tests
nvnieuwk Feb 12, 2025
abbddee
update changelog and docs
nvnieuwk Feb 13, 2025
f0955c4
fix linting
nvnieuwk Feb 13, 2025
afcf2c9
make toml channel a value channel
nvnieuwk Feb 17, 2025
7442b8e
Merge branch 'dev' into fix/107
nvnieuwk Feb 21, 2025
85785d1
Update CHANGELOG.md
nvnieuwk Feb 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
6. Removed the old output publishing code and used the new workflow output definitions instead
7. Bumped the minimal nextflow version to 24.10.0
8. VCFanno will now run when `--vcfanno_toml` has been given and `--annotate` has not been given. You still need to supply `--annotate` to get the full annotation, but this can be used check for common variants without having to perform full annotation.
9. Removed the `--delly_sv_types` parameter.
9. Changed the `--annotations_filter` parameter to a `--filter` parameter. This parameter takes an argument of `bcftools filter` to filter the resulting VCFs.
10. Removed the `--delly_sv_types` parameter.


### `Fixed`

Expand Down
12 changes: 9 additions & 3 deletions conf/modules.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,12 +251,18 @@ process {

withName: "^.*VCF_ANNOTATE_VCFANNO:VCFANNO\$" {
ext.prefix = {"${meta.id}.${meta.variant_type}.vcfanno"}
ext.args = "-permissive-overlap -ends"
ext.args = "-ends"
}

withName: "^.*VCF_ANNOTATE_VCFANNO:BCFTOOLS_FILTER\$" {
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SV AND CNV FILTERING
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*/

withName: "^.*STRUCTURAL:BCFTOOLS_FILTER\$" {
ext.prefix = {"${meta.id}.${meta.variant_type}.filtered"}
ext.args = "${params.annotations_filter} --output-type z --write-index=tbi"
ext.args = "${params.filter} --output-type z --write-index=tbi"
}

/*
Expand Down
2 changes: 1 addition & 1 deletion docs/parameters.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ Options specific to the execution of this pipeline
| `cnv_callers_support` | The minimum amount of CNV callers that should detect a variant. All variants that have a lower amount of callers supporting it, will be removed. (Only used when more than one caller is given) | `integer` | 1 | | |
| `annotate` | Run the annotation with Ensembl VEP and AnnotSV (and optionally VCFanno). | `boolean` | | | |
| `concat_output` | Also output a concatenated VCF with all variant types analysed included. | `boolean` | | | |
| `annotations_filter` | The filter arguments to use after annotating. <details><summary>Help</summary><small>A common use case for this is to filter out common variants. You can supply a VCF file with common variants to VCFanno and filter out the matching variants using this parameter</small></details> | `string` | | | |
| `filter` | The filter options to perform on SV and CNV VCF files as postprocessing <details><summary>Help</summary><small>A common use case for this is to filter out common variants. You can supply a VCF file with common variants to VCFanno and filter out the matching variants using this parameter</small></details> | `string` | | | |

## Delly parameters

Expand Down
2 changes: 1 addition & 1 deletion main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ workflow {
params.species,
params.vep_assembly,
params.vep_cache_version,
params.annotations_filter,
params.filter,
params.outdir
)
//
Expand Down
2 changes: 1 addition & 1 deletion nextflow.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ params {
cnv_callers_support = 1
annotate = false
concat_output = false
annotations_filter = null
filter = null
bedpe = false

// Delly parameters
Expand Down
4 changes: 2 additions & 2 deletions nextflow_schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -331,9 +331,9 @@
"type": "boolean",
"description": "Also output a concatenated VCF with all variant types analysed included."
},
"annotations_filter": {
"filter": {
"type": "string",
"description": "The filter arguments to use after annotating.",
"description": "The filter options to perform on SV and CNV VCF files as postprocessing",
"help_text": "A common use case for this is to filter out common variants. You can supply a VCF file with common variants to VCFanno and filter out the matching variants using this parameter"
},
"bedpe": {
Expand Down
17 changes: 2 additions & 15 deletions subworkflows/local/vcf_annotate_vcfanno/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ workflow VCF_ANNOTATE_VCFANNO {
ch_sample_specific_resources // channel: [optional] [ val(meta), path(vcf), path(tbi) ] Files containing resources that are sample-specific
ch_vcfanno_lua // channel: [optional] [ path(lua) ] => The lua script to influence VCFanno
val_vcfanno_resources // list: [optional] [ path(file1, file2, file3...) ] => The extra VCFanno files
filter // string: [optional] => A filter pattern to use after annotating
vcfanno_toml // file: [optional] => A vcfanno TOML config
default_vcfanno_tomls // list: [mandatory] => A list of default vcfanno configs to be concatenated with the input TOML
annotate // boolean: [mandatory] => Whether or not to run the full annotation or only the specified annotations
Expand All @@ -31,7 +30,7 @@ workflow VCF_ANNOTATE_VCFANNO {
}
ch_vcfanno_input = ch_vcfs.join(ch_collected_specific_resources, failOnMismatch:true, failOnDuplicate:true)
} else {
ch_vcfanno_toml = Channel.fromPath(vcfanno_toml)
ch_vcfanno_toml = Channel.fromPath(vcfanno_toml).collect()
ch_vcfanno_input = ch_vcfs.map { meta, vcf, tbi ->
[ meta, vcf, tbi, [] ]
}
Expand All @@ -45,19 +44,7 @@ workflow VCF_ANNOTATE_VCFANNO {
)
ch_versions = ch_versions.mix(VCFANNO.out.versions)

def ch_vcfanno_output = VCFANNO.out.vcf.join(VCFANNO.out.tbi, failOnDuplicate:true, failOnMismatch:true)

def ch_annotated_vcfs = Channel.empty()
if(filter) {
BCFTOOLS_FILTER(
ch_vcfanno_output
)
ch_versions = ch_versions.mix(BCFTOOLS_FILTER.out.versions.first())

ch_annotated_vcfs = BCFTOOLS_FILTER.out.vcf.join(BCFTOOLS_FILTER.out.tbi, failOnDuplicate:true, failOnMismatch:true)
} else {
ch_annotated_vcfs = ch_vcfanno_output
}
def ch_annotated_vcfs = VCFANNO.out.vcf.join(VCFANNO.out.tbi, failOnDuplicate:true, failOnMismatch:true)

emit:
vcfs = ch_annotated_vcfs // channel: [ val(meta), path(vcf), path(tbi) ]
Expand Down
69 changes: 8 additions & 61 deletions tests/subworkflows/local/vcf_annotate_vcfanno/main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,57 +10,9 @@ nextflow_workflow {

options "-stub"

test("homo_sapiens - no_filter, annotate") {
test("homo_sapiens - annotate") {

when {
params {
annotations_filter = null
}
workflow {
"""
def meta = [id:"test", sample:"test", sex:"male", family: "test1", family_count:1, hpo:[], variant_type:'sv']
input[0] = Channel.of([
meta,
file(params.sv_vcf1, checkIfExists:true),
file(params.sv_tbi1, checkIfExists:true)
])
input[1] = Channel.of([
meta,
[],
[]
])
input[2] = []
input[3] = []
input[4] = params.annotations_filter
input[5] = []
input[6] = files("\${params.vcfanno_tomls}/*.toml", checkIfExists:true)
input[7] = true
"""
}
}

then {
assertAll(
{ assert workflow.success },
{ assert snapshot(
workflow.out.collectEntries { key, value ->
if(key.matches("^\\d+\$") || key.matches('versions')) {
return null
}
return [ key, value ]
}.findAll { it != null }
).match() }
)
}

}

test("homo_sapiens - filter, no_annotate") {

when {
params {
annotations_filter = "-i FILTER='PASS'"
}
workflow {
"""
def meta = [id:"test", sample:"test", sex:"male", family: "test1", family_count:1, hpo:[], variant_type:'sv']
Expand All @@ -76,10 +28,9 @@ nextflow_workflow {
])
input[2] = []
input[3] = []
input[4] = params.annotations_filter
input[5] = files("\${params.vcfanno_tomls}/*.toml", checkIfExists:true)[0]
input[6] = files("\${params.vcfanno_tomls}/*.toml", checkIfExists:true)
input[7] = false
input[4] = []
input[5] = files("\${params.vcfanno_tomls}/*.toml", checkIfExists:true)
input[6] = true
"""
}
}
Expand All @@ -100,12 +51,9 @@ nextflow_workflow {

}

test("homo_sapiens - filter, annotate") {
test("homo_sapiens - no_annotate") {

when {
params {
annotations_filter = null
}
workflow {
"""
def meta = [id:"test", sample:"test", sex:"male", family: "test1", family_count:1, hpo:[], variant_type:'sv']
Expand All @@ -121,10 +69,9 @@ nextflow_workflow {
])
input[2] = []
input[3] = []
input[4] = params.annotations_filter
input[5] = []
input[6] = files("\${params.vcfanno_tomls}/*.toml", checkIfExists:true)
input[7] = true
input[4] = files("\${params.vcfanno_tomls}/*.toml", checkIfExists:true)[0]
input[5] = files("\${params.vcfanno_tomls}/*.toml", checkIfExists:true)
input[6] = false
"""
}
}
Expand Down
36 changes: 4 additions & 32 deletions tests/subworkflows/local/vcf_annotate_vcfanno/main.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,33 +1,5 @@
{
"homo_sapiens - filter, no_annotate": {
"content": [
{
"vcfs": [
[
{
"id": "test",
"sample": "test",
"sex": "male",
"family": "test1",
"family_count": 1,
"hpo": [

],
"variant_type": "sv"
},
"test.sv.filtered.vcf.gz:md5,68b329da9893e34099c7d8ad5cb9c940",
"test.sv.filtered.vcf.gz.tbi:md5,d41d8cd98f00b204e9800998ecf8427e"
]
]
}
],
"meta": {
"nf-test": "0.9.1",
"nextflow": "24.10.4"
},
"timestamp": "2025-02-10T16:55:00.763124808"
},
"homo_sapiens - filter, annotate": {
"homo_sapiens - no_annotate": {
"content": [
{
"vcfs": [
Expand All @@ -53,9 +25,9 @@
"nf-test": "0.9.1",
"nextflow": "24.10.4"
},
"timestamp": "2025-02-10T16:55:09.970621986"
"timestamp": "2025-02-12T14:07:01.425728203"
},
"homo_sapiens - no_filter, annotate": {
"homo_sapiens - annotate": {
"content": [
{
"vcfs": [
Expand All @@ -81,6 +53,6 @@
"nf-test": "0.9.1",
"nextflow": "24.10.4"
},
"timestamp": "2025-02-10T16:54:50.521405941"
"timestamp": "2025-02-12T14:06:51.583328891"
}
}
22 changes: 17 additions & 5 deletions workflows/structural.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ include { ANNOTSV_INSTALLANNOTATIONS } from '../modules/nf-core/annotsv/i
include { UNTAR as UNTAR_ANNOTSV } from '../modules/nf-core/untar/main'
include { UNTAR as UNTAR_BWA } from '../modules/nf-core/untar/main'
include { NGSBITS_SAMPLEGENDER } from '../modules/nf-core/ngsbits/samplegender/main'
include { BCFTOOLS_FILTER } from '../modules/nf-core/bcftools/filter/main'
include { SVTOOLS_VCFTOBEDPE } from '../modules/nf-core/svtools/vcftobedpe/main'
include { MULTIQC } from '../modules/nf-core/multiqc/main'

Expand Down Expand Up @@ -95,7 +96,7 @@ workflow STRUCTURAL {
species // The species to be used by VEP
vep_assembly // The genome assembly to be downloaded for VEP
vep_cache_version // The version of the VEP cache to use
annotations_filter // The filter pattern to use after annotation
filter // The filter pattern to use after annotation
outdir // The output directory of the pipeline

main:
Expand Down Expand Up @@ -409,22 +410,33 @@ workflow STRUCTURAL {
ch_annotation_output = ch_annotation_input
}

def ch_outputs = Channel.empty()
def ch_vcfanno_output = Channel.empty()
if(annotate || vcfanno_toml) {
VCF_ANNOTATE_VCFANNO(
ch_annotation_output,
ch_annotsv_vcfs,
ch_vcfanno_lua,
val_vcfanno_resources,
annotations_filter,
val_vcfanno_toml,
val_default_vcfanno_tomls,
annotate
)
ch_versions = ch_versions.mix(VCF_ANNOTATE_VCFANNO.out.versions)
ch_outputs = ch_outputs.mix(VCF_ANNOTATE_VCFANNO.out.vcfs)
ch_vcfanno_output = VCF_ANNOTATE_VCFANNO.out.vcfs
} else {
ch_vcfanno_output = ch_annotation_input
}

def ch_outputs = Channel.empty()
if(filter) {
BCFTOOLS_FILTER(
ch_vcfanno_output
)
def ch_filter_output = BCFTOOLS_FILTER.out.vcf.join(BCFTOOLS_FILTER.out.tbi, failOnMismatch:true, failOnDuplicate:true)
ch_outputs = ch_outputs.mix(ch_filter_output)
ch_versions = ch_versions.mix(BCFTOOLS_FILTER.out.versions)
} else {
ch_outputs = ch_outputs.mix(ch_annotation_input)
ch_outputs = ch_outputs.mix(ch_vcfanno_output)
}

//
Expand Down
Loading