Adds the Lock Transformation #1

emarinier · 2025-02-05T21:59:15Z

Adds the lock transformation, which simply writes the passed metadata to a CSV file and parses that CSV file with the nf-iridanext plugin. This pull request requires the nf-iridanext plugin PR to be completed and released first:

phac-nml/nf-iridanext#21

github-actions · 2025-02-05T21:59:25Z

This PR is against the `main` branch ❌

Do not close this PR
Click Edit and change the base to dev
This CI test will remain failed until you push a new commit

Hi @emarinier,

It looks like this pull-request is has been made against the phac-nml/metadatatransformation main branch.
The main branch on phac-nml repositories should always contain code from the latest release.
Because of this, PRs to main are only allowed if they come from the phac-nml/metadatatransformation dev branch.

You do not need to close this PR, you can change the target branch to dev by clicking the "Edit" button at the top of this page.
Note that even after this, the test will continue to show as failing until you push a new commit.

Thanks again for your contribution!

github-actions · 2025-02-05T22:00:27Z

`nf-core pipelines lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 2cf979b

+| ✅ 139 tests passed       |+
#| ❔  27 tests were ignored |#
!| ❗   6 tests had warnings |!

❗ Test warnings:

files_exist - File not found: conf/igenomes_ignored.config
files_exist - File not found: ro-crate-metadata.json
nextflow_config - nf-validation has been detected in the pipeline. Please migrate to nf-schema: https://nextflow-io.github.io/nf-schema/latest/migration_guide/
schema_lint - Schema 'description' should be 'Metadata Transformation Pipeline'
Found: 'IRIDA Next Example Pipeline'
local_component_structure - samplesheet_check.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - input_check.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

files_exist - File is ignored: assets/nf-core-metadatatransformation_logo_light.png
files_exist - File is ignored: docs/images/nf-core-metadatatransformation_logo_light.png
files_exist - File is ignored: docs/images/nf-core-metadatatransformation_logo_dark.png
files_exist - File is ignored: .github/workflows/awstest.yml
files_exist - File is ignored: .github/workflows/awsfulltest.yml
files_exist - File is ignored: lib/Utils.groovy
files_exist - File is ignored: lib/WorkflowMain.groovy
files_exist - File is ignored: lib/NfcoreTemplate.groovy
files_exist - File is ignored: lib/WorkflowMetadatatransformation.groovy
nextflow_config - Config variable ignored: manifest.name
nextflow_config - Config variable ignored: manifest.homePage
nextflow_config - Config variable ignored: params.max_cpus
files_unchanged - File ignored due to lint config: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
files_unchanged - File ignored due to lint config: assets/email_template.html
files_unchanged - File ignored due to lint config: assets/email_template.txt
files_unchanged - File ignored due to lint config: assets/sendmail_template.txt
files_unchanged - File does not exist: assets/nf-core-metadatatransformation_logo_light.png
files_unchanged - File does not exist: docs/images/nf-core-metadatatransformation_logo_light.png
files_unchanged - File does not exist: docs/images/nf-core-metadatatransformation_logo_dark.png
files_unchanged - File ignored due to lint config: docs/README.md
actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/metadatatransformation/metadatatransformation/.github/workflows/awstest.yml
actions_awsfulltest - actions_awsfulltest
pipeline_name_conventions - pipeline_name_conventions

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: modules.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-metadatatransformation_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Found nf-validation plugin
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: 1.0.0dev
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.transformation= lock
nextflow_config - Config default value correct: params.metadata_1_header= metadata_1
nextflow_config - Config default value correct: params.metadata_2_header= metadata_2
nextflow_config - Config default value correct: params.metadata_3_header= metadata_3
nextflow_config - Config default value correct: params.metadata_4_header= metadata_4
nextflow_config - Config default value correct: params.metadata_5_header= metadata_5
nextflow_config - Config default value correct: params.metadata_6_header= metadata_6
nextflow_config - Config default value correct: params.metadata_7_header= metadata_7
nextflow_config - Config default value correct: params.metadata_8_header= metadata_8
nextflow_config - Config default value correct: params.max_cpus= 4
nextflow_config - Config default value correct: params.max_memory= 2.GB
nextflow_config - Config default value correct: params.max_time= 1.h
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.validate_params= true
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
readme - README Zenodo placeholder was replaced with DOI.
pipeline_todos - No TODO strings found
plugin_includes - No wrong validation plugin imports have been found
template_strings - Did not find any Jinja template strings (0 files)
schema_lint - Schema lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: branch.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
base_config - CUSTOM_DUMPSOFTWAREVERSIONS found in conf/base.config and Nextflow scripts.
modules_config - conf/modules.config found and not ignored.
modules_config - CUSTOM_DUMPSOFTWAREVERSIONS found in conf/modules.config and Nextflow scripts.
modules_config - LOCK_METADATA found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 3.2.0

Run details

nf-core/tools version 3.2.0
Run at 2025-02-19 18:58:41

…ort other transformations.

apetkau

Thanks so much for this work @emarinier 😄 . I have added some in-line comments.

tests/pipelines/integration.nf.test

apetkau

This looks great. Thanks so much @emarinier

nextflow.config

apetkau

This looks great to me. Thanks so much @emarinier 😄

emarinier added 13 commits January 23, 2025 12:59

Adding locking parameter.

10828a0

Basic CSV writing, IRIDA plugin not working.

6805867

Adding lock module.

e09d346

Channel operator for manually adding IDs to valid IDs.

32f44d0

Adding metadata.

f2fd862

Adding support for sample IDs and Irida IDs.

26e35e1

Updating to match latest plugin code, outputting both IDs.

b8c3112

Linting fixes.

3a182e4

Missing newlines.

0e7860e

Setting up basic tests.

20c1f9c

Parameter tests.

cdf4a54

More tests.

2acfcea

Updating docs.

da16c13

emarinier self-assigned this Feb 5, 2025

emarinier changed the base branch from main to dev February 5, 2025 22:00

emarinier added 3 commits February 7, 2025 09:51

Changing lock/locked.csv to transformation/transformation.csv to supp…

cc134da

…ort other transformations.

Prettier

50aa0cf

Formatting

e0accb3

apetkau requested changes Feb 10, 2025

View reviewed changes

tests/pipelines/integration.nf.test Outdated Show resolved Hide resolved

tests/pipelines/integration.nf.test Outdated Show resolved Hide resolved

tests/pipelines/integration.nf.test Outdated Show resolved Hide resolved

Correcting ID/sample/sample_name problems.

2de1af1

apetkau approved these changes Feb 12, 2025

View reviewed changes

nextflow.config Outdated Show resolved Hide resolved

emarinier added 4 commits February 18, 2025 13:30

Moving iridanext config to different file.

a432a04

nf-core lint files changed

a0140a2

Adding ignores.

cbf319e

Setting nf-core version.

2cf979b

apetkau approved these changes Feb 19, 2025

View reviewed changes

emarinier merged commit f2d736b into dev Feb 20, 2025
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds the Lock Transformation #1

Adds the Lock Transformation #1

emarinier commented Feb 5, 2025

github-actions bot commented Feb 5, 2025

github-actions bot commented Feb 5, 2025 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

apetkau left a comment

apetkau left a comment

apetkau left a comment

Adds the Lock Transformation #1

Adds the Lock Transformation #1

Conversation

emarinier commented Feb 5, 2025

github-actions bot commented Feb 5, 2025

This PR is against the main branch ❌

github-actions bot commented Feb 5, 2025 • edited Loading

nf-core pipelines lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

apetkau left a comment

Choose a reason for hiding this comment

apetkau left a comment

Choose a reason for hiding this comment

apetkau left a comment

Choose a reason for hiding this comment

This PR is against the `main` branch ❌

github-actions bot commented Feb 5, 2025 •

edited

Loading

`nf-core pipelines lint` overall result: Passed ✅ ⚠️