Strobealign evaluation

This repository provides runnable workflows for evaluating strobealign.

The Snakefile in this folder only generates simulated datasets. Some paper-specific evaluation workflows that use (a subset of) these datasets are available in the following subfolders:

• minmax/: Reproduces the min/max evaluation experiments in the paper "Designing efficient randstrobes for sequence similarity analyses" by Karami et al., 2023.
• mcs/: Reproduces the read alignment benchmarks in "Multi-context seeds enable fast and high-accuracy read mapping" by Tolstoganov et al., 2026.

The following workflow is not paper-specific:

• simx/: Run strobealign and other read mappers on a configurable list of datasets (sim0, sim4, ...) and libraries (fruitfly-100, CHM13-200, ...), measure accuracy, and plot the results.

Please see the README.md file in the respective subfolder.

To reproduce exact results from one of the papers, you should not use the main branch, but the correct exact commit (as documented in the READMEs).

The rest of this document describes the datasets generated by the workflow in this folder.

The workflow

The Snakefile in this folder downloads reference genomes and generates libraries of simulated reads, which are grouped into datasets named SIM0-SIM6, where SIM0 has the lowest mutation rate and SIM6 the highest.

Original paper

The original workflow was created for the following paper:

Sahlin, K. Strobealign: flexible seed size enables ultra-fast and accurate read alignment. Genome Biology 23, 260 (2022). https://doi.org/10.1186/s13059-022-02831-7

See the Supplementary Information in that paper, section "Note A: Simulations" for a description, and the original file at https://github.com/ksahlin/alignment_evaluation/blob/master/evaluation/Snakefile

Various changes have been made to the workflow since then, but the names of datasets have been kept to make comparisons across papers possible.

Some differences:

• The number of reads per library has been reduced to 1 million.
• The original paper mentions only SIM3 and SIM4 datasets.
• The original paper mentions only drosophila, maize, CHM13, rye genomes.
• Read lengths greater than 500 nt have been added.

Datasets

• SIM0: No variants. No read errors. (The reads are substrings of the reference.)
• SIM1: No variants. Simulated Illumina sequencing errors.
• SIM3: Low variation.
• SIM4: Medium variation.
• SIM5: High variation.
• SIM6: Very high variation.

See the Snakefile for the exact parameters (to mason_variator).

Libraries

Reads for the following genomes are generated:

• fruitfly (D. melanogaster)
• maize
• CHM13
• rye
• chrY (chromosome Y of CHM13)
• ecoli50 (simulated E. coli pangenome, see below)

Read lengths:

• Paired-end reads with 75, 100, 150, 200, 300 and 500,nt per read. Each library contains 1 million reads.
• Single-end reads with 1000, 5000 and 10,000,nt per read. Per-library read counts are 500,000, 100,000 and 50,000, respectively.

E. coli pangenome

The E. coli pangenome ecoli50 consists of 50 randomly selected E. coli assemblies from RefSeq. Because RefSeq changes, querying it is not reproducible. Therefore, a pre-generated list of 100 E. coli assembly accessions can be found in ecoli-accessions.txt. It was generated in the following way:

ncbi-genome-download --dry-run --assembly-levels complete --taxid 562 bacteria | sed 1d | cut -f1 | shuf | head -n 100 > ecoli-accessions.txt

The ecoli50 datasets uses the first 50 accessions from that list.

Running the workflow

First, make the required software available:

conda env create --file environment.yml
conda activate strobealign

Then run the workflow to download the genomes and simulate the reads:

snakemake --cores=all

This generates the folders downloads/, genomes/ and datasets/. The downloads/ folder contains intermediate files and can be deleted if desired.