Tag in output FASTQ file

[ChangqingW]

Would be great if you cold have the CB and UB tags in the output FASTQ file:

head -n1 test.fastq
@TCTTCCTTCAATCTCT_GTATACCCGCTT#m84073_240809_024049_s1/255004042/ccs/16_2868_+1of2_C    CB:Z:TCTTCCTTCAATCTCT   UB:Z:GTATACCCGCTT
head -n1 test_polished.fastq
@consensus_0 MI:Z:TCTTCCTTCAATCTCT_GTATACCCGCTT XT:Z:single XC:i:1 XR:i:1

Also, the tags should be delimited by tabs instead of spaces, otherwise minimap2 will treat it as read ID and the resulting BAM file would not include them as AUX fields.

[olliecheng]

Hi! This is already implemented in development and will be available in the next release.

Currently, the MI (molecular identifier) tag is used instead of CB and UB, reflecting the fact that:

• MI is a combination of both CB and UB
• Users do not always have a binary identifier consisting of a cell-specific barcode and molecule-specific UMI i.e. split-seq

What do you think? Is CB/UB important?

[ChangqingW]

I see, that make sense. I was asking because oarfish's single-cell quantification requires the CB tag but I guess I could always just pipe the output to awk.
btw do you have an eta for gzipped input support?

[ChangqingW]

Are you going to put this on crates.io? If you do then I could smuggle a nailpolish binary to FLAMES via cargo

[olliecheng]

oarfish's single-cell quantification requires the CB tag

I see. In this case, it is specifically important that CB represents the barcode only (so that i.e. a single cell is identifiable by barcode)? If it is necessary that barcode and UMI are individually separated, I can perhaps implement named capture group support in the header so a format like (?<CB>[ATCG]{16})_(?<UB>[ATCG]{16}) produces the CB: and UB: tags?

(As an aside: would this be useful in Flexiplex as well??)

Re: putting Nailpolish on crates.io, it's a possibility. If cross-platform support isn't too important, it might be faster to download a premade binary. It is unclear whether Nailpolish will compile on Windows; I haven't tested it. At the very least, CMake will be necessary on the target machine to compile the SPOA Rust bindings.

At the moment, it's probably best to fix FLAMES to a set build such as https://github.com/DavidsonGroup/nailpolish/releases/download/v0.2.0/nailpolish-linux, since the API is changing rapidly. I also have nightly builds which auto-build on every commit.

In the next few days I should have 0.2.1 ready, which will change the output format to:

@processed_12112_1      MI:Z:GTCGTTCCACTTTATC_TTCCAATTGCGC      nI:i:12112      nT:Z:consensus  nC:i:1  nL:i:2
...
@processed_12112_2      MI:Z:GTCGTTCCACTTTATC_TTCCAATTGCGC      nI:i:12112      nT:Z:consensus  nC:i:2  nL:i:2

Of note: in this case, there are 4 reads which have the same barcode and UMI, but have been clustered into two consensus sequences, so they share the same MI but have different cluster indices.

[olliecheng]

gzipped input support is in the works but probably a week or more away from completion.

[ChangqingW]

oarfish currently only use the CB tag, to identify single-cells. Since nailpolish already collapsed the UMIs I guess the UB tag is not that important.

[olliecheng]

In that case, what do you think is the most useful implementation? Something like this?

I can perhaps implement named capture group support in the header regex so a format like (?[ATCG]{16})_(?[ATCG]{16}) produces the CB: and UB: tags?

[ChangqingW]

That sounds very handy but I am not sure if I understood this correctly, is this --barcode-regex how you specify nailpolish to identify barcodes and UMIs from the read ID? And you were proposing adding a capture group so nailpolish would know which bit is the barcode / UMI so it could output them?
I guess the bit I am confused is that, does nailpolish currently treat BC-UMI as a whole thing? I was assuming that for consensus read calling, exact match were used for BC and fuzzy matching were used to group the reads.

[olliecheng]

Nailpolish treats BC-UMI as the whole thing, because fuzzy matching the UMIs would require an alignment (either reference-free or otherwise) and the computational requirements of that technique might be really high. For each duplicate group of BC-UMIs, Nailpolish uses further heuristics to separate 'true duplicates' from 'false duplicates' (which have the same BC-UMI but have different biological origin). So a 'true duplicate' group may be clustered into multiple subgroups.

[ChangqingW]

I see, in that case yes a capture group for BC would probably be helpful.

[olliecheng]

Implemented and enabled by default in 29ffe26, this will be available in v0.2.1. As an example of how to use it, see src/cli/preset.rs:

impl PresetBarcodeFormats {
    /// Returns the corresponding regex for the preset
    pub fn to_regex(&self) -> Result<Regex, regex::Error> {
        Regex::new(match self {
            PresetBarcodeFormats::BcUmi => r"^(?<CB>[ATCGNX]{16})_(?<UB>[ATCGNX]{12})",
            PresetBarcodeFormats::UmiTools => r"_(?<UB>[ATCGNX]+)$",
            PresetBarcodeFormats::Illumina => r":(?<UB>[ATCGNX]+)$",
            PresetBarcodeFormats::SamTaggedCB => r"\t:CB:Z:(?<CB>[ATCGNX]+)",
            // this one is a bit annoying. it's two regexes, connected with an OR:
            //   regex 1: \t:CB:Z:(?<CB>[ATCGNX]+).*\t:UB:Z:(?<UB>[ATCGNX]+
            //   regex 2: \t:UB:Z:(?<UB>[ATCGNX]+).*\t:CB:Z:(?<CB>[ATCGNX]+)
            // this accounts for the fact that the order of CB and UB may be switched around.
            PresetBarcodeFormats::SamTaggedCBUB => {
                r"(?:\t:CB:Z:(?<CB>[ATCGNX]+).*\t:UB:Z:(?<UB>[ATCGNX]+))|(?:\t:UB:Z:(?<UB>[ATCGNX]+).*\t:CB:Z:(?<CB>[ATCGNX]+))"
            }
        })
    }
}

This also supports the :CB:Z: and :UB:Z: tags in the source .fastq using the sam-tagged-cb and sam-tagged-cbub preset:

Create an index file from a demultiplexed .fastq

Usage: nailpolish index [OPTIONS] <INPUT> [PRESET]

Arguments:
  <INPUT>
          the input .fastq file

  [PRESET]
          [default: bc-umi]

          Possible values:
          - bc-umi:          @BARCODE_UMI format as produced by Flexiplex for 10x3 chemistry
          - umi-tools:       `_<UMI>` format as produced by `umi-tools extract`
          - illumina:        bcl2fastq format, which has `:<UMI>` at the end of the read ID
          - sam-tagged-cbub: .sam tag format with barcode and UMI, which uses the :CB:Z:____ and :UB:Z:____ tag format
          - sam-tagged-cb:   .sam tag format with only barcode, which uses the :CB:Z:___ format

Options:
      --overwrite
          overwrite an existing index file, if it exists
    ...

[olliecheng]

You can get a build here for your testing for now:

https://github.com/DavidsonGroup/nailpolish/actions/runs/16106521030