Validate multi-ancestor approaches at scale

[jeromekelleher]

Support exists for defining multiple ancestors VCZ files, and combining these. Check that this works at scale and add more testing.

[Duncan-JR]

What is the application you have in mind for this feature? My understanding was that, in the case of multiple data sources, we integrate the variants together before inferring a single ancestor VCZ file for the chromosome. Is it for cases where the user already has a bunch of ancestors VCZ files for non-overlapping regions of a chromosome, and wants to match against the union of them to form a single ARG?

[jeromekelleher]

The basic idea is that you might want to run ancestor inference separately on different datasets, for whatever reason. This is in addition to the application where we combine multiple sources to generate ancestors as you describe.

[gregorgorjanc]

Would this be useful for datasets that have samples from deeply split populations (I have Bos Taurus and Bos Indicus in mind or European vs Asian pig)? If so, even deeply split populations coalesce at some point - is there a step of "merging" the two ancestor files. We could give this a try on our ~1000 diploid samples data or are you looking for larger test cases?

[jeromekelleher]

This was the type of thing I was thinking about. Take the high frequency variants from both to generate joint ancestors and generate the ancestors separately for the lower frequency variants. You can then match across all three sets.

This should be straightforward to express now in a single config file, using bcftools filters to define the sources.

[jeromekelleher]

One issue that'll need to be resolved is how we define the sequence intervals. Currently it's computed inline during ancestor generation using the max_gap_length parameter, but this won't work across multiple ancestor files. More sophisticated approaches based on site density over the filtered/unfiltered sites will likely be required also.