Idea: DNA Anotation in Gramps Web

[PaulJ00]

I would like to propose a feature that allows users to upload their Whole Genome Sequencing (WGS) data in VCF format and perform basic, understandable analysis without requiring deep DNA or bioinformatics knowledge.
The following concept first describes a basic DNA annotation but can be expanded into a comprehensive ancestry report.

Feature Idea

The idea is to enable users to upload their VCF file and then:

• Match detected SNPs against open-source genomic databases
• Display global distribution and frequency information for variants
• Provide simple, visual insights rather than complex raw data

Possible Data Sources

The following open-source databases could be used:

• gnomAD – variant frequencies across populations
• 1000 Genomes Project – population-level reference data
• Single-gene frequency and distribution analysis

Optionally:

• ClinVar – basic health-related variant annotations

  • This would be strictly informational and only shown if allowed by the admin
  • I understand this may be considered off-topic or sensitive

Motivation

One reason this idea is important to me is that there is currently no open-source, self-hostable solution aimed at non-DNA experts that allows simple analysis of personal WGS data.

Most existing tools are:

• Highly technical
• Pipeline-based
• Focused on bioinformatics professionals

This feature could bridge the gap between genealogy, ancestry, and basic genomic exploration.

Technical Feasibility

From a technical perspective:

• Matching SNPs from a VCF file to databases is already well-supported by existing bioinformatics frameworks
• Many open-source tools and libraries make this process relatively straightforward
• Creating a full ancestry report is much more complex could be addet in the future

Scope Consideration

I am aware that DNA analysis may be partially outside classic genealogy.
However, a limited, well-defined scope (basic matching and visualization) could add value without overcomplicating the project.

Thank you for considering this idea.

[DavidMStraub]

Although I don't think this feature would be used by most current Gramps Web users, I personally find it super interesting, so if there are open source libraries we can leverage, why not?

Some questions:

• Is there some sample data that could be used to develop such a feature?
• Can you recommend a WGS provider?

I think the VCF files would be stored as media objects in Gramps, so the first step in implementing such a feature would be to decide what (read-only) REST API endpoints we could set up that take the raw data (and whatever freely available external information we need) and compute/derive some JSON data from it that we could then use for visualization down the line.

Could you sketch something for this (Python-only) part?

[PaulJ00]

Is there some sample data that could be used to develop such a feature?

I would simply create a new VCF file with the header copied from a real one and paste some SNPs into it just for testing.

Can you recommend a WGS provider?

I personally like:

• TellMeGen

  • Good for privacy because of EU privacy laws
  • Based in Spain / Lab in Denmark
  • No need for a real name or identification
  • Test-Kit can be bought on Amazon to avoid sharing payment details to the WGS provider

• YSEQ

  • Small provider from Germany with no direct insight, but if you only need the DNA files, it sounds good

Other larger providers are:

• Nebula Genomics (USA-based)
• Dante Labs (international)

Could you sketch something for this (Python-only) part?

I can sketch some concrete examples in the near future.

I think it would be useful to use Ensembl VEP to annotate the VCF.

From the command line, it would only take the following to annotate the VCF with two SNP databases (ChatGPT example):

vep \
  -i input.vcf.gz \
  -o annotated.vcf.gz \
  --vcf --compress_output bgzip \
  --cache --offline --assembly GRCh38 \
  --af_gnomad \
  --custom file=clinvar.vcf.gz,short_name=ClinVar,format=vcf,type=exact,coords=0,fields=CLNSIG%CLNREVSTAT%CLNDN

After that, you can find information such as allele frequencies and pathogenicity in the VCF file.

An alternative to Ensembl VEP would be bcftools.

One important thing to check prior to annotation is whether the VCF is based on the human reference genome GRCh38 or GRCh37.

Youtube-Video about Ensemble VEP

[PaulJ00]

As I am currently in the exploration phase of this idea, I will paste some repositories here that are worth looking into:

• https://github.com/bcgsc/ntRoot
  • A Python framework that computes a genome-wide ancestry report.
  • Possible Reports could look like this:

-

Circular genome representations of ancestry inference using WGA for (a) European HuRef (Levy et al. 2007) and HuRefPrime haplotypes, (b) Han Chinese CN1 (Yang et al. 2023) (mat: maternal, pat: paternal) haplotypes and (c) simulated HuRef/CN1 diploid rearrangement. The outer track shows ntRoot local ancestry assignments (LAI) along each chromosome using 5Mbp tiles (AMR: Admixed American; AFR: African; EUR: European; SAS: South Asian; EAS: East Asian), with each subsequent inner track showing the corresponding SNV density for each super-population. Inner pie charts report the predicted global ancestry composition (GAI).

• Source: https://academic.oup.com/bioinformaticsadvances/article/5/1/vbaf287/8317431

Another thing to consider is that these computations require large amounts of data such as reference genomes and possible compute.
If we implement functionality like this, only users who actually need it should be required to download the data.

[PaulJ00]

Another repo to consider is ADMIXTOOLS.
It offers many tools and can, for example, calculate genetic distances to ancient populations.

Here are some descriptions of the tools it has to offer (AI-generated).

smartpca
Use PCA to place your sample relative to reference populations and visualize “where you cluster.” It’s the clearest, most report-friendly first view.

qp3Pop (outgroup f3)
Use it to rank which reference populations share the most drift with your sample (a “closest matches” list). It’s a simple, interpretable similarity metric.

qpDstat
Use D-statistics to test specific claims like “am I closer to Reference A or Reference B?” It’s a clean way to confirm/deny borderline PCA impressions.

qpAdm
Use it to produce mixture proportion estimates (your “X% A + Y% B” style output), but only relative to the sources you choose. It’s the closest thing in ADMIXTOOLS to ancestry percentages—still model-dependent.

convertf
Use it to get your data into EIGENSTRAT format so the tools above can run. It’s not “analysis,” but it’s essential plumbing.

Example:
https://www.eupedia.com/forum/threads/using-admixtools2-to-model-admixture.44759/

[PaulJ00]

I will report some of my learnings:

There are many tests that can be used to estimate ancient DNA ancestry. The two most common are PCA-based and admixture-based tests. For identifying close relationships, there are also tools like ChromoPainter and fineSTRUCTURE, but I won’t focus on these here.

PCA
This creates and plot you on a graph, where you can measure the distance to other clusters. An example is G25 (Global25), where you can calculate and project your own position on the G25 plot.
There you see your distance the other populations.

Admixture
This gives you an output like “15% Albanian” and “30% Turkish.” There are global datasets like Dodecad K12b, where you get ancestry weights across 12 populations.
There is also ADMIXTOOLS2, an R-based framework for building ancestry models. It is often combined with the AADR 1240K dataset (Harvard), which contains standardized ancient DNA data. You can create models with ADMIXTOOLS2-qpAdm and evaluate them yourself in terms of significance and error, or compute many models and use an algorithm to select the best-fitting one.
I tried it myself and it worked, but it’s difficult to create these models because you have to choose the right parameters and populations to get a statistically significant result without an huge standard error.

Right now, only ADMIXTOOLS 2 (with qpAdm) seems open source to me.
To get your G25 coordinates, you apparently have to pay.
Dodecad is from 2012, and I’m not sure what the licensing terms are.

It would be possible to create your own PCA plot using PLINK 2.0 or flashpca and a dataset like 1000 Genomes or AADR.
But that would make the whole process much more complex than simply using an existing library and integrating your sample.