Deep Learning Framework for Transposable Element Discovery and Annotation

Table of Contents

• Introduction
• State-Of-The-Art Tool Performance
• Installation
• Testing
• Usage
• • Training
• • Inference
• • Library
• • Evaluation
• Supported TE tasks
• Integrating Custom models to PanTEon
• Computacional resources needed
• References
• Funding
• Citation

Introduction

PanTEon

PanTEon is an integrated framework for the identification, curation, classification, and library construction of transposable elements (TEs). It combines a large, high-quality TE database with a modular machine learning and deep learning software designed for scalable and reproducible TE analysis across diverse taxa.

PanTEon Database

The PanTEon Database is a freely available collection of almost 240,000 automatically curated transposable elements, spanning animals, plants, and fungi, and covering all major TE orders. The database was designed to maximize sequence fidelity, taxonomic diversity, and methodological consistency. It integrates:

• Curated sequences from Dfam (version 3.9)
• Automatically curated sequences from APTEdb (Pedro et al., 2021)
• Uncurated sequences from Dfam
• TE sequences from the Ensembl 2025 release (Dyer et al., 2025)

All uncurated sequences were originally generated by RepeatModeler2 (Flynn et al., 2020) and subsequently automatically curated using MCHelper (Orozco-Arias et al., 2024). Only TEs showing evidence of structural completeness and expected length profiles were retained, ensuring a high-confidence dataset suitable for training and benchmarking machine learning models.

The PanTEon Database is publicly available at: 🔗 (https://zenodo.org/records/18039746)

PanTEon Software

PanTEon includes a modular software framework that enables users to apply state-of-the-art machine learning and deep learning approaches to transposable element analysis. The software is organized into complementary modules:

• Training Module

  • Allows users to train one or multiple ML/DL models simultaneously from a FASTA file
  • Supports custom classification schemes, enabling classification beyond standard TE hierarchies
  • Facilitates fair comparison across different algorithms using the same training data

• Inference Module

  • Performs TE classification using one or multiple trained models (up to nine architectures in parallel)
  • Enables ensemble-based predictions and model comparison

• Library Construction Module

  • Generates custom TE libraries tailored to specific analytical needs
  • Supports filtering by taxonomic level (kingdom, phylum, class, order, family, or species)
  • Supports filtering by TE classification level (class, order, or superfamily)

This functionality is particularly useful for reducing the computational cost of genome annotation, as annotating genomes with very large TE libraries can be prohibitively expensive. PanTEon enables users to extract only the relevant subset of TEs, optimizing both accuracy and performance.

State-Of-The-Art Tool Performance

Below we present the most recent benchmarking results of state-of-the-art TE classification tools, evaluated using the PanTEon framework. All benchmarks were conducted with the PanTEon Database v1.6.1 (benchmark edition) and the built-in models provided by the PanTEon framework, ensuring a fair and standardized comparison across methods.

Benchmarking task

The evaluated task was TE superfamily classification, performed under a controlled and consistent setup:

• All species: 30 superfamilies
• Animalia: 30 superfamilies
• Plantae: 24 superfamilies
• Fungi: 24 superfamilies

Tool	Model type	# trainable parameters	Used features	F1 All species	F1 Animalia	F1 Plantae	F1 Fungi
BERTE	Transformers	9,199,830	Embedded DNA + k-mer frequencies	53.9	59	69	45
ClassifyTE	Stacking-based ML model	502,133	K-mer frequencies	87.4	88	90	69
CREATE	CNN+RNN	72,191,518	One-hot + k-mer frequencies	88.7	88	94	72
DeepTE	CNN	59,075,823	K-mer frequencies	88.7	88	94	70
Inpactor2_Class	CNN	546,830	K-mer frequencies	84.7	87	94	69
NeuralTE	CNN	603,710	Structural features + K-mers frequencies	92.2	92	94	82
TEclass2	Transformers	75,470,622	Embedded DNA	81.5	74	59	37
TERL	CNN	889,138	One-hot	73.7	77	87	64
Terrier	CNN	2,127,894	Embedded DNA	89.2	88	89	69

Installation

PanTEon can be installed using one of three different approaches:

1. Apptainer/Singularity container (recommended)
2. Miniforge/Miniconda environment YAML file
3. Manual installation using Miniforge/Miniconda

Although PanTEon can run on CPUs, we strongly recommend using GPUs whenever possible, particularly for model training. For convenience, this repository provides both GPU-enabled and CPU-only Apptainer/Singularity containers, as well as Conda environment files for each configuration.

First, clone the latest version of the repository:

git clone https://github.com/simonorozcoarias/PanTEon.git
cd PanTEon

1. Installation using Apptainer/Singularity (recommended)

First, install Apptainer (https://apptainer.org/) or Singularity on your system.

To build the GPU-enabled container:

apptainer build --fakeroot PanTEon_gpu.sif containers/PanTEon_gpu.def

To build the CPU-only container:

apptainer build --fakeroot PanTEon_cpu.sif containers/PanTEon_cpu.def

Once built, the container can be executed using:

apptainer exec --nv -B $PWD:/PanTEon PanTEon_gpu.sif python /PanTEon/PanTEon.py --help

For systems without GPUs:

apptainer exec -B $PWD:/PanTEon PanTEon_cpu.sif python /PanTEon/PanTEon.py --help

---

2. Installation using a Miniforge/Miniconda environment file

PanTEon provides pre-configured Conda environments for both GPU and CPU users.

GPU-enabled environment (recommended)

CONDA_OVERRIDE_CUDA="12.6" conda env create -f PanTEon_env.yml
conda activate PanTEon

Note: Replace 12.6 with the CUDA version available on your system if necessary.

CPU-only environment

conda env create -f PanTEon_env_cpu.yml
conda activate PanTEon

Warning: CPU execution is supported but may be considerably slower for training/inferring deep learning models.

---

3. Manual installation using Miniforge/Miniconda

For advanced users who require full control over package versions, PanTEon can also be installed by manually creating a Conda environment and installing all dependencies individually.

Detailed installation instructions are provided below.

1. batch: Environment creation

conda create -n PanTEon_env python=3.10.18 -c conda-forge -y
conda activate PanTEon_env

2. batch: tensorflow + CUDA (Replace the CONDA_OVERRIDE_CUDA with your own version if you are using GPUs, otherwise you can safely keep that intact).

CONDA_OVERRIDE_CUDA=12.2 conda install -c conda-forge -c bioconda \
  "tensorflow=2.18.0=cuda*" "tensorflow-base=2.18.0=cuda*" \
  "tensorflow-estimator=2.18.0=cuda*" tensorboard=2.18.0 \
  cudnn cuda-version=12 numpy=2.2.6 pandas=2.3.1 \
  scikit-learn=1.7.1 scipy=1.15.2 matplotlib=3.10.5 \
  seaborn=0.13.2 h5py=3.14.0 biopython=1.85 \
  openpyxl=3.1.5 tqdm joblib requests statsmodels \
  -y

3. batch: Bioinformatics and base packages

conda install -c conda-forge -c bioconda \
  blast entrez-direct openjdk perl wget \
  graphviz pydot xgboost \
  -y

4. batch: Pytoch and Hugging Face installation by pip.

pip install \
  torch==2.8.0 \
  torchvision==0.23.0 \
  torchmetrics==1.8.2 \
  transformers==4.55.0 \
  accelerate==1.10.0 \
  tokenizers==0.21.4 \
  safetensors==0.6.2 \
  huggingface-hub==0.34.4 \
  tf-keras==2.18.0

5. batch: Final PanTEon's dependencies.

pip install graphviz==0.21 plotly==6.5.0
pip install --no-deps hierarchicalsoftmax==1.4.4

pip install \
  anytree==2.13.0 \
  attrs==25.4.0 \
  click==8.3.1 \
  dm-tree==0.1.9 \
  filelock==3.18.0 \
  fsspec==2025.7.0 \
  jinja2==3.1.6 \
  networkx==3.4.2 \
  psutil==7.0.0 \
  regex==2025.7.34 \
  sympy==1.14.0 \
  typer==0.20.0

Now download the latest version of the PanTEon Database ((https://zenodo.org/records/18039747)) and place it in the data/ folder.

Next step consists in giving execution permissions to extra tools:

chmod +x tools/ltrsearch
chmod +x tools/itrsearch
chmod +x tools/RMout_to_bed.pl

Finally, access to PanTEon Database ((https://zenodo.org/records/18039747)) and download the trained models you are interested in (e.g. All species, Animalia, Plantae, etc). This directory is the one you need to indicate in the -d parameter when using the inference module.

Testing:

After successfully installing PanTEon_env, you can test it using the testing data contained in this repository.

Conda/Miniforge installation

First you must activate the conda environment:

conda activate PanTEon_env

or for CPU users:

conda activate PanTEon_env_cpu

In some cases, you will need to update some environment variables to point to the correct paths inside the conda environment (do this only one time per session):

export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH

Then, you should run the training module, using the following command. Some algorithms could take some minutes to complete the training, so please be patient:

python3 PanTEon.py training -f Test_data/sequences_toy.fasta -t 2 -d Test_data/testing_models -w Test_data/work_dir -n all

After the training is complete, you can now test the inference module:

python3 PanTEon.py inference -f Test_data/sequences_toy.fasta -t 2 -d Test_data/testing_models -w Test_data/work_dir/ -n all -p testing

Apptainer/Singularity installation

If you are using the GPU container:

apptainer exec --nv -B $PWD:/PanTEon PanTEon_gpu.sif python /PanTEon/PanTEon.py training -f /PanTEon/Test_data/sequences_toy.fasta -t 2 -d /PanTEon/Test_data/testing_models -w /PanTEon/Test_data/work_dir -n all

For inference:

apptainer exec --nv -B $PWD:/PanTEon PanTEon_gpu.sif python /PanTEon/PanTEon.py inference -f /PanTEon/Test_data/sequences_toy.fasta -t 2 -d /PanTEon/Test_data/testing_models -w /PanTEon/Test_data/work_dir -n all -p testing

For CPU-only users:

apptainer exec -B $PWD:/PanTEon PanTEon_cpu.sif python /PanTEon/PanTEon.py training -f /PanTEon/Test_data/sequences_toy.fasta -t 2 -d /PanTEon/Test_data/testing_models -w /PanTEon/Test_data/work_dir -n all

For inference:

apptainer exec -B $PWD:/PanTEon PanTEon_cpu.sif python /PanTEon/PanTEon.py inference -f /PanTEon/Test_data/sequences_toy.fasta -t 2 -d /PanTEon/Test_data/testing_models -w /PanTEon/Test_data/work_dir -n all -p testing

Usage

PanTEon provides four main modules:

• training
• inference
• library
• evaluation

Each module has its own set of parameters and options. We recommend consulting the help message before running a specific module.

Conda/Miniforge installation

Activate the environment:

conda activate PanTEon_env

Then, display the help message for a specific module:

python3 PanTEon.py training -h
python3 PanTEon.py inference -h
python3 PanTEon.py library -h
python3 PanTEon.py evaluation -h

Apptainer/Singularity installation

Display the general help message:

apptainer exec --nv \
    -B $PWD:/PanTEon \
    PanTEon_gpu.sif \
    python /PanTEon/PanTEon.py -h

Display the help message for a specific module:

apptainer exec --nv -B $PWD:/PanTEon PanTEon_gpu.sif python /PanTEon/PanTEon.py training -h

apptainer exec --nv -B $PWD:/PanTEon PanTEon_gpu.sif python /PanTEon/PanTEon.py inference -h

apptainer exec --nv -B $PWD:/PanTEon PanTEon_gpu.sif python /PanTEon/PanTEon.py library -h

apptainer exec --nv -B $PWD:/PanTEon PanTEon_gpu.sif python /PanTEon/PanTEon.py evaluation -h

For CPU-only containers, simply replace:

PanTEon_gpu.sif

with:

PanTEon_cpu.sif

in all commands above.

Note: The repository directory is mounted inside the container using:

-B $PWD:/PanTEon

This allows the containerized PanTEon installation to access input data, output directories, and configuration files directly from the host filesystem.

Training

usage: PanTEon.py training [-h] -f FASTA [-w WORK_DIR] -t THREADS [-n MODELS] -d MODELS_DIRECTORY [-z MIN_PROB] [-b BASED_MODELS] [-g GPUS]

Options

• -h, --help : Show this help message and exit

• -f FASTA, --fasta FASTA : Path to the TE FASTA file

• -w WORK_DIR, --work-dir WORK_DIR : Path to the working directory

• -t THREADS, --threads THREADS : Number of CPU threads to use

• -n MODELS, --models MODELS : Models to use (comma-separated)

  Available models:

  • All
  • NeuralTE
  • Terrier
  • CREATE
  • ClassifyTE
  • DeepTE
  • Inpactor2_Class
  • TERL
  • BERTE
  • TEClass2

• -d MODELS_DIRECTORY, --models_directory MODELS_DIRECTORY : Directory where trained models will be stored

• -z MIN_PROB, --min_prob MIN_PROB : Minimum probability threshold for TE classification

• -b BASED_MODELS : Pre-trained models used as initialization for transfer learning

• -g GPUS, --gpus GPUS : Number of GPUs used for training (default = 1)

Important: We strongly recommend creating a separate output directory for each execution to avoid overwriting results from previous runs.

Inference

usage: PanTEon.py inference [-h] -f FASTA -t THREADS -w WORK_DIR [-n MODELS] -d MODELS_DIRECTORY -p PREFIX [-z MIN_PROB]

Options

• -h, --help : Show this help message and exit
• -f FASTA, --fasta FASTA : Path to the TE FASTA file
• -t THREADS, --threads THREADS : Number of CPU threads to use
• -w WORK_DIR, --work-dir WORK_DIR : Path to the working directory
• -n MODELS, --models MODELS : Models to use (comma-separated)
• -d MODELS_DIRECTORY, --models_directory MODELS_DIRECTORY : Directory containing trained models
• -p PREFIX, --prefix PREFIX : Prefix for output files
• -z MIN_PROB, --min_prob MIN_PROB : Minimum classification probability (default = 0.6)

---

Library

usage: PanTEon.py library [-h] [--taxon TAXON] [--req_class REQ_CLASS] [--view_only]

Options

• -h, --help : Show this help message and exit
• --taxon TAXON : Taxonomic group (e.g. Plantae, Chordata, Fungi)
• --req_class REQ_CLASS : TE classification (e.g. ClassI, LTR, Gypsy, Helitron)
• --view_only : Display the report without generating a FASTA file

Evaluation

usage: PanTEon.py evaluation [-h] --true_fasta TRUE_FASTA --pred_fasta PRED_FASTA [--level LEVEL] [--out_confusion OUT_CONFUSION] [--out_report OUT_REPORT]

Supported TE tasks

PanTEon was designed as a standardized and modular platform covering the full spectrum of TE analysis, from detection from genomes to curated TE libraries and annotations, while explicitly enabling the simultaneous training, inference, and comparison of multiple machine learning and deep learning models. PanTEon also allows users to seamlessly integrate their own custom neural networks into the framework.

TE Classification

What it does

TE classification assigns curated sequences to hierarchical TE categories such as class, order, superfamily, or any user-defined classification scheme (e.g. TEs vs non-TEs, chimeric sequences vs intact sequences, etc).

How it is performed in PanTEon (--task classification parameter)

Within PanTEon, inputs required to perform the classification are:

• a fasta file containing sequences, with the classification integrated into the IDs (after a "#" symbol).

The PanTEon's output are:

• A set of trained models inside a directory (specified by the -d parameter; training module only)
• A directory with several training plots for each trained ML/DL algorithm (training module only)
• a merged report summaring the performance of all the trained algorithms (training module only)
• a fasta file with the sequences and the predicted classifications (inference module only)
• a CSV fil containing the predicted classification per sequence provided by each trained algorithm (inference module only)

Which ML/DL approaches are integrated into PanTEon

• BERTE (Y. Chen et al., 2024)
• CREATE (Qi et al., 2025)
• ClassifyTE (Panta et al., 2021)
• DeepTE (Yan et al., 2020)
• Inpactor2_Class (Orozco-Arias et al,. 2023)
• NeuralTE (Hu et al., 2024)
• TEClass2 (Bickmann et al., 2023)
• TERL (da Cruz et al., 2021)
• Terrier (Turnbull et al., 2025b)

TE Identification

What it does

The identification task focuses on detecting candidate TE sequences and their genomic coordinates, providing the initial input for downstream analysis.

How it is performed in PanTEon (--task identification parameter)

Within PanTEon, inputs required to perform the identification are:

• a fasta file containing long sequences containing (or not) TEs.
• A GFF file with the coordinate information of each TE inserted into the sequences contained in the fasta file (training module only)

The PanTEon's output are:

• A set of trained models inside a directory (specified by the -d parameter; training module only)
• A directory with several training plots for each trained ML/DL algorithm (training module only)
• a GFF containing the predicted coordinates for all detected TEs (inference module only)

Which ML/DL approaches are integrated into PanTEon

• No approach have been developed to date to tack this task

TE trimming

What it does

This task focuses on finding the correct TE ends in a longer sequences, in the case of over-extended or chimeric sequences.

How it is performed in PanTEon (--task trimming parameter)

Within PanTEon, inputs required to perform the classification are:

• a fasta file containing sequences, with the correct starting and ending positions (between 0-1) integrated into the IDs, separated by "_" symbol (e.g. Sequence1_0.0227_0.25).

The PanTEon's output are:

• A set of trained models inside a directory (specified by the -d parameter; training module only)
• A directory with several training plots for each trained ML/DL algorithm (training module only)
• a merged report summaring the performance of all the trained algorithms (training module only)
• a fasta file with the trimming sequences (inference module only)
• a CSV fil containing the predicted starting and ending positions per sequence provided by each trained algorithm (inference module only)

Which ML/DL approaches are integrated into PanTEon

• No approach have been developed to date to tack this task

Integrating Custom models to PanTEon

PanTEon is designed to be a flexible and extensible framework, allowing users to develop and integrate their own ML or DL architectures seamlessly alongside the built-in models.

Custom Model Requirements

To be compatible with PanTEon, custom models must meet the following requirements:

• Be implemented using TensorFlow or PyTorch as the deep learning framework
• Be written as a Python script
• Define three mandatory functions: load_data, get_model, and run_experiment (see figure below for a detailed description of their expected behavior)
• Define two attributes: DL_FRAMEWORK and superf_dict
• Be placed inside the Custom_classifiers/ directory of the PanTEon framework

Automatic Model Discovery

PanTEon automatically detects and loads custom models at runtime. Every time the training or inference module is executed:

• All Python scripts located in the Custom_classifiers/ folder are automatically imported
• Custom models become immediately available for training, and inference modules
• User-defined architectures will be evaluated and compared under the same conditions as the built-in models

This design allows researchers to rapidly prototype new architectures and directly assess their performance within a standardized and reproducible TE analysis framework.

Why This Matters

By supporting custom models, PanTEon functions not only as a TE analysis tool, but also as a benchmarking platform for ML/DL research on TEs, enabling fair comparisons between novel and established architectures.

You can fin two examples (one script using tensorflow and another using pytorch) in the Custom_classifiers folder.

References

If you use any of the ML/DL algorithms integrated into PanTEon, please cite them as following:

• Chen, Y., Qi, Y., Wu, Y., Zhang, F., Liao, X., & Shang, X. (2024). BERTE: High-precision hierarchical classification of transposable elements by a transfer learning method with BERT pre-trained model and convolutional neural network. BioRxiv, 2024-01.
• Qi, Y., Chen, Y., Wu, Y., Guo, Y., Gao, M., Zhang, F., Liao, X., & Shang, X. (2025). CREATE: a novel attention-based framework for efficient classification of transposable elements. Briefings in Bioinformatics, 26(6), bbaf608.
• Panta, M., Mishra, A., Hoque, M. T., & Atallah, J. (2021). ClassifyTE: a stacking-based prediction of hierarchical classification of transposable elements. Bioinformatics, 37(17), 2529-2536.
• Yan, H., Bombarely, A., & Li, S. (2020). DeepTE: a computational method for de novo classification of transposons with convolutional neural network. Bioinformatics, 36(15), 4269-4275.
• Orozco-Arias, S., Humberto Lopez-Murillo, L., Candamil-Cortés, M. S., Arias, M., Jaimes, P. A., Rossi Paschoal, A., Tabares-Soto, R., Isaza, G., & Guyot, R. (2022). Inpactor2: a software based on deep learning to identify and classify LTR-retrotransposons in plant genomes. Briefings in Bioinformatics.
• Hu, K., Xu, M., Gao, X., & Wang, J. (2024). NeuralTE: an accurate approach for Transposable Element superfamily classification with multi-feature fusion. Proceedings of the 15th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, 1-10.
• Bickmann, L., Rodriguez, M., Jiang, X., & Makałowski, W. (2025). Transformer-Based Classification of Transposable Element Consensus Sequences with TEclass2. Biology, 15(1), 59.
• da Cruz, M. H. P., Domingues, D. S., Saito, P. T. M., Paschoal, A. R., & Bugatti, P. H. (2021). TERL: classification of transposable elements by convolutional neural networks. Briefings in bioinformatics, 22(3), bbaa185.
• Turnbull, R., Young, N. D., Tescari, E., Skerratt, L. F., & Kosch, T. A. (2025). Terrier: A deep learning repeat classifier. Briefings in Bioinformatics, 26(4), bbaf442.

Other resources used in this work:

• Storer, J., Hubley, R., Rosen, J., Wheeler, T. J., & Smit, A. F. (2021). The Dfam community resource of transposable element families, sequence models, and genome annotations. Mobile DNA, 12(1), 1-14.
• Flynn, J. M., Hubley, R., Goubert, C., Rosen, J., Clark, A. G., Feschotte, C., & Smit, A. F. (2020). RepeatModeler2 for automated genomic discovery of transposable element families. Proceedings of the National Academy of Sciences, 117(17), 9451-9457.
• Orozco-Arias, S., Sierra, P., Durbin, R., & González, J. (2024). MCHelper automatically curates transposable element libraries across eukaryotic species. Genome Research, 34(12), 2256-2268.
• Pedro, D. L. F., Amorim, T. S., Varani, A., Guyot, R., Domingues, D. S., & Paschoal, A. R. (2021). An atlas of plant transposable elements. F1000Research, 10, 1194.
• Dyer, S. C., Austine-Orimoloye, O., Azov, A. G., Barba, M., Barnes, I., Barrera-Enriquez, V. P., Becker, A., Bennett, R., Beracochea, M., Berry, A., & others. (2025). Ensembl 2025. Nucleic acids research, 53(D1), D948-D957.

Funding

Simon Orozco-Arias is supported by a fellowship within the “Generación D” initiative, Red.es, Ministerio para la Transformación Digital y de la Función Pública, for talent attraction (C005/24-ED CV1). Funded by the European Union NextGenerationEU funds, through PRTR.

Citation

if you use this framework, the database or any part of it (as the neural network architectures, functions, etc), please cite us as following:

• Orozco-Arias, S., Ferrer-Pomer, I., Rodrigues de Goes, F., Gaviria-Orrego, S., Gomiz-Fernandez, J., Llatser-Torres, J., Paschoal, A. R., Guyot, R., & Gabaldon, T. (2026). PanTEon: a cross-kingdom framework to guide the design of transposable element classifiers. bioRxiv, 2026-04.