Skip to content

Add TabPFN3-ESMC600M supervised DMS baseline - #108

Open
someguysomewhereoverthere wants to merge 4 commits into
OATML-Markslab:mainfrom
someguysomewhereoverthere:codex/add-tabpfn3-supervised-baseline
Open

Add TabPFN3-ESMC600M supervised DMS baseline#108
someguysomewhereoverthere wants to merge 4 commits into
OATML-Markslab:mainfrom
someguysomewhereoverthere:codex/add-tabpfn3-supervised-baseline

Conversation

@someguysomewhereoverthere

@someguysomewhereoverthere someguysomewhereoverthere commented Jul 1, 2026

Copy link
Copy Markdown

Summary

This PR adds the TabPFN3-ESMC600M supervised DMS substitution baseline.

The baseline uses TabPFN3 as the supervised tabular regressor with precomputed ESMC600M-derived protein features. It adds:

  • proteingym/baselines/tabpfn3_esmc600m/
  • scripts/scoring_DMS_supervised/scoring_TabPFN3_ESMC600M_substitutions.sh
  • supervised substitutions registration in config.json
  • supervised model metadata in proteingym/constants.json

Associated new model issue:

https://github.com/OATML-Markslab/ProteinGym/issues/109

The PR does not include raw ProteinGym data, derived feature matrices, model checkpoints, or generated score archives. Feature matrices are hosted on Zenodo, and TabPFN3 weights are obtained through the official Prior Labs / TabPFN license-acceptance flow.

Reference

Guan, D., Zhang, L., Wijesinghe, A., Zhu, A., Zhao, H., Power, H.,
Ahmed, F. H., Warden, A., Ong, C. S., & Steinberg, D. M. (2026).
Can Tabular In-Context Learners Generalize to Biomolecular Property Prediction?
arXiv:2606.31126. https://arxiv.org/abs/2606.31126

Reproduction

One-assay scoring:

CSV=/path/to/ProteinGym/cv/<DMS_id>.csv \
FEATURE_ROOT=/path/to/extracted/all_zs \
FOLD_COLUMN=fold_random_5 \
scripts/scoring_DMS_supervised/scoring_TabPFN3_ESMC600M_substitutions.sh

Full reproduction instructions and validation metadata are provided in the linked new-model issue.

External Artifacts

Feature artifact:

Record: https://zenodo.org/records/21137791
DOI: 10.5281/zenodo.21137791
Concept DOI: 10.5281/zenodo.21137790
File: proteingym_tabpfn3_esmc600m_all_zs_features_20260702.tar.zst
SHA256: 445f1b1dd966abf86799df8acde0d4b0c27d21b8f51fa3e68a64d2281e4b04bd

Complete model scores for all submitted mutants:

Record: https://zenodo.org/records/21146906
DOI: 10.5281/zenodo.21146906
Concept DOI: 10.5281/zenodo.21146905
Download: https://zenodo.org/records/21146906/files/tabpfn3_esmc600m_supervised_substitutions_scores_20260703.zip?download=1
File: tabpfn3_esmc600m_supervised_substitutions_scores_20260703.zip
SHA256: 37494a73e0a74d11e83b9796705c29ff260e4986566010dd6a2665d8ab442cc5
Zenodo MD5: d806c538c46e6d240458cca0c7ed8d2a

Score archive layout:

model_scores/supervised_substitutions/<cv_scheme>/tabpfn3_esmc600m/<DMS_id>.csv

Each score CSV contains:

mutant,y,y_pred,fold

Validation

The submitted TabPFN3-ESMC600M score package was validated as:

n_assays = 217
n_expected_files = 651
n_written_files = 651
n_issues = 0

@someguysomewhereoverthere someguysomewhereoverthere changed the title [New Model] TabPFN3 supervised DMS baseline Add TabPFN3-ESMC600M supervised DMS baseline Jul 3, 2026
@someguysomewhereoverthere

Copy link
Copy Markdown
Author

Updated this submission from the preliminary TabPFN3 entry to TabPFN3-ESMC600M. The new score archive is complete for 217 supervised substitution assays across fold_random_5, fold_modulo_5, and fold_contiguous_5, with 651/651 score files and zero validation issues. The display name and score location are now TabPFN3-ESMC600M / tabpfn3_esmc600m.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant