Skip to content

Add XSignal zero-shot baseline - #112

Open
Billwanttobetop wants to merge 1 commit into
OATML-Markslab:mainfrom
Billwanttobetop:add-xsignal-baseline-audited
Open

Add XSignal zero-shot baseline#112
Billwanttobetop wants to merge 1 commit into
OATML-Markslab:mainfrom
Billwanttobetop:add-xsignal-baseline-audited

Conversation

@Billwanttobetop

Copy link
Copy Markdown

Summary

This PR adds XSignal, a zero-shot evidence-integration baseline for ProteinGym substitution assays. XSignal derives 14 mutation-scoring channels from ProteinGym MSAs, MSA weights, precomputed AlphaFold2 structures, and fixed amino-acid physicochemical descriptors. Each channel is transformed to assay-wise percentile ranks and the final score is an equal-weight average.

The final scorer reads only variant-key columns from DMS assay CSVs (mutant, mutated_sequence). It does not read DMS labels, binary labels, supervised folds, public baseline score packages, or official merged-score files during final scoring. XSignal uses no pretrained protein language model weights; it does use precomputed AlphaFold2 structures and ten self-generated intermediate evidence score directories.

Score Package

Audited release: https://github.com/Billwanttobetop/ProteinGym/releases/tag/xsignal-proteingym-scores-20260723-audited

Score asset: XSignal_scores_substitutions_20260723.zip

  • 217 ProteinGym substitution assays
  • 2,465,767 variants
  • columns: mutant, mutated_sequence, XSignal_score
  • SHA256: 0e3c9fa348d83b60e23ee3f57a7c1d517ea1109cc277556bdc8598762538b367

Evaluation

Official ProteinGym wrapper metrics on the full substitution benchmark:

Metric Score
Spearman 0.476
AUC 0.761
MCC 0.369
NDCG 0.778
Top recall 0.213

A full final-fusion reproduction regenerated 217 assay score files / 2,465,767 variants from the archived upstream XSignal sensor outputs plus ProteinGym MSA weights and AF2 structures, matching the frozen internal candidate PureGraphGEContactPair_v1 with max_abs_diff = 0.0. The release also includes a compact audit/provenance bundle.

Scope Boundary

This submission is for the frozen XSignal substitution score package only. Public RefSeq MSA expansion experiments and private/non-public datasets are not included.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant