|
| 1 | +# MOOZY: A Patient-First Foundation Model for Computational Pathology |
| 2 | + |
| 3 | +<p align="center"> |
| 4 | + <a href="https://atlasanalyticslab.github.io/MOOZY/"><img src="https://img.shields.io/badge/Project-Page-4285F4?logo=googlechrome&logoColor=white" alt="Project Page"></a> |
| 5 | + <a href="https://arxiv.org/abs/2603.27048"><img src="https://img.shields.io/badge/arXiv-2603.27048-B31B1B?logo=arxiv" alt="arXiv"></a> |
| 6 | + <a href="https://huggingface.co/AtlasAnalyticsLab/MOOZY"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace-Model-yellow" alt="HuggingFace"></a> |
| 7 | + <a href="https://pypi.org/project/moozy/"><img src="https://img.shields.io/pypi/v/moozy?logo=pypi&logoColor=white&label=PyPI" alt="PyPI"></a> |
| 8 | + <a href="https://github.com/AtlasAnalyticsLab/MOOZY/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-lightgrey" alt="License"></a> |
| 9 | + <a href="https://www.python.org/"><img src="https://img.shields.io/badge/Python-3.10%2B-blue?logo=python&logoColor=white" alt="Python 3.10+"></a> |
| 10 | +</p> |
| 11 | + |
| 12 | +<p align="center"> |
| 13 | + <img src="https://raw.githubusercontent.com/AtlasAnalyticsLab/MOOZY/main/assets/paper_figures/data_scale_overview.png" alt="MOOZY: a patient-first foundation model for computational pathology, whole-slide image encoding, and case-level representation learning"> |
| 14 | +</p> |
| 15 | + |
| 16 | +**MOOZY is a foundation model for computational pathology that treats the patient case, not the individual slide, as the fundamental unit of representation.** It encodes one or more whole-slide images (WSIs) into a single 768-dimensional case-level embedding that captures dependencies across all slides from the same patient. Trained entirely on public data with 85.8M parameters (14x smaller than GigaPath), MOOZY outperforms larger models on classification tasks across diverse organs and cancer types. |
| 17 | + |
| 18 | +--- |
| 19 | + |
| 20 | +## Table of Contents |
| 21 | + |
| 22 | +- [News](#news) |
| 23 | +- [Quick Start](#quick-start) |
| 24 | + - [Environment Setup](#environment-setup) |
| 25 | + - [Using the Output](#using-the-output) |
| 26 | +- [Method Overview](#method-overview) |
| 27 | +- [Training](#training) |
| 28 | + - [Scripts](#scripts) |
| 29 | + - [SLURM Jobs](#slurm-jobs) |
| 30 | +- [Notes from the Authors](#notes-from-the-authors) |
| 31 | + - [On using Linear vs non-Linear classifier](#on-using-linear-vs-non-linear-classifier) |
| 32 | + - [On the strength of Stage 1 alone](#on-the-strength-of-stage-1-alone) |
| 33 | +- [Acknowledgment](#acknowledgment) |
| 34 | +- [Citation](#citation) |
| 35 | +- [Contact](#contact) |
| 36 | +- [License](#license) |
| 37 | + |
| 38 | +## News |
| 39 | + |
| 40 | +- **[2026/04]** MOOZY is now public! |
| 41 | + |
| 42 | +## Quick Start |
| 43 | + |
| 44 | +```bash |
| 45 | +pip install moozy |
| 46 | +``` |
| 47 | + |
| 48 | +Model weights download automatically on first use. No access gates, no manual downloads, no HuggingFace approval. |
| 49 | + |
| 50 | +```bash |
| 51 | +# Encode a patient case from pre-extracted H5 feature files |
| 52 | +moozy encode slide_1.h5 slide_2.h5 --output case_embedding.h5 |
| 53 | + |
| 54 | +# Encode directly from raw whole-slide images |
| 55 | +moozy encode slide_1.svs slide_2.svs --output case_embedding.h5 |
| 56 | +``` |
| 57 | + |
| 58 | +Or use the Python API: |
| 59 | + |
| 60 | +```python |
| 61 | +from moozy.encoding import run_encoding |
| 62 | + |
| 63 | +run_encoding( |
| 64 | + slide_paths=["slide_1.h5", "slide_2.h5"], |
| 65 | + output_path="case_embedding.h5", |
| 66 | +) |
| 67 | +``` |
| 68 | + |
| 69 | +The output H5 file contains a 768-d case-level embedding ready for downstream tasks: classification, survival prediction, or retrieval. |
| 70 | + |
| 71 | +All encoding arguments (data, runtime, raw WSI options, mixed precision) are documented in [docs/encode.md](https://github.com/AtlasAnalyticsLab/MOOZY/blob/main/docs/encode.md). |
| 72 | + |
| 73 | +### Environment Setup |
| 74 | + |
| 75 | +```bash |
| 76 | +conda create -n moozy python=3.12 -y |
| 77 | +conda activate moozy |
| 78 | +pip install moozy |
| 79 | +``` |
| 80 | + |
| 81 | +<details> |
| 82 | +<summary><b>venv</b></summary> |
| 83 | + |
| 84 | +```bash |
| 85 | +python -m venv moozy-env |
| 86 | +source moozy-env/bin/activate |
| 87 | +pip install moozy |
| 88 | +``` |
| 89 | + |
| 90 | +</details> |
| 91 | + |
| 92 | +<details> |
| 93 | +<summary><b>uv</b></summary> |
| 94 | + |
| 95 | +```bash |
| 96 | +uv venv moozy-env |
| 97 | +source moozy-env/bin/activate |
| 98 | +uv pip install moozy |
| 99 | +``` |
| 100 | + |
| 101 | +</details> |
| 102 | + |
| 103 | +### Using the Output |
| 104 | + |
| 105 | +The output is a standard H5 file. Load it with `h5py`: |
| 106 | + |
| 107 | +```python |
| 108 | +import h5py |
| 109 | + |
| 110 | +with h5py.File("case_embedding.h5", "r") as f: |
| 111 | + embedding = f["features"][:] # (768,) float32 case-level embedding |
| 112 | + |
| 113 | +# Use the embedding for downstream tasks |
| 114 | +# e.g., as input to a linear probe, k-NN, MLP probe, or clustering |
| 115 | +``` |
| 116 | + |
| 117 | +## Method Overview |
| 118 | + |
| 119 | +MOOZY is a two-stage pipeline that first learns slide-level representations through self-supervised learning, then aligns them with clinical meaning through multi-task supervision. |
| 120 | + |
| 121 | +**Stage 1: Self-supervised slide encoder.** A vision transformer learns context-aware spatial representations from 77,134 unlabeled public histopathology slides (~1.67 billion patches across 23 anatomical sites) using masked self-distillation. No labels are used. The slide encoder captures tissue morphology, spatial context, and inter-region relationships across the whole slide. |
| 122 | + |
| 123 | +**Stage 2: Patient-aware multi-task alignment.** The pretrained slide encoder is fine-tuned end-to-end with a case transformer that models dependencies across all slides from the same patient. A learnable [CASE] token aggregates per-slide embeddings into a single case-level representation. Multi-task supervision across 333 tasks (205 classification, 128 survival) from 56 public datasets provides broad clinical grounding. All task heads are discarded after training, leaving a general-purpose patient encoder. |
| 124 | + |
| 125 | +For detailed model specifications, see the [model card](https://github.com/AtlasAnalyticsLab/MOOZY/blob/main/MODEL_CARD.md). |
| 126 | + |
| 127 | +## Training |
| 128 | + |
| 129 | +Both training stages are fully open-source and reproducible using only public data. All training arguments (data, model, optimization, checkpointing, logging, runtime) are documented in the [Stage 1](https://github.com/AtlasAnalyticsLab/MOOZY/blob/main/docs/stage_1.md) and [Stage 2](https://github.com/AtlasAnalyticsLab/MOOZY/blob/main/docs/stage_2.md) training docs. |
| 130 | + |
| 131 | +### Scripts |
| 132 | + |
| 133 | +For local multi-GPU training, use the launch scripts in [`scripts/`](https://github.com/AtlasAnalyticsLab/MOOZY/tree/main/scripts): |
| 134 | + |
| 135 | +```bash |
| 136 | +# Stage 1: Self-supervised pretraining |
| 137 | +GPU_IDS=0,1,2,3,4,5,6,7 bash scripts/train_stage1.sh |
| 138 | + |
| 139 | +# Stage 2: Multi-task alignment |
| 140 | +GPU_IDS=0,1,2,3,4,5,6,7 bash scripts/train_stage2.sh |
| 141 | +``` |
| 142 | + |
| 143 | +### SLURM Jobs |
| 144 | + |
| 145 | +SLURM job templates are provided in [`slurm/`](https://github.com/AtlasAnalyticsLab/MOOZY/tree/main/slurm) for cluster environments: |
| 146 | + |
| 147 | +| Script | Description | |
| 148 | +|---|---| |
| 149 | +| [`slurm/single_gpu.sh`](https://github.com/AtlasAnalyticsLab/MOOZY/blob/main/slurm/single_gpu.sh) | Single-GPU training | |
| 150 | +| [`slurm/multi_gpu.sh`](https://github.com/AtlasAnalyticsLab/MOOZY/blob/main/slurm/multi_gpu.sh) | Multi-GPU training on one node | |
| 151 | +| [`slurm/multi_node.sh`](https://github.com/AtlasAnalyticsLab/MOOZY/blob/main/slurm/multi_node.sh) | Multi-node distributed training | |
| 152 | +| [`slurm/inference.sh`](https://github.com/AtlasAnalyticsLab/MOOZY/blob/main/slurm/inference.sh) | Patient encoding | |
| 153 | + |
| 154 | +## Notes from the Authors |
| 155 | + |
| 156 | +### On using Linear vs non-Linear classifier |
| 157 | + |
| 158 | +A few readers have asked us why the main tables in the paper use a non-linear (MLP) probe rather than the more conventional linear probe. We wanted to share the reasoning here. |
| 159 | + |
| 160 | +Slide-encoder embeddings are not guaranteed to be linearly separable. Some encoders (e.g. contrastive or aligned multimodal models) are explicitly trained to structure features along linear axes, while others organize information through higher-order interactions that a linear classifier cannot access. A linear probe rewards the former and can underrepresent the latter even when both carry the same useful information. We chose the MLP probe as the primary benchmark because it treats every encoder symmetrically. The classifier is free to use whichever structure is present in the features, without requiring it to be linearly separable. In pathology, clinically relevant phenotypes depend on nonlinear mixtures of cellular morphology and its spatial context, so a linear head on top of frozen slide features is expected to leave real signal unread. Linear-probe results still matter, since they are a more conservative measure of how features transfer to downstream pipelines that use a simple logistic-regression head, so we report both. |
| 161 | + |
| 162 | +The full linear-probe version of the slide-encoder comparison (L2-regularized multinomial logistic regression on the same frozen features) is reported in the appendix of our paper, alongside the per-task breakdowns. MOOZY's numbers drop when the classifier is swapped from an MLP head to a linear head, but that is true of every slide encoder we evaluated, not just MOOZY. Averaged across all six encoders (CHIEF, GigaPath, PRISM, Madeleine, TITAN, and MOOZY), the mean macro-average loss when moving from MLP to linear is about 0.097 on weighted F1, 0.027 on weighted ROC-AUC, and 0.087 on balanced accuracy. The much smaller drop on ROC-AUC is consistent with the rest of this note. A linear head preserves the ordering of predictions across the board but loses the clean decision boundary that an MLP head can find. |
| 163 | + |
| 164 | +The same linear-vs-MLP question can also be asked against the patch-encoder plus trained-MIL baselines. In the table below, each non-MOOZY row pairs a frozen patch encoder with a task-specific MIL aggregator trained from scratch (MeanMIL, ABMIL, CLAM, DSMIL, TransMIL) and averages across the five architectures. The *Backbone* row uses the same ViT-S/8 Lunit DINOv2 patch encoder that MOOZY itself uses internally (Kang et al. 2023), so this row isolates what MOOZY's slide and case encoder add on top of the shared patch features. |
| 165 | + |
| 166 | +**Linear classifier on MOOZY vs. trained MIL aggregators (macro average over 5 MIL architectures).** |
| 167 | + |
| 168 | +| Patch encoder | Weighted F1 | Weighted ROC-AUC | Balanced Accuracy | |
| 169 | +|---|---|---|---| |
| 170 | +| Backbone (MOOZY's patch encoder) | 0.733 | 0.735 | 0.686 | |
| 171 | +| UNI v2 | 0.716 | 0.719 | 0.660 | |
| 172 | +| Phikon v2 | 0.715 | 0.724 | 0.654 | |
| 173 | +| CONCH v1.5 | **0.746** | 0.751 | **0.696** | |
| 174 | +| MUSK | 0.729 | 0.725 | 0.679 | |
| 175 | +| **MOOZY** (linear probe) | 0.698 | **0.778** | 0.674 | |
| 176 | + |
| 177 | +<p align="center"><sub>Macro averages across the eight held-out tasks. Non-MOOZY rows use frozen patch features with a trained MIL aggregator head, averaged over five architectures. MOOZY uses a linear classifier on top of its frozen case embedding, with no MIL training.</sub></p> |
| 178 | + |
| 179 | +MOOZY's slide and case encoder add real signal on top of the shared patch features, but that signal is non-linearly structured. A linear head recovers the ordering but not the decision boundary, so MOOZY keeps the top ROC-AUC under the linear probe (+0.027 over CONCH v1.5) while trailing CONCH v1.5 on weighted F1 and balanced accuracy. The Backbone row tells the same story from a different angle, since its frozen features are the same ones MOOZY is built on. Under MLP, the Backbone-to-MOOZY gap is +0.068 F1, +0.080 ROC-AUC, and +0.072 balanced accuracy. Under linear, only the +0.043 ROC-AUC lift survives, and F1 and balanced accuracy fall by -0.035 and -0.012. |
| 180 | + |
| 181 | +### On the strength of Stage 1 alone |
| 182 | + |
| 183 | +A related question we have heard is how much of MOOZY's gain comes from Stage 1 (the self-supervised slide encoder) versus Stage 2 (the patient-aware multi-task alignment). We find that Stage 1 on its own is already competitive with fully-trained slide encoder baselines, while being one of the smallest pipelines in the comparison and using no paired text, no cross-stain supervision, and no slide-level labels. |
| 184 | + |
| 185 | +**Stage 1 only (MOOZY SSL) vs. other slide encoders (macro average over 8 held-out tasks, MLP probe).** |
| 186 | + |
| 187 | +| Slide encoder | Training signal | Params (total) | Weighted F1 | Weighted ROC-AUC | Balanced Accuracy | |
| 188 | +|---|---|---|---|---|---| |
| 189 | +| CHIEF | Vision SSL + weakly-supervised slide labels | **28.71M** | 0.745 | 0.761 | **0.711** | |
| 190 | +| GigaPath | Vision SSL (masked autoencoder) | 1.22B | 0.730 | 0.728 | 0.668 | |
| 191 | +| PRISM | Vision-language (paired clinical text) | 742.06M | 0.736 | 0.736 | 0.690 | |
| 192 | +| Madeleine | Multimodal (cross-stain supervision) | 400.23M | 0.758 | 0.751 | 0.706 | |
| 193 | +| TITAN | Vision-language (paired clinical captions) | 354.65M | 0.746 | **0.773** | 0.703 | |
| 194 | +| **MOOZY SSL (Stage 1)** | Vision SSL (masked self-distillation) | 64.47M | **0.760** | 0.753 | 0.701 | |
| 195 | + |
| 196 | +<p align="center"><sub>Macro averages across the eight held-out tasks. MOOZY SSL refers to the slide encoder after Stage 1 only, with no Stage 2 multi-task alignment and no case aggregator. Total params include the slide encoder plus its frozen patch encoder (21.67M ViT-S/8 Lunit DINOv2).</sub></p> |
| 197 | + |
| 198 | +Stage 1 on its own is the top weighted-F1 encoder in the table. Its weighted ROC-AUC trails TITAN by 0.020, and its balanced accuracy trails CHIEF by 0.010, which places Stage 1 alone in the same performance band as slide encoders that use paired captions (TITAN, PRISM), cross-stain supervision (Madeleine), or weak slide-level labels (CHIEF). It gets there with no paired supervision at all, just masked self-distillation on 77,134 unlabeled public slides. |
| 199 | + |
| 200 | +Also the closest methodological analogue is GigaPath, which is also vision-only SSL. MOOZY SSL beats it by +0.030 weighted F1, +0.025 weighted ROC-AUC, and +0.033 balanced accuracy while running at roughly 5% of its total parameter count (64.47M vs 1.22B). That gap is largely a question of where the capacity is spent. GigaPath puts almost all of its parameters into a 1.1B-parameter tile encoder, whereas MOOZY keeps a compact 21.67M ViT-S/8 patch encoder frozen and routes the remaining budget into slide-level modeling. This is the most direct evidence we have for a hypothesis we raise in the paper, that slide- and context-level modeling, not patch-level capacity, is the real bottleneck in computational pathology. It also means a useful public slide encoder can be trained on public data with self-distillation without needing paired text, IHC pairs, or labeled slides. |
| 201 | + |
| 202 | +## Acknowledgment |
| 203 | + |
| 204 | +This work was supported by NSERC-DG RGPIN-2022-05378 [M.S.H], Amazon Research Award [M.S.H], and Gina Cody RIF [M.S.H], FRQNT scholarship [Y.K]. Computational resources were provided in part by [Calcul Québec](https://www.calculquebec.ca) and the [Digital Research Alliance of Canada](https://www.alliancecan.ca). |
| 205 | + |
| 206 | +## Citation |
| 207 | + |
| 208 | +If you find MOOZY useful, please cite: |
| 209 | + |
| 210 | +```bibtex |
| 211 | +@misc{kotp2026moozypatientfirstfoundationmodel, |
| 212 | + title={MOOZY: A Patient-First Foundation Model for Computational Pathology}, |
| 213 | + author={Yousef Kotp and Vincent Quoc-Huy Trinh and Christopher Pal and Mahdi S. Hosseini}, |
| 214 | + year={2026}, |
| 215 | + eprint={2603.27048}, |
| 216 | + archivePrefix={arXiv}, |
| 217 | + primaryClass={cs.CV}, |
| 218 | + url={https://arxiv.org/abs/2603.27048}, |
| 219 | +} |
| 220 | +``` |
| 221 | + |
| 222 | +## Contact |
| 223 | + |
| 224 | +For questions, bug reports, or just to say hi, my inbox is open at [yousefkotp@outlook.com](mailto:yousefkotp@outlook.com). I am a human who reads every email, even the ones that start with "I know you're probably busy, but...". Feel free to reach out about anything related to MOOZY, computational pathology, or just to chat about deep learning and its applications in medicine. I also welcome feedback on the codebase and any suggestions for improvement. |
| 225 | + |
| 226 | +## License |
| 227 | + |
| 228 | +This project is licensed under [CC BY-NC-SA 4.0](https://github.com/AtlasAnalyticsLab/MOOZY/blob/main/LICENSE). |
0 commit comments