Franca: Nested Matryoshka Clustering for Scalable Visual Representation Learning

Shashanka Venkataramanan*, Valentinos Pariza*, Mohammadreza Salehi, Lukas Knobel, Spyros Gidaris, Elias Ramzi, Andrei Bursuc†, Yuki M. Asano†

*: equal contribution †: equal advising

Valeo.ai, Paris; Fundamental AI Lab, UTN; VIS Lab, University of Amsterdam

---

Welcome to the official codebase for Franca (pronounced Fran-ka), the first fully open-source vision foundation model—including data, code, and pretrained weights.

Franca matches or surpasses the performance of leading proprietary models such as DINOv2, CLIP, and SigLIPv2. Built on a fully transparent training pipeline leveraging publicly available datasets like ImageNet-21K and LAION-600M, Franca advances the state of self-supervised learning (SSL) in vision.

Key contributions include:

• Nested Matryoshka Clustering: A parameter-efficient, multi-head clustering projector that refines feature representations into increasingly fine-grained clusters without increasing model size. This approach improves performance while reducing memory overhead for downstream applications.

• RASA (Relative Absolute Spatial Attention): A novel positional disentanglement strategy that explicitly removes positional biases from dense representations, enhancing semantic encoding.

• CyclicMask: A simple masking strategy with circular shifts that overcomes spatial imbalance in masking augmentations.

Despite training on large-scale, uncurated open-source internet data, Franca demonstrates strong generalization across scales and excels on a wide range of downstream tasks, including: In-context learning with HummingBird benchmark, Out-of-distribution detection, 3D understanding, various image classification benchmarks.

Franca is released as a research project to promote transparency, reproducibility, and broad accessibility in vision foundation models. We aim to establish a new benchmark for open and generalizable AI models that empower the scientific community.

---

News

• [July 2025] Official code and pretrained models released! 🔥

---

Pretrained models

model	# of params	Dataset	Resolution	HRFT + RASA	ImageNet-1K k-NN	ImageNet-1K linear	ADE20K in-context	DAVIS VOS	download	download
DINOv2 ViT-B/14	86 M	In21K	224	No	77.0%	81.2%	30.0%	63.1%	backbone only	not available
DINOv2 ViT-B/14	86 M	In21K	518	Yes	76.8%	81.0%	32.4%	69.2%	backbone only	RASA head
DINOv2 ViT-L/14	300 M	In21K	224	No	82.1%	84.0%	33.5%	64.2%	backbone only	not available
DINOv2 ViT-L/14	300 M	In21K	518	Yes	80.7%	84.5%	37.9%	66.6%	backbone only	RASA head
Franca ViT-B/14	86 M	In21K	224	No	77.5%	82.0%	31.6%	65.5%	backbone only	not available
Franca ViT-B/14	86 M	In21K	518	Yes	79.6%	82.6%	35.0%	70.6%	backbone only	RASA head
Franca ViT-L/14	300 M	LAION-600M	224	No	82.2%	84.5%	34.5%	68.0%	backbone only	not available
Franca ViT-L/14	300 M	LAION-600M	518	Yes	82.5%	85.2%	39.6%	70.0%	backbone only	RASA head
Franca ViT-g/14	1,100 M	LAION-600M	224	RASA only	not reported	not reported	not reported	not reported	part 1, part 2, part 3	RASA head

Pretrained backbones (via PyTorch Hub)

Please follow the instructions here to install PyTorch (the only required dependency for loading the model). Installing PyTorch with CUDA support is strongly recommended.

import torch

repo = 'valeoai/Franca:v1.1.0'

# DINOv2 baselines -- In21K
dinov2_vitb14 = torch.hub.load(repo, 'franca_vitb14', weights='DINOV2_IN21K', use_rasa_head=False)
dinov2_vitb14_518 = torch.hub.load(
    repo, 'franca_vitb14', weights='DINOV2_IN21K_518', use_rasa_head=True
)
dinov2_vitl14 = torch.hub.load(repo, 'franca_vitl14', weights='DINOV2_IN21K', use_rasa_head=False)
dinov2_vitl14_518 = torch.hub.load(
    repo, 'franca_vitl14', weights='DINOV2_IN21K_518', use_rasa_head=True
)

# Franca
franca_vitb14 = torch.hub.load(repo, 'franca_vitb14', weights='IN21K_224', use_rasa_head=False)
franca_vitb14_518 = torch.hub.load(repo, 'franca_vitb14', weights='IN21K', use_rasa_head=True)
franca_vitl14 = torch.hub.load(repo, 'franca_vitl14', weights='LAION_224', use_rasa_head=False)
franca_vitl14_518 = torch.hub.load(repo, 'franca_vitl14', weights='LAION_518', use_rasa_head=True)

# Existing Franca ViT-g/14 -- LAION-600M
franca_vitg14 = torch.hub.load(repo, 'franca_vitg14', weights='LAION', use_rasa_head=True)

The evaluation results above are from Table 2 of the Franca paper. The Franca-L checkpoints use LAION-600M. Legacy RASA checkpoints with eight preprocessing projections and current checkpoints with nine are both detected and loaded automatically.

Installation

git clone https://github.com/valeoai/Franca.git
cd Franca
# To install Franca and RASA you can use the following command
pip install -e ".[franca]"
# To install RASA seperately you can use the following command
pip install -e .

We recommend install torch separately to match your specific config. Similarly, Franca relies on xFormers / cuML and RASA relies on faiss-gpu, which we also recommend installing on your side.

Otherwise you can use the following commands:

# Install franca with additional dependencies
pip install -e ".[franca,torch,cuml,xformers]"
# Install rasa with additional dependencies
pip install -e ".[torch,faiss]"

Inference code

To load a Franca model directly using the checkpoint from [link](#coming soon), use the example below:

import torch
from PIL import Image
from torchvision import transforms
from franca.hub.backbones import _make_franca_model
from rasa.src.rasa_head import RASAHead

# --- Step 1: Choose model config ---
arch_name = "vit_large"
img_size = 224
ckpt_path = "<your path>/franca_vitl14_In21K.pth"
rasa_ckpt_path = "<your path>/franca_vitl14_In21K_rasa.pth"

# Define image transformation
transform = transforms.Compose([
    transforms.Resize(256, interpolation=transforms.InterpolationMode.BICUBIC),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225))
])

# --- Step 2: Build and load model ---
model = _make_franca_model(
    arch_name=arch_name,
    img_size=img_size,
    pretrained=True,
    local_state_dict=ckpt_path,
    RASA_local_state_dict=rasa_ckpt_path,
    use_rasa_head=True
)


# --- Step 3: Forward pass ---
model.cuda()
model.eval()

image = Image.open("assets/dog.jpg")
x = transform(image).unsqueeze(0).cuda()


with torch.no_grad():
    feats = model.forward_features(x, use_rasa_head=True)
    cls_token = feats["x_norm_clstoken"]
    patch_tokens = feats["x_norm_patchtokens"]
    patch_tokens_debiased = feats["patch_token_rasa"]

print("CLS token shape:", cls_token.shape)
print("Patch token shape:", patch_tokens.shape)
print("Patch token RASA shape:", patch_tokens_debiased.shape)

Structure

• Dataset preparation: Dataset prep
• Training: Training Details
• Model card: Models
• RASA Usage: RASA README

Citation

If you use Franca in your research, please cite:

@article{venkataramanan2025franca,
  title={Franca: Nested Matryoshka Clustering for Scalable Visual Representation Learning},
  author={Venkataramanan, Shashanka and Pariza, Valentinos and Salehi, Mohammadreza and Knobel, Lukas and Gidaris, Spyros and Ramzi, Elias and Bursuc, Andrei and Asano, Yuki M.},
  journal={arXiv preprint arXiv:2507.14137},
  year={2025}
}

Acknowledgments

We thank the DINOv2 team for their excellent codebase. We also gratefully acknowledge the authors of OpenOOD, Open-hummingbird, Probe3D, and Feat2GS for their open-sourced codes, which we used for downstream evaluations.