Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ work can use `skip_vertices=True` to avoid preparing rest vertices.
- Anatomicals: SKEL, MyoFullBody
- Heads: FLAME
- Hands: MANO
- Robots: BrainCo, G1
- Robots: BrainCo, G1, SmplHumanoid

See the [model docs](https://abcamiletto.github.io/body-models/#supported-models)
for setup, supported backends, inputs, and model-specific behavior.
Expand Down
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ uv add "body-models[jax]"
| --- | --- | --- |
| [BrainCo](models/brainco.md) | BrainCo Revo 2 robotic hand | auto-download |
| [G1](models/g1.md) | Unitree G1 rigid links | auto-download |
| [SmplHumanoid](models/smpl-humanoid.md) | SMPL-compatible humanoid MJCF variants | auto-download |

## Common Usage

Expand Down
4 changes: 3 additions & 1 deletion docs/models/anny.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ ANNY is a phenotype-driven body model with configurable rig and topology variant

## Setup

ANNY downloads automatically on first use. To prefetch and save the path:
ANNY downloads automatically on first use from `https://huggingface.co/abcamiletto/body-models`.
The Hugging Face repo records the original ANNY Apache 2.0 and MPFB2 CC0 provenance. To prefetch
and save the path:

```bash
# Download the ANNY assets and store their path in the body-models config.
Expand Down
4 changes: 3 additions & 1 deletion docs/models/brainco.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ BrainCo is a rigid articulated model of the BrainCo Revo 2 robotic hand using th

## Setup

BrainCo downloads automatically on first use. To prefetch and save the path:
BrainCo downloads from the public [`abcamiletto/body-models`](https://huggingface.co/abcamiletto/body-models) Hugging Face repository on first use. To prefetch and save the path:

```bash
# Download the BrainCo MuJoCo XML and STL assets.
Expand All @@ -13,6 +13,8 @@ body-models download brainco

When passed manually, `model_path` should contain `left.xml`, `right.xml`, and `meshes/{left,right}/*.STL`.

The original BrainCo Revo2 description license is included with the hosted assets.

## Usage

```python
Expand Down
4 changes: 3 additions & 1 deletion docs/models/g1.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ G1 is a rigid articulated Unitree G1 model with STL link meshes attached to the

## Setup

G1 downloads automatically on first use. To prefetch and save the path:
G1 downloads automatically on first use from `https://huggingface.co/abcamiletto/body-models`.
The Hugging Face repo records the original GR00T-WholeBodyControl / LeRobot provenance. To prefetch
and save the path:

```bash
# Download the Unitree G1 XML and link meshes.
Expand Down
5 changes: 3 additions & 2 deletions docs/models/garment-measurements.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@ GarmentMeasurements is a PCA body model with an FBX-derived skeleton and skinnin

## Setup

GarmentMeasurements downloads its preprocessed asset from
`https://huggingface.co/datasets/abcamiletto/body-models-assets` on first use. To prefetch and save the path:
GarmentMeasurements downloads its preprocessed assets from
`https://huggingface.co/abcamiletto/body-models` on first use. The Hugging Face repo records
the original SOMA-X Apache 2.0 provenance for the source asset. To prefetch and save the path:

```bash
# Download the preprocessed GarmentMeasurements body-model asset.
Expand Down
6 changes: 5 additions & 1 deletion docs/models/mhr.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,17 @@ MHR is an expressive full-body model with neural pose correctives.

## Setup

MHR downloads automatically on first use. To prefetch and save the path:
MHR downloads from the public [`abcamiletto/body-models`](https://huggingface.co/abcamiletto/body-models) Hugging Face repository on first use. The hosted package keeps the original MHR checkpoint for the default LOD 1 path and adds preprocessed FBX-derived mesh assets for LODs 0 through 6.

To prefetch and save the path:

```bash
# Download the MHR assets and store their path in the body-models config.
body-models download mhr
```

The original MHR license is included with the hosted assets.

## API

::: body_models.bodies.mhr.numpy.MHR
3 changes: 2 additions & 1 deletion docs/models/myofullbody.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ MyoFullBody is a MuJoCo-derived musculoskeletal full-body model from `amathislab

## Setup

MyoFullBody downloads automatically on first use. To prefetch and save the path:
MyoFullBody downloads automatically on first use from `https://huggingface.co/abcamiletto/body-models`.
The Hugging Face repo records the original MuscleMimic Apache 2.0 provenance. To prefetch and save the path:

```bash
# Download the MyoFullBody MJCF and referenced mesh assets.
Expand Down
18 changes: 18 additions & 0 deletions docs/models/smpl-humanoid.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# SmplHumanoid

SmplHumanoid is a rigid articulated humanoid model loaded from SMPL-compatible MJCF XML variants.

## Setup

SmplHumanoid downloads its XML assets from the public [`abcamiletto/body-models`](https://huggingface.co/abcamiletto/body-models) Hugging Face repository. To prefetch and save the hosted path:

```bash
# Download the SmplHumanoid MJCF XML assets.
body-models download smpl-humanoid
```

The hosted folder includes license/provenance notes for the XML variants.

## API

::: body_models.robots.smpl_humanoid.numpy.SmplHumanoid
3 changes: 2 additions & 1 deletion docs/models/soma.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ SOMA provides a native implementation for SOMA-X assets with identity, pose, and

## Setup

SOMA downloads automatically on first use. To prefetch and save the path:
SOMA downloads automatically on first use from `https://huggingface.co/abcamiletto/body-models`.
The Hugging Face repo records the original SOMA-X Apache 2.0 provenance. To prefetch and save the path:

```bash
# Download the SOMA-X assets used by the native SOMA implementation.
Expand Down
8 changes: 8 additions & 0 deletions src/body_models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ def main() -> None:
from .config import CONFIG_FILE, MODELS, get_model_path, set_model_path, unset_model_path
from .robots.brainco.io import download_model as download_brainco_model
from .robots.g1.io import download_model as download_g1_model
from .robots.smpl_humanoid.io import download_model as download_smpl_humanoid_model
from .skeletons.myofullbody.io import download_model as download_myofullbody_model

Model = Literal[
Expand All @@ -88,6 +89,7 @@ def main() -> None:
"flame",
"brainco",
"g1",
"smpl-humanoid",
"soma",
"garment-measurements",
"myofullbody",
Expand Down Expand Up @@ -130,6 +132,7 @@ def download(
"brainco",
"mhr",
"g1",
"smpl-humanoid",
"soma",
"garment-measurements",
"myofullbody",
Expand Down Expand Up @@ -234,6 +237,11 @@ def download(
set_model_path("g1", str(path))
print(f"Set g1 = {path}")

if model in ("smpl-humanoid", "all"):
path = download_smpl_humanoid_model()
set_model_path("smpl-humanoid", str(path))
print(f"Set smpl-humanoid = {path}")

if model in ("soma", "all"):
path = download_soma_model()
set_model_path("soma", str(path))
Expand Down
6 changes: 3 additions & 3 deletions src/body_models/bodies/anny/io.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@
from nanomanifold import SO3

from body_models import config
from body_models.cache import download_and_extract, get_cache_dir
from body_models.cache import HF_MODEL_BASE_URL, download_and_extract, get_cache_dir

PathLike = Path | str
Array = Any

Front = tuple[list[int], list[int]] # One FK depth level: (joint_indices, parent_indices).

ANNY_URL = "https://github.com/naver/anny/archive/refs/heads/main.zip"
ANNY_URL = f"{HF_MODEL_BASE_URL}/anny/assets.zip"

PHENOTYPE_VARIATIONS = {
"race": ["african", "asian", "caucasian"],
Expand Down Expand Up @@ -83,7 +83,7 @@ def get_model_path(model_path: PathLike | None = None) -> Path:
def download_model() -> Path:
cache_dir = get_cache_dir() / "anny"
print(f"Downloading ANNY model to {cache_dir}...")
download_and_extract(url=ANNY_URL, dest=cache_dir, extract_subdir="anny-main/src/anny/")
download_and_extract(url=ANNY_URL, dest=cache_dir)
print("Done")
return cache_dir

Expand Down
6 changes: 3 additions & 3 deletions src/body_models/bodies/garment_measurements/io.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,15 @@
from jaxtyping import Float, Int

from body_models import config
from body_models.cache import HF_DATASET_BASE_URL, download_file, get_cache_dir
from body_models.cache import HF_MODEL_BASE_URL, download_and_extract, get_cache_dir

PathLike = Path | str
Array = Any

Front = tuple[list[int], list[int]] # One FK depth level: (joint_indices, parent_indices).

PREPROCESSED_FILENAME = "garment_measurements.npz"
GARMENT_MEASUREMENTS_URL = f"{HF_DATASET_BASE_URL}/garment_measurements/{PREPROCESSED_FILENAME}"
GARMENT_MEASUREMENTS_URL = f"{HF_MODEL_BASE_URL}/garment_measurements/assets.zip"


@dataclass(frozen=True)
Expand Down Expand Up @@ -65,7 +65,7 @@ def download_model() -> Path:
"""Download preprocessed GarmentMeasurements data assets."""
cache_dir = get_cache_dir() / "garment_measurements"
print(f"Downloading GarmentMeasurements model to {cache_dir}...")
download_file(GARMENT_MEASUREMENTS_URL, cache_dir / PREPROCESSED_FILENAME)
download_and_extract(GARMENT_MEASUREMENTS_URL, cache_dir)
print("Done")
return cache_dir

Expand Down
59 changes: 51 additions & 8 deletions src/body_models/bodies/mhr/io.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

from body_models import config
from body_models.common import simplify_mesh
from body_models.cache import download_and_extract, get_cache_dir
from body_models.cache import HF_MODEL_BASE_URL, download_and_extract, get_cache_dir

PathLike = Path | str

Expand All @@ -28,7 +28,8 @@
"load_pose_correctives",
]

MHR_URL = "https://github.com/facebookresearch/MHR/releases/download/v1.0.0/assets.zip"
MHR_URL = f"{HF_MODEL_BASE_URL}/mhr/assets.zip"
SUPPORTED_LODS = tuple(range(7))


@dataclass(frozen=True)
Expand Down Expand Up @@ -71,35 +72,40 @@ def get_model_path(model_path: PathLike | None = None) -> Path:
return validate_path(model_path)

cache_path = get_cache_dir() / "mhr"
if (cache_path / "mhr_model.pt").exists():
if _has_hosted_assets(cache_path):
return cache_path

return download_model()


def download_model() -> Path:
"""Download MHR model from GitHub releases."""
"""Download MHR model assets."""
cache_dir = get_cache_dir() / "mhr"
print(f"Downloading MHR model to {cache_dir}...")
download_and_extract(url=MHR_URL, dest=cache_dir, extract_subdir="assets/")
download_and_extract(url=MHR_URL, dest=cache_dir)
print("Done")
return cache_dir


def load_model_data(asset_dir: Path, *, lod: int = 1, simplify: float = 1.0) -> MhrWeights:
if simplify < 1.0:
raise ValueError("simplify must be >= 1.0")
if lod != 1:
raise ValueError("MHR lod values other than 1 are not supported.")
if lod not in SUPPORTED_LODS:
raise ValueError(f"MHR lod must be one of {SUPPORTED_LODS}, got {lod}")

data = _load_raw_model_data(asset_dir)
shared_data = _load_raw_model_data(asset_dir)
data = shared_data if lod == 1 else _load_preprocessed_lod_data(asset_dir, lod, shared_data)
base_vertices = data["base_vertices"]
blendshape_dirs = data["blendshape_dirs"]
skin_weights = data["skin_weights"]
skin_indices = data["skin_indices"].astype(np.int64)
faces = data["faces"].astype(np.int64)
corrective_weights = load_pose_correctives_weights(asset_dir, lod)
corrective_W2 = corrective_weights["W2"]
if corrective_W2.shape[0] != len(base_vertices) * 3:
raise ValueError(
f"MHR lod{lod} corrective W2 has {corrective_W2.shape[0]} rows, expected {len(base_vertices) * 3}"
)

if simplify > 1.0:
target_faces = int(len(faces) / simplify)
Expand Down Expand Up @@ -169,6 +175,43 @@ def _load_raw_model_data(asset_dir: Path) -> dict[str, Any]:
}


def _load_preprocessed_lod_data(asset_dir: Path, lod: int, shared_data: dict[str, Any]) -> dict[str, Any]:
path = asset_dir / f"mhr_lod{lod}.npz"
with np.load(path, allow_pickle=False) as asset:
joint_names = [str(name) for name in asset["skin_joint_names"].tolist()]
checkpoint_joint_index = {name: index for index, name in enumerate(shared_data["joint_names"])}
missing = sorted(name for name in joint_names if name not in checkpoint_joint_index)
if missing:
raise ValueError(f"{path} references joints missing from mhr_model.pt: {missing}")

skin_joint_indices = np.asarray(asset["skin_joint_indices"], dtype=np.int64)
mapped_joint_indices = np.asarray([checkpoint_joint_index[joint_names[index]] for index in skin_joint_indices])
base_vertices = np.asarray(asset["base_vertices"], dtype=np.float32)
skin_indices, skin_weights = _build_dense_skinning(
asset["skin_vertex_indices"],
mapped_joint_indices,
asset["skin_weights"],
len(base_vertices),
)

return shared_data | {
"base_vertices": base_vertices,
"blendshape_dirs": np.asarray(asset["blendshape_dirs"], dtype=np.float32),
"skin_weights": skin_weights,
"skin_indices": skin_indices,
"faces": np.asarray(asset["faces"], dtype=np.int64),
}


def _has_hosted_assets(model_path: Path) -> bool:
asset_names = [
"mhr_model.pt",
"corrective_activation.npz",
*(f"mhr_lod{lod}.npz" for lod in SUPPORTED_LODS),
]
return all((model_path / name).is_file() for name in asset_names)


def _get_attr(obj: Any, path: str) -> Any:
cur = obj
for part in path.split("."):
Expand Down
Loading
Loading