Add WorldPlay2 synthesis backend using shared Wan components

[iChubai]

Summary

Add native WorldPlay2 Fast, AR and BI generation from an image, navigation actions and scheduled scene prompts. WorldPlay2-specific conditioning, compressed causal memory, transformer extensions and sampling live in worldfoundry/synthesis/visual_generation/worldplay2/.

The implementation reuses the shared Wan2.2 transformer, UMT5 encoder, Wan2.1 VAE, checkpoint loader, runtime policy, attention backends and media IO. Shared changes extract Wan image conditioning, expose an output-head hook and add request-owned streaming codec caches.

Released-checkpoint short replay declarations cover the default Fast route and Fast/AR/BI variants. make test-infer-tensors selects all 42 WorldPlay2 CPU contracts; the official numeric fixture uses the public JSON format.

Generated recipe metadata records a portable Python interpreter default, with matching output under Python 3.10 and 3.12.

User-Visible Behavior

• WorldPlay2Pipeline.from_pretrained(...) selects Fast, AR or BI expert weights through one public adapter.
• Actions support movement, look and special-action combinations in first-person or third-person views. Named prompts can change at chunk boundaries.
• Fast defaults to four steps, 125 frames at 832×448 and 16 FPS. AR and BI use 40-step UniPC sampling.
• Calls return video or latents and can export MP4. English/Chinese documentation, model catalog variants and runtime bindings expose the same options.

Affected Pipeline / Benchmark

• Pipeline(s): WorldPlay2; shared native visual adapter and Wan components.
• Benchmark(s): N/A; no benchmark or metric implementation changes.
• Runtime profile(s): worldplay2, worldplay2-fast, worldplay2-ar, worldplay2-bi.
• Public entrypoint(s): worldfoundry.pipelines.worldplay2:WorldPlay2Pipeline.

Change Type

• Bug fix
• Model integration
• Benchmark integration
• Pipeline/runtime change
• Documentation only
• Test/QA tooling

Asset / API / GPU Requirements

• Downloads or large assets: selected WorldPlay2 high/low expert safetensors, plus Wan2.2-I2V-A14B UMT5, tokenizer, Wan2.1 VAE and expert configuration files.
• API keys or quota: none.
• GPU / simulator requirements: CUDA hardware sized for two A14B experts and shared text/VAE components; no simulator.
• Official repository or checkpoint assumptions: WorldPlay2 source revision c5d83e32099116ff3a1437a8a05c764d579f704b; model and support-asset revisions are pinned in the recipe and catalog.

Commands Run

Validation covers WorldPlay2 Fast/AR/BI CPU parity, shared Wan/runtime contracts, HelixWorld coexistence, checkpoint selection, video exports and replay declarations. The combined selection passes all 124 tests. Representative commands:

Command	Result	Notes
PYTHONPATH=/tmp/worldfoundry-helix-test-deps:. OMP_NUM_THREADS=2 MKL_NUM_THREADS=2 /mnt/dolphinfs/ssd_pool/docker/user/hadoop-nlp-hl02/hadoop-aipnlp/3A/multimodal/yangboxue/arena/conda/envs/ltx23/bin/python -m pytest -q tests/synthesis/test_worldplay2.py tests/synthesis/test_worldplay2_compressor_pdd.py tests/synthesis/test_worldplay2_upstream_parity.py tests/eval_core/test_worldplay2_integration.py tests/core/test_video_tensor_regression.py::test_public_native_video_adapter_applies_defaults_only_for_none tests/runtime/test_geometry_regression_impact.py::test_public_matrix_has_complete_declarations_and_known_hy2_path	Pass: 44 cases	Integration, adapter and matrix contracts.
PYTHONPATH=/tmp/worldfoundry-helix-test-deps:. /mnt/dolphinfs/ssd_pool/docker/user/hadoop-nlp-hl02/hadoop-aipnlp/3A/multimodal/yangboxue/arena/conda/envs/ltx23/bin/python docs/fumadocs/scripts/generate-model-recipes.py --check	Pass	Generated recipe data and model pages.
PYTHONPATH=/tmp/worldfoundry-helix-test-deps:. /mnt/dolphinfs/ssd_pool/docker/user/hadoop-nlp-hl02/hadoop-aipnlp/3A/multimodal/yangboxue/arena/conda/envs/ltx23/bin/python -m pytest -q tests/synthesis/test_worldplay2_upstream_parity.py	Pass: 8 cases	Pinned official CPU fixtures, including a three-chunk Fast rollout.
PATH=/tmp/worldatlas-ffmpeg-runtime/bin:$PATH PYTHONPATH=/tmp/worldfoundry-helix-test-deps:. /mnt/dolphinfs/ssd_pool/docker/user/hadoop-nlp-hl02/hadoop-aipnlp/3A/multimodal/yangboxue/arena/conda/envs/ltx23/bin/python -m pytest -q tests/eval_core/test_helixworld_integration.py::test_helixworld_exports_fhwc_float_video_with_audio	Pass: 2 cases	Shared adapter preserves HelixWorld video/audio export.
/home/hadoop-aipnlp/.local/bin/ruff check worldfoundry/synthesis/visual_generation/worldplay2 worldfoundry/pipelines/worldplay2 worldfoundry/base_models/diffusion_model/models/autoencoders/wan/component.py worldfoundry/base_models/diffusion_model/models/initializers/wan/component.py worldfoundry/base_models/diffusion_model/models/networks/wan/model.py worldfoundry/base_models/diffusion_model/recipes/registry.py worldfoundry/base_models/diffusion_model/runners/strategies.py worldfoundry/pipelines/native_diffusion.py tests/synthesis/test_worldplay2.py tests/synthesis/test_worldplay2_compressor_pdd.py tests/synthesis/test_worldplay2_upstream_parity.py tests/eval_core/test_worldplay2_integration.py tests/pipelines/test_native_diffusion_runtime_policy.py	Pass	Changed Python files.
git diff --cached --check	Pass	Staged integration diff.

The CPU checks use Python 3.12, Torch 2.7.1 and Transformers 4.57.6. The committed reference fixture makes official parity tests independent of the upstream checkout.

The generated recipe data and pages also pass generate-model-recipes.py --check under the dependency-light system Python 3.10 and the inference Python 3.12 environment.

Checkpoint / API Key Needs

• Checkpoints / weights: aejion/WorldPlay2-Fast, aejion/WorldPlay2-AR or aejion/WorldPlay2-BI, with Wan-AI/Wan2.2-I2V-A14B support assets.
• Required environment variables: none for the public pipeline.
• API providers: none.
• Local cache assumptions: expert directories are passed through model_path; shared assets through base_model_path. Offline CPU tests require only the committed fixture.

Sample Artifact Evidence

• Artifact path or link: tests/fixtures/worldplay2/upstream_cpu.json, documented in tests/fixtures/worldplay2/README.md.
• What it demonstrates: official BI and cached AR forwards, memory features, incremental KV updates, both compact Fast PDD heads and a 12-latent-frame/45-pixel-frame Fast rollout. The three-chunk rollout has maximum latent error 8.34e-7 against the pinned official small graph.
• Checkpoint structure: all six released expert headers contain 1,305 tensors each, with zero missing keys, unexpected keys or shape mismatches against the full native meta graph.
• Known limitations: see Limitations.

Validation Matrix Status

Area	Status	Evidence / Command
Unit or focused regression	Pass	124 combined WorldPlay2, HelixWorld, shared runtime, checkpoint-selection, export and replay-declaration cases.
Real inference validation	Not run	See Limitations.
Streaming or multi-turn path	Pass	Three-chunk official CPU rollout, incremental KV and request-owned VAE cache tests.
Benchmark runner / metric path	N/A	Benchmark and metric implementations are unchanged.
Docs build or link check	Pass	Generated model data and authored-page checks.
GPU/API-key dependent path	Not run	See Limitations.

Leaderboard Validity Impact

• Readiness status changed: native integration is available with status native_small_graph_cpu_parity_validated_checkpoint_gpu_pending.
• leaderboard_valid impact: no benchmark score or validity promotion.
• Scorecard/preflight evidence: catalog, public target and runtime-profile references resolve for all three modes.
• Demo or contract-only limitations: official runner parity remains pending.

Compatibility And Risk

• Backward compatibility impact: shared Wan behavior is covered by 62 existing regression cases; coexistence with HelixWorld Add native HelixWorld audio-video inference #13 covers registry resolution and audio/video output.
• Expected resource cost: two A14B experts plus UMT5/VAE assets; full-model memory and throughput are not measured.
• Failure modes or rollout concerns: expert weights must match the selected mode, and action durations and prompt boundaries must align with its chunk length.

Limitations

• Full-checkpoint CUDA generation and official full-size output parity remain pending. CPU parity uses a small FP32 transformer with deterministic text and codec fixtures.

• WorldPlay2-TAE weights are unreleased; generation uses the shared Wan2.1 VAE.

• WorldPlay2 code and weights use CC BY-NC 4.0. Wan support assets retain Apache-2.0 terms; attribution is included in THIRD-PARTY-NOTICES.

• Repository CI is blocked by Kandinsky6 license consolidation, missing native replay recipes, and model-home-page source/locale metadata. The inference replay gate requires a configured pinned accepted reference index and successful checkpoint replay receipts for this exact commit.

Checklist

• I kept the change narrowly scoped.
• I did not commit secrets, API keys, large checkpoints, or generated cache files.
• I reused fixtures from data/test_cases/ or assets from data/benchmarks/ where practical.
• I updated docs when public behavior, install steps, or entry commands changed.
• I documented API/GPU/checkpoint/official-repo requirements or confirmed none are needed.
• I did not promote demo, contract-only, normalizer-only, API-blocked, or partial evidence to leaderboard validity.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.