fix(data): harden process_token_dict_to_mappings with explicit dtypes, O(1) lookup, and assertions#304

Open

eppaneamd wants to merge 2 commits into

Tencent:mainfrom

eppaneamd:fix/eagle3-vocab-mapping-dtype

Commits on May 11, 2026

fix(data): explicit dtypes, O(1) set lookup, and assertions in process_token_dict_to_mappings
eppaneamd
committed

Commits on May 12, 2026

style: apply black formatting
eppaneamd
committed