You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When testing the latest version of llama2.mojo, the Stories models are working fine, but the Tinyllama-1.1B model is producing strange results. Here is an example:
mojo llama2.mojo tl-chat.bin \
-z tok_tl-chat.bin \
-n 256 -t 0 -s 100 -i "<|im_start|>user\nHello. What is your name?<|im_end|>\n<|im_start|>assistant\n"
num parallel workers: 8 SIMD width: 16
Total bytes read: 4400767004 Estimated checkpoint size: 4196 MB
n layers: 22 | vocab size: 32003
<|im_start|>user\nHello. What is your name?<|im_end|>\n<|im_start|>assistant\nqué<0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2><0xE2>
achieved tok/s: 29.109589041095887
or
mojo llama2.mojo tl-chat.bin \
-z tok_tl-chat.bin \
-n 256 -t 0 -s 100 -i "<|im_start|>user\nList all integer numbers from 0 to 150.<|im_end|>\n<|im_start|>assistant\n"
num parallel workers: 8 SIMD width: 16
Total bytes read: 4400767004 Estimated checkpoint size: 4196 MB
n layers: 22 | vocab size: 32003
<|im_start|>user\nList all integer numbers from 0 to 150.<|im_end|>\n<|im_start|>assistant\n<0xE2>сосо<0x0A>Que me puedes ayudar?<|im_end|><0x0A><|im_end|><0x0A><|im_start|>assistant<0x0A>¡Claro! ¿Cuál es tu tarea?<|im_end|><0x0A>
achieved tok/s: 28.91844997108155
The text was updated successfully, but these errors were encountered:
When testing the latest version of llama2.mojo, the Stories models are working fine, but the Tinyllama-1.1B model is producing strange results. Here is an example:
or
The text was updated successfully, but these errors were encountered: