llama.cpp server core dumps in docker #9
Description
Why did you build llama.cpp service with AMX_INT8 = 1 ? Is it so popular?
Chat fails without any clear message.
{code}
AMX is not ready to be used!
Illegal instruction (core dumped)
{code}
Also why no sudo? Or at least vim.
docker image id: ee6133cdf762