Max retries exceeded with url: /v1/session

[LIUTAOGE]

Continuous call will appear this error, how to solve

[SiriusNEO]

Hi @LIUTAOGE , could you please provide a minimal reproduction for this problem? Thanks!

[LIUTAOGE]

when I use the main branch of code, I start serve also the sam error.
attn_func "xformers_fill_vllm_paged_attention_generate" or "xformers_fill_shared_prompts_generate" result to this error. Only when i start serve with the attn_func is "xformers_with_buffer", the serve is ok

why? Can i fix it with some param edit ?

[LIUTAOGE]

[LIUTAOGE]

@SiriusNEO Thank you

[SiriusNEO]

@LIUTAOGE Hi! According to your description, this may due to incorrect installation of vLLM / parrot kernels. The ServeCore throws connection error since the engine is broken due to its internal error. You can try to modify and run the code under https://github.com/microsoft/ParrotServe/blob/main/benchmark/bench_kernel.py to check whether the kernel works.