llama.cpp rocm segfaults with R9700 (gfx1201) if integrated GPU (gfx1036) also detected

[kaerumy]

Operating System

Other Linux (specify below)

OS Details (if "Other" or additional info)

Ubuntu 26.04

GPU Model

AMD Radeon AI PRO R9700

llamacpp-rocm Build Version

B8935

Model Used

unsloth/Qwen3.6-27B-GGUF-UD-Q4_K_XL

Does this issue also occur with Vulkan on upstream llama.cpp?

Yes — also broken on Vulkan / upstream llama.cpp

Issue Description

On Ubuntu Linux 26.04 llama.cpp rocm backend run on platform with one GPU (R9700, gfx1201) and integrated graphics (9950X3D, gfx1036) , llama.cpp rocm backend will segfault with default options. This is behaviour also happens with upstream llama.cpp build.

Additional Information

It will load successfully if main GPU selected and pipeline parallelism disabled `-mg 0 -sm none`

llama.cpp vulkan backend loads successfully

Log output:


2026-05-17 12:33:05.652 [Info] (Process) ............................................................................................
2026-05-17 12:33:05.655 [Info] (Process) common_init_result: added <|endoftext|> logit bias = -inf
2026-05-17 12:33:05.655 [Info] (Process) common_init_result: added <|im_end|> logit bias = -inf
2026-05-17 12:33:05.655 [Info] (Process) common_init_result: added <|fim_pad|> logit bias = -inf
2026-05-17 12:33:05.655 [Info] (Process) common_init_result: added <|repo_name|> logit bias = -inf
2026-05-17 12:33:05.655 [Info] (Process) common_init_result: added <|file_sep|> logit bias = -inf
2026-05-17 12:33:05.655 [Info] (Process) llama_context: constructing llama_context
2026-05-17 12:33:05.655 [Info] (Process) llama_context: n_seq_max     = 4
2026-05-17 12:33:05.655 [Info] (Process) llama_context: n_ctx         = 131072
2026-05-17 12:33:05.655 [Info] (Process) llama_context: n_ctx_seq     = 131072
2026-05-17 12:33:05.655 [Info] (Process) llama_context: n_batch       = 2048
2026-05-17 12:33:05.655 [Info] (Process) llama_context: n_ubatch      = 512
2026-05-17 12:33:05.655 [Info] (Process) llama_context: causal_attn   = 1
2026-05-17 12:33:05.655 [Info] (Process) llama_context: flash_attn    = auto
2026-05-17 12:33:05.655 [Info] (Process) llama_context: kv_unified    = true
2026-05-17 12:33:05.655 [Info] (Process) llama_context: freq_base     = 10000000.0
2026-05-17 12:33:05.655 [Info] (Process) llama_context: freq_scale    = 1
2026-05-17 12:33:05.655 [Info] (Process) llama_context: n_ctx_seq (131072) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
2026-05-17 12:33:05.656 [Info] (Process) llama_context:  ROCm_Host  output buffer size =     3.79 MiB
2026-05-17 12:33:05.657 [Info] (Process) llama_kv_cache:      ROCm0 KV buffer size =  2048.00 MiB
2026-05-17 12:33:05.944 [Info] (Process) llama_kv_cache:      ROCm1 KV buffer size =  6144.00 MiB
2026-05-17 12:33:06.082 [Info] (Process) llama_kv_cache: size = 8192.00 MiB (131072 cells,  16 layers,  4/1 seqs), K (f16): 4096.00 MiB, V (f16): 4096.00 MiB
2026-05-17 12:33:06.082 [Info] (Process) llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 256
2026-05-17 12:33:06.082 [Info] (Process) llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 256
2026-05-17 12:33:06.083 [Info] (Process) llama_memory_recurrent:      ROCm0 RS buffer size =   162.09 MiB
2026-05-17 12:33:06.115 [Info] (Process) llama_memory_recurrent:      ROCm1 RS buffer size =   436.41 MiB
2026-05-17 12:33:06.116 [Info] (Process) llama_memory_recurrent: size =  598.50 MiB (     4 cells,  64 layers,  4 seqs), R (f32):   22.50 MiB, S (f32):  576.00 MiB
2026-05-17 12:33:06.116 [Info] (Process) llama_context: pipeline parallelism enabled
2026-05-17 12:33:06.116 [Info] (Process) sched_reserve: reserving ...
2026-05-17 12:33:06.120 [Info] (Process) sched_reserve: Flash Attention was auto, set to enabled
2026-05-17 12:33:06.120 [Info] (Process) sched_reserve: resolving fused Gated Delta Net support:
2026-05-17 12:33:06.120 [Info] (Process) sched_reserve: fused Gated Delta Net (autoregressive) enabled
2026-05-17 12:33:06.121 [Info] (Process) sched_reserve: fused Gated Delta Net (chunked) enabled
2026-05-17 12:33:06.331 [Info] (Process) sched_reserve:      ROCm0 compute buffer size =  1336.06 MiB
2026-05-17 12:33:06.331 [Info] (Process) sched_reserve:      ROCm1 compute buffer size =  1047.07 MiB
2026-05-17 12:33:06.331 [Info] (Process) sched_reserve:  ROCm_Host compute buffer size =  1044.08 MiB
2026-05-17 12:33:06.331 [Info] (Process) sched_reserve: graph nodes  = 3657
2026-05-17 12:33:06.331 [Info] (Process) sched_reserve: graph splits = 3
2026-05-17 12:33:06.331 [Info] (Process) sched_reserve: reserve took 215.60 ms, sched copies = 4
2026-05-17 12:33:06.331 [Info] (Process) common_init_from_params: KV cache shifting is not supported for this context, disabling KV cache shifting
2026-05-17 12:33:06.331 [Info] (Process) common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
2026-05-17 12:33:06.561 [Error] (WrappedServer) llama-server process has terminated with exit code: -1
2026-05-17 12:33:06.561 [Error] (WrappedServer) This usually means:
2026-05-17 12:33:06.561 [Error] (WrappedServer)   - Missing required drivers or dependencies
2026-05-17 12:33:06.561 [Error] (WrappedServer)   - Incompatible model file
2026-05-17 12:33:06.561 [Error] (WrappedServer)   - Try running the server manually to see the actual error
2026-05-17 12:33:11.570 [Warn] (ProcessManager) Process did not respond to SIGTERM, using SIGKILL
2026-05-17 12:33:11.570 [Info] (ProcessManager) Process terminated, waiting for GPU driver cleanup...
2026-05-17 12:33:13.570 [Error] (Router) Retry also failed: llama-server failed to start
2026-05-17 12:33:13.570 [Info] (LlamaCpp) Unloading model...
2026-05-17 12:33:13.570 [Info] (LlamaCpp) Unloading model...
2026-05-17 12:33:13.570 [Error] (Router) Failed to load model: llama-server failed to start
2026-05-17 12:33:13.570 [Error] (Server) Failed to load model: llama-server failed to start
2026-05-17 12:33:13.570 [Error] (Server) Error 500: POST /api/v1/load
2026-05-17 12:56:26.478 [Info] (Server) Log level changed to: debug


rocminfo:


ROCk module is loaded
=====================    
HSA System Attributes    
=====================    
Runtime Version:         1.1
Runtime Ext Version:     1.14
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE                              
System Endianness:       LITTLE                             
Mwaitx:                  DISABLED
XNACK enabled:           NO
DMAbuf Support:          YES
VMM Support:             YES

==========               
HSA Agents               
==========               
*******                  
Agent 1                  
*******                  
  Name:                    AMD Ryzen 9 9950X3D 16-Core Processor
  Uuid:                    CPU-XX                             
  Marketing Name:          AMD Ryzen 9 9950X3D 16-Core Processor
  Vendor Name:             CPU                                
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                              
  Node:                    0                                  
  Device Type:             CPU                                
  Cache Info:              
    L1:                      49152(0xc000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   5756                               
  BDFID:                   0                                  
  Internal Node ID:        0                                  
  Compute Unit:            32                                 
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:1                                  
  Memory Properties:       
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    127097132(0x793592c) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    127097132(0x793592c) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    127097132(0x793592c) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 4                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    127097132(0x793592c) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*******                  
Agent 2                  
*******                  
  Name:                    gfx1201                            
  Uuid:                    GPU-7c3bbefc1ef95918               
  Marketing Name:          AMD Radeon AI PRO R9700            
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    1                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      32(0x20) KB                        
    L2:                      8192(0x2000) KB                    
    L3:                      65536(0x10000) KB                  
  Chip ID:                 30033(0x7551)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          256(0x100)                         
  Max Clock Freq. (MHz):   2350                               
  BDFID:                   768                                
  Internal Node ID:        1                                  
  Compute Unit:            64                                 
  SIMDs per CU:            2                                  
  Shader Engines:          4                                  
  Shader Arrs. per Eng.:   2                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Memory Properties:       
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          32(0x20)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    1024(0x400)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        2147483647(0x7fffffff)             
    y                        65535(0xffff)                      
    z                        65535(0xffff)                      
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 218                                
  SDMA engine uCode::      662                                
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    33406976(0x1fdc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx1201         
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)             
        y                        65535(0xffff)                      
        z                        65535(0xffff)                      
      FBarrier Max Size:       32                                 
    ISA 2                    
      Name:                    amdgcn-amd-amdhsa--gfx12-generic   
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)             
        y                        65535(0xffff)                      
        z                        65535(0xffff)                      
      FBarrier Max Size:       32                                 
*******                  
Agent 3                  
*******                  
  Name:                    gfx1036                            
  Uuid:                    GPU-XX                             
  Marketing Name:          AMD Ryzen 9 9950X3D 16-Core Processor
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    2                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      16(0x10) KB                        
    L2:                      256(0x100) KB                      
  Chip ID:                 5056(0x13c0)                       
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          128(0x80)                          
  Max Clock Freq. (MHz):   2200                               
  BDFID:                   31488                              
  Internal Node ID:        2                                  
  Compute Unit:            2                                  
  SIMDs per CU:            2                                  
  Shader Engines:          1                                  
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Memory Properties:       APU
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          32(0x20)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    1024(0x400)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        2147483647(0x7fffffff)             
    y                        65535(0xffff)                      
    z                        65535(0xffff)                      
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 26                                 
  SDMA engine uCode::      9                                  
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    63548564(0x3c9ac94) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    63548564(0x3c9ac94) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx1036         
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)             
        y                        65535(0xffff)                      
        z                        65535(0xffff)                      
      FBarrier Max Size:       32                                 
    ISA 2                    
      Name:                    amdgcn-amd-amdhsa--gfx10-3-generic 
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)             
        y                        65535(0xffff)                      
        z                        65535(0xffff)                      
      FBarrier Max Size:       32                                 
*** Done ***

[danielholanda]

@ppanchad-amd Can you take a look? Looks like this might need to be fixed in ggml-org/llamacpp

[realsheox-max]

Hey, I have the same GPU (R9700, gfx1201) and ran into similar issues.

What's happening is that ROCm detects all GPU agents on your system, including the integrated graphics of your 9950X3D (gfx1036). So llama.cpp sees two ROCm devices and automatically tries to split the workload across both — which causes the segfault because the iGPU simply can't handle it.

You can see it in your log:
ROCm0 KV buffer size = 2048 MiB → R9700 ✅
ROCm1 KV buffer size = 6144 MiB → iGPU ❌ crash

First check which device index is which on your system:

llama-server --list-devices

Then just tell llama.cpp to only use the R9700 (usually ROCm0):

ROCR_VISIBLE_DEVICES=0 llama-server --device ROCm0 -mg 0 -sm none -m model.gguf -ngl 99

ROCR_VISIBLE_DEVICES=0 hides the iGPU from ROCm entirely, --device ROCm0 targets the R9700 explicitly, and -mg 0 -sm none disables the automatic multi-GPU split. Together they make sure llama.cpp only ever sees and uses the R9700.

In my case I have an RTX 4070 as display GPU so ROCm only picks up the R9700 and I don't hit this issue — but the fix above should work for your setup. Probably needs a proper fix upstream in ggml-org/llama.cpp to either skip iGPUs automatically or at least warn before trying to use them.

For reference, here is my working command with the R9700 on Windows using the Lemonade ROCm build — 35B model, 180k context, stable at ~88 t/s:

.\llama-server.exe --device ROCm0 --model "model.gguf" --ctx-size 180000 --threads 8 --parallel 1 --n-gpu-layers -1 --host 127.0.0.1 --port 8080 --mlock --no-mmap --no-warmup --cache-ram 8192

The syntax should be identical on Linux, just replace .\llama-server.exe with ./llama-server:

./llama-server --device ROCm0 --model "model.gguf" --ctx-size 180000 --threads 8 --parallel 1 --n-gpu-layers -1 --host 127.0.0.1 --port 8080 --mlock --no-mmap --no-warmup --cache-ram 8192

The key part is --device ROCm0 which makes sure only the R9700 is used and ignores the iGPU completely. Works great — 88 t/s on a 35B model with 180k context, all 32GB VRAM utilized.

[mapatel-amd]

@kaerumy, is it possible for you to disable your integrated graphics in BIOS? This is often the quickest solution to get things working.

[kaerumy]

@mapatel-amd I already provided the workaround in my bug filing, and also by another in the reply.

To add -mg 0 -sm none in LLAMA CPP arguments in lemonade to manually set main GPU and disable split mode, if you want to only use your main GPU. It's less drastic then disabling iGPU entirely in BIOS.

For those with multiple GPUs + iGPU, they can list specific devices and not include the iGPU:

llama-server --list-devices

  Available devices:
  ROCm0: AMD Radeon AI PRO R9700 (32624 MiB, 2974 MiB free)
  ROCm1: AMD Radeon AI PRO R9700 (32624 MiB, 15865 MiB free)
  ROCm2: AMD Ryzen 9 9950X3D 16-Core Processor (62059 MiB, 103150 MiB free)

llama-server --device ROCm0,ROCm1 (in this example if you have two or more similar GPUs)

Adding this in documentation somewhere might be helpful for others.

Our use case is that we use iGPU for basic desktop GUI, so that GPU can be fully dedicated for AI and compute use.

[mapatel-amd]

@kaerumy, thanks for the help and clarification. The llama.cpp arguments help me figure out why the iGPU was still being targeted. The issue above was filed and now this pr should help with getting the ROCm backend working. Will now investigate into why the Vulkan backend is also not working. Just wanted to give an update.

[kaerumy]

Also related lemonade-sdk/lemonade#2371 as for lemonade 10.8.1

-mg 0 -sm none work around won't work anymore because lemonade now installs gfx1036 libs for gfx1201

2026-06-25 15:58:48.943 [Debug] (Server) GET /api/v1/system-info
2026-06-25 15:58:49.311 [Debug] (Server) POST /api/v1/load
2026-06-25 15:58:49.311 [Debug] (Server) ===== LOAD ENDPOINT ENTERED (Thread: 136382283478720) =====
2026-06-25 15:58:49.311 [Info] (Server) Ensuring model loaded: Qwen3.6-27B-MTP-GGUF 
2026-06-25 15:58:49.311 [Debug] (Router) Loading model: Qwen3.6-27B-MTP-GGUF (checkpoint: unsloth/Qwen3.6-27B-MTP-GGUF:Qwen3.6-27B-UD-Q4_K_XL.gguf, recipe: llamacpp, type: llm, device: gpu)
2026-06-25 15:58:49.311 [Warn] (Router) Pruning unavailable backend for model: Qwen3.6-27B-MTP-GGUF (state=watchdog_reset)
2026-06-25 15:58:49.311 [Info] (Router) Evicting model: Qwen3.6-27B-MTP-GGUF
2026-06-25 15:58:49.311 [Info] (LlamaCpp) Unloading model...
2026-06-25 15:58:49.311 [Info] (LlamaCpp) Unloading model...
2026-06-25 15:58:49.311 [Info] (Router) Evicted model: Qwen3.6-27B-MTP-GGUF
2026-06-25 15:58:49.311 [Debug] (Router) Effective settings: ctx_size=131072, llamacpp_device=(none), llamacpp_backend=rocm, llamacpp_args=-mg 0 -sm none  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-device ROCm0   --presence-penalty 0.0 --repeat-penalty 1.0 --temp 0.6 --top-p 0.95  --top-k 10  --min-p 0.05, merge_args=true, auto_evict=(none), evict_idle_timeout=300, downsize_idle_timeout=60, evict_weight_factor=1.000000, pinned=false
2026-06-25 15:58:49.311 [Debug] (Router) Creating LlamaCpp backend
2026-06-25 15:58:49.311 [Debug] (Router) Starting backend (this may take a moment)...
2026-06-25 15:58:49.311 [Info] (LlamaCpp) Loading model: Qwen3.6-27B-MTP-GGUF
2026-06-25 15:58:49.311 [Debug] (LlamaCpp) Per-model settings: ctx_size=131072, llamacpp_device=(none), llamacpp_backend=rocm, llamacpp_args=-mg 0 -sm none  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-device ROCm0   --presence-penalty 0.0 --repeat-penalty 1.0 --temp 0.6 --top-p 0.95  --top-k 10  --min-p 0.05, merge_args=true, auto_evict=(none), evict_idle_timeout=300, downsize_idle_timeout=60, evict_weight_factor=1.000000, pinned=false
2026-06-25 15:58:49.311 [Info] (LlamaCpp) Using LlamaCpp Backend: rocm-stable
2026-06-25 15:58:49.311 [Debug] (BackendManager) Installing llamacpp:rocm-stable
2026-06-25 15:58:49.311 [Debug] (BackendUtils) No system-wide ROCm installation detected at /opt/rocm
2026-06-25 15:58:49.312 [Debug] (llamacpp Server) Found executable at: /var/lib/lemonade/.cache/lemonade/bin/llamacpp/rocm-stable/llama-server
2026-06-25 15:58:49.312 [Debug] (LlamaCpp) Using GGUF: /var/lib/lemonade/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/0da008d9b980b76520a952bb3aa4abbaa0955647/Qwen3.6-27B-UD-Q4_K_XL.gguf
2026-06-25 15:58:49.312 [Debug] (WrappedServer) llama-server will use port: 8002
2026-06-25 15:58:49.312 [Debug] (LlamaCpp) Using backend: rocm-stable
2026-06-25 15:58:49.312 [Debug] (LlamaCpp) [LlamaCpp] Use GPU: true
2026-06-25 15:58:49.312 [Info] (LlamaCpp) Model uses MTP, adding draft decoding defaults
2026-06-25 15:58:49.312 [Debug] (LlamaCpp) Adding custom arguments: -mg 0 -sm none  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-device ROCm0   --presence-penalty 0.0 --repeat-penalty 1.0 --temp 0.6 --top-p 0.95  --top-k 10  --min-p 0.05
2026-06-25 15:58:49.312 [Info] (LlamaCpp) Starting llama-server...
2026-06-25 15:58:49.312 [Debug] (BackendUtils) Returning TheRock runtime path: /var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib
2026-06-25 15:58:49.312 [Debug] (LlamaCpp) Setting LD_LIBRARY_PATH=/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib:/var/lib/lemonade/.cache/lemonade/bin/llamacpp/rocm-stable
2026-06-25 15:58:49.312 [Debug] (ProcessManager) Starting process with filtered output: /var/lib/lemonade/.cache/lemonade/bin/llamacpp/rocm-stable/llama-server -m /var/lib/lemonade/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/0da008d9b980b76520a952bb3aa4abbaa0955647/Qwen3.6-27B-UD-Q4_K_XL.gguf --ctx-size 131072 --port 8002 --jinja --metrics --mmproj /var/lib/lemonade/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/0da008d9b980b76520a952bb3aa4abbaa0955647/mmproj-F16.gguf --reasoning-format auto --spec-draft-p-min 0.75 --no-webui --no-mmap -mg 0 -sm none --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-device ROCm0 --presence-penalty 0.0 --repeat-penalty 1.0 --temp 0.6 --top-p 0.95 --top-k 10 --min-p 0.05
2026-06-25 15:58:49.312 [Info] (ProcessManager) Process started successfully, PID: 248062
2026-06-25 15:58:49.313 [Debug] (WrappedServer) Waiting for llama-server to be ready...
2026-06-25 15:58:49.422 [Info] (Process) 0.00.105.034 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
2026-06-25 15:58:49.422 [Info] (Process) 0.00.105.037 I device_info:
2026-06-25 15:58:49.422 [Info] (Process) 0.00.105.053 I   - ROCm0   : AMD Radeon AI PRO R9700 (32624 MiB, 32556 MiB free)
2026-06-25 15:58:49.422 [Info] (Process) 0.00.105.055 I   - CPU     : AMD Ryzen 9 9950X3D 16-Core Processor (124118 MiB, 124118 MiB free)
2026-06-25 15:58:49.422 [Info] (Process) 0.00.105.079 I system_info: n_threads = 16 (n_threads_batch = 16) / 32 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
2026-06-25 15:58:49.422 [Info] (Process) 0.00.105.084 I srv  llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
2026-06-25 15:58:49.422 [Info] (Process) 0.00.105.106 I srv          init: running without SSL
2026-06-25 15:58:49.422 [Info] (Process) 0.00.105.143 I srv          init: using 31 threads for HTTP server
2026-06-25 15:58:49.422 [Info] (Process) 0.00.105.145 I srv          init: The UI is disabled
2026-06-25 15:58:49.422 [Info] (Process) 0.00.105.145 I srv          init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable
2026-06-25 15:58:49.422 [Info] (Process) 0.00.105.214 I srv         start: binding port with default address family
2026-06-25 15:58:49.423 [Info] (Process) 0.00.106.344 I srv  llama_server: loading model
2026-06-25 15:58:49.423 [Info] (Process) 0.00.106.347 I srv    load_model: loading model '/var/lib/lemonade/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/0da008d9b980b76520a952bb3aa4abbaa0955647/Qwen3.6-27B-UD-Q4_K_XL.gguf'
2026-06-25 15:58:49.462 [Info] (Process) 0.00.145.500 I srv    load_model: [mtmd] estimated worst-case memory usage of mmproj is 1157.64 MiB (took 39.12 ms)
2026-06-25 15:58:49.697 [Info] (Process) 0.00.380.424 I srv    load_model: [spec] estimated memory usage of MTP context is 708.02 MiB
2026-06-25 15:58:49.697 [Info] (Process) 0.00.380.443 I common_init_result: fitting params to device memory ...
2026-06-25 15:58:49.697 [Info] (Process) 0.00.380.443 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
2026-06-25 15:58:52.582 [Info] (Process) 0.03.265.612 W llama_context: n_ctx_seq (131072) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
2026-06-25 15:58:52.715 [Info] (Process) 0.03.398.279 W sched_reserve: layer 0 is assigned to device CPU but the fused Gated Delta Net tensor is assigned to device ROCm0 (usually due to missing support)
2026-06-25 15:58:52.715 [Info] (Process) 0.03.398.284 W sched_reserve: fused Gated Delta Net (chunked) not supported, set to disabled
2026-06-25 15:58:52.744 [Info] (Process) 0.03.427.805 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
2026-06-25 15:58:52.879 [Info] (Process) 0.03.562.493 I srv    load_model: creating MTP draft context against the target model '/var/lib/lemonade/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/0da008d9b980b76520a952bb3aa4abbaa0955647/Qwen3.6-27B-UD-Q4_K_XL.gguf'
2026-06-25 15:58:52.879 [Info] (Process) 0.03.562.569 W llama_context: n_ctx_seq (131072) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
2026-06-25 15:58:52.904 [Info] (Process) 0.03.587.442 W load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
2026-06-25 15:58:52.904 [Info] (Process) 0.03.587.446 W load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
2026-06-25 15:58:52.904 [Info] (Process) 0.03.587.446 W load_hparams: more info: https://github.com/ggml-org/llama.cpp/issues/16842
2026-06-25 15:58:53.009 [Info] (Process) 0.03.692.052 I srv    load_model: loaded multimodal model, '/var/lib/lemonade/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/0da008d9b980b76520a952bb3aa4abbaa0955647/mmproj-F16.gguf'
2026-06-25 15:58:53.009 [Info] (Process) 0.03.692.069 I srv    load_model: initializing slots, n_slots = 4
2026-06-25 15:58:53.120 [Info] (Process) 0.03.803.769 I common_context_can_seq_rm: the context supports bounded partial sequence removal
2026-06-25 15:58:53.181 [Info] (Process) 0.03.863.951 I common_speculative_impl_draft_mtp: adding speculative implementation 'draft-mtp'
2026-06-25 15:58:53.181 [Info] (Process) 0.03.863.958 I common_speculative_impl_draft_mtp: - n_max=4, n_min=0, p_min=0.75, n_embd=5120, backend_sampling=1
2026-06-25 15:58:53.181 [Info] (Process) 0.03.863.961 I common_speculative_impl_draft_mtp: - gpu_layers=-1, cache_k=f16, cache_v=f16, ctx_tgt=yes, ctx_dft=yes, devices=[ROCm0]
2026-06-25 15:58:53.181 [Info] (Process) 0.03.864.075 W llama_sampler_backend_support: device 'ROCm0' does not have support for op TOP_K needed for sampler 'top-k'
2026-06-25 15:58:53.181 [Info] (Process) 0.03.864.080 W llama_sampler_backend_support: device 'ROCm0' does not have support for op TOP_K needed for sampler 'top-k'
2026-06-25 15:58:53.181 [Info] (Process) 0.03.864.081 W llama_sampler_backend_support: device 'ROCm0' does not have support for op TOP_K needed for sampler 'top-k'
2026-06-25 15:58:53.181 [Info] (Process) 0.03.864.083 W llama_sampler_backend_support: device 'ROCm0' does not have support for op TOP_K needed for sampler 'top-k'
2026-06-25 15:58:53.217 [Info] (Process) 0.03.900.583 I srv    load_model: speculative decoding context initialized
2026-06-25 15:58:53.217 [Info] (Process) 0.03.900.588 I slot   load_model: id  0 | task -1 | new slot, n_ctx = 131072
2026-06-25 15:58:53.217 [Info] (Process) 0.03.900.593 I slot   load_model: id  1 | task -1 | new slot, n_ctx = 131072
2026-06-25 15:58:53.217 [Info] (Process) 0.03.900.594 I slot   load_model: id  2 | task -1 | new slot, n_ctx = 131072
2026-06-25 15:58:53.217 [Info] (Process) 0.03.900.594 I slot   load_model: id  3 | task -1 | new slot, n_ctx = 131072
2026-06-25 15:58:53.217 [Info] (Process) 0.03.900.620 I srv    load_model: prompt cache is enabled, size limit: 8192 MiB
2026-06-25 15:58:53.217 [Info] (Process) 0.03.900.621 I srv    load_model: use `--cache-ram 0` to disable the prompt cache
2026-06-25 15:58:53.217 [Info] (Process) 0.03.900.621 I srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
2026-06-25 15:58:53.217 [Info] (Process) 0.03.900.622 I srv    load_model: context checkpoints enabled, max = 32, min spacing = 256
2026-06-25 15:58:53.217 [Info] (Process) 0.03.900.633 I srv          init: idle slots will be saved to prompt cache and cleared upon starting a new task
2026-06-25 15:58:53.227 [Info] (Process) 0.03.910.574 I init: chat template, example_format: '<|im_start|>system
2026-06-25 15:58:53.227 [Info] (Process) You are a helpful assistant<|im_end|>
2026-06-25 15:58:53.227 [Info] (Process) <|im_start|>user
2026-06-25 15:58:53.227 [Info] (Process) Hello<|im_end|>
2026-06-25 15:58:53.227 [Info] (Process) <|im_start|>assistant
2026-06-25 15:58:53.227 [Info] (Process) Hi there<|im_end|>
2026-06-25 15:58:53.227 [Info] (Process) <|im_start|>user
2026-06-25 15:58:53.227 [Info] (Process) How are you?<|im_end|>
2026-06-25 15:58:53.227 [Info] (Process) <|im_start|>assistant
2026-06-25 15:58:53.227 [Info] (Process) <think>
2026-06-25 15:58:53.227 [Info] (Process) '
2026-06-25 15:58:53.235 [Info] (Process) 0.03.918.086 I srv          init: init: chat template, thinking = 1
2026-06-25 15:58:53.235 [Info] (Process) 0.03.918.126 I srv  llama_server: model loaded
2026-06-25 15:58:53.235 [Info] (Process) 0.03.918.130 I srv  llama_server: server is listening on http://127.0.0.1:8002
2026-06-25 15:58:53.268 [Info] (WrappedServer) llama-server is ready!
2026-06-25 15:58:53.269 [Info] (BackendWatchdog) Started watchdog for llama-server using http://127.0.0.1:8002/health (streaming=on, non_streaming=on)
2026-06-25 15:58:53.269 [Debug] (LlamaCpp) Model loaded on port 8002
2026-06-25 15:58:53.269 [Debug] (Router) Backend started successfully in 3957ms
2026-06-25 15:58:53.269 [Info] (Router) Model loaded successfully. Total loaded: 1
2026-06-25 15:58:53.269 [Debug] (Server) POST /api/v1/load - 200
2026-06-25 15:58:54.004 [Debug] (Server) GET /api/v1/models
2026-06-25 15:59:11.557 [Debug] (Server) POST /api/v1/chat/completions
2026-06-25 15:59:11.557 [Debug] (Server) No tools in request
2026-06-25 15:59:11.557 [Info] (Server) Model already loaded: Qwen3.6-27B-MTP-GGUF
2026-06-25 15:59:11.557 [Info] (Server) POST /api/v1/chat/completions - Streaming
2026-06-25 15:59:11.569 [Info] (Process) 0.22.252.407 I srv    operator(): Chat format: peg-native
2026-06-25 15:59:11.569 [Info] (Process) 0.22.252.667 I slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
2026-06-25 15:59:11.569 [Info] (Process) 0.22.252.669 I srv  get_availabl: updating prompt cache
2026-06-25 15:59:11.569 [Info] (Process) 0.22.252.677 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
2026-06-25 15:59:11.569 [Info] (Process) 0.22.252.684 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 131072 tokens, 8589934592 est)
2026-06-25 15:59:11.569 [Info] (Process) 0.22.252.685 I srv  get_availabl: prompt cache update took 0.01 ms
2026-06-25 15:59:11.569 [Info] (Process) 0.22.252.782 I slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
2026-06-25 15:59:11.569 [Info] (Process) 0.22.252.784 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
2026-06-25 15:59:11.569 [Info] (Process) 0.22.252.786 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
2026-06-25 15:59:11.570 [Info] (Process) 0.22.252.938 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
2026-06-25 15:59:11.570 [Info] (Process) 0.22.252.939 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
2026-06-25 15:59:11.570 [Info] (Process) 0.22.253.072 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
2026-06-25 15:59:11.570 [Info] (Process) 0.22.253.072 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
2026-06-25 15:59:11.593 [Error] (Process) rocblaslt error: Cannot read "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/hipblaslt/library/TensileLibrary_lazy_gfx1201.dat": No such file or directory
2026-06-25 15:59:11.593 [Error] (Process) rocblaslt error: Could not load "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/hipblaslt/library/TensileLibrary_lazy_gfx1201.dat"
2026-06-25 15:59:11.593 [Info] (Process) hipModuleLoad failed: /var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/hipblaslt/library/Kernels.so-000-gfx1201.hsaco
2026-06-25 15:59:11.593 [Error] (Process)  error: file not found
2026-06-25 15:59:11.593 [Info] (Process) hipModuleLoad failed: /var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/hipblaslt/library/Kernels.so-000-gfx1201-xnack-.hsaco
2026-06-25 15:59:11.593 [Error] (Process)  error: file not found
2026-06-25 15:59:11.593 [Info] (Process) hipModuleLoad failed: /var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/hipblaslt/library/Kernels.so-000-gfx1201-xnack+.hsaco
2026-06-25 15:59:11.593 [Error] (Process)  error: file not found
2026-06-25 15:59:11.594 [Error] (Process) rocBLAS error: Cannot read /var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary.dat: Illegal seek for GPU arch : gfx1201
2026-06-25 15:59:11.594 [Info] (Process)  List of available TensileLibrary Files : 
2026-06-25 15:59:11.594 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1032.dat"
2026-06-25 15:59:11.594 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1031.dat"
2026-06-25 15:59:11.594 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1036.dat"
2026-06-25 15:59:11.594 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1034.dat"
2026-06-25 15:59:11.594 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1030.dat"
2026-06-25 15:59:11.595 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1035.dat"
2026-06-25 15:59:11.595 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1033.dat"
2026-06-25 15:59:11.666 [Warn] (HttpClient) Stream ended with: Transferred a partial file (response code: 200)
2026-06-25 15:59:11.666 [Error] (WrappedServer) Streaming request failed: backend connection failed during SSE stream before DONE: CURL error: Transferred a partial file
2026-06-25 15:59:11.666 [Error] (BackendWatchdog) llama-server backend marked unavailable: backend connection failed during streaming request: backend connection failed during SSE stream before DONE: CURL error: Transferred a partial file; terminating and reaping backend process PID 248062
2026-06-25 15:59:11.766 [Info] (ProcessManager) Process terminated, waiting for GPU driver cleanup...
2026-06-25 15:59:13.767 [Warn] (Router) Reloading model after backend watchdog reset: Qwen3.6-27B-MTP-GGUF
2026-06-25 15:59:13.767 [Debug] (Router) Loading model: Qwen3.6-27B-MTP-GGUF (checkpoint: unsloth/Qwen3.6-27B-MTP-GGUF:Qwen3.6-27B-UD-Q4_K_XL.gguf, recipe: llamacpp, type: llm, device: gpu)
2026-06-25 15:59:13.767 [Warn] (Router) Pruning unavailable backend for model: Qwen3.6-27B-MTP-GGUF (state=watchdog_reset)
2026-06-25 15:59:13.767 [Info] (Router) Evicting model: Qwen3.6-27B-MTP-GGUF
2026-06-25 15:59:13.767 [Info] (LlamaCpp) Unloading model...
2026-06-25 15:59:13.767 [Info] (LlamaCpp) Unloading model...
2026-06-25 15:59:13.767 [Info] (Router) Evicted model: Qwen3.6-27B-MTP-GGUF
2026-06-25 15:59:13.767 [Debug] (Router) Effective settings: ctx_size=131072, llamacpp_device=(none), llamacpp_backend=rocm, llamacpp_args=--min-p 0.05 --presence-penalty 0.0 --repeat-penalty 1.0 --spec-draft-device ROCm0 --spec-draft-n-max 4 --spec-type draft-mtp --temp 0.6 --top-k 10 --top-p 0.95 -mg 0 -sm none, merge_args=true, auto_evict=(none), evict_idle_timeout=300, downsize_idle_timeout=60, evict_weight_factor=1.000000, pinned=false
2026-06-25 15:59:13.767 [Debug] (Router) Creating LlamaCpp backend
2026-06-25 15:59:13.767 [Debug] (Router) Starting backend (this may take a moment)...
2026-06-25 15:59:13.767 [Info] (LlamaCpp) Loading model: Qwen3.6-27B-MTP-GGUF
2026-06-25 15:59:13.767 [Debug] (LlamaCpp) Per-model settings: ctx_size=131072, llamacpp_device=(none), llamacpp_backend=rocm, llamacpp_args=--min-p 0.05 --presence-penalty 0.0 --repeat-penalty 1.0 --spec-draft-device ROCm0 --spec-draft-n-max 4 --spec-type draft-mtp --temp 0.6 --top-k 10 --top-p 0.95 -mg 0 -sm none, merge_args=true, auto_evict=(none), evict_idle_timeout=300, downsize_idle_timeout=60, evict_weight_factor=1.000000, pinned=false
2026-06-25 15:59:13.767 [Info] (LlamaCpp) Using LlamaCpp Backend: rocm-stable
2026-06-25 15:59:13.767 [Debug] (BackendManager) Installing llamacpp:rocm-stable
2026-06-25 15:59:13.767 [Debug] (BackendUtils) No system-wide ROCm installation detected at /opt/rocm
2026-06-25 15:59:13.767 [Debug] (llamacpp Server) Found executable at: /var/lib/lemonade/.cache/lemonade/bin/llamacpp/rocm-stable/llama-server
2026-06-25 15:59:13.767 [Debug] (LlamaCpp) Using GGUF: /var/lib/lemonade/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/0da008d9b980b76520a952bb3aa4abbaa0955647/Qwen3.6-27B-UD-Q4_K_XL.gguf
2026-06-25 15:59:13.768 [Debug] (WrappedServer) llama-server will use port: 8001
2026-06-25 15:59:13.768 [Debug] (LlamaCpp) Using backend: rocm-stable
2026-06-25 15:59:13.768 [Debug] (LlamaCpp) [LlamaCpp] Use GPU: true
2026-06-25 15:59:13.768 [Info] (LlamaCpp) Model uses MTP, adding draft decoding defaults
2026-06-25 15:59:13.768 [Debug] (LlamaCpp) Adding custom arguments: --min-p 0.05 --presence-penalty 0.0 --repeat-penalty 1.0 --spec-draft-device ROCm0 --spec-draft-n-max 4 --spec-type draft-mtp --temp 0.6 --top-k 10 --top-p 0.95 -mg 0 -sm none
2026-06-25 15:59:13.768 [Info] (LlamaCpp) Starting llama-server...
2026-06-25 15:59:13.768 [Debug] (BackendUtils) Returning TheRock runtime path: /var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib
2026-06-25 15:59:13.768 [Debug] (LlamaCpp) Setting LD_LIBRARY_PATH=/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib:/var/lib/lemonade/.cache/lemonade/bin/llamacpp/rocm-stable
2026-06-25 15:59:13.768 [Debug] (ProcessManager) Starting process with filtered output: /var/lib/lemonade/.cache/lemonade/bin/llamacpp/rocm-stable/llama-server -m /var/lib/lemonade/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/0da008d9b980b76520a952bb3aa4abbaa0955647/Qwen3.6-27B-UD-Q4_K_XL.gguf --ctx-size 131072 --port 8001 --jinja --metrics --mmproj /var/lib/lemonade/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/0da008d9b980b76520a952bb3aa4abbaa0955647/mmproj-F16.gguf --reasoning-format auto --spec-draft-p-min 0.75 --no-webui --no-mmap --min-p 0.05 --presence-penalty 0.0 --repeat-penalty 1.0 --spec-draft-device ROCm0 --spec-draft-n-max 4 --spec-type draft-mtp --temp 0.6 --top-k 10 --top-p 0.95 -mg 0 -sm none
2026-06-25 15:59:13.768 [Info] (ProcessManager) Process started successfully, PID: 248121
2026-06-25 15:59:13.769 [Debug] (WrappedServer) Waiting for llama-server to be ready...
2026-06-25 15:59:13.884 [Info] (Process) 0.00.111.584 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
2026-06-25 15:59:13.884 [Info] (Process) 0.00.111.587 I device_info:
2026-06-25 15:59:13.884 [Info] (Process) 0.00.111.604 I   - ROCm0   : AMD Radeon AI PRO R9700 (32624 MiB, 32556 MiB free)
2026-06-25 15:59:13.884 [Info] (Process) 0.00.111.610 I   - CPU     : AMD Ryzen 9 9950X3D 16-Core Processor (124118 MiB, 124118 MiB free)
2026-06-25 15:59:13.884 [Info] (Process) 0.00.111.636 I system_info: n_threads = 16 (n_threads_batch = 16) / 32 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
2026-06-25 15:59:13.884 [Info] (Process) 0.00.111.641 I srv  llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
2026-06-25 15:59:13.884 [Info] (Process) 0.00.111.667 I srv          init: running without SSL
2026-06-25 15:59:13.884 [Info] (Process) 0.00.111.708 I srv          init: using 31 threads for HTTP server
2026-06-25 15:59:13.884 [Info] (Process) 0.00.111.710 I srv          init: The UI is disabled
2026-06-25 15:59:13.884 [Info] (Process) 0.00.111.710 I srv          init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable
2026-06-25 15:59:13.884 [Info] (Process) 0.00.111.783 I srv         start: binding port with default address family
2026-06-25 15:59:13.885 [Info] (Process) 0.00.112.936 I srv  llama_server: loading model
2026-06-25 15:59:13.885 [Info] (Process) 0.00.112.940 I srv    load_model: loading model '/var/lib/lemonade/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/0da008d9b980b76520a952bb3aa4abbaa0955647/Qwen3.6-27B-UD-Q4_K_XL.gguf'
2026-06-25 15:59:13.926 [Info] (Process) 0.00.154.272 I srv    load_model: [mtmd] estimated worst-case memory usage of mmproj is 1157.64 MiB (took 41.29 ms)
2026-06-25 15:59:14.169 [Info] (Process) 0.00.396.664 I srv    load_model: [spec] estimated memory usage of MTP context is 708.02 MiB
2026-06-25 15:59:14.169 [Info] (Process) 0.00.396.684 I common_init_result: fitting params to device memory ...
2026-06-25 15:59:14.169 [Info] (Process) 0.00.396.685 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
2026-06-25 15:59:17.058 [Info] (Process) 0.03.286.093 W llama_context: n_ctx_seq (131072) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
2026-06-25 15:59:17.191 [Info] (Process) 0.03.418.696 W sched_reserve: layer 0 is assigned to device CPU but the fused Gated Delta Net tensor is assigned to device ROCm0 (usually due to missing support)
2026-06-25 15:59:17.191 [Info] (Process) 0.03.418.701 W sched_reserve: fused Gated Delta Net (chunked) not supported, set to disabled
2026-06-25 15:59:17.220 [Info] (Process) 0.03.447.950 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
2026-06-25 15:59:17.354 [Info] (Process) 0.03.581.697 I srv    load_model: creating MTP draft context against the target model '/var/lib/lemonade/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/0da008d9b980b76520a952bb3aa4abbaa0955647/Qwen3.6-27B-UD-Q4_K_XL.gguf'
2026-06-25 15:59:17.354 [Info] (Process) 0.03.581.740 W llama_context: n_ctx_seq (131072) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
2026-06-25 15:59:17.378 [Info] (Process) 0.03.606.029 W load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
2026-06-25 15:59:17.378 [Info] (Process) 0.03.606.033 W load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
2026-06-25 15:59:17.378 [Info] (Process) 0.03.606.033 W load_hparams: more info: https://github.com/ggml-org/llama.cpp/issues/16842
2026-06-25 15:59:17.482 [Info] (Process) 0.03.710.041 I srv    load_model: loaded multimodal model, '/var/lib/lemonade/.cache/huggingface/hub/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/0da008d9b980b76520a952bb3aa4abbaa0955647/mmproj-F16.gguf'
2026-06-25 15:59:17.482 [Info] (Process) 0.03.710.055 I srv    load_model: initializing slots, n_slots = 4
2026-06-25 15:59:17.594 [Info] (Process) 0.03.821.608 I common_context_can_seq_rm: the context supports bounded partial sequence removal
2026-06-25 15:59:17.654 [Info] (Process) 0.03.881.769 I common_speculative_impl_draft_mtp: adding speculative implementation 'draft-mtp'
2026-06-25 15:59:17.654 [Info] (Process) 0.03.881.776 I common_speculative_impl_draft_mtp: - n_max=4, n_min=0, p_min=0.75, n_embd=5120, backend_sampling=1
2026-06-25 15:59:17.654 [Info] (Process) 0.03.881.779 I common_speculative_impl_draft_mtp: - gpu_layers=-1, cache_k=f16, cache_v=f16, ctx_tgt=yes, ctx_dft=yes, devices=[ROCm0]
2026-06-25 15:59:17.654 [Info] (Process) 0.03.881.892 W llama_sampler_backend_support: device 'ROCm0' does not have support for op TOP_K needed for sampler 'top-k'
2026-06-25 15:59:17.654 [Info] (Process) 0.03.881.896 W llama_sampler_backend_support: device 'ROCm0' does not have support for op TOP_K needed for sampler 'top-k'
2026-06-25 15:59:17.654 [Info] (Process) 0.03.881.897 W llama_sampler_backend_support: device 'ROCm0' does not have support for op TOP_K needed for sampler 'top-k'
2026-06-25 15:59:17.654 [Info] (Process) 0.03.881.899 W llama_sampler_backend_support: device 'ROCm0' does not have support for op TOP_K needed for sampler 'top-k'
2026-06-25 15:59:17.686 [Info] (Process) 0.03.914.235 I srv    load_model: speculative decoding context initialized
2026-06-25 15:59:17.686 [Info] (Process) 0.03.914.240 I slot   load_model: id  0 | task -1 | new slot, n_ctx = 131072
2026-06-25 15:59:17.686 [Info] (Process) 0.03.914.247 I slot   load_model: id  1 | task -1 | new slot, n_ctx = 131072
2026-06-25 15:59:17.686 [Info] (Process) 0.03.914.247 I slot   load_model: id  2 | task -1 | new slot, n_ctx = 131072
2026-06-25 15:59:17.686 [Info] (Process) 0.03.914.248 I slot   load_model: id  3 | task -1 | new slot, n_ctx = 131072
2026-06-25 15:59:17.686 [Info] (Process) 0.03.914.298 I srv    load_model: prompt cache is enabled, size limit: 8192 MiB
2026-06-25 15:59:17.686 [Info] (Process) 0.03.914.299 I srv    load_model: use `--cache-ram 0` to disable the prompt cache
2026-06-25 15:59:17.686 [Info] (Process) 0.03.914.300 I srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
2026-06-25 15:59:17.686 [Info] (Process) 0.03.914.300 I srv    load_model: context checkpoints enabled, max = 32, min spacing = 256
2026-06-25 15:59:17.686 [Info] (Process) 0.03.914.312 I srv          init: idle slots will be saved to prompt cache and cleared upon starting a new task
2026-06-25 15:59:17.696 [Info] (Process) 0.03.924.319 I init: chat template, example_format: '<|im_start|>system
2026-06-25 15:59:17.696 [Info] (Process) You are a helpful assistant<|im_end|>
2026-06-25 15:59:17.696 [Info] (Process) <|im_start|>user
2026-06-25 15:59:17.696 [Info] (Process) Hello<|im_end|>
2026-06-25 15:59:17.696 [Info] (Process) <|im_start|>assistant
2026-06-25 15:59:17.696 [Info] (Process) Hi there<|im_end|>
2026-06-25 15:59:17.696 [Info] (Process) <|im_start|>user
2026-06-25 15:59:17.697 [Info] (Process) How are you?<|im_end|>
2026-06-25 15:59:17.697 [Info] (Process) <|im_start|>assistant
2026-06-25 15:59:17.697 [Info] (Process) <think>
2026-06-25 15:59:17.697 [Info] (Process) '
2026-06-25 15:59:17.704 [Info] (Process) 0.03.931.878 I srv          init: init: chat template, thinking = 1
2026-06-25 15:59:17.704 [Info] (Process) 0.03.931.918 I srv  llama_server: model loaded
2026-06-25 15:59:17.704 [Info] (Process) 0.03.931.923 I srv  llama_server: server is listening on http://127.0.0.1:8001
2026-06-25 15:59:17.744 [Info] (WrappedServer) llama-server is ready!
2026-06-25 15:59:17.744 [Info] (BackendWatchdog) Started watchdog for llama-server using http://127.0.0.1:8001/health (streaming=on, non_streaming=on)
2026-06-25 15:59:17.744 [Debug] (LlamaCpp) Model loaded on port 8001
2026-06-25 15:59:17.744 [Debug] (Router) Backend started successfully in 3977ms
2026-06-25 15:59:17.744 [Info] (Router) Model loaded successfully. Total loaded: 1
2026-06-25 15:59:17.754 [Info] (Process) 0.03.981.767 I srv    operator(): Chat format: peg-native
2026-06-25 15:59:17.754 [Info] (Process) 0.03.982.002 I slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
2026-06-25 15:59:17.754 [Info] (Process) 0.03.982.007 I srv  get_availabl: updating prompt cache
2026-06-25 15:59:17.754 [Info] (Process) 0.03.982.015 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
2026-06-25 15:59:17.754 [Info] (Process) 0.03.982.021 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 131072 tokens, 8589934592 est)
2026-06-25 15:59:17.754 [Info] (Process) 0.03.982.024 I srv  get_availabl: prompt cache update took 0.01 ms
2026-06-25 15:59:17.754 [Info] (Process) 0.03.982.118 I slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
2026-06-25 15:59:17.754 [Info] (Process) 0.03.982.119 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
2026-06-25 15:59:17.754 [Info] (Process) 0.03.982.121 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
2026-06-25 15:59:17.754 [Info] (Process) 0.03.982.239 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
2026-06-25 15:59:17.754 [Info] (Process) 0.03.982.240 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
2026-06-25 15:59:17.754 [Info] (Process) 0.03.982.347 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
2026-06-25 15:59:17.754 [Info] (Process) 0.03.982.349 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
2026-06-25 15:59:17.770 [Error] (Process) rocblaslt error: Cannot read "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/hipblaslt/library/TensileLibrary_lazy_gfx1201.dat": No such file or directory
2026-06-25 15:59:17.771 [Error] (Process) rocblaslt error: Could not load "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/hipblaslt/library/TensileLibrary_lazy_gfx1201.dat"
2026-06-25 15:59:17.771 [Info] (Process) hipModuleLoad failed: /var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/hipblaslt/library/Kernels.so-000-gfx1201.hsaco
2026-06-25 15:59:17.771 [Error] (Process)  error: file not found
2026-06-25 15:59:17.771 [Info] (Process) hipModuleLoad failed: /var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/hipblaslt/library/Kernels.so-000-gfx1201-xnack-.hsaco
2026-06-25 15:59:17.771 [Error] (Process)  error: file not found
2026-06-25 15:59:17.771 [Info] (Process) hipModuleLoad failed: /var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/hipblaslt/library/Kernels.so-000-gfx1201-xnack+.hsaco
2026-06-25 15:59:17.771 [Error] (Process)  error: file not found
2026-06-25 15:59:17.771 [Error] (Process) rocBLAS error: Cannot read /var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary.dat: Illegal seek for GPU arch : gfx1201
2026-06-25 15:59:17.771 [Info] (Process)  List of available TensileLibrary Files : 
2026-06-25 15:59:17.771 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1032.dat"
2026-06-25 15:59:17.771 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1031.dat"
2026-06-25 15:59:17.771 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1036.dat"
2026-06-25 15:59:17.771 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1034.dat"
2026-06-25 15:59:17.771 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1030.dat"
2026-06-25 15:59:17.772 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1035.dat"
2026-06-25 15:59:17.772 [Info] (Process) "/var/lib/lemonade/.cache/lemonade/bin/therock/gfx1036-7.13.0/lib/rocblas/library/TensileLibrary_lazy_gfx1033.dat"
2026-06-25 15:59:17.833 [Warn] (HttpClient) Stream ended with: Transferred a partial file (response code: 200)
2026-06-25 15:59:17.833 [Error] (WrappedServer) Streaming request failed: backend connection failed during SSE stream before DONE: CURL error: Transferred a partial file
2026-06-25 15:59:17.833 [Error] (BackendWatchdog) llama-server backend marked unavailable: backend connection failed during streaming request: backend connection failed during SSE stream before DONE: CURL error: Transferred a partial file; terminating and reaping backend process PID 248121
2026-06-25 15:59:17.933 [Info] (ProcessManager) Process terminated, waiting for GPU driver cleanup...
2026-06-25 15:59:19.933 [Debug] (Server) POST /api/v1/chat/completions - 200

[mapatel-amd]

Sorry for the long wait, as of this recent PR, this issue should be solved I believe. Please let me know if you're still encountering this issue using the new llamacpp build.

[kaerumy]

Vulkan thankfully has been working with recent builds (b9747), even with the wrong iGPU architecture libs being downloaded providing a temporary fallback workaround while this issue is being resolved. Will test with upstream build with the recently merged fix to confirm.