fix(autogptq): do not use_triton with qwen-vl (#1985)

* Enhance autogptq backend to support VL models * update dependencies for autogptq * remove redundant auto-gptq dependency * Convert base64 to image_url for Qwen-VL model * implemented model inference for qwen-vl * remove user prompt from generated answer * fixed write image error * fixed use_triton issue when loading Qwen-VL model --------- Co-authored-by: Binghua Wu <[email protected]>
mudler · Apr 11, 2024 · 0004ec8 · 0004ec8
1 parent d692b2c
commit 0004ec8
Showing 1 changed file with 0 additions and 1 deletion.
diff --git a/backend/python/autogptq/autogptq.py b/backend/python/autogptq/autogptq.py
@@ -39,7 +39,6 @@ def LoadModel(self, request, context):
                 self.model_name = "Qwen-VL-Chat"
                 model = AutoModelForCausalLM.from_pretrained(model_path, 
                     trust_remote_code=request.TrustRemoteCode,
-                    use_triton=request.UseTriton,
                     device_map="auto").eval()
             else:
                 model = AutoGPTQForCausalLM.from_quantized(model_path,