[chrome extension] wrong cursor position

[songkeys]

when asking "help me search today's news on google" on action, it keeps clicking on wrong position of webpage.

llm output:

. The Google homepage is displayed, with the search bar prominently located in the center of the screen.\n2. To proceed with searching for today's news, I need to click on the search bar to activate it and bring up the keyboard for text input.\n3. Clicking on the search bar will allow me to type in my query and initiate the search process.\nAction: click(start_box='<|box_start|>(459,978)<|box_end|>')

report: https://raw.githubusercontent.com/songkeys/oss/refs/heads/master/mac/midscene_report.html

[yuyutaotao]

Hi @songkeys ,which version of vlm-ui-tars are you using ?

[songkeys]

@yuyutaotao hi. it's https://huggingface.co/mradermacher/UI-TARS-7B-DPO-GGUF

[yuyutaotao]

According to the README of official UI-TARS model team, the GGUF version is no longer recommended. You may try other versions.

https://github.com/bytedance/UI-TARS?tab=readme-ov-file#%EF%B8%8F-important-announcement-gguf-model-performance

[songkeys]

when i tried searching UI TARS on lm-studio, it's all GGUF version. what alternative version would you recommend? could you share a huggingface repo? many thanks!

[yuyutaotao]

This is the official hugging face repo of ui tars: https://huggingface.co/bytedance-research/UI-TARS-7B-DPO

[songkeys]

thank you. i don't know if this could be solved by running the model you provided. on my macbook (m1), lm-studio and ollama only support GGUFs model. i have to run it with vllm but it's extremely slow and frozed my computer. not a single request was responsed. i'll leave this until i find a way to run it.

[yuyutaotao]

@songkeys you can try deploying it on cloud. M1 macbook can not handle the 7b model well.