Autocomplete does not call correct template

[alabamagan]

Problem

After last major update (which gives us the new model config page), I haven't been able to get autocopmlete working. From Continue console, it seems that the template is not correctly used. My guess is there's some residual code overriding this, or perhaps that openrouter is not supported.

Current setting

My setting is simple:

  - name: OR-Qwen 
    model: qwen/qwen-2.5-coder-32b-instruct
    provider: openrouter
    apiKey: XXXXX
    apiBase: https://openrouter.ai/api/v1
    roles:
      - apply
      - autocomplete
    requestOptions:
      extraBodyProperties:
        think: false  # turning off the thinking

Obtained results

And the console response is weird like the model is not getting the system prompt at all.

What I've tried.

• reinstalled continue
• reload vscode window

[letanloc1998]

#8128 (comment)

Please choose the model support /completions instead of /response for workaround now

[ml-w]

#8128 (comment)

Please choose the model support /completions instead of /response for workaround now

Thanks for your suggestion but I am not sure I follow. Qwen and Mercury are both suggested on the official docs here, and they both supporting /completions endpoint.

Just to add a bit of info, I meant that whenever Continue autocomplete made an API, the Continue console shows the model response indicates its initiating a chat rather than a completion. I have no clue why this happens.

[mjdiff]

Similar here, was working but stopped. What do you mean by "Please choose the model support /completions instead of /response for workaround now"?

[bdougie]

I found some existing related conversations. Let me know if these are helpful:

#	Title	Status
#7738	Autocomplete in VSCode not working	🟢 Open
#7567	Autocomplete doesn't work	🟢 Open
#7896	Autocomplet does not work with VSCODE and ollama	✅ Answered
#7504	Local Ollama autocomplete keeps aborting	🟢 Open
#7219	vscode extension select model hangs.	🟢 Open
#7590	Code complete is not working!	🟢 Open
#7732	cannot assign model to autocomplete	🟢 Open

[mjdiff]

I think this is a bug. I have the same problem with OpenAI api. Before opening the issue, I just want to confirm.

[alabamagan]

@mjdiff Of note openrouter provider doesn't work as well despite the correct model.

@bdougie Thanks for following this up! I have had a close loo at those issues you posted, perhaps #7738 and #7567 are related. But seems like there's not a solution yet.

[mjdiff]

The #7590 seems to be this same issue as here instead of code complete where FIM marking is the response is like in chat.

[alabamagan]

The #7590 seems to be this same issue as here instead of code complete where FIM marking is the response is like in chat.

Could this be related to specifying the provider and openai instead of ollama once again? He seems to be using an ollama server while specifying openai provider?

[mjdiff]

I don't think so. I tested this with the Ollama server using both the OpenAI API and the Ollama API, as well as with another provider using the OpenAI API. In both cases, autocomplete does not work with the OpenAI API; the responses are similar to those in chat mode, ignoring the FIM markers.

[RomneyDa]

Thank you all for the details, this is a high priority for me.