How do I make an agent start with a static string and autocomplete the rest?

[PyroGenesis]

I have a Coder agent, and the only thing I want it to do is write code. I want to encourage this by adding a static string at the beginning and letting the model autocomplete it.

The starting string would be something like this:

'''python
import pandas as pd
df = pd.read_csv('data.csv')

and the rest should be completed by the Agent / LLM.

This is very similar to the 'start reply with' feature in text-generation-webui. How do I do this with AutoGen?

[rickyloynd-microsoft]

The key is to use the LLM for Completion rather than ChatCompletion. The API details for this have changed in the latest version of the openai package, so you can save yourself the time of porting your code from the old version to the new version by starting with the new version. As of today, that's openai==1.3.5, and pyautogen==0.2.0b6.

The documentation is here, and this is the most relevant code snippet from that page:

from autogen import OpenAIWrapper
# OpenAI endpoint
client = OpenAIWrapper()
# ChatCompletion
response = client.create(messages=[{"role": "user", "content": "2+2="}], model="gpt-3.5-turbo")
# extract the response text
print(client.extract_text_or_function_call(response))
# Azure OpenAI endpoint
client = OpenAIWrapper(api_key=..., base_url=..., api_version=..., api_type="azure")
# Completion
response = client.create(prompt="2+2=", model="gpt-3.5-turbo-instruct")
# extract the response text
print(client.extract_text_or_function_call(response))

[PyroGenesis]

Is there a way to do this in AutoGen v1? I'm using AutoGen + FastChat for local LLMs and haven't upgraded yet.

[rickyloynd-microsoft]

Yes, along the general lines shown in the documentation above, but using the older API. Then after you upgrade later, you'll have to change your code again to use the new API.

[PyroGenesis]

Thanks, I implemented it using the v1 API like so:

from autogen import oai

# create a text completion request
response = oai.Completion.create(
    config_list=config_list,
    prompt = (
        "'''python\n"
        "import pandas as pd\n"
        "df = pd.read_csv('data.csv')\n"
    ),
)
print(oai.Completion.extract_text_or_function_call(response))

then changed it to to the following after upgrading to v2:

from autogen import OpenAIWrapper

client = OpenAIWrapper(config_list=config_list)
response = client.create(prompt=(
    "'''python\n"
    "import pandas as pd\n"
    "df = pd.read_csv('data.csv')\n"
))
print(client.extract_text_or_function_call(response))

As for using it for an agent, I wrapped it up in a function and registered it as a reply function at the position just before the base reply function.

coder.register_reply(autogen.Agent, reply_func=generate_oai_textcompletion_reply, position=3)

[rickyloynd-microsoft]

I'll resolve this now. Please reopen to continue the discussion.