SystemMessage / UserMessage / AssistantMessage with @prompt_chain

[alexchandel]

chatprompt supports the very nice SystemMessage, UserMessage, and AssistantMessage classes for input roles, which the API server will translate to <|im_start|>system / <|im_end|> / whatever formatting. However it doesn't support chained function calls, meaning you must resolve them and send the output back.

On the other hand, prompt_chain seems to support chained function calls, but only takes a template string (no message list), and sends the message as "role": "user". How can I use SystemMessage, UserMessage, and AssistantMessage with prompt_chain, or achieve the same effect?

[jackmpcollins]

@alexchandel Unfortunately there is no chatprompt_chain decorator - I'm open to adding this.

All that prompt_chain is doing internally is creating a Chat (not yet documented, see #317) and then looping until it gets no more function calls.

magentic/src/magentic/prompt_chain.py

Lines 82 to 93 in d7ae5c2

	chat = Chat.from_prompt(prompt_function, *args, **kwargs).submit()
	num_calls = 0
	while isinstance(chat.last_message.content, FunctionCall):
	if max_calls is not None and num_calls >= max_calls:
	msg = (
	f"Function {func.__name__} reached limit of"
	f" {max_calls} function calls"
	)
	raise MaxFunctionCallsError(msg)
	chat = chat.exec_function_call().submit()
	num_calls += 1
	return cast(R, chat.last_message.content)

Based on this you could do something like:

from magentic import UserMessage, FunctionCall
from magentic.chat import Chat

chat = Chat(
    messages=[UserMessage(...), ...],
    functions=[my_func],
    output_types=[str, list[int], FunctionCall],  # Note: FunctionCall is needed here
).submit()  # .submit() adds an AssistantMessage by querying the LLM
while isinstance(chat.last_message.content, FunctionCall):
    chat = chat.exec_function_call().submit()
return chat.last_message.content

---

Update: Chat is now documented, including the loop example above, at https://magentic.dev/chat

[alexchandel]

I like the loop idea, but is there any way to make it a decorator, such that UserMessage(...) could be parameterized over query or similar? Ofc it's simplest just to make it a function, but decorators are pretty

[alexchandel]

Actually it looks like there is maybe a bug in how magentic submits responses that include an assistant's tool call:

  File "C:\Users\me\Projects\OpenBB\main.py", line 85, in tool_prompt
    chat = chat.exec_function_call().submit()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\Projects\OpenBB\.venv\Lib\site-packages\magentic\chat.py", line 96, in submit
    output_message: AssistantMessage[Any] = self.model.complete(
                                            ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\Projects\OpenBB\.venv\Lib\site-packages\magentic\chat_model\openai_chat_model.py", line 459, in complete
    response: Iterator[ChatCompletionChunk] = self._client.chat.completions.create(
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\Projects\OpenBB\.venv\Lib\site-packages\openai\_utils\_utils.py", line 275, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\Projects\OpenBB\.venv\Lib\site-packages\openai\resources\chat\completions.py", line 859, in create
    return self._post(
           ^^^^^^^^^^^
  File "C:\Users\me\Projects\OpenBB\.venv\Lib\site-packages\openai\_base_client.py", line 1280, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\Projects\OpenBB\.venv\Lib\site-packages\openai\_base_client.py", line 957, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "C:\Users\me\Projects\OpenBB\.venv\Lib\site-packages\openai\_base_client.py", line 1061, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': "Invalid 'content': 'content' field must be a string or an array of objects.."}

In particular, it looks like when the assistant sent no context but just a bare tool call, magentic adds a "content": null field to the assistant's response, which the server doesn't like.

2024-12-18 21:49:18 [DEBUG] 
Received request: POST to /v1/chat/completions with body {
  "messages": [
    {
      "role": "user",
      "content": "Get the secret number, using any tool you have access to."
    },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "6b17b9368",
          "type": "function",
          "function": {
            "name": "get_secret_number",
            "arguments": "{}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "6b17b9368",
      "content": "42.69"
    }
  ],
  "model": "qwen2.5-7b-instruct",
  "parallel_tool_calls": false,
  "stream": true,
  "stream_options": {
    "include_usage": true
  },
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_secret_number",
        "parameters": {
          "properties": {},
          "type": "object"
        },
        "description": "Get the secret number."
      }
    }
  ]
}
2024-12-18 21:49:18 [ERROR] 
[Server Error] Invalid 'content': 'content' field must be a string or an array of objects.

Any ideas for how to work around this for now?

[alexchandel]

Update. workaround was to comment out line 112 of openai_chat_model.py ("content": None,), although changing it to return an empty string also works ("content": '',).

Now, not sure if this is correct, first of all because "content": null may be valid in the JSON schema (although I don't think so...), but also because the LLM might have sent some content back that magentic dropped, whereas the existing implementation of @message_to_openai_message.register(AssistantMessage) had no context hardcoded...

[jackmpcollins]

@alexchandel Thanks for debugging this! What model are you using? OpenAI docs indicate "content" is optional, so it should probably be left out. Though I'm pretty sure it had to be set to null/None for an earlier model or maybe for the pydantic model (otherwise I wouldn't have explicitly set it to None). If the "content": None can now be removed then they should be! Interestingly it is returned as null in the chatcompletions response (at least as shown in the docs).

https://platform.openai.com/docs/api-reference/chat/create

---

As a workaround in your own code to avoid editing magentic directly you could just register a new handler for the AssistantMessage. This will take precedence over the existing handler.

from magentic.chat_model.openai_chat_model import message_to_openai_message

@message_to_openai_message.register(AssistantMessage)
def _(message: AssistantMessage[Any]) -> ChatCompletionMessageParam:
	# same code as existing but without `"content": None`

[alexchandel]

Hmm if it sent us a "content": null in the tool call, could very well be a bug in LM Studio (0.3.5 build 9). Current model is qwen2.5-7b-instruct.

[alexchandel]

I opened an issue with LM Studio: lmstudio-ai/lmstudio-bug-tracker#261

So how would write a chatprompt_chain-like decorator?

[alexchandel]

#392

[jackmpcollins]

I like the loop idea, but is there any way to make it a decorator, such that UserMessage(...) could be parameterized over query or similar? Ofc it's simplest just to make it a function, but decorators are pretty

Released now in https://github.com/jackmpcollins/magentic/releases/tag/v0.37.0

from magentic import prompt_chain, UserMessage

def get_current_weather(location, unit="fahrenheit"):
    """Get the current weather in a given location"""
    return {"temperature": "72", "forecast": ["sunny", "windy"]}

@prompt_chain(
    template=[UserMessage("What's the weather like in {city}?")],
    functions=[get_current_weather],
)
def describe_weather(city: str) -> str: ...

describe_weather("Boston")
'The weather in Boston is currently 72°F with sunny and windy conditions.'