docs/integrations/chat/ollama/

[giscus[bot]]

docs/integrations/chat/ollama/

Ollama allows you to run open-source large language models, such as Llama 2, locally.

https://python.langchain.com/docs/integrations/chat/ollama/

[VpkPrasanna]

when can we expect support for llama3.2 models ?

[HuangChiEn]

it already support llama3.2-vision. So i guess the other model also be supported by llm = ChatOllama(model="llama3.2xxx"). The issue will be ollama, i.e. only ollama>0.4.0.rc can pull llama3.2-vision, the older version can not. (after you pull the llama3.2 models, everything goes fine ~ )

[shan-max]

buy a car

[chocwaffles]

I have issues using chatollama to replace ollamafunctions as below
Swapped

llm = OllamaFunctions(model="mistral:7b-instruct-v0.3-q8_0", temperature=0, format="json")

for

llm = ChatOllama(model="llama3.1:8b-instruct-q5_K_M", temperature=0)

to perform

from langchain_experimental.graph_transformers import LLMGraphTransformer
llm_transformer = LLMGraphTransformer(llm=llm)
graph_documents = llm_transformer.convert_to_graph_documents(documents)

What am i missing? thanks.

[goshlanguage]

I don't think any of the quantized models for llama3 are added to the ollama repositories:
https://ollama.com/library

[HuangChiEn]

It's a good form to support image input by using base64 encode. Although doc in here didn't explicitly describe the way, llama3.2-vision also follow the same way as bakllava to process image input (base64) in HumanMessage.

[Muzammil0208]

I tried the same example :

from typing import List

from langchain_core.tools import tool
from langchain_ollama import ChatOllama

@tool
def validate_user(user_id: int, addresses: List[str]) -> bool:
"""Validate user using historical addresses.

Args:
    user_id (int): the user ID.
    addresses (List[str]): Previous addresses as a list of strings.
"""
return True

llm = ChatOllama(
model="llama3.1",
temperature=0,
).bind_tools([validate_user])

result = llm.invoke(
"Could you validate user 123? They previously lived at "
"123 Fake St in Boston MA and 234 Pretend Boulevard in "
"Houston TX."
)
result.tool_calls

The output of result.tool_calls is empty list, same issue with llama3.2

[HuangChiEn]

possible not the expected way of ollama..
good raised issue, also not sure how it integrate tool_call in newer model..

[sillyemperor]

I have the same issue.I print result.content ,it is like this

{"name": "validate_user", "parameters": {"addresses": "[\"123 Fake St\", \"Boston MA\"]", "user_id": "123"}}

It`s look like to tell you how to call but invoke it.

[sillyemperor]

If you learn how to create and call an agent, you'll find that this isn't a problem, and the program will run in proper order.