Deprecate LlamaStackAsLibraryClient?

[ehhuang]

The history predates my time here but I think the reason for introducing LlamaStackAsLibraryClient was to have a good DevX in notebook environment. Stack has since evolved quite a bit and now might be a good time to revisit the necessity of this.

1. There's been a refocus to the story of a 'hosted stack' with developers using either Stack or OAI SDK now that we have OAI-compat endpoints.
2. LlamaStackAsLibraryClient no longer fits this narrative as there's no production scenario that makes use of it.
3. Using the library client may even be detrimental as it hides potential cases where DevX is suboptimal with stack server.
4. Yet we still run tests with the library client and have to maintain it in future changes, slowing down development.

Question for the group: do we still have a valid case for keeping LlamaStackAsLibraryClient? If we do want something like this, we could create a helper function that spins up the stack server in the background.

[ashwinb]

I think the main argument against it is that it just adds a significant amount of complexity to the basic infrastructure of the stack and the surrounding test suite. The use case (being able to "embed the stack as a library") is nearly non-existent right now.

[mattf]

there's something to be said for a low barrier to entry. for some, spinning up containers, adjusting config files, and managing process trees is second nature. instantiating a class is second nature for everyone.

^^ is a reasonable way to look at it pre-nov 2022. however, today the bar is how effectively can an llm answer "how do i get started with llama stack?"

sep 2025, it doesn't look good -

with some help, it still doesn't look good -

imho, LlamaStackAsLibraryClient can assist here by keeping the skill scope narrow. however, AsLibraryClient doesn't need the full capabilities of stack to do that. cast as a getting started tool, it doesn't make sense to support providers that rely on the user to setup a second system, aka llm to generate instruction for multi-process setup. inline providers make sense. supporting some remote providers makes sense: openai / lllama-api / together yes, but not vllm / ollama / nim / tgi; chroma & faiss yes, but not pgvector & weaviate.

[tisnik]

Several teams in Red Hat leveraged Llama Stack as a library, mainly because it is much easier to deploy + run it that way in different environments. Running Llama Stack in separate process is usually fine, but there are more possible problems that have to be checked and managed (network issues, different versions, whole deployment process etc.). If I may I'd vote to keep it there.

There's also one non-technical aspect: we were able to move some projects from using langchain/langraph etc. into Llama Stack, because "just" the inference call is different (sorry, simplification), everything else in the project can remain as is. This encourages the teams to try Llama Stack in the 1st place.

[eranco74]

I'd like to add my voice in strong support of keeping LlamaStackAsLibraryClient.

For our use case, the library client isn't just a tool for notebooks or local development; it's a deliberate architectural choice for our containerized deployments that significantly reduces operational complexity.
Using the library client simplifies our workflow at every stage. At build time, it's treated as a simple Python dependency in our pyproject.toml, removing the need to maintain a separate container image. This carries through to deployment, where it eliminates the overhead of managing a separate pod, service, and network configuration, keeping our architecture lean and simple.

While I understand the goal of focusing on a "hosted stack", this pattern of embedding the stack as a library directly translates to minimal operational overhead. It lowers the barrier not just for initial experimentation, as @mattf pointed out, but also for production deployment.
Deprecating it would remove a valuable architectural pattern that simplifies production deployments. I strongly vote to keep it.

[ashwinb]

Thanks for the feedback here. It sounds like we should keep it and strive to make the underlying infrastructure of managing the library client easier.