How many servers and tools it can handle?

[HardikJain02]

Pre-submission Checklist

• I have checked that this question would not be more appropriate as an issue in a specific repository
• I have searched existing discussions and documentation for answers

Question Category

• Protocol Specification
• SDK Usage
• Server Implementation
• General Implementation
• Documentation
• Other

Your Question

If I’ve got millions of tools across thousands of servers, the MAX_TOOLS limit is a looming question—where does it all start to collapse for the client? It feels like the bottleneck is more on the LLM side? I’m toying with the idea of structuring it like a RAG pipeline—embedding, vectordb, hierarchical chunking with nodes. But even then, I’m skeptical. RAG might not be robust enough to handle. It’s a gamble, and I’m not entirely convinced it’ll hold.

Two key issues:
Selecting the right tools among million of tools.
Executing them in parallel—say, 1000 at once. What’s the known limit before it collapses? All remote apis.

Any help will be appreciated.

Edit: When a query comes in, does it first search for the relevant servers and then look for the tools within them, or does it go straight to searching for the tools directly?

[sufiyan-ahmed]

Not an answer for your question but I have been exploring MCPs myself and I came across something called OctoTools last week.
https://octotools.github.io/#introduction
They propose an automated algorithm to optimize the toolset configuration for each task using a greedy search strategy. I would suggest you to give it a read.

[omrilahav]

I think this is a broader question beyond just MCP; it applies to any agentic system. LLMs (currently) can easily become confused when handling excessively large and broad contexts. Therefore, it is recommended to break agents down into mission-specific units and manage them within a hierarchy. This way, each agent can focus on specific tasks using dedicated tools, improving its effectiveness. From this perspective, providing LLMs with millions of tools can make it difficult for them to perform their tasks correctly.

[Mehdi-Bl]

The more tools you add, the more information you add to the context and overload the model with informations.
There is reported issues when reaching 80-90. Using currently 30-40 no issues but trying to separate the MCP's by groups to disable them when not needed.

Don't forget to use an MCP you need to add it to the context and an effective tool have a short description explaining the arguments otherwise Sonnet for example get confused how to use the tool or pass wrong parameter types.