How to properly handle a lot of data while using tools?

[robiningelbrecht]

I implemented an agent using tools extensively. My specific use case in the fitness industry where users can ask questions about their workout history and ask for tips. A virtual workout assistant if you will.

every user has it's own history of workouts and the agent uses those to properly formulate answers to questions like:

• In the past 3 months, what has been my most intensive workout and how many times a week should I keep this intensity going?
• What's my average bike mileage per ride over the last year?
• ....

I have a 2 main tools implemented to achieve this:

• GetAllActivityIdsTool which fetches all available activity ids
• GetActivityDetailsTool which returns detailed information of an activity by a given activity id

The issue is that, depending on how much activities a user has, sometimes the LLM throws a "400 Bad Request" because the API call exceeded the max number of tokens allowed.

I am wondering how I could approach these questions that require knowledge of a big part of the user's activities, without having the problem described above?

Anyone had a similar issue?

[ilvalerione]

Exceed the max number of tokens it should be avoided by the chat history component if it refers to the context window.

Is this your case? What model are you using?

Remember that all chat history implementation allows you to specify the size of the context window in the constructor parameters.