DB Connection pool is getting exhausted

[JonahSussman]

Despite #833 , a race condition still might exist in the solution server. Below is a description provided by a user:

Today, we conducted a load test on our Solution Server with six concurrent users. Two of the six users encountered the following connection error.

Here is the error message

INFO:     10.0.1.197:46444 - "POST / HTTP/1.1" 200 OK
INFO:     10.0.1.197:42678 - "POST / HTTP/1.1" 200 OK
INFO:     10.0.1.197:60462 - "POST / HTTP/1.1" 200 OK
INFO:     10.0.1.197:58285 - "POST / HTTP/1.1" 200 OK
INFO:     10.0.1.197:39838 - "POST / HTTP/1.1" 200 OK
INFO:     10.0.1.197:48651 - "POST / HTTP/1.1" 200 OK
INFO:     10.0.1.29:57621 - "POST / HTTP/1.1" 200 OK
INFO:     10.0.1.29:59981 - "POST / HTTP/1.1" 200 OK
INFO:     10.0.1.197:56534 - "POST / HTTP/1.1" 200 OK
INFO:     10.0.1.197:55045 - "POST / HTTP/1.1" 200 OK
Task exception was never retrievedfuture: <Task finished name='Task-81323' coro=<generate_hint_v3() done, defined at /home/ec2-user/kai/kai_mcp_solution_server/src/kai_mcp_solution_server/server.py:541> exception=APIConnectionError('Connection error.')>Traceback (most recent call last):  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions    yield  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpx/_transports/default.py", line 394, in handle_async_request    resp = await self._pool.handle_async_request(req)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request    raise exc from None  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request    response = await connection.handle_async_request(               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request    raise exc  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpcore/_async/connection.py", line 78, in handle_async_request    stream = await self._connect(request)             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpcore/_async/connection.py", line 156, in _connect    stream = await stream.start_tls(**kwargs)             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpcore/_backends/anyio.py", line 67, in start_tls    with map_exceptions(exc_map):         ^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib64/python3.12/contextlib.py", line 158, in __exit__    self.gen.throw(value)  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions    raise to_exc(exc) from exchttpcore.ConnectErrorThe above exception was the direct cause of the following exception:Traceback (most recent call last):  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/openai/_base_client.py", line 1529, in request    response = await self._client.send(               ^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpx/_client.py", line 1629, in send    response = await self._send_handling_auth(               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpx/_client.py", line 1657, in _send_handling_auth    response = await self._send_handling_redirects(               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects    response = await self._send_single_request(request)               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpx/_client.py", line 1730, in _send_single_request    response = await transport.handle_async_request(request)               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpx/_transports/default.py", line 393, in handle_async_request    with map_httpcore_exceptions():         ^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib64/python3.12/contextlib.py", line 158, in __exit__    self.gen.throw(value)  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions    raise mapped_exc(message) from exchttpx.ConnectErrorThe above exception was the direct cause of the following exception:Traceback (most recent call last):  File "/home/ec2-user/kai/kai_mcp_solution_server/src/kai_mcp_solution_server/server.py", line 615, in generate_hint_v3    response = await kai_ctx.model.ainvoke(prompt)               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 405, in ainvoke    llm_result = await self.agenerate_prompt(                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/ec2-user/kai/kai_mcp_solution_server/.venv/lib64/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 1017, in agenerate_prompt    return