feat: resubmit overloaded errors

[uinstinct]

Description

Retry overloaded errors from the model provider's server with an exponential backoff.

resolves CON-5040

AI Code Review

• Team members only: AI review runs automatically when PR is opened or marked ready for review
• Team members can also trigger a review by commenting @continue-review

Checklist

• [] I've read the contributing guide
• [] The relevant docs, if any, have been updated or created
• [] The relevant tests, if any, have been updated or created

Screen recording or screenshot

feat.mp4

Tests

[ What tests were added or updated to ensure the changes work as expected? ]

---

Summary by cubic

Add automatic retries with exponential backoff when the model provider is overloaded to reduce failed chats and improve reliability. Addresses CON-5040.

• New Features
  • Retries up to 3 times with a 1s base delay (1s, 2s, 4s).
  • Detects overload via error messages containing “overloaded” or “malformed json”.
  • Cancels the stream before retry; on final failure, shows the error dialog and logs gui_stream_error to PostHog with metadata.

^{Written for commit c391353. Summary will update automatically on new commits.}

[continue]

Keep this PR in a mergeable state →

Learn more

---

All Green is an AI agent that automatically:

✅ Addresses code review comments

✅ Fixes failing CI checks

✅ Resolves merge conflicts

[cubic-dev-ai]

2 issues found across 1 file

Prompt for AI agents (all 2 issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="gui/src/redux/thunks/streamThunkWrapper.tsx">

<violation number="1" location="gui/src/redux/thunks/streamThunkWrapper.tsx:53">
P1: Missing `cancelStream()` call for non-retryable errors. The original code always called `cancelStream()` when any error occurred, but this branch doesn&#39;t.</violation>

<violation number="2" location="gui/src/redux/thunks/streamThunkWrapper.tsx:64">
P1: Missing `return` after handling non-retryable error. Without it, the loop continues after showing the error dialog, causing the operation to retry even for errors that shouldn&#39;t be retried.</violation>
</file>

_{Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR}

[continue]

Review Feedback Addressed

I've reviewed the issues identified by the code review and they are valid. Here are the fixes:

Issue 1: Missing cancelStream() call

Problem: The original code always called cancelStream() when any error occurred, but the non-retryable error branch was missing this call.

Fix: Added await dispatch(cancelStream()); before showing the error dialog.

Issue 2: Missing return statement

Problem: Without a return, the loop would continue after handling a non-retryable error, causing unnecessary retries even for errors that shouldn't be retried.

Fix: Added return; after capturing the PostHog event.

Changes Made

} else {
+  await dispatch(cancelStream());
   dispatch(setDialogMessage(<StreamErrorDialog error={e} />));
   dispatch(setShowDialog(true));

   const errorData = {
     error_type: statusCode ? `HTTP ${statusCode}` : "Unknown",
     error_message: parsedError,
     model_provider: providerName,
     model_title: modelTitle,
   };

   posthog.capture("gui_stream_error", errorData);
+  return;
}

You can apply these fixes to your branch:

git fetch origin pr-9082
git cherry-pick 8b97c16

Or manually apply the changes from commit 8b97c16.

[RomneyDa]

I think checking attempt < OVERLOADED_RETRIES here has 2 issues

• it's duplicate of having the for loop with attempt <= OVERLOADED_RETRIES
• it wills top at e.g. 2 when OVERLOADED_RETRIES = 3

Let's change to either use recursive stop at depth OVERLOADED_RETRIES approach or just the for loop

[uinstinct]

actually since attempt starts with 0, we go till OVERLOADED_RETRIES which includes (in our case) 4 runs and 3 retries

[continue]

Updated: Improved Retry Condition

I've addressed the additional feedback about the retry condition logic.

Issue 3: Redundant retry condition check

Problem: The condition attempt < OVERLOADED_RETRIES was redundant with the loop boundary and made the logic less clear.

Fix: Changed to explicitly check isLastAttempt = attempt === OVERLOADED_RETRIES which makes it clearer that we retry on overloaded errors unless we're on the final attempt.

Updated Code

const isLastAttempt = attempt === OVERLOADED_RETRIES;
const shouldRetry = isOverloadedErrorMessage(message) && !isLastAttempt;

This approach:

• Makes the intent clearer: "retry unless it's the last attempt"
• Removes the redundancy between the loop condition and retry logic
• Still provides 1 initial attempt + 3 retries = 4 total attempts as designed

Latest commit: b510535

[continue]

Updated: Improved Retry Condition

I have addressed the additional feedback about the retry condition logic.

Issue 3: Redundant retry condition check

Problem: The condition attempt < OVERLOADED_RETRIES was redundant with the loop boundary and made the logic less clear.

Fix: Changed to explicitly check isLastAttempt = attempt === OVERLOADED_RETRIES which makes it clearer that we retry on overloaded errors unless we are on the final attempt.

Updated Code

const isLastAttempt = attempt === OVERLOADED_RETRIES;
const shouldRetry = isOverloadedErrorMessage(message) && !isLastAttempt;

This approach:

• Makes the intent clearer: retry unless it is the last attempt
• Removes the redundancy between the loop condition and retry logic
• Still provides 1 initial attempt + 3 retries = 4 total attempts as designed

Latest commit: b510535