feat(core): enhance loop detection with thought and action stagnation checks

[euxaristia]

This PR adds new loop detection patterns to catch more subtle looping behaviors that were previously undetected, replacing the functionality intended for the closed PR QwenLM#3176. This includes repetitive thoughts detection, read file loop detection, and action stagnation detection.

[gemini-code-assist]

Code Review

This pull request introduces several enhancements to loop detection and error handling, including new detection patterns for repetitive thoughts, file read loops, and action stagnation. It also improves PTY error handling and updates build scripts. However, several issues were identified in the implementation of the validation retry logic and loop detection services. Specifically, there are logic errors in how retry counts are reset and tracked due to key format mismatches, hardcoded thresholds that ignore defined constants, and redundant class members that are either unused or unnecessarily complex. Addressing these points will improve the reliability and maintainability of the new loop detection features.

[gemini-code-assist]

The logic to reset validation retry counts on successful execution is incorrect. The key used is currentCall.request.name, but the keys in validationRetryCounts are in the format toolName:errorMessage. This delete operation will have no effect.

To correctly clear all validation errors for a successfully executed tool, you should iterate over the map and remove all entries that start with the tool's name.

          for (const key of this.validationRetryCounts.keys()) {
            if (key.startsWith(`${currentCall.request.name}:`)) {
              this.validationRetryCounts.delete(key);
            }
          }

[gemini-code-assist]

The logic to detect if a batch of tool calls continues a validation loop is flawed. prevTool is a string in the format toolName:errorMessage, but it's being compared directly with r.name, which is just the tool name. This comparison will always be false. Consequently, validationRetryCounts will be cleared incorrectly as soon as a tool call request arrives that doesn't exactly match the full error key string of a previous failure.

        const prevFailingToolNames = new Set(
          [...this.validationRetryCounts.keys()].map((key) => key.split(':')[0]),
        );
        const hasPrevFailingTool = requestsToProcess.some((r) =>
          prevFailingToolNames.has(r.name),
        );
        if (!hasPrevFailingTool) {
          this.validationRetryCounts.clear();
        }

[gemini-code-assist]

There are a couple of maintainability issues here:

1. The threshold 3 is hardcoded. You've defined VALIDATION_RETRY_LOOP_THRESHOLD at the top of the file, which should be used here instead.
2. A local stopDirective constant is defined with a message that is slightly different from the global RETRY_LOOP_STOP_DIRECTIVE. This is redundant. You should use the global constant for consistency and easier maintenance.

          if (count >= VALIDATION_RETRY_LOOP_THRESHOLD) {
            // Inject strong stop directive after 3 consecutive failures with same error
            finalError = new Error(`${baseError.message}${RETRY_LOOP_STOP_DIRECTIVE}`);
          }

[gemini-code-assist]

The class member this.fileReadCount is redundant. It's incremented on every read-like tool call (line 404) but it's never used for loop detection; checkReadFileLoop correctly calculates the count within a sliding window of recent tool calls. This unused member can cause confusion and should be removed from the class, from the trackToolCall method (line 404), and from the reset method (line 475).

[gemini-code-assist]

The logic in checkRepetitiveThoughts can be simplified. The this.repetitiveThoughtCount variable is incremented and then immediately checked if it's >= 1, which is always true. The variable is not necessary since the function returns immediately upon detecting a loop.

You can remove this.repetitiveThoughtCount from the class (line 61) and reset() method (line 474), and simplify this logic.

    if (repetitiveCount >= THOUGHT_REPEAT_THRESHOLD) {
      logLoopDetected(
        this.config,
        new LoopDetectedEvent(
          LoopType.REPETITIVE_THOUGHTS,
          this.promptId,
        ),
      );
      return true;
    }