Skip to content

fix: return best attempt on heal loop exhaustion (v1.14.4)#192

Merged
devonakelley merged 1 commit into
mainfrom
fix/heal-loop-return-best-attempt
May 22, 2026
Merged

fix: return best attempt on heal loop exhaustion (v1.14.4)#192
devonakelley merged 1 commit into
mainfrom
fix/heal-loop-return-best-attempt

Conversation

@devonakelley

Copy link
Copy Markdown
Contributor

Problem

When the heal loop exhausts all paths, it raises RuntimeError, causing benchmarks to record a hard failure with model="error". A partial or low-quality response is better than an exception for most callers.

Fix

In kalibr/router.py, when heal_result['success'] is False but heal_result['response'] is not None, return the last attempted response instead of raising.

New response attributes set on exhausted return:

  • kalibr_healed = True — the heal loop was triggered
  • kalibr_heal_exhausted = True — all paths were tried without passing Gate 1/2
  • kalibr_heal_count — number of repair attempts made
  • kalibr_models_tried — list of models attempted
  • kalibr_model_used — final model tried

RuntimeError is still raised if last_response is None (no bytes were ever received — e.g. pure network failure).

Testing

Callers can detect exhaustion via:

response = router.completion(messages=[...], healing=True)
if getattr(response, 'kalibr_heal_exhausted', False):
    # degraded path — log, alert, or fallback
    ...

Checklist

  • Fix applied in kalibr/router.py
  • Version bumped: 1.14.31.14.4 in pyproject.toml
  • CHANGELOG.md updated
  • Do NOT merge without review
  • Do NOT publish to PyPI

When all heal-loop paths are exhausted, return the last attempted response
instead of raising RuntimeError. Sets kalibr_heal_exhausted=True on the
response so callers can detect the exhausted state. Only raises if no
response was ever received (e.g. network failure before any bytes arrived).

Prevents benchmarks from counting partial/low-quality results as hard
failures with model='error'.

Closes: heal-loop-exhaustion-raises
@devonakelley devonakelley merged commit 8a93bd6 into main May 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant