Skip to content

fix(billing-cost-management-mcp-server): point a filtered forecast's no-data error at the filter - #4940

Merged
finklen-amazon merged 6 commits into
awslabs:mainfrom
finklen-amazon:fix/bcm-forecast-no-match-hint
Nov 1, 2026
Merged

finklen-amazon merged 6 commits into
awslabs:mainfrom
finklen-amazon:fix/bcm-forecast-no-match-hint

Conversation

@finklen-amazon

@finklen-amazon finklen-amazon commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Fixes

Summary

Cost Explorer returns DataUnavailableException: Insufficient amount of historical data. in two cases: when there is too little history to forecast, and when the forecast filter as a whole matches no past costs. That covers a misspelled value, the wrong case, a value from another dimension, or a partial value used with STARTS_WITH, CONTAINS or CASE_INSENSITIVE. The service accepts those MatchOptions but does not apply them; the DimensionValues reference says MatchOptions applies only to cost category and anomaly subscription actions.

The shared DataUnavailableException hint already names an unmatched filter as one possible cause, but it maps to do_not_retry. The forecast notes described only the history case. So a caller with a mistyped filter was told not to retry and reported that no forecast was possible, while the same account forecasts fine with a correct filter.

Changes

  • getCostForecast / getUsageForecast: when a filter was sent and the service returns DataUnavailableException, the response now has retry_strategy: modify_request and a forecast-specific hint: do not resend the same request; check the values with getDimensionValues (getTagsOrValues for tags, getCostCategories for cost categories) and retry with corrected values; if the values exist and can occur together, report that there is too little history. Without a filter, the shared do_not_retry guidance is unchanged.
  • Forecast notes: filter values must match exactly, including case; those MatchOptions are accepted but not applied; a filter that matches no past costs returns the same error as too little history.

No tool, parameter or schema is renamed or removed.

User experience

Before: {"operation": "getCostForecast", "metric": "UNBLENDED_COST", "granularity": "MONTHLY", "filter": {"Dimensions": {"Key": "SERVICE", "Values": ["Amazon Elastic Compute Cloud - Compte"]}}} returned DataUnavailableException "Insufficient amount of historical data." with retry_strategy: do_not_retry, and an agent reported that the account cannot be forecast.

After: the same call returns the same error_type and message, with retry_strategy: modify_request and a hint to check the filter values first. Unfiltered calls are unchanged.

Testing

  • Unit: new tests for the hint, the notes and the guards: an unfiltered call keeps do_not_retry, an empty filter ({}, '{}') is a validation_error without the hint, and other errors with a filter are unchanged. 246 tests in the two touched test files pass; ruff and pyright are clean.
  • Live (read-only, against a real account, through the MCP server over stdio):
    • On the change, all claims passed: the notes as served; a valid SERVICE filter forecasts successfully (control); a one-letter typo, a STARTS_WITH partial value and a misspelled USAGE_TYPE_GROUP each return DataUnavailableException with modify_request and the new hint.
    • On the base, the same harness failed 4 of 5 (old notes; do_not_retry with the generic hint). Only the valid-filter control passed.
    • Independently re-checked with direct API calls: CONTAINS and CASE_INSENSITIVE are ignored (same error); a value from another dimension, a misspelled tag value and the wrong case match nothing; Not of a misspelled value and Or of a valid and a misspelled value both forecast normally.

Merge conflict

#4919 merged as a squash commit, so this branch, which was stacked on #4919's branch, conflicted with main in cost_explorer_operations.py and cost_explorer_tools.py: the forecast lines #4919 changed showed up as edits to the same lines this PR extends. I rebased this PR's single commit onto main, dropping #4919's commits, which are now upstream. The commit applied with no manual edits. #4919's period_note text and helper are kept as merged, and this PR's changed lines are identical to the reviewed revision. Unit tests (3252), ruff and pyright pass on the rebased branch.

After this PR was approved, #4936 merged and added its billing_view_arn tests at the end of tests/tools/test_cost_explorer_tools.py, where this PR adds its forecast-notes test, so the branch conflicted with main again. I rebased onto main and kept both: #4936's tests unchanged, followed by this PR's test. No source or tool text conflicted, and this PR's changed lines are identical to the approved revision. Unit tests (3442), ruff and pyright pass on the rebased branch.

Main then moved again and this branch conflicted a third time. This time I merged main into the branch (a merge commit, no force-push):

The tool code of this PR is unchanged. The live check passes 5 of 5 on the merge, and the follow-up commits change only that empty-filter test. The touched tests, ruff and pyright pass. One test in test_recommendation_details_tools.py fails on main too, without this PR.

Checklist

If your change doesn't seem to apply, please leave them unchecked.

  • I have reviewed the contributing guidelines
  • I have performed a self-review of this change
  • Changes have been tested
  • Changes are documented

Is this a breaking change? (Y/N) N

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.

@codecov

codecov Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.79%. Comparing base (c6951b9) to head (7b517b7).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4940   +/-   ##
=======================================
  Coverage   93.79%   93.79%           
=======================================
  Files        1080     1080           
  Lines       94586    94597   +11     
  Branches    15402    15403    +1     
=======================================
+ Hits        88715    88726   +11     
  Misses       3498     3498           
  Partials     2373     2373           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@finklen-amazon
finklen-amazon force-pushed the fix/bcm-forecast-no-match-hint branch from c24cb72 to a9cf4f0 Compare October 30, 2026 00:02
anniewn
anniewn previously approved these changes Oct 30, 2026
…no-data error at the filter

Cost Explorer returns DataUnavailableException "Insufficient amount of
historical data." both when there is too little history and when a
forecast filter as a whole matches no past costs: a misspelled value, the
wrong case, a value from another dimension, or a partial value with
STARTS_WITH, CONTAINS or CASE_INSENSITIVE, which the service accepts but
does not apply (the API reference says MatchOptions applies only to cost
category and anomaly subscription actions). The shared DataUnavailable
hint names an unmatched filter as one possible cause, but maps to
do_not_retry, and the forecast notes described only the history case, so
a caller with a mistyped filter concluded that no forecast was possible.

- getCostForecast and getUsageForecast: when a filter was sent and the
  service returns DataUnavailableException, set retry_strategy to
  modify_request with a hint not to resend the same request, to check the
  values with getDimensionValues, getTagsOrValues or getCostCategories,
  and to report too little history only if the values exist and can
  occur together. Without a filter the shared do_not_retry guidance is
  unchanged.
- Forecast notes: filter values match exactly, including case; those
  MatchOptions are accepted but not applied; an unmatched filter returns
  the same error as too little history.
Resolve conflicts with main:
- cost_explorer_tools.py getUsageForecast: take main's Example 3 (one
  Region filter instead of group_by, which forecasts now reject) and keep
  this branch's added Notes sentence on unmatched filters.
- test_cost_explorer_tools.py: keep both sides' appended tests.
- test_cost_explorer_operations.py: an empty filter is now a
  validation_error on main, so the unfiltered guard uses None only, and a
  new test checks that an empty filter never gets the no-match hint.
…ck in the forecast hint

Use 'filters is not None', as main now does when deciding to send Filter,
and check the empty-filter test through add_retry_guidance.
… on a sent filter

Revert the previous commit's 'filters is not None' check to 'if filters'.
parse_filter already rejects empty filters, so both forms send the hint
for the same requests; the truthy form is the one already reviewed.
@finklen-amazon
finklen-amazon added this pull request to the merge queue Oct 30, 2026
Pick up main's DynamoDB rec-details test fix so CI re-runs. No conflicts.
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 30, 2026
@finklen-amazon
finklen-amazon added this pull request to the merge queue Nov 1, 2026
Merged via the queue into awslabs:main with commit dc8221e Nov 1, 2026
138 checks passed
@finklen-amazon
finklen-amazon deleted the fix/bcm-forecast-no-match-hint branch November 1, 2026 23:20
@github-project-automation github-project-automation Bot moved this from To triage to Done in awslabs/mcp Project Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants