added a warning when n_trials budget is depleted but ask still return trials

[sagar1sharma]

…s trials

Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: https://github.com/automl/SMAC3/blob/main/CONTRIBUTING.md

Reference Issues/PRs

Example: Fixes #1234. See also #3456.
Please use keywords (e.g., Fixes) to create link to the issues or pull requests you resolved, so that they will automatically be closed when your pull request is merged. See https://github.com/blog/1506-closing-issues-via-pull-requests

Description, Motivation and Context

Provide a short description of your changes.

Changes

Explain your changes in detail here.

Type Of Changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

What is Missing?

Please provide any further information that is needed to close this PR.
This could be another PR that is blocking this PR, or any further changes
that need to be addressed in the future.

Checklist

Go over all the following points, and put an x in all the boxes that apply.
Should there be a good argument to differ, please provide a reasoned explanation.

• My change is based on the latest stage of the develop branch.
• My change required a change of the documentation, which has been done.
• I checked that the documentation can be build, visualizes everything as expected, and does not contain any warnings.
• I have added/adapted tests to cover my changes.
• The tests can be executed successfully.
• I have added a description of the changes to CHANGELOG.rst.

Any other comments?

We appreciate your effort, but please be aware that we are not working full-time on this project. We welcome any contribution and value your effort, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process.

Thanks for contributing!

[Copilot]

Pull request overview

This pull request adds a warning mechanism to alert users when ask() is called after the optimization budget has been exhausted. The change maintains backward compatibility by continuing to return trials while warning users about this behavior, and updates documentation to explain the ask-and-tell interface behavior.

Changes:

• Added a warning flag and logic to warn users once when calling ask() after budget exhaustion
• Updated documentation to clarify ask-and-tell behavior after budget depletion
• Properly reset the warning flag in the reset() method

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
smac/main/smbo.py	Added _warned_on_ask_after_budget_exhausted flag, warning logic in ask() method, and reset logic
docs/advanced_usage/5_ask_and_tell.md	Added warning section explaining ask-and-tell behavior after budget exhaustion

Comments suppressed due to low confidence (4)

smac/main/smbo.py:164

• The warning message doesn't specify which type of budget was exhausted. Since budget_exhausted can be True due to walltime, cputime, or trial count limits, the message should clarify which specific budget(s) caused the exhaustion. Consider adding information about remaining_walltime and remaining_cputime in addition to remaining_trials, similar to the logging in the optimize() method at lines 350-354.

            logger.warning(
                "ask() was called after the scenario budget was exhausted."
                f"remaining trials: {self.remaining_trials}). "
                "SMAC will continue returning trials for backward compatibility."
            )

smac/main/smbo.py:165

• The new warning functionality when ask() is called after budget exhaustion lacks test coverage. While test_ask_and_tell_after_optimization in tests/test_ask_and_tell/test_ask_and_tell_intensifier.py exercises this code path, it doesn't verify that the warning is emitted or that the warning flag is set correctly. Consider adding a test that captures and verifies the logger.warning call.

        if self.budget_exhausted and not self._warned_on_ask_after_budget_exhausted:
            logger.warning(
                "ask() was called after the scenario budget was exhausted."
                f"remaining trials: {self.remaining_trials}). "
                "SMAC will continue returning trials for backward compatibility."
            )
            self._warned_on_ask_after_budget_exhausted = True

docs/advanced_usage/5_ask_and_tell.md:21

• The documentation states "SMAC does not hard-stop ask() when n_trials is depleted" but this is misleading. The budget_exhausted check includes walltime, cputime, and n_trials budgets, not just n_trials. The documentation should clarify that ask() continues to return trials after any budget type is exhausted, not just when n_trials is depleted.

    In pure ask-and-tell usage, SMAC does not hard-stop `ask()` when `n_trials` is depleted.
    This means `ask()` can still return additional trials after budget exhaustion.

smac/main/smbo.py:82

• Trailing whitespace detected at the end of line 82. This should be removed to maintain code cleanliness and avoid potential issues with version control and linting tools.

        self._warned_on_ask_after_budget_exhausted = False  

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.