Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/advanced_usage/5_ask_and_tell.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,13 @@ and report the results of the trial.
different budgets, they, obviously, can not be considered. However, all user-provided configurations will flow
into the intensification process.

!!! warning

In pure ask-and-tell usage, SMAC does not hard-stop `ask()` when `n_trials` is depleted. (This is true for any kind of budget exhaustion and not only `n_trials` eg. walltime, cputime. warning logs all the budget variables in the case of exhaustion).
This means `ask()` can still return additional trials after budget exhaustion.
SMAC now emits a runtime warning in this case and keeps this behaviour for backward compatibility.
If you want strict stopping in your loop, stop calling `ask()` when the optimizer reports no remaining budget (for example, `smac.optimizer.budget_exhausted` or `smac.optimizer.remaining_trials <= 0`)

Notice: if you are exclusively using the ask-and-tell interface and do not use `smac.optimize()`, then smac no longer
is responsible for the evaluation of the trials and therefore the Facade no longer will require a specified `target_algorithm` argument.

Expand Down
12 changes: 12 additions & 0 deletions smac/main/smbo.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ def __init__(
self._finished = False
self._stop = False # Gracefully stop SMAC
self._callbacks: list[Callback] = []
self._warned_on_ask_after_budget_exhausted = False

# Stats variables
self._start_time: float | None = None
Expand Down Expand Up @@ -155,6 +156,16 @@ def ask(self) -> TrialInfo:
"""
logger.debug("Calling ask...")

if self.budget_exhausted and not self._warned_on_ask_after_budget_exhausted:
logger.warning(
"ask() was called after the scenario budget was exhausted."
f"(remaining wallclock time: {self.remaining_walltime}, "
f"remaining cpu time: {self.remaining_cputime}, "
f"remaining trials: {self.remaining_trials}). "
"SMAC will continue returning trials for backward compatibility."
)
self._warned_on_ask_after_budget_exhausted = True

for callback in self._callbacks:
callback.on_ask_start(self)

Expand Down Expand Up @@ -371,6 +382,7 @@ def reset(self) -> None:
self._used_target_function_walltime = 0
self._used_target_function_cputime = 0
self._finished = False
self._warned_on_ask_after_budget_exhausted = False

# We also reset runhistory and intensifier here
self._runhistory.reset()
Expand Down
Loading