Add the support for XGBoost 3.0.0 #216
Description
Expected behavior
XGBoost 3.0.0 was released two weeks ago, causing the workflows to fail as follows.
- https://pypi.org/project/xgboost/#history
- https://github.com/optuna/optuna-integration/actions/runs/13981048377/job/39146195909
I will open a PR to add the version constraint, xgboost<=3.0.0, as a hotfix. This issue suggests fixing the above error and remove the version constraint.
Environment
- Optuna version: main
- Optuna Integration version: main
- Python version: 3.9
- OS: ubuntu-latest
- (Optional) Other libraries and their versions: xgboost 3.0.0
Error messages, stack traces, or logs
_______________________ test_xgboost_pruning_callback_cv _______________________
def test_xgboost_pruning_callback_cv() -> None:
def objective(trial: optuna.trial.Trial) -> float:
dtrain = xgb.DMatrix(np.ones((2, 1)), label=[1.0, 1.0])
params = {
"objective": "binary:logistic",
}
pruning_callback = XGBoostPruningCallback(trial, "test-logloss")
xgb.cv(params, dtrain, callbacks=[pruning_callback], nfold=2)
return 1.0
study = optuna.create_study(pruner=DeterministicPruner(True))
> study.optimize(objective, n_trials=1)
tests/xgboost/test_xgboost.py:73:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/optuna/study/study.py:475: in optimize
_optimize(
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/optuna/study/_optimize.py:63: in _optimize
_optimize_sequential(
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/optuna/study/_optimize.py:160: in _optimize_sequential
frozen_trial = _run_trial(study, func, catch)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/optuna/study/_optimize.py:248: in _run_trial
raise func_err
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/optuna/study/_optimize.py:197: in _run_trial
value_or_values = func(trial)
tests/xgboost/test_xgboost.py:69: in objective
xgb.cv(params, dtrain, callbacks=[pruning_callback], nfold=2)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/core.py:729: in inner_f
return func(**kwargs)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/training.py:581: in cv
booster.update(i, obj)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/training.py:229: in update
fold.update(iteration, obj)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/training.py:215: in update
self.bst.update(self.dtrain, iteration, fobj)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/core.py:2246: in update
_check_call(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ret = -1
def _check_call(ret: int) -> None:
"""Check the return value of C API call
This function will raise exception when error occurs.
Wrap every API call with this function
Parameters
----------
ret :
return value from API calls
"""
if ret != 0:
> raise XGBoostError(py_str(_LIB.XGBGetLastError()))
E xgboost.core.XGBoostError: [23:18:15] /workspace/src/objective/./regression_loss.h:69: Check failed: base_score > 0.0f && base_score < 1.0f: base_score must be in (0,1) for logistic loss, got: 1
E Stack trace:
E [bt] (0) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x2a6acc) [0x7fb07cea6acc]
E [bt] (1) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0xed8329) [0x7fb07dad8329]
E [bt] (2) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x681223) [0x7fb07d281223]
E [bt] (3) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x6815ec) [0x7fb07d2815ec]
E [bt] (4) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x68bb3b) [0x7fb07d28bb3b]
E [bt] (5) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(XGBoosterUpdateOneIter+0x77) [0x7fb07cdb6ba7]
E [bt] (6) /lib/x86_64-linux-gnu/libffi.so.8(+0x7b16) [0x7fb0b0825b16]
E [bt] (7) /lib/x86_64-linux-gnu/libffi.so.8(+0x43ef) [0x7fb0b08223ef]
E [bt] (8) /lib/x86_64-linux-gnu/libffi.so.8(ffi_call+0x12e) [0x7fb0b08250be]
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/core.py:310: XGBoostError
----------------------------- Captured stderr call -----------------------------
[I 2025-03-20 23:18:15,587] A new study created in memory with name: no-name-fbb9dd4f-9283-4813-9952-b27d7124cc69
[W 2025-03-20 23:18:15,589] Trial 0 failed with parameters: {} because of the following error: XGBoostError('[23:18:15] /workspace/src/objective/./regression_loss.h:69: Check failed: base_score > 0.0f && base_score < 1.0f: base_score must be in (0,1) for logistic loss, got: 1\nStack trace:\n [bt] (0) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x2a6acc) [0x7fb07cea6acc]\n [bt] (1) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0xed8329) [0x7fb07dad8329]\n [bt] (2) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x681223) [0x7fb07d281223]\n [bt] (3) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x6815ec) [0x7fb07d2815ec]\n [bt] (4) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x68bb3b) [0x7fb07d28bb3b]\n [bt] (5) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(XGBoosterUpdateOneIter+0x77) [0x7fb07cdb6ba7]\n [bt] (6) /lib/x86_64-linux-gnu/libffi.so.8(+0x7b16) [0x7fb0b0825b16]\n [bt] (7) /lib/x86_64-linux-gnu/libffi.so.8(+0x43ef) [0x7fb0b08223ef]\n [bt] (8) /lib/x86_64-linux-gnu/libffi.so.8(ffi_call+0x12e) [0x7fb0b08250be]\n\n').
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/optuna/study/_optimize.py", line 197, in _run_trial
value_or_values = func(trial)
File "/home/runner/work/optuna-integration/optuna-integration/tests/xgboost/test_xgboost.py", line 69, in objective
xgb.cv(params, dtrain, callbacks=[pruning_callback], nfold=2)
File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/core.py", line 729, in inner_f
return func(**kwargs)
File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/training.py", line 581, in cv
booster.update(i, obj)
File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/training.py", line 229, in update
fold.update(iteration, obj)
File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/training.py", line 215, in update
self.bst.update(self.dtrain, iteration, fobj)
File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/core.py", line 2246, in update
_check_call(
File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/core.py", line 310, in _check_call
raise XGBoostError(py_str(_LIB.XGBGetLastError()))
xgboost.core.XGBoostError: [23:18:15] /workspace/src/objective/./regression_loss.h:69: Check failed: base_score > 0.0f && base_score < 1.0f: base_score must be in (0,1) for logistic loss, got: 1
Stack trace:
[bt] (0) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x2a6acc) [0x7fb07cea6acc]
[bt] (1) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0xed8329) [0x7fb07dad8329]
[bt] (2) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x681223) [0x7fb07d281223]
[bt] (3) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x6815ec) [0x7fb07d2815ec]
[bt] (4) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x68bb3b) [0x7fb07d28bb3b]
[bt] (5) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(XGBoosterUpdateOneIter+0x77) [0x7fb07cdb6ba7]
[bt] (6) /lib/x86_64-linux-gnu/libffi.so.8(+0x7b16) [0x7fb0b0825b16]
[bt] (7) /lib/x86_64-linux-gnu/libffi.so.8(+0x43ef) [0x7fb0b08223ef]
[bt] (8) /lib/x86_64-linux-gnu/libffi.so.8(ffi_call+0x12e) [0x7fb0b08250be]Steps to reproduce
Please checkout the CI logs below:
https://github.com/optuna/optuna-integration/actions/runs/13981048377/job/39146195909
Additional context (optional)
No response