index planning: implement branch and bound #13568
Open
+219
−105
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
dimitarvdimitrov
force-pushed
the
dimitar/index-planning/branch-and-bound-proper
branch
3 times, most recently
from
958ba9d to
9c471e6
Compare
dimitarvdimitrov
force-pushed
the
dimitar/index-planning/branch-and-bound-proper
branch
2 times, most recently
from
9a9da0c to
142f2ad
Compare
dimitarvdimitrov
changed the base branch from
main
to
dimitar/index-planning/add-virtual-predicates
1 task
dimitarvdimitrov
added
the
changelog-not-needed
dimitarvdimitrov
force-pushed
the
dimitar/index-planning/branch-and-bound-proper
branch
2 times, most recently
from
0c02b81 to
fccd6f3
Compare
This will help us test the branch and bound planner with more complex queries. Signed-off-by: Dimitar Dimitrov <[email protected]>
Signed-off-by: Dimitar Dimitrov <[email protected]>
…d-bound Remove the partialPlan struct and its duplicated cost calculation methods. Instead, use the plan struct directly with its virtualPredicate() method and numDecidedPredicates field to compute lower bound costs. Changes: - Removed partialPlan struct - Removed all cost methods from partialPlan (now inherited from plan) - Changed partialPlans heap type to plans - Updated pools to use plan instead of partialPlan - Branch-and-bound now uses plan directly with numDecidedPredicates tracking - TotalCost() automatically computes lower bound when numDecidedPredicates < len(predicates) This simplifies the code by eliminating duplication while maintaining the same behavior. Signed-off-by: Dimitar Dimitrov <[email protected]>
dimitarvdimitrov
force-pushed
the
dimitar/index-planning/branch-and-bound-proper
branch
from
fccd6f3 to
17d3132
Compare
Signed-off-by: Dimitar Dimitrov <[email protected]>
chencs
reviewed
Nov 19, 2025
| heap.Push(prospectPlans, newPlanWithCost(current.plan, current.numDecidedPredicates+1)) | ||
| } | ||
|
|
||
| // Fall back to index-only plan to ensure that our code doesn't choose a more expensive plan than the naive plan. |
Contributor
There was a problem hiding this comment.
Would it make sense to use the cost of the indexOnlyPlan as the starting bestCompleteCost to further narrow the possible number of generated plans/branches?
Contributor
Author
There was a problem hiding this comment.
yep, it would 👍 let me do that now
Contributor
Author
There was a problem hiding this comment.
done in 42ec853 ; this also changed the tests. but I checked and the costs of the old and the new plans is the same - 11.5, 0, 0. So adding the index-only plan at the start just influences which equivalent plan with the same cost we choose, but they're all in theory equally cheap
Signed-off-by: Dimitar Dimitrov <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does
The exhaustive planner generates all 2^N possible plans for N matchers, which becomes prohibitively expensive for queries with many matchers.
This change implements a branch-and-bound algorithm that:
For undecided predicates, the lower bound calculation:
Benchmarks
The tradeoff is that for smaller plans the new exploration is more expensive
Details
Which issue(s) this PR fixes or relates to
closes #11921
based on the changes from #13572
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]. If changelog entry is not needed, please add thechangelog-not-neededlabel to the PR.about-versioning.mdupdated with experimental features.Note
Replaces exhaustive 2^N plan generation with a branch-and-bound priority-queue explorer, refactors planner to stream plans, and updates tracing, pooling, and tests (incl. large-matcher cases).
generatePlansBranchAndBoundwith a min-heap over plan lower-bound costs; prunes branches and merges remaining candidates (branch_and_bound.go).iter.Seq[plan]); selects first plan containing index matchers.newIndexOnlyPlan(),hasAnyIndexPredicate(), and virtual predicate tweaks for non-zero minimal costs; addsGetPlans()slab-backed PQ; removes "too many matchers" early abort.planningDisabled(ctx)and context helper; returns input plan unchanged when disabled.TestCostBasedPlannerWithManyMatchers(1000 matchers); adds preservation and preference tests.planner_test_cases.csvwith high-cardinality 20-matcher scenarios and sharded variants.Written by Cursor Bugbot for commit 42ec853. This will update automatically on new commits. Configure here.