Skip to content

GCRANSAC::run: don't read models[0] when the final fit returns no model - #56

Open
f-dy wants to merge 1 commit into
danini:masterfrom
f-dy:fix-empty-models
Open

f-dy wants to merge 1 commit into
danini:masterfrom
f-dy:fix-empty-models

Conversation

@f-dy

@f-dy f-dy commented Nov 9, 2026

Copy link
Copy Markdown

Fixes #55.

When estimateModelNonminimal returns no model at the end of GCRANSAC::run, the code copied models[0] of the empty vector (L615-L616). These two lines are removed: the best model stays the one found before the final fit, and the returned inliers are already its inliers.

Tested in HomLib, which uses GC-RANSAC with its own solvers (macOS arm64, AddressSanitizer): the container overflow at this line, reported in 11 of 20 runs of its GC-RANSAC tests, is gone in 20 runs, and the tests pass.

When estimateModelNonminimal returns no model, run() copied the
descriptor of models[0], an element of the empty (cleared) vector: a
read of a destroyed Eigen matrix, which aborts at random on macOS. The
best model now stays the one found before the final fit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GCRANSAC::run reads models[0] of an empty vector when the final fit returns no model

1 participant