Estimate camera extrinsic/intrinsic parameters from 3d scene #1528

aleksas · 2021-09-10T11:21:02Z

aleksas
Sep 10, 2021

Having constructed photogrammetry 3D scene from ~300 photos using camera A, is it possible to estimate camera B intrinsic and extrinsic parameters providing single camera B photo without metadata (IP camera) using Meshroom? If so, suggestions how to do that would be more than welcome.

Answered by natowi

Sep 16, 2021

Sorry, might have been not the best node for your use case after all. Did you try to augment your reconstruction with your image? (drag and drop the image in the augmentation area on import) In the new cameraInit you can define "fisheye" for the image and than try to match it to the existing sfm...

View full answer

natowi · 2021-09-10T15:51:16Z

natowi
Sep 10, 2021
Collaborator

Does the node https://meshroom-manual.readthedocs.io/en/latest/feature-documentation/nodes/CameraLocalization.html help?
(Somewhat related #1486)

0 replies

aleksas · 2021-09-13T08:00:50Z

aleksas
Sep 13, 2021
Author

Great, this should work!

[...]

Just tried that but it seams I'm not doing it correctly...

I've connected SfM Data and Descriptor Path node parameters to FeatureExtraction and StructureFromMotion nodes respectively. The result is Localized 0/1 images

The initial set of images that the 3d scene was built from was taken using smartphone wide lens camera.
The camera to localize is mounted on the ceiling and is a half-fisheye (one axis view angle is wide, the other is narrow)

What am I doing wrong?

The log output and node connection screenshots provided below.

Here is how I've connected the nodes:

Program called with the following parameters:
 * algorithm = "AllResults"
 * calibration = ""
 * commonviews = 3
 * descriptorPath = "/absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/FeatureExtraction/aab4f74a7ba285133bc53787a7631955e9aa46a7"
 * globalBundle = 1
 * matchDescTypes = "sift"
 * matchingError = 4
 * matchingEstimator =  Unknown Type "N11aliceVision16robustEstimation16ERobustEstimatorE"
 * maxResults = 10
 * mediafile = "/absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/test/76fce5ef-0ba8-41b1-8ac0-6b0eb11c0864.jpg"
 * minPointVisibility = 2
 * nNearestKeyFrames = 5
 * nbFrameBufferMatching = 10
 * nbImageMatch = 8
 * noBArefineIntrinsics = 0
 * noDistortion = 0
 * outputAlembic = "/absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/CameraLocalization/6b4cce859420be55d49b332ad444539fff8ba321/trackedCameras.abc"
 * outputJSON = "/absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/CameraLocalization/6b4cce859420be55d49b332ad444539fff8ba321/trackedCameras.json"
 * preset =  Unknown Type "N11aliceVision7feature21EImageDescriberPresetE"
 * randomSeed = 5489 (default)
 * refineIntrinsics = 1
 * reprojectionError = 8
 * resectionEstimator =  Unknown Type "N11aliceVision16robustEstimation16ERobustEstimatorE"
 * robustMatching = 1
 * sfmdata = "/absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/StructureFromMotion/c06d41fe037f6a07e7f33bb1594024ff442659e2/sfm.abc"
 * visualDebug = ""
 * voctree = "/absolute/path/Meshroom-2021.1.0-linux-cuda10/Meshroom-2021.1.0-av2.4.0-centos7-cuda10.2/aliceVision/share/aliceVision/vlfeat_K80L3.SIFT.tree"
 * voctreeWeights = ""

[2021-09-13 10:30:51.936531] [0x00007fa14d504880] [debug]   0 samples found in this animated xform.
[2021-09-13 10:30:51.936746] [0x00007fa14d504880] [debug]   0 samples found in this animated xform.
[2021-09-13 10:30:51.951291] [0x00007fa14d504880] [debug]   0 samples found in this animated xform.
[2021-09-13 10:30:51.951859] [0x00007fa14d504880] [debug]   0 samples found in this animated xform.
[2021-09-13 10:30:51.951866] [0x00007fa14d504880] [debug]   0 samples found in this animated xform.
[2021-09-13 10:30:52.006388] [0x00007fa14d504880] [trace]   305 extrinsics declared and 318 extrinsics used.
[2021-09-13 10:30:52.046669] [0x00007fa14d504880] [info]    Supported CUDA-Enabled GPU detected.
[2021-09-13 10:30:52.071267] [0x00007fa14d504880] [debug]   Loading vocabulary tree...
[2021-09-13 10:30:52.095047] [0x00007fa14d504880] [debug]   tree loaded with 3 levels and 80 branching factors
[2021-09-13 10:30:52.095057] [0x00007fa14d504880] [debug]   Creating the database...
[2021-09-13 10:30:52.098473] [0x00007fa14d504880] [debug]   No weights specified, skipping...
[2021-09-13 10:30:52.098479] [0x00007fa14d504880] [debug]   Build observations per view

- Load Features and Descriptors per view -
0%   10   20   30   40   50   60   70   80   90   100%
|----|----|----|----|----|----|----|----|----|----|
[2021-09-13 10:30:52.136210] [0x00007fa14d504880] [trace]   Features filename: /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/FeatureExtraction/aab4f74a7ba285133bc53787a7631955e9aa46a7/4634976.sift.feat
[2021-09-13 10:30:52.136221] [0x00007fa14d504880] [trace]   Descriptors filename: /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/FeatureExtraction/aab4f74a7ba285133bc53787a7631955e9aa46a7/4634976.sift.desc
[2021-09-13 10:30:52.136227] [0x00007fa128f60700] [trace]   Features filename: /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/FeatureExtraction/aab4f74a7ba285133bc53787a7631955e9aa46a7/685208576.sift.feat
[2021-09-13 10:30:52.136236] [0x00007fa10ffff700] [trace]   Features filename: /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/FeatureExtraction/aab4f74a7ba285133bc53787a7631955e9aa46a7/1484311176.sift.feat
[2021-09-13 10:30:52.136246] [0x00007fa10ffff700] [trace]   Descriptors filename: /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/FeatureExtraction/aab4f74a7ba285133bc53787a7631955e9aa46a7/1484311176.sift.desc
[2021-09-13 10:30:52.136254] [0x00007fa128f60700] [trace]   Descriptors filename: /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/FeatureExtraction/aab4f74a7ba285133bc53787a7631955e9aa46a7/685208576.sift.desc
[2021-09-13 10:30:52.156232] [0x00007fa14d504880] [trace]   Region count: 20000
[2021-09-13 10:30:52.161682] [0x00007fa128f60700] [trace]   Region count: 20000
[2021-09-13 10:30:52.162314] [0x00007fa10ffff700] [trace]   Region count: 20000
*[2021-09-13 10:30:52.648844] [0x00007fa128f60700] [trace]   Features filename: /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/FeatureExtraction/aab4f74a7ba285133bc53787a7631955e9aa46a7/685337739.sift.feat

.......
.......
.......



[2021-09-13 10:31:44.555572] [0x00007fa10ffff700] [trace]   Region count: 20000
[2021-09-13 10:31:45.040218] [0x00007fa10ffff700] [trace]   Features filename: /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/FeatureExtraction/aab4f74a7ba285133bc53787a7631955e9aa46a7/2144040141.sift.feat
[2021-09-13 10:31:45.040224] [0x00007fa10ffff700] [trace]   Descriptors filename: /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/MeshroomCache/FeatureExtraction/aab4f74a7ba285133bc53787a7631955e9aa46a7/2144040141.sift.desc
[2021-09-13 10:31:45.059326] [0x00007fa10ffff700] [trace]   Region count: 20000
*[2021-09-13 10:31:45.542135] [0x00007fa14d504880] [debug]   /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/test/76fce5ef-0ba8-41b1-8ac0-6b0eb11c0864.jpg
[2021-09-13 10:31:45.584920] [0x00007fa14d504880] [trace]   Read image /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/test/76fce5ef-0ba8-41b1-8ac0-6b0eb11c0864.jpg (encoded in sRGB colorspace).
******************************
FRAME 0000
******************************
[2021-09-13 10:31:45.602612] [0x00007fa14d504880] [debug]   [features]	Extract Regions from query image
Choosing device 0: GeForce GTX 1650 SUPER
[2021-09-13 10:31:45.756641] [0x00007fa14d504880] [trace]   PopSIFT features count: 8038, descriptors count: 9338

[2021-09-13 10:31:45.757247] [0x00007fa14d504880] [trace]   aliceVision PopSIFT feature count : 9338

[2021-09-13 10:31:45.757295] [0x00007fa14d504880] [debug]   [features]	Extract sift done: found 9338 features in 154 [ms]
[2021-09-13 10:31:45.757304] [0x00007fa14d504880] [debug]   [database]	Request closest images from voctree
[2021-09-13 10:31:45.803769] [0x00007fa14d504880] [debug]   [matching]	Building the matcher
[2021-09-13 10:31:45.839424] [0x00007fa14d504880] [trace]   [matching]	Trying to match the query image with /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/IMG_20210903_185004.jpg
[2021-09-13 10:31:45.839429] [0x00007fa14d504880] [trace]   [matching]	It has 979 available features to match
[2021-09-13 10:31:45.846797] [0x00007fa14d504880] [trace]   RelativePoseKernel: x1: 2x21, x2: 2x21
[2021-09-13 10:31:45.862902] [0x00007fa14d504880] [debug]   [matching]	Geometric validation failed.
[2021-09-13 10:31:45.862909] [0x00007fa14d504880] [trace]   [matching]	Trying to match the query image with /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/IMG_20210903_185255_1.jpg
[2021-09-13 10:31:45.862911] [0x00007fa14d504880] [trace]   [matching]	It has 662 available features to match
[2021-09-13 10:31:45.867039] [0x00007fa14d504880] [trace]   RelativePoseKernel: x1: 2x13, x2: 2x13
[2021-09-13 10:31:45.881883] [0x00007fa14d504880] [debug]   [matching]	Geometric validation failed.
[2021-09-13 10:31:45.881888] [0x00007fa14d504880] [trace]   [matching]	Trying to match the query image with /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/IMG_20210903_185132_1.jpg
[2021-09-13 10:31:45.881890] [0x00007fa14d504880] [trace]   [matching]	It has 3107 available features to match
[2021-09-13 10:31:45.897299] [0x00007fa14d504880] [trace]   RelativePoseKernel: x1: 2x52, x2: 2x52
[2021-09-13 10:31:45.914623] [0x00007fa14d504880] [debug]   [matching]	Geometric validation failed.
[2021-09-13 10:31:45.914628] [0x00007fa14d504880] [trace]   [matching]	Trying to match the query image with /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/IMG_20210903_184806_1.jpg
[2021-09-13 10:31:45.914630] [0x00007fa14d504880] [trace]   [matching]	It has 3206 available features to match
[2021-09-13 10:31:45.931197] [0x00007fa14d504880] [trace]   RelativePoseKernel: x1: 2x49, x2: 2x49
[2021-09-13 10:31:45.948153] [0x00007fa14d504880] [debug]   [matching]	Geometric validation failed.
[2021-09-13 10:31:45.948158] [0x00007fa14d504880] [trace]   [matching]	Trying to match the query image with /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/IMG_20210903_185638.jpg
[2021-09-13 10:31:45.948160] [0x00007fa14d504880] [trace]   [matching]	It has 1077 available features to match
[2021-09-13 10:31:45.955054] [0x00007fa14d504880] [trace]   RelativePoseKernel: x1: 2x6, x2: 2x6
[2021-09-13 10:31:45.955058] [0x00007fa14d504880] [debug]   [matching]	Geometric validation failed.
[2021-09-13 10:31:45.955062] [0x00007fa14d504880] [trace]   [matching]	Trying to match the query image with /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/IMG_20210903_184901.jpg
[2021-09-13 10:31:45.955064] [0x00007fa14d504880] [trace]   [matching]	It has 214 available features to match
[2021-09-13 10:31:45.956630] [0x00007fa14d504880] [trace]   RelativePoseKernel: x1: 2x7, x2: 2x7
[2021-09-13 10:31:45.956633] [0x00007fa14d504880] [debug]   [matching]	Geometric validation failed.
[2021-09-13 10:31:45.956637] [0x00007fa14d504880] [trace]   [matching]	Trying to match the query image with /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/IMG_20210903_185324.jpg
[2021-09-13 10:31:45.956639] [0x00007fa14d504880] [trace]   [matching]	It has 3663 available features to match
[2021-09-13 10:31:45.971665] [0x00007fa14d504880] [trace]   RelativePoseKernel: x1: 2x37, x2: 2x37
[2021-09-13 10:31:45.987861] [0x00007fa14d504880] [debug]   [matching]	Geometric validation failed.
[2021-09-13 10:31:45.987866] [0x00007fa14d504880] [trace]   [matching]	Trying to match the query image with /absolute/path/Meshroom-2021.1.0-linux-cuda10/PROJECT/IMG_20210903_185612_1.jpg
[2021-09-13 10:31:45.987868] [0x00007fa14d504880] [trace]   [matching]	It has 854 available features to match
[2021-09-13 10:31:45.993199] [0x00007fa14d504880] [trace]   RelativePoseKernel: x1: 2x17, x2: 2x17
[2021-09-13 10:31:46.008653] [0x00007fa14d504880] [debug]   [matching]	Geometric validation failed.
[2021-09-13 10:31:46.008657] [0x00007fa14d504880] [debug]   [matching]	Using frameBuffer matching: matching with the past 10 frames
[2021-09-13 10:31:46.008898] [0x00007fa14d504880] [debug]   [poseEstimation]	Estimating camera pose...
[2021-09-13 10:31:46.008918] [0x00007fa14d504880] [debug]   Resection status is false:
	- resection_data.vec_inliers.size() = 0
	- minimumSamples = 6
[2021-09-13 10:31:46.008934] [0x00007fa14d504880] [info]    Robust Resection information:
	- resection status: false
	- threshold (error max): 0
	- # points used for resection: 0
	- # points validated by robust resection: 0
[2021-09-13 10:31:46.008936] [0x00007fa14d504880] [debug]   [poseEstimation]	Resection failed

Localization took  406 [ms]
Unable to localize frame 0
[2021-09-13 10:31:46.009093] [0x00007fa14d504880] [debug]   next frame 1


******************************
Localized 0/1 images
Images localized with the number of 2D/3D matches during localization :
Processing took 0.406 [s] overall
Mean time for localization:   406 [ms]
Max time for localization:   406 [ms]
Min time for localization:   406 [ms]

0 replies

natowi · 2021-09-16T17:11:58Z

natowi
Sep 16, 2021
Collaborator

Sorry, might have been not the best node for your use case after all. Did you try to augment your reconstruction with your image? (drag and drop the image in the augmentation area on import) In the new cameraInit you can define "fisheye" for the image and than try to match it to the existing sfm...

0 replies

aleksas · 2021-09-17T10:33:07Z

aleksas
Sep 17, 2021
Author

This looks quite alright. Meshroom was able to localize the camera. Will try to check how accurate that is.

Correct me if I'm wrong intrinsic and extrinsic information is located in MeshroomCache/StructureFromMotion/[NEEW HASH]/cameras.sfm . I've managed to locate pose and intrinsic from new image filename within cameras.sfm.

Also in camerLocalisation node there is an option to "Refine intrinsics" is it part of the augmented reconstruction or this is something to consider?

0 replies

aleksas · 2021-09-17T12:56:47Z

aleksas
Sep 17, 2021
Author

Also is there a reason why setting camera type to brown causes StructureFromMotion computation to stuck in a loop - wasn't able to compute in over 2h? On the other hand if camera is initialized with radial1 or radial3 SfM is computed in under a minute.

P.S.

brown is of interest because this is the distortion correction method non-fisheye cameras were calibrated with. Getting calibration parameters from meshroom would allow in place parameter replacement.
The "augmenting" picture i'm refering to is not from fisheye camera, but a regular durveillance camera. Will try fisheye later though.
The original model smartphone cameras were initialized as fisheye4

UPDATE

adding more images from different cameras allowed for some of them to compute SfM and localize cameras. Might be that images surveillance cameras have too many differences in scene (different capture time) causing meshroom to discard them?

Also different cameras would have different brown intrinsic parameters, but by adding multiple images from different cameras meshroom adds just two intrinsic entries.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Estimate camera extrinsic/intrinsic parameters from 3d scene #1528

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 5 comments

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Estimate camera extrinsic/intrinsic parameters from 3d scene #1528

aleksas Sep 10, 2021

Replies: 5 comments

natowi Sep 10, 2021 Collaborator

aleksas Sep 13, 2021 Author

natowi Sep 16, 2021 Collaborator

aleksas Sep 17, 2021 Author

aleksas Sep 17, 2021 Author

UPDATE

aleksas
Sep 10, 2021

natowi
Sep 10, 2021
Collaborator

aleksas
Sep 13, 2021
Author

natowi
Sep 16, 2021
Collaborator

aleksas
Sep 17, 2021
Author

aleksas
Sep 17, 2021
Author