Why is the projmatrix in cam the same to first frame?

[Yoona12]

Thanks for your great work!
When i read the code, i have following question.
With camera moving, the viewmatrix of camera is changing,and thus the projmatrix is changing.
But in the code, with camera moving, the cam is the same to the first frame all the time.

[Nik-V9]

Hi, Thanks for the interest in our work!

In SplaTAM, we fix the viewing camera to the first frame (this is the world frame). For subsequent cameras, rather than defining a new viewing camera, we project the Gaussians to the first frame (that is the world frame). Hence, we don't need to change the viewing camera for the rasterizer.

Please refer to the following comments and let me know if this answers your query:

1. Confusion about 'transformed_params2depthplussilhouette' function #52 (comment)
2. Q about pose estimation #28 (comment)

[DeepDuke]

Hi, I have the same question. Before rendering, the Gaussians are transformed to the current frame. If the viewing matrix is fixed to the first frame, shouldn't we render the transformed Gaussians in the current frame? It looks strange to render at the first frame view. Could u please help explain more, thanks a lot! @Nik-V9

[DeepDuke]

Another question is about calculating the loss. If the rendered images are viewed from the first frame, how do u compute the loss with ground truth images that are observed at different frames?

[Nik-V9]

Hi, as I mentioned before, the reference frame is fixed to the first frame, and the Gaussians (pointclouds) are just transformed into the frame of the reference frame for rasterization.

"For subsequent cameras, rather than defining a new viewing camera, we project the Gaussians to the first frame (the world frame). Hence, we don't need to change the viewing camera for the rasterizer."

Hence, the renderings for the current view will represent the current camera, and you can apply rendering losses.