Abstract
Gaussian Splatting and its dynamic extensions are effective for reconstructing 3D scenes from 2D images when there is significant camera movement to facilitate motion parallax and when scene objects remain relatively static. However, in many real-world scenarios, these conditions are not met. As a consequence, data-driven semantic and geometric priors have been favored as regularizers, despite their bias toward training data and their neglect of broader movement dynamics.
Departing from this practice, we propose a novel analytical approach that adapts the classical Lucas-Kanade method to dynamic Gaussian splatting. By leveraging the intrinsic properties of the forward warp field network, we derive an analytical velocity field that, through time integration, facilitates accurate scene flow computation. This enables the precise enforcement of motion constraints on warp fields, thus constraining both 2D motion and 3D positions of the Gaussians. Our method excels in reconstructing highly dynamic scenes with minimal camera movement, as demonstrated through experiments on both synthetic and real-world scenes.
Gaussian Splatting LK regulates the warp field to enhance Gaussian Splatting scene geometry.
Animation
Interpolating motions
We can also animate the scene by interpolating the integrated trajectories. Use the slider here to linearly interpolate between the left frame and the right frame.
Start Frame
End Frame
Related Links
This work is inspired by many recent works in the field of neural rendering and dynamic scene modeling and bijective warp field in point cloud alignment.
PointNetLK introduces a PointNet-derived analytical warp field for point cloud alignment.
NSFF, NR-NeRF and FSDNeRF all use flow supervisions to model non-rigid NeRF scenes.
Recent dynamic Gaussian Splatting frameworks such as DeformableGS, 4DGaussians, and DynamicGS have paved the way for dynamic scene modeling with explicit Gaussian represenations.
We derive a general analytical scene flow representation for Gaussian Splatting and show that it can be used to enhance dynamic motions and scene structural integrity.