SV-Gaussians: 4D Gaussian Splatting for High-Fidelity Dynamic Reconstruction of Single-View Scenes

With the introduction of differentiable rendering frameworks, many advanced reconstruction techniques are proposed for both static and dynamic scenes, such as NeRFs and 3D Gaussian Splatting methods. However, there is a type of video on social platforms that is shot by users through fixed monocular cameras. For the reconstruction of such dynamic scenes under single-view observation, existing reconstruction techniques are difficult to produce satisfactory results. The problems include reconstruction blur in high-frequency details of objects and inaccurate estimation of geometry. These issues motivate us to study high-quality reconstruction method for single-view videos. Inspired by 3D Gaussian Splatting, we propose a framework specifically designed for single-view dynamic scene reconstruction and editing, SV-Gaussians. Our method utilizes sparse point cloud of the scene as initialization for 3D Gaussians, and designs a deformation module to model the changes in Gaussian properties over time. We propose deformation regularization to ensure the consistency of adjacent Gaussian motions. To remove obstacles in the scene during reconstruction, we define color constraints based on obstacle mask to help Gaussians learn the true appearance of invisible regions. We conduct extensive experiments to evaluate our method and compare it with SOTA single-view reconstruction methods. Experiments show that our method has strong advantages in reconstruction quality and can achieve real-time rendering.