Lee DongWon

3DGS Keyframe Distillation

3D Gaussian Splatting (gsplat) on TUM RGB-D sequences.

A sharpness-and-coverage keyframe selection for 3D Gaussian Splatting: a modest gain over uniform sampling from a small share of the frames, reported with its limits.

The project in 17 seconds. No sound.

Overview

3D Gaussian Splatting trains on posed images, and a handheld RGB-D sequence contains many blurry or near-duplicate frames. I designed a keyframe selection that scores sharpness and trajectory coverage together, and compared it with uniform sampling at budgets of 30 and 60 frames.

A sharpness curve over frame id; uniform picks sometimes land on blurry dips while the proposed picks sit on sharp frames spread across the sequence
The idea: pick sharp frames, spread across the trajectory (schematic curve, not measured data).

Setup

Renderer
3D Gaussian Splatting with gsplat, 7k iterations
Data
Three TUM RGB-D sequences: freiburg1_desk, freiburg1_room, freiburg2_xyz
Budgets
30 and 60 training frames, which is 1.7–21.1% of the available frames
Baselines
Uniform sampling, random, blur-only selection, and all frames
Seeds
Two per setting

Results

Across the six scene and budget settings, the selection averaged +0.55 dB PSNR (range 0.0–1.5) and +0.015 SSIM (0.004–0.025) over uniform sampling.

360-degree turntable renders at a 30-frame budget, seed 0: ground-truth fusion, this method, uniform sampling, and all frames, for each sequence.
Per-sequence PSNR and SSIM bars for this method, uniform, random and blur-only selection at 30 and 60 frames, with the sharpness of the training pool and the selected frames
PSNR and SSIM per sequence (mean and standard deviation over two seeds), and where each method picks frames.

Limits

  • Results varied by scene. On desk at 60 frames, the PSNR difference was within seed noise.
  • Blur-only selection outperformed this method on fr2_xyz.
  • LPIPS was not measured.

More projects