FixAnything: Repairing 3D Rendering Artifacts With a Video Generation Prior

Background and Context

A persistent problem in 3D computer graphics is that rendering from scene representations produces visible artifacts whenever the input views are sparse or the target view lies far from the captured angles. Whether the underlying representation is Gaussian splatting (3DGS), a neural radiance field (NeRF), a mesh, or a point cloud, these formats tend to expose structural chaos, blur, or misalignment under sparse input or large-angle novel-view synthesis. The consequence is a rendering that looks broken from viewpoints the system never saw during training or capture.

Existing diffusion-based prior methods address this issue, but they typically tailor their architecture to a single representation or demand heavy retraining. The practical result is that each 3D representation requires its own dedicated inpainting pipeline, multiplying maintenance and extension costs. Researchers have therefore lacked a unified approach that could repair artifacts across representations without rebuilding the model for each one.

The paper introduces FixAnything, a general-purpose model designed to repair a broad class of rendering artifacts rather than a single format. Its central contribution is treating inpainting as a transferable task rather than a per-representation engineering problem. By converging scattered work into one framework, the authors aim to reduce the cost of supporting new 3D formats in the future.

Deep Analysis

The authors' key insight is that even noisy renderings preserve camera motion and coarse scene structure. Because the temporal sequence retains how the scene shifts as the camera moves, inpainting can be reformulated as a video-to-video translation task. This reframing lets researchers reuse already-trained video generation models with minimal architectural change and lightweight fine-tuning, transferring cross-representation repair capability directly.

FixAnything repurposes a pretrained video generation model, exploiting the implicit multi-view prior it has absorbed from large video corpora rather than designing a network from scratch. Video models learn how scenes change with camera motion, a规律 that inherently encodes consistency between views. To tell the model which structures must be preserved and which need refinement, the authors introduce a binary mask marking clean pixels. This mask anchors the output to high-quality inputs, such as the training views, while concentrating effort on refining unrepaired regions, preventing both the corruption of correct areas and structural distortion in blurry ones.

To ensure repaired renderings support downstream 3D reconstruction, the authors use camera pose accuracy as a reward signal combined with direct preference optimization (DPO). Pose accuracy is recovered via structure-from-motion, so a rendering that enables more accurate reconstruction of the camera trajectory is treated as superior. This geometric-consistency reward unifies visual quality and geometric reliability within a single training objective, explaining why a seemingly simple video-to-video framework can satisfy both demands.

Industry Impact

The paper evaluates FixAnything across four distinct 3D representations—Gaussian splatting, NeRF, meshes, and point clouds—to verify its generality. Results show that on each representation, the model stably improves rendering quality through lightweight fine-tuning rather than working for only one format. This cross-representation consistency directly reflects the framework's design goal and demonstrates that a single general video prior can replace multiple purpose-built inpainting pipelines.

Ablation and mechanistic findings trace back to two core designs: the anchoring effect of the binary mask and the DPO reward based on camera pose accuracy. The mask refines remaining regions while preserving high-quality pixels, avoiding structural damage from over-generation, while preference optimization explicitly folds the geometric consistency required by downstream reconstruction into training. Together these explain how the framework balances image quality and geometric reliability.

For the open-source community, repurposing a pretrained video model with lightweight fine-tuning lowers the barrier to reproduction and further development, letting later researchers test new generation models at low cost. For industrial deployment, tasks such as 3D reconstruction, visualization, or content generation that require rendering under sparse or large-angle conditions benefit from a single repair model reducing the human and compute cost of maintaining separate pipelines per representation.

Outlook

The authors emphasize that the framework's simplicity means stronger video generation models can be integrated directly in the future without redesigning the architecture, converting performance gains into immediate improvements. This evolution-ready design lets the current work continuously benefit from progress across the broader video generation field.

The demonstrated transferability of the implicit multi-view prior in video models, along with the feasibility of using geometric consistency as a reward signal, offers a reusable path for coupling generation priors more tightly with 3D recovery tasks. Researchers can now explore stronger video backbones and richer reward formulations without rebuilding the repair system.

Ultimately, FixAnything reframes 3D artifact repair as a problem of transferring mature video priors rather than reinventing per-format solutions. As video generation models grow more capable, the framework is positioned to inherit those gains directly, making cross-representation rendering repair increasingly economical and broadly applicable.