An evaluation of SVBRDF Prediction from Generative Image Models for Appearance Modeling of 3D Scenes

Eurographics Symposium on Rendering 2025 (Symposium Track)

Alban Gauthier1, Valentin Deschaintre2, Alexandre Lanvin1, Fredo Durand3,
Adrien Bousseau1, George Drettakis1,
1Inria & Université Côte d'Azur, 2Adobe Research, 3MIT

1 2 3
Paper Video Digital Library Code Group Publ. Page


We evaluate design choices for implementing a fast pipeline to model appearance of 3D scene assets.

Given the geometry of the scene and an example image (left), we use generative image diffusion models and SVBRDF predictors to obtain multiview physically-based material maps that are merged into a texture atlas for the scene, enabling rendering under arbitrary view and lighting (right).

Abstract

Digital content creation is experiencing a profound change with the advent of deep generative models. For texturing, conditional image generators now allow the synthesis of realistic RGB images of a 3D scene that align with the geometry of that scene. For appearance modeling, SVBRDF prediction networks recover material parameters from RGB images. Combining these technologies allows us to quickly generate SVBRDF maps for multiple views of a 3D scene, which can be merged to form a SVBRDF texture atlas of that scene. In this paper, we analyze the challenges and opportunities for SVBRDF prediction in the context of such a fast appearance modeling pipeline. On the one hand, single-view SVBRDF predictions might suffer from multiview incoherence and yield inconsistent texture atlases. On the other hand, generated RGB images, and the different modalities on which they are conditioned, can provide additional information for SVBRDF estimation compared to photographs. We compare neural architectures and conditions to identify designs that achieve high accuracy and coherence. We find that, surprisingly, a standard UNet is competitive with more complex designs.

Method Overview

Pipeline overview

We first generate a sequence of views of the scene using an image diffusion model conditioned on depth and contours (Step 1). While the first view is generated entirely (top left), subsequent views are filled in by re-projecting existing views and inpainting holes revealed by disocclusions (bottom left). We then estimate SVBRDF maps for each view (Step 2), and merge them to form a texture atlas for the scene, enabling physically-based rendering with diverse materials (Step 3).

Video

Single-view SVBRDF prediction

Metrics table

PBR texturing results

sampletext
Livingroom
sampletext
Bathroom

Concurrent Work

The following concurrent works also propose to repurpose diffusion models for PBR texturing:

- Jointly Generating Multi-view Consistent PBR Textures using Collaborative Control

- Material Anything: Generating Materials for Any 3D Object via Diffusion

- SuperMat: Physically Consistent PBR Material Estimation at Interactive Rates

- VideoMat: Extracting PBR Materials from Video Diffusion Models

BibTeX


      @inproceedings{
        gauthier2025evaluation,
        booktitle = {Eurographics Symposium on Rendering},
        title = {{An evaluation of SVBRDF Prediction from Generative Image Models for Appearance Modeling of 3D Scenes}},
        author = {Gauthier, Alban and Deschaintre, Valentin and Lanvin, Alexandre and Durand, Fredo and Bousseau, Adrien and Drettakis, George},
        year = {2025},
        publisher = {The Eurographics Association},
      }

Acknowledgments and Funding

This work was funded by the European Research Council (ERC) Advanced Grant NERPHYS, number 101141721 https://project.inria.fr/nerphys. The authors are grateful to the OPAL infrastructure of the Université Côte d'Azur for providing resources and support, as well as Adobe and NVIDIA for software and hardware donations. This work was granted access to the HPC resources of IDRIS under the allocation AD011015561 made by GENCI. F. Durand acknowledges funding from Google, Amazon, and MIT-GIST.