Generative Base Material: An Open-Source Prototype for PBR Material Estimation Debuting at SIGGRAPH Asia 2025

Ubisoft La Forge, Ubisoft’s R&D network, is excited to open the weights of the material estimation model from its paper “CHORD: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Images”, accepted at SIGGRAPH Asia 2025.

What Are Base Materials in Video Games?

Base materials are modular and reusable assets to give the final appearance of 3D objects in video games worlds. They are independent of the 3D objects’ shape; they can be tiled on infinite surfaces and can be combined to create sophisticated textures.

Open-world games typically use hundreds of base materials that require a significant amount of time to produce and a strong expertise in 3D software. They are created by texture artists using various technics such as photogrammetry, scans, or procedural tools like Adobe Substance 3D. While all these techniques have proven their effectiveness, recent advances in image generation allow us to create images from text prompts, suggesting a potential new tool for material creation. This is why La Forge has been collaborating with material artists and technical artists from game development teams to develop Generative Base Material, an R&D prototype to assist material artists in their workflow.

“Creating high‑quality PBR materials demands a rare blend of artistic vision and technical precision—a skill set that’s hard to find and harder to scale. The promise of AI assistance is exciting because it can make this pipeline more accessible and empower teams to focus on creativity. Ultimately, it could help us better realize the visual ambitions of our games.” - Gilles Fleury, Senior Technical Art Director

Example of a base material for a rock surface created with traditional technics, the material maps include (from left to right) the base color, normal, height, and roughness.

Prototype Overview

The objective of our prototype is to create an accessible pipeline that generates base materials for Physically Based Rendering (PBR) from inputs such as text or images. Our focus is to provide creators with assistance to accelerate their workflows while maximizing their control on the results. To do so, we designed a multi-stage pipeline composed of 3 distinct AI models.

Stage 1: Texture Image Synthesis

Given a text prompt and optional conditioning images such as linearts, sketches or height maps, our first AI model generates a seamless tileable 2D texture depicting the material under fixed lighting. This model is a custom diffusion model trained on internal assets allowing artists to leverage existing tools and technics to control the generation of texture images such as text-to-image, image-to-image, ControlNets, Adapters, etc.

Stage 2: Image-To-Material Estimation

In the second stage, our prototype converts the generated texture image into PBR material maps. This step uses our custom model architecture “CHORD”, which performs a chain of rendering decomposition. You can read more details on this model in the following part of this blog post.

Stage 3: Material Upscaling

Finally, because game assets often require 2K or 4K resolution textures, our third AI model enables 2x or 4x upscaling of the PBR material maps.

Prototype overview – Our 3 AI models combined allow to go from prompt to a full, upscaled texture set that can then be fine-tuned by texture artists.

How CHORD Advances the Field of PBR Material Estimation

CHORD is a diffusion-based model that predicts material maps from a single RGB image. It takes a texture image as input and produces standard svBRDF channels such as base color, normal, height, roughness, and metalness. Our model architecture incorporates several novelties:

Chained Decomposition: CHORD organizes rendering decomposition into a chain that simplifies multi‑modal prediction by breaking it into manageable sub‑tasks. The base color is predicted first; The approximated irradiance map is computed by removing color information from the input image and used to predict the normal map. The normal map is used to compute the height map via integration. Then all predicted maps are fed into a renderer to generate a synthetic image, and the difference from the input image is minimized to recover the approximated roughness and metalness parameters used to predict the final roughness and metalness maps.

LEGO-conditioning: The LEGO-conditioning solves weight conflicts in multi-modal prediction, allowing shared backbone weights for spatial alignment but modality-specific processing. Inspired by the work of (Zeng et al., 2024) on RGB↔X, CHORD uses the CLIP text embeddings of keywords "Basecolor", "Normal", "Roughness", "Metalness" as switches to handle different modalities in a unified framework.

Single-step Training: Finally, the single-step training approach improves efficiency and output quality by avoiding the typical multi-step denoising in diffusion models.

Our training procedure follows a two‑phases approach. A pretraining phase that minimizes loss in latent space. And a single‑step phase where the Loss is computed directly in image space, omitting certain noise terms for efficiency.

CHORD architecture

Training Dataset

The open-weights version of our model is trained using MatSynth, a large open-source dataset of approximately 5700 PBR materials, widely used in the computer graphics community.

In the data preparation phase, we augment our training dataset by rotating materials from 4 different angles resulting in 22800 samples, and during training, we further augment the data at runtime by randomly cropping and resizing materials.

Leveraging Open-Source Resources for Implementation

Considering the multi-stage nature of our prototype, ComfyUI provides us with an efficient framework to build integrated workflows doing texture image synthesis, material estimation and material upscaling. This also enables us to leverage state-of-the-art generative models and the powerful features of ComfyUI that provide fine-grain control to creators with ControlNets, image guidance, inpainting, and countless other options.

As open-source technology enables us to build and augment the potential of our prototype, we felt it was natural to give back to the community. This is why in addition to our model weights, we are pleased to open-source a set of custom nodes to run CHORD directly in ComfyUI. You can read more about the nodes usage and find examples of workflows in this blog post: Ubisoft Open-Sources the CHORD Model and ComfyUI Nodes for End-to-End PBR Material Generation.

Results

Our quantitative benchmarks demonstrate that CHORD outperforms other methods on most material maps, in terms of PSNR and LPIPS scores. More details and ablation studies are available in our paper.

From a qualitative standpoint, CHORD produces state-of-the-art results in its category of multi-modal PBR maps generation models, as shown in the visualizations below. However, general feedback from artists highlights the fact that results are not production-ready yet, but a solid first step toward our goal of empowering them.

A texture of pesto pasta, generated with text-to-image in the first step, and image-to-material in the second step.

A texture of marble bas relief with peony flowers. The roughness map is discarded and replaced by a fixed value to improve the visual quality in the 3D render.

A texture of slate roof tiles. A ControlNet with a lineart image is used to guide the tiles pattern in the first step.

A texture of wolf fur. A ControlNet with a height map of animal fur is used to guide the generation. The same height map is reused in the 3D render.

Current Limitations

While we are excited by the results we obtain with our prototype, the output quality still has room for improvement.

“Although the quality isn’t where it needs to be for AAA video games material, it shows a lot of promise, and I’m confident it’s only a matter of time before it delivers the expected results.” – Greg Baran, Expert Material Artist

Additionally, our CHORD model is optimized for images with a resolution of 1024, when high-quality game textures are usually in 2048 or 4096 resolutions. Upscaling material maps after their generation provide an effective solution but suffer from error accumulation along the pipeline.

Our tests demonstrate our prototype performs well for organic materials, but shows limitations for estimating svBRDF maps from input images with strong specular properties such as metal. The metalness map is notably difficult to predict with high accuracy.

Takeaways

Developing Generative Base Material to become an actual production tool still requires more research, but our prototype demonstrates potential for assisting creators in multiple use cases. For instance in conception stage for creating references of textures, or concept assets, and in production for placeholder materials.

The multi-stage pipeline also provides great flexibility and control. Artists can use the entire pipeline for quickly prototyping materials and dressing up a scene, but each AI model can also be used separately to mix and integrate generated outputs into traditional workflows.

“The modular approach blends very well into existing material creation workflows, making adoption easier by positioning AI as an additional tool into existing pipelines rather than a disruptive change. This is reassuring for artists who may be hesitant about this new technology.” – Gilles Fleury, Senior Technical Art Director

As we release CHORD to the research community and beyond, we can’t wait to see what creatives will produce with our model. Reach out to us and let us know at laforge[at]ubisoft.com.

Get started with CHORD material estimation:

• Project site
• Arxiv page
• Paper
• Supplementary
• Code repository
• ComfyUI implementation