Ubisoft La Forge Research | CVPR 2026
The Problem with Capturing a Human Face
Creating a realistic digital human head for games comes with a heavy price, not just in money, but also in time and infrastructure. The industry gold standard relies on photogrammetry: surrounding an actor with dozens, sometimes hundreds, of synchronized cameras to triangulate every pore, wrinkle, and facial contour into a dense 3D mesh. The results are impressive, but the setup requires a dedicated light stage, extensive compute time, and skilled artists to clean up artifacts, especially around hair and fine facial structures.
One alternative has been to lean on AI. Recent AI foundation models can reconstruct a 3D face from a single image in seconds. Unfortunately, they trade away person-specific detail for this efficiency. The subtle geometry that makes a face uniquely identifiable: the exact depth of a laugh line, the specific fold of an eyelid, gets averaged out. The result is plausible, but it isn't accurate.
At Ubisoft La Forge, we asked: what if you didn't have to compromise?
A Hybrid Approach: The Core Idea
Skullptor is built around a simple but powerful insight: data-driven foundation models and direct optimization are not competing approaches. Instead, they're complementary ones.
Foundation models are fast and they encode a deep understanding of human faces from large-scale training data. Optimization-based methods are precise and can lock onto person-specific geometry by directly minimizing a geometric error. The weakness of each is the strength of the other.
So, we built a two-stage pipeline that uses them in sequence.
In the first stage, a multi-view aware neural network takes a sparse set of images, as few as three, up to ten, and predicts surface normal maps for each viewpoint. Surface normals represent the orientation of every point on the skin surface, essentially encoding a dense geometric blueprint of the face. Crucially, this stage runs with a single feed-forward pass taking roughly 1.5 seconds.
In the second stage, those predicted normals become the input to an inverse rendering optimization. Starting from a sphere, a mesh is iteratively deformed so that its rendered normals match the predicted ones, with adaptive remeshing applied at every step to recover fine surface detail. The whole optimization runs in under 30 seconds.
The result is a complete, detailed 3D head mesh with wrinkles, skin folds and person-specific surface geometry reconstructed in seconds from a handful of cameras.
Making the AI Prediction Geometrically Consistent
Using a foundation model for normal prediction comes with a catch. These models were designed to process one image at a time. When fed the same face from three different angles, each prediction is made independently, with no awareness of the others. A skin fold visible from the front view and the same fold visible from the side view will produce normals that disagree in 3D space. When you try to reconstruct geometry from these conflicting normals, these features average out and are smoothed over.
To solve this, we built a model on top of DAViD, a monocular foundation model for facial normal estimation, extending it with view-aware cross-attention. Between each transformer block, we insert a cross-attention layer that lets every viewpoint attend to all others simultaneously. Each image's feature tokens become the query; the keys and values are constructed from the concatenated features of the entire set of images. Camera pose is encoded as a positional embedding and injected directly into this process, so the model knows from which viewpoint each token originates.
This way, the model predicts normals that are geometrically consistent across all viewpoints. This consistency is what makes the downstream optimization high-quality and stable.
From Normal Maps to a 3D Mesh
Once we have consistent normal maps, we can start the optimization stage. We initialize a mesh as a unit sphere and iteratively optimize vertex positions to minimize the difference between the mesh's rendered normals and the predicted ones.
Two design choices make this work well in practice.
Firstly, we weight the normal loss by camera-facing angle. Normals predicted for surface regions that face the camera directly are more reliable than those at grazing angles, and the optimization reflects this by making frontal regions contribute more to the reconstruction loss.
Secondly, we use Continuous Remeshing's optimizer throughout the reconstruction. At every iteration, edge splits, collapses, and flips are applied to dynamically adapt the mesh resolution to local geometric complexity. Additionally, this optimizer adapts to the scale of each vertex's local geometry. This prevents the mesh from degrading into self-intersections or collapsed faces, and allowing fine details like wrinkles to emerge cleanly.
The full optimization runs for 300 steps and completes in under 30 seconds on a single consumer grade GPU.
Why the Hybrid Approach Matters
The key reason for the success of this project was choosing to use surface normals rather than depth maps or explicit 3D geometry.
First, surface normals are incredibly well-suited for AI foundation models. Unlike complex 3D geometry, normals can be easily rendered from synthetic data to easily create high-quality training sets. Because predicting them is a dense, per-pixel regression task, standard dense prediction transformers (DPTs) can be naturally adapted to the problem.
Second, normals are a superior geometric representation for multi-view optimization. While depth maps only tell you how far away a point is, normals encode the orientation of the surface. This means they capture higher-frequency details---like the fine edge of a wrinkle or the sharp contour of an eyelid---far better than depth can. Crucially, normals are scale-invariant. They describe the local shape rather than absolute distances, which side-steps the scale-matching issues that tend break multi-view reconstruction pipelines.
By utilizing geometrically consistent normals as our optimization target, Skullptor bridges the gap between AI speed and classical precision. It allows us to operate in a regime where traditional photogrammetry fails entirely, such as using just three cameras, while still recovering the sharp, identity-defining details that make a digital human feel real.
Results
We evaluated Skullptor on two publicly available datasets: NPHM, which contains high-quality structured-light scans, and Multiface, which provides lightstage multi-view video captures.
Across both datasets, Skullptor matches photogrammetry-level geometric quality while using a fraction of the input views and completing reconstruction in seconds rather than minutes. It substantially outperforms recent Gaussian splatting-based reconstruction methods (2DGS and SuGaR) on all metrics, including depth accuracy, angular normal error, and preservation of high-frequency surface detail.
We wish to highlight the sparse-view ablation: as the number of input views drops from 23 down to 3, Skullptor degrades gracefully and quality stays high even at three cameras. Classical photogrammetry, however, degrades sharply below 16 views and produces near-unusable results at three.
Qualitative comparisons tell a similar story: Skullptor recovers wrinkles, skin folds, and person-specific surface variation that Gaussian splatting methods cannot reproduce.
Conclusion
Skullptor demonstrates that combining a data-driven normal predictor with a fast inverse rendering optimization closes the gap between the speed of AI foundation models and the quality of dense-view photogrammetry. From fewer than ten cameras, in under 30 seconds, it produces complete, high-fidelity 3D head geometry with wrinkle-level detail.
The current method is designed for controlled capture conditions, synchronized cameras and reasonably consistent lighting. Strong specular reflections, motion blur, or occlusions can affect the final result. Next steps include extending the framework toward full appearance capture, jointly predicting normals and albedo and incorporating lighting estimation to enable relighting.
We're releasing code to support future research. More results, including 4D sequence reconstructions, are available on the project page.