Disentangling Facial Expressions and Identities through Deep Learning

In this article, we present a new method for reconstructing detailed 3D faces using neural networks. Our approach combines the advantages of implicit neural representations (INRs) with a novel blending technique to create an efficient and accurate model.
Firstly, we explain how INRs can be used to build a sophisticated nonlinear 3D face mesh model. Essentially, these networks learn to fit a continuous function that represents surfaces through level-sets. This allows for the encoding of dynamic high-frequency details within an established template space.
Next, we introduce the blending technique, which softly partitions the entire facial surface into distinct regions. A lightweight module called the Fusion Network is then used to adaptively blend these regions, resulting in a comprehensive Neural Blend-Field. This allows for more accurate and detailed reconstruction of 3D faces compared to previous methods.
To illustrate how our method works, we provide an example of a face scan being reconstructed using the Neural Blend-Field. The process involves automatically learning deformations linked to identity and expression while simultaneously establishing correspondences between different individuals. It then learns to encode dynamic high-frequency details within the established template space.
In summary, our approach leverages INRs to build a detailed nonlinear 3D face mesh model that can efficiently represent facial geometry. The Neural Blend-Field technique enables more accurate and comprehensive reconstruction of 3D faces by adaptively blending local field functions. This method has significant potential in various applications, including 3D avatars, virtual try-on, and facial analysis.