Intelligent Systems

Neural Head Avatars from Monocular RGB Videos

2022

Conference Paper

ncs

We present Neural Head Avatars, a novel neural representation that explicitly models the surface geometry and appearance of an animatable human avatar that can be used for teleconferencing in AR/VR or other applications in the movie or games industry that rely on a digital human. Our representation can be learned from a monocular RGB portrait video that features a range of different expressions and views. Specifically, we propose a hybrid representation consisting of a morphable model for the coarse shape and expressions of the face, and two feed-forward networks, predicting vertex offsets of the underlying mesh as well as a view- and expression-dependent texture. We demonstrate that this representation is able to accurately extrapolate to unseen poses and view points, and generates natural expressions while providing sharp texture details. Compared to previous works on head avatars, our method provides a disentangled shape and appearance model of the complete human head (including hair) that is compatible with the standard graphics pipeline. Moreover, it quantitatively and qualitatively outperforms current state of the art in terms of reconstruction quality and novel-view synthesis.

Author(s): Philip-William Grassal and Malte Prinzler and Titus Leistner and Carsten Rother and Matthias Nießner and Justus Thies
Book Title: 2022 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR)
Pages: 18632--18643
Year: 2022
Department(s): Neural Capture and Synthesis
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference
DOI: 10.1109/CVPR52688.2022.01810
Event Name: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022)
Event Place: New Orleans, Louisiana
Digital: True
State: Published
URL: https://philgras.github.io/neural_head_avatars/neural_head_avatars.html
Links: Code
Video
Video:

BibTex 

@inproceedings{nha2022,
  title = {Neural Head Avatars from Monocular RGB Videos},
  author = {Grassal, Philip-William and Prinzler, Malte and Leistner, Titus and Rother, Carsten and Nießner, Matthias and Thies, Justus},
  booktitle = {2022 IEEE/CVF Conf.~on Computer Vision and Pattern Recognition (CVPR) },
  pages = {18632--18643 },
  year = {2022},
  doi = {10.1109/CVPR52688.2022.01810},
  url = {https://philgras.github.io/neural_head_avatars/neural_head_avatars.html}
}