Intelligent Systems

Instant Volumetric Head Avatars

2023

Conference Paper

ncs

ps


We present Instant Volumetric Head Avatars (INSTA),a novel approach for reconstructing photo-realistic digital avatars instantaneously. INSTA models a dynamic neural radiance field based on neural graphics primitives embedded around a parametric face model. Our pipeline is trained on a single monocular RGB portrait video that observes the subject under different expressions and views. While state-of-the-art methods take up to several days to train an avatar, our method can reconstruct a digital avatar in less than 10 minutes on modern GPU hardware, which is orders of magnitude faster than previous solutions. In addition, it allows for the interactive rendering of novel poses and expressions. By leveraging the geometry prior of the underlying parametric face model, we demonstrate that INSTA extrapolates to unseen poses. In quantitative and qualitative studies on various subjects, INSTA outperforms state-of-the-art methods regarding rendering quality and training time.

Author(s): Wojciech Zielonka and Timo Bolkart and Justus Thies
Book Title: IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR)
Year: 2023
Month: June

Department(s): Neural Capture and Synthesis, Perceiving Systems
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Event Name: CVPR 2023
Event Place: Vancouver

Links: pdf
project
video
code
face tracker code
dataset
Video:

BibTex

@inproceedings{INSTA,
  title = {Instant Volumetric Head Avatars},
  author = {Zielonka, Wojciech and Bolkart, Timo and Thies, Justus},
  booktitle = {IEEE/CVF Conf.~on Computer Vision and Pattern Recognition (CVPR)},
  month = jun,
  year = {2023},
  doi = {},
  month_numeric = {6}
}