Coarse-to-Fine:
Facial Structure Editing of Portrait Images
via Latent Space Classifications

ACM SIGGRAPH 2021
1State Key Lab of CAD&CG, Zhejiang University, 2ZJU-Tencent Game and Intelligent Graphics InnovationTechnology Joint Lab, 3University of Bath
teaser

Our method can generate a new facial structure without double chin
while consistently leaving other regions unchanged.

Abstract

Facial structure editing of portrait images is challenging given the facial variety, the lack of ground-truth, the necessity of jointly adjusting color and shape, and the requirement of no visual artifacts.

In this paper, we investigate how to perform chin editing as a case study of editing facial structures. We present a novel method that can automatically remove the double chin effect in portrait images. Our core idea is to train a fine classification boundary in the latent space of the portrait images. This can be used to edit the chin appearance by manipulating the latent code of the input portrait image while preserving the original portrait features. To achieve such a fine separation boundary, we employ a carefully designed training stage based on latent codes of paired synthetic images with and without a double chin. In the testing stage, our method can automatically handle portrait images with only a refinement to subtle misalignment before and after double chin editing.

Our model enables alteration to the neck region of the input portrait image while keeping other regions unchanged, and guarantees the rationality of neck structure and the consistency of facial characteristics. To the best of our knowledge, this presents the first effort towards an effective application for editing double chins. We validate the efficacy and efficiency of our approach through extensive experiments and user studies.

Video

Pipeline

The pipeline of our model.

Pipeline

Animation

Interpolating states

By changing the hyper-parameter in our method, we can further quantify the degree of double chin removal as show in the video.

Interpolate start reference image.

Start Frame

Loading...
Interpolation end reference image.

End Frame


BibTeX

@article{10.1145/3450626.3459814,
author = {Wu, Yiqian and Yang, Yong-Liang and Xiao, Qinjie and Jin, Xiaogang},
title = {Coarse-to-fine: facial structure editing of portrait images via latent space classifications},
year = {2021},
issue_date = {August 2021},
publisher = {Association for Computing Machinery},
volume = {40},
number = {4},
issn = {0730-0301},
journal = {ACM Trans. Graph.},
month = {jul},
articleno = {46},
numpages = {13},
}