시맨틱 이미지 생성을 사용한 3D 장면 색칠

Abstract

We propose a novel approach to 3D scene painting using a configurable 3D scene layout. Our approach takes a 3D scene with semantic class labels as input and trains a 3D scene painting network that synthesizes color values for the input 3D scene. With the trained painting network, a colored 3D scene can be rendered and manipulated. To teach the painting network without explicit color supervision, we exploit an off-the-shelf 2D semantic image synthesis method. Experiments show that our approach produces images with geometrically correct structures and supports scene manipulation, such as the change of viewpoint, object poses, and painting style. Our approach provides rich controllability on synthesized images in the aspect of 3D geometry.