PointActor: Animatable Human Radiance Field from Monocular Video

Abstract

We present PointActor, a method capable of efficiently generating novel view images from an animated person in a video. PointActor first represents a T-shaped human point cloud in a canonical space and then maps it to an observation space that faithfully characterizes the geometry of the person. Human appearance is represented as a combination of a multi-level hash encoding and a shallow MLP. Contrary to other approaches that use backward mapping, we leverage forward mapping to render free-viewpoint images alongside novel human poses while effectively disentangling complex human gestures such as overlapping limbs. On top of obtaining those natural poses, we further switch from a volumetric rendering approach to a splat rendering approach which, coupled with the point cloud space representation, allows us to perform inference up to 57$\times$ faster than previous methods.