This paper proposes an uncalibrated photometric stereo method for non-Lambertian scenes based on deep learning. Unlike previous approaches that heavily rely on assumptions of specific reflectances and light source distributions, our method is able to determine both shape and light directions of a scene with unknown arbitrary reflectances observed under unknown varying light directions. To achieve this goal, we propose a two-stage deep learning architecture, called SDPS-Net, which can effectively take advantage of intermediate supervision, resulting in reduced learning difficulty compared to a single-stage model. Experiments on both synthetic and real datasets show that our proposed approach significantly outperforms previous uncalibrated photometric stereo methods.

The network architecture of SDPS-Net is composed of (a) Lighting Calibration Network and (b) Normal Estimation Network. Kernel sizes for all convolutional layers are 3 × 3, and values above the layers indicate the number of feature channels.

We adopted the publicly available synthetic PS Blobby and Sculpture datasets for training. Blobby and Sculpture datasets provide surfaces with complex normal distributions and diverse materials from MERL dataset. Effects of cast shadow and inter-reflection were considered during rendering using the physically based raytracer Mitsuba. There are 85,212 samples in total. Each sample was rendered under 64 distinct light directions sampled from the upper-hemisphere with uniform light intensity, resulting in 5,453,568 images (85,212 X 64). The rendered images have a dimension of 128 X 128.

Up to Down: Sample Input, Object, GT Normal, Results of SDPS-Net, and Error Map

Up to Down: Sample Input, Object, Results of SDPS-Net and UPS-FCN

Webpage template borrowed from Split-Brain Autoencoders, CVPR 2017.