This is code for the paper Neural 3D Mesh Renderer by Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada.

For more details, please visit project page.

This repository only contains the core component and simple examples. Related repositories are:

• Neural Renderer (this repository)
  • Single-image 3D mesh reconstruction
  • 2D-to-3D style transfer
  • 3D DeepDream

sudo python setup.py install

python ./examples/example1.py
python ./examples/example2.py
python ./examples/example3.py
python ./examples/example4.py

Transforming the silhouette of a teapot into a rectangle. The loss function is the difference between the rendered image and the reference image.

Reference image, optimization, and the result.

Matching the color of a teapot with a reference image.

Reference image, result.

The derivative of images with respect to camera pose can be computed through this renderer. In this example the position of the camera is optimized by gradient descent.

From left to right: reference image, initial state, and optimization process.

@InProceedings{kato2018renderer
    title={Neural 3D Mesh Renderer},
    author={Kato, Hiroharu and Ushiku, Yoshitaka and Harada, Tatsuya},
    booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2018}
}