Welcome to our instructional guide for inference and realtime DNN vision library for NVIDIA Jetson Nano/TX1/TX2/Xavier NX/AGX Xavier.

This repo uses NVIDIA TensorRT for efficiently deploying neural networks onto the embedded Jetson platform, improving performance and power efficiency using graph optimizations, kernel fusion, and FP16/INT8 precision.

Vision primitives, such as imageNet for image recognition, detectNet for object detection, and segNet for semantic segmentation, inherit from the shared tensorNet object. Examples are provided for streaming from live camera feed and processing images. See the API Reference section for detailed reference documentation of the C++ and Python libraries.

Follow the Hello AI World tutorial for running inference and transfer learning onboard your Jetson, including collecting your own datasets and training your own models. It covers image classification, object detection, and segmentation.

• Hello AI World
• API Reference
• Code Examples
• Pre-Trained Models
• System Requirements
• Change Log

>   Jetson Nano 2GB and JetPack 4.4.1 is now supported in the repo.
>   Try the new Re-training SSD-Mobilenet object detection tutorial!
>   See the Change Log for the latest updates and new features.

Hello AI World can be run completely onboard your Jetson, including inferencing with TensorRT and transfer learning with PyTorch. The inference portion of Hello AI World - which includes coding your own image classification and object detection applications for Python or C++, and live camera demos - can be run on your Jetson in roughly two hours or less, while transfer learning is best left to leave running overnight.

• Setting up Jetson with JetPack
• Running the Docker Container
• Building the Project from Source

• Classifying Images with ImageNet
  • Using the ImageNet Program on Jetson
  • Coding Your Own Image Recognition Program (Python)
  • Coding Your Own Image Recognition Program (C++)
  • Running the Live Camera Recognition Demo
• Locating Objects with DetectNet
  • Detecting Objects from Images
  • Running the Live Camera Detection Demo
  • Coding Your Own Object Detection Program
• Semantic Segmentation with SegNet
  • Segmenting Images from the Command Line
  • Running the Live Camera Segmentation Demo

• Transfer Learning with PyTorch
• Classification/Recognition (ResNet-18)
  • Re-training on the Cat/Dog Dataset
  • Re-training on the PlantCLEF Dataset
  • Collecting your own Classification Datasets
• Object Detection (SSD-Mobilenet)
  • Re-training SSD-Mobilenet
  • Collecting your own Detection Datasets

• Camera Streaming and Multimedia
• Image Manipulation with CUDA
• Deep Learning Nodes for ROS/ROS2

Below are links to reference documentation for the C++ and Python libraries from the repo:

• C++
• Python

These libraries are able to be used in external projects by linking to libjetson-inference and libjetson-utils.

Introductory code walkthroughs of using the library are covered during these steps of the Hello AI World tutorial:

• Coding Your Own Image Recognition Program (Python)
• Coding Your Own Image Recognition Program (C++)

Additional C++ and Python samples for running the networks on static images and live camera streams can be found here:

note: for working with numpy arrays, see Converting to Numpy Arrays and Converting from Numpy Arrays

These examples will automatically be compiled while Building the Project from Source, and are able to run the pre-trained models listed below in addition to custom models provided by the user. Launch each example with --help for usage info.

The project comes with a number of pre-trained models that are available through the Model Downloader tool:

• If the resolution is omitted from the CLI argument, the lowest resolution model is loaded
• Accuracy indicates the pixel classification accuracy across the model's validation dataset
• Performance is measured for GPU FP16 mode with JetPack 4.2.1, nvpmodel 0 (MAX-N)

• Jetson Nano Developer Kit with JetPack 4.2 or newer (Ubuntu 18.04 aarch64).
• Jetson Nano 2GB Developer Kit with JetPack 4.4.1 or newer (Ubuntu 18.04 aarch64).
• Jetson Xavier NX Developer Kit with JetPack 4.4 or newer (Ubuntu 18.04 aarch64).
• Jetson AGX Xavier Developer Kit with JetPack 4.0 or newer (Ubuntu 18.04 aarch64).
• Jetson TX2 Developer Kit with JetPack 3.0 or newer (Ubuntu 16.04 aarch64).
• Jetson TX1 Developer Kit with JetPack 2.3 or newer (Ubuntu 16.04 aarch64).

The Transfer Learning with PyTorch section of the tutorial speaks from the perspective of running PyTorch onboard Jetson for training DNNs, however the same PyTorch code can be used on a PC, server, or cloud instance with an NVIDIA discrete GPU for faster training.

In this area, links and resources for deep learning are listed:

• ros_deep_learning - TensorRT inference ROS nodes
• NVIDIA AI IoT - NVIDIA Jetson GitHub repositories
• Jetson eLinux Wiki - Jetson eLinux Wiki

note: the DIGITS/Caffe tutorial from below is deprecated. It's recommended to follow the Transfer Learning with PyTorch tutorial from Hello AI World.

^{© 2016-2019 NVIDIA | Table of Contents}