The Raspberry Pi Urban Mobility Tracker is the simplest way to track and count pedestrians, cyclists, scooters, and vehicles. For more information, see the original blog post [here].

Primary Components

1. Raspberry Pi (ideally v4-b)
2. Raspberry Pi camera (ideally v2)
3. Google Coral Accelerator (Not required, but strongly encouraged)

Secondary Components

1. Ballhead mount: https://www.amazon.com/gp/product/B00DA38C3G
2. Clear lens: https://www.amazon.com/gp/product/B079JW114G
3. Weatherproof enclosure: https://www.amazon.com/gp/product/B005UPAN0W
4. 30000mAh battery: https://www.amazon.com/gp/product/B01M5LKV4T

Notes

• The mounts located in geometry/ are currently represented as stl files which are 3d printer ready. I don't currently have a 3d printer so I used the crowd sourced printing service https://printathing.com/ which yielded great results (kind of a sales pitch, but not really. I just like the service).
• The original FreeCAD file is also included just in case you want to modify the geometry.
• The only cutting necessary is through the plastic case to allow for the lens. This joint should then be sealed using silicone caulk to prevent any moisture from entering.
• All the secondary components listed are just suggestions which worked well for my build. Feel free to use what ever you want.

3D printed mounts	mounts with attached hardware

Final setup (open)	Front (closed)

First install the required dependencies for cv2

sudo apt-get install libhdf5-dev libhdf5-serial-dev libhdf5-103
sudo apt-get install libqtgui4 libqtwebkit4 libqt4-test python3-pyqt5
sudo apt-get install libatlas-base-dev
sudo apt-get install libjasper-dev

Next, create and initialize a virtual environment using virtualenv and python 3.7

sudo apt-get install virtualenv
virtualenv -p python3.7 venv_umt
source venv_umt/bin/activate

Now, install the required python libraries

pip install filterpy
pip install imutils
pip install matplotlib
pip install numpy
pip install pillow
pip isntall python-opencv
pip install scipy
pip install scikit-image
pip install tensorflow

To run models using TensorFlow Lite, you'll need to install the interpreter which can be found [here]. If your on a linux (x86-64) machine running Python 3.7, it's simply the following:

pip3 install https://dl.google.com/coral/python/tflite_runtime-2.1.0.post1-cp37-cp37m-linux_x86_64.whl

If you're planning using a Raspberry Pi running Raspbian Buster make sure to use the Python wheel built for Linux (ARM 64) and Python 3.7 which can achieved with the following:

pip3 install https://dl.google.com/coral/python/tflite_runtime-2.1.0.post1-cp37-cp37m-linux_armv7l.whl

In order to utilize the Coral TPU, the Edge TPU runtime and TensorFlow Lite interpreter library must be installed. The TensorFlow Lite installation is the same as above, and the Edge TPU can be installed by following the directions [here].

Lastly, install this repo with the following:

pip install git+https://github.com/nathanrooy/rpi-urban-mobility-tracker

The default deep learning model is the MobileNet v1 which has been trained on the COCO dataset and quantized for faster performance on edge deployments. Another good model choice is PedNet which is also a quantized MobileNet v1 however, it's been optimized specifically for pedestrians, cyclsts, and vehicles. To use PedNet, simply download it from its repo here: https://github.com/nathanrooy/ped-net or clone it.

git clone https://github.com/nathanrooy/ped-net

Once the model and labels have been downloaded, simply use the modelpath and labelmap flags to specify a non-default model setup. As an example:

umt -camera -modelpath pednet_20200326_tflite_graph.tflite -labelmap labels.txt

Since this code is configured as a cli, everything is accessible via the umt command on your terminal. To run while using the Raspberry Pi camera (or laptop camera) data source run the following:

umt -camera

To run the tracker on an image sequence, append the -imageseq flag followed by a path to the images. Included in this repo are the first 300 frames from the MOT (Multiple Object Tracking Benchmark) Challenge PETS09-S2L1 video. To use them, simply download/clone this repo and cd into the main directory.

umt -imageseq data/images/PETS09-S2L1/

To view the bounding boxes and tracking ability of the system, append the -display flag to view a live feed. Note that this will greatly slow down the fps and is only recommended for testing purposes.

umt -imageseq data/images/PETS09-S2L1/ -display

By default, only the first 10 frames will be processed. To increase or decrease this value, append the -nframes flag followed by an integer value.

umt -imageseq data/images/PETS09-S2L1/ -display -nframes 20

To persist the image frames and detections, use the -save flag. Saved images are then available in the output/ directory.

umt -imageseq data/images/PETS09-S2L1/ -save -nframes 20

To run the tracker using a video file input, append the -video flag followed by a path to the video file. Included in this repo are two video clips of vehicle traffic.

umt -video data/videos/highway_01.mp4

In certain instances, you may want to override the default object detection threshold (default=0.5). To accompish this, append the -threshold flag followed by a float value in the range of [0,1]. A value closer to one will yield fewer detections with higher certainty while a value closer to zero will result in more detections with lower certainty. It's usually better to error on the side of lower certainty since these objects can always be filtered out during post processing.

umt -video data/videos/highway_01.mp4 -display -nframes 100 -threshold 0.4

To get the highest fps possible, append the -tpu flag to use the Coral USB Accelerator for inferencing.

umt -imageseq data/images/PETS09-S2L1/ -tpu

@inproceedings{Wojke2017simple,
  title={Simple Online and Realtime Tracking with a Deep Association Metric},
  author={Wojke, Nicolai and Bewley, Alex and Paulus, Dietrich},
  booktitle={2017 IEEE International Conference on Image Processing (ICIP)},
  year={2017},
  pages={3645--3649},
  organization={IEEE},
  doi={10.1109/ICIP.2017.8296962}
}

@inproceedings{Wojke2018deep,
  title={Deep Cosine Metric Learning for Person Re-identification},
  author={Wojke, Nicolai and Bewley, Alex},
  booktitle={2018 IEEE Winter Conference on Applications of Computer Vision (WACV)},
  year={2018},
  pages={748--756},
  organization={IEEE},
  doi={10.1109/WACV.2018.00087}
}

Additionally, the following repo was referenced during the DeepSORT implementation:

https://github.com/theAIGuysCode/yolov4-deepsort