Submission to the Microsoft Azure AI Hackathon 2021

Installation/setup docs coming soon!

HealthEye is an IoT system that allows you to monitor the number of people and social distancing violations in a certain area. It uses a Raspberry Pi and camera system to analyze the number of people and the number of social distancing violations for a specific area. Additionally, it is able to forecast the number of people and social distancing violations for a future time that a user can select.

As proof of concept, we also included the ability for the system to input video files as well as camera feed.

We utilized Azure Custom Vision API to detect people and their location in a video frame. We decided to use the Custom Vision AI rather than the Object Detection API as we wanted to train our model on a diverse data set consisting of images of libraries, classrooms, and outdoor scenery.

In addition to the Custom Vision model, we used the CV2 library to help us estimate the true distance between people given the bounding boxes from the vision model. We used image transformations to help us do so

We used CosmosDB to store our data, and used Azure Functions to abstract the code to write/read from the CosmosDB instance. For the final website, we created it using Python Flask and ReactJS, and deployed as a Azure Web App

We utilized a multivariate time series model (VAR) to predict the number of individuals in a given location and the number of socially distanced violations for a distance threshold that a user could specify. We used data extracted from the CosmosDB database containing the same metrics, extracted from video frames.

Coming soon!

Our originally planned architecture consisted of this:

We envisioned this solution to be scalable and modular, where it was easy to add a new Raspberry Pi (or any new device) to the system and add more data. Azure IoT Hub and IoT Edge offered this type of scalability, and thus we wanted to utilize it to its max potential. After failure to properly install IoT Edge on the Raspberry Pi 4 correctly, we set it as a project for the future (after the hackathon demo). However, we still plan to use it to make this entire thing much more scalable and modular.

We hope to refine our Azure CV model to be more accurate and more accepting of a wider dataset.

We'd like to improve the user experience on the website, as most of our work revolved around the backend, thus resulting in an awkward-to-use interface. We know that if this was going into production, the design and UX would have to drastically improve.