An additional part of our project is a hardware component, running our object detection model on a Jetson Nano in real time so that the drone operators can tell if the video they’re collecting is working well with the model, or if they need to adjust the camera settings. This week, we’ve gotten the hardware set up and have begun testing how to run our software on the relevant hardware.
One of the first issues we encountered was power constraints. With the lower level hardware, it often wasn’t able to both power the various peripherals and run complex computations. This was addressed by simply using a Bluetooth keyboard (which has its own battery) saving it enough power to operate normally. The final product will not need to have these peripherals anyway, as the relevant programs will already be loaded onto the device and will only need to run.
With the power concerns addressed, we were able to run a basic face detection algorithm:
As you can see, the video runs quite slowly at the moment due to the processing required to run the machine learning model. To address this, we have a few options including finding ways to simplify the model, or exporting the model to another machine learning framework designed to run on lower level hardware such as TensorFlow Lite.