Deploy an edge-compute Machine Vision camera in your projects, using the Firefly DL.
FLIR Systems as a company has been around since the late 70s. They’re the world leader in the design, manufacture, and marketing of thermal imaging infrared cameras.
FLIR as an acronym stands for Forward Looking Infrared. While we might think of infrared cameras being useful for night vision, and that’s certainly true, Infrared imaging has a vast and growing array of applications from defense to recreational boating. More recently they have also found applications in the fight against COVID-19.
When it comes to the Firefly DL, however, Infrared isn’t part of the story. The DL stands for deep learning. Yes, that’s right, this beauty is a camera with onboard deep learning capability for edge-compute image recognition and processing.
INITIAL THOUGHTS
The camera is quite weighty for its size - no doubt due to the hardware onboard. However we note it’s not excessively heavy, just solid and constructed well. It has an onboard Micro USB SuperSpeed connector as you’d usually find on a portable hard drive. However, this one is a locking type, which was a pleasant surprise indeed. It certainly makes sense for a whole stack of applications, and even just when testing, knowing that it’s not going to unplug itself while you reach for a coffee.
The camera is fitted with an S-mount lens. This standardised fitting often seen in small cameras including surveillance equipment, makes it easy to access the range of lenses available to customise for your application. This flexibility is certainly something we’re seeing more of in traditional maker-type products too, such as the new High Quality Raspberry Pi cameras and something that truly provides value in machine vision applications.
USING THE CAMERA
There’s a lot of control options here, and seemingly endless configuration options. Unfortunately, this can create some startup hurdles too.
FLIR has created a software application called SpinView which allows you to get going, to run a bunch of tests on the camera. This is part of the Spinnaker SDK which is used for a range of FLIR cameras. The application is compiled for Windows, Linux, and MacOS environments, which is a great effort at providing cross-platform support.
Unfortunately, after some frustrating testing, we discovered our MacOS version lacked the UI nuance the Windows version did, so it did create some substantial challenges in getting things to work. It also requires manual installation of some software dependencies, which is understandable, however, the software could be a little more elegant in its handling of this, to provide better feedback around what’s going on.
PRELOADED MACHINE VISION MODEL
An important part of the “plug and play” nature of this device is the preloaded model. The Firefly DL comes preloaded with a Cat vs Dog classifier.
You need to pay fairly close attention to the setup of the camera using SpinView. It’s one instance where you really need to pay attention and read the documentation (since many of us disregard it by default).
Once you get the right configuration however, things really start to work rather flawlessly and inference is made quickly and reliably when we show the camera a series of cat / dog pictures.
DEEP LEARNING ON BOARD
Obviously the truly standout feature here is that the Machine Vision is built right into the device. There’s no need for a Raspberry Pi or other processor which we’d normally deploy into this type of environment.
TensorFlow and Caffe deep learning frameworks are supported, so training and deployment of models can be done separately from the device itself. You can even leverage pre-trained models depending on your application and entirely remove the need for training, you simply need to convert them as appropriate.
APPLICATIONS AND DEPLOYMENT
We can see a huge range of deployment benefits here, having a self-contained and robust device. The GPIO provides a handful of output options which can be used to drive other hardware or software triggers (think sorting machines, etc).
The camera unit generates a reasonable amount of heat, but that’s expected. It’s doing substantial processing under the hood, but would be a consideration under deployment.
While it’s not really priced into the maker market for experimentation such as say a Raspberry Pi camera, it’s still certainly affordable for the right applications.
SUMMARY
For the right application, this system will far outperform a camera and processing system (of comparable processing power) and with a mere fraction of the power consumption. While you could theoretically achieve similar results with a Raspberry Pi and suitable camera, you’re dealing with a more complex software overhead and potentially less robust operational platform.
The camera also comes with a robust 3yr warranty, which really speaks to the quality of this unit and how long FLIR expects it to last.
The price point may prohibit it from being explored in the maker market for those just dipping their toes into image processing with Machine Learning. Ultimately however, for someone serious about developing a robust product this camera’s specifications meet, the combination of robustness, speed, and edge compute, makes it hard to look past.
If FLIR decides to push further into the maker world with Machine Vision, it would be great to have a few less hurdles in the “experimentation” phase. If you have knowledge of Machine Learning and C programming, you’ll easily be able to make use of the examples included and get going. If this is your first foray into machine vision you should still manage to get there, but expect to invest a little time.
We have only just scratched the surface of what this camera can do, and for the right applications we see huge potential.
SPECIFICATIONS
- RESOLUTION: 1440 x 1080 @ 60fps
- SENSOR: Sony IMX296 Colour
- LENS TYPE: S-mount
- FLASH MEMORY: 24MB non-volatile
- POWER: 2.2W (5V via USB)
- SIZE: 27 x 27 x 14mm excluding lens
Shopping List
Firely DL Deep Learning Camera: www.flir.com/firefly-dl
- Firefly DL FFY-16S2M-S-DL US$299