We show you how easy it is to set up a portable web server and to perform object tracking using an ESP32-CAM with its digital camera and onboard processing.
BUILD TIME: 90 MINUTES
DIFFICULTY RATING: Intermediate
The original Arduino Uno was released over a decade ago in 2010 and remains one of the go-to boards for maker projects with its extensive documentation and support. However, recently we have seen a new generation of microcontrollers emerge with increased performance and connectivity. These new microcontrollers were created to fill the demand for smart IoT devices and advanced maker projects. We have been amazed by just how much is possible with these new microcontrollers including machine learning and audio/visual processing.
These new microcontrollers allow us to create more advanced projects that feature integrated video processing and streaming all in a tiny low-power package which makes it perfect for low-cost IoT devices. The current champion of these connected microcontrollers is the ESP32 series.
The ESP32 is a series of low-cost, low-power chips developed by Espressif Systems and serve as the more powerful successor to the ESP8266. The ESP32 was launched in 2016 and features a dual-core design with WiFi and dual-mode Bluetooth connectivity. The microcontroller features a 32-bit Tensilica Xtensa LX6 microprocessor running at 240 MHz and an ultra-low power co-processor.
The ESP32 module includes 520KB of SRAM and is paired with 4- 6MB of external flash memory for programs. The processor runs on 3.3VDC power and includes support for 802.11 b/g/n WiFi, Bluetooth 4.2, and Bluetooth Low Energy.
THE ESP32-CAM
Several custom boards featuring the ESP32 chip have been developed for a variety of applications. We will be looking at one such board, the ESP32-CAM which has been designed to include a small digital camera that can be streamed over the network along with performing basic onboard processing.
The ESP32-CAM is a compact development board that includes a microSD card slot and camera ribbon cable connector. The ESP32-CAM features the low-cost 1/4" 2MP OV2640 camera which was released in 2005 and boasts a maximum resolution of 1600 x 1200 at 15fps, 800 x 600 at 30fps, or 352 x 288 at 60fps. While the camera sensor is old and only features a limited resolution it can be had for extremely cheap and is well supported.
The ESP32-S found in our ESP32-CAM is manufactured by Ai-Thinker and uses the common ESP32-D0WDQ6 chip which is comparable to the ESP32-WROOM-32 from Espressif.
| Parts Required: | Jaycar | ||
|---|---|---|---|
| 1 x ESP32-CAM Development Board | - | ||
| 1 x FTDI Basic Breakout 3.3/5V Board | - |
| OPTIONAL: (For external power) | Jaycar | ||
|---|---|---|---|
| 1 x Lithium Rechargeable Battery (400mAh) | - | ||
| 1 x PowerBoost 1000 Charger | - |
| OPTIONAL: (For extended wifi range) | Jaycar | ||
|---|---|---|---|
| 1 x Mini Flexible WiFi Antenna | - |
ESP32-CAM Setup:
The ESP32-CAM comes packaged in two parts with the board and camera. The camera attaches to the mainboard by lifting the black flap on the connector up, carefully inserting the camera ribbon cable, then locking it in place by lowering the connector bar.
Programming
The ESP32-CAM doesn’t include any USB connectors for programming to reduce cost and design complexity. To program the onboard ESP32 chip we will need to use a USB to Serial FTDI breakout to connect the processor to our computer.
When selecting an FTDI board be sure to watch out for the voltage used or choose one that has an option to select 3.3V or 5V. The ESP32-CAM supports both 3.3V and 5V programming but we find 5V is more stable, so we will be using that.
To connect the FTDI breakout to the ESP32-CAM, we will use some breadboard jumper cables to make the connections.
We need to connect GND on the FTDI board to GND on the ESP32-CAM, and VCC on the FTDI to 5V on the ESP32-CAM. This will supply 5V power to the board.
To program the board, we need to connect TXO on the FTDI board to U0T on the ESP32-CAM, and RXI on the FTDI to U0R on the ESP32-CAM. Finally, we also need to connect GND and IO0 on the ESP32-CAM to set the board to programming mode.
We will be using the Arduino IDE to program and upload code to the ESP32-CAM. You can download the latest version of the Arduino IDE for Windows, Mac, or Linux from https://www.arduino.cc/en/software.
Once the Arduino IDE is installed, install and set up the ESP32-CAM board. To do this, download ESP32 Add-on.
With the Arduino IDE open, navigate to File > Preferences and locate the Additional Board Managers URLs field. Enter the following URL in the field and select OK: https://dl.espressif.com/dl/package_esp32_index.json
Install the added board manager from the Arduino IDE Boards Manager. Navigate to Tools > Board > Boards Manager, and search for esp32 by Espressif Systems. Once this is installed, it will be ready to connect to our ESP32-CAM
With the FTDI board connected to the ESP32-CAM, connect the FTDI board to your computer with a USB cable. In the Arduino IDE, select your board by navigating to Tools > Board and selecting AI-Thinker ESP32-CAM.
Set CPU Frequency to 240MHz (WiFi/BT), Flash Frequency to 80MHz, and Flash Mode to QIO. Finally, select the correct COM port. You are now ready to upload firmware to the device.
Web Server
We will be installing the expanded ESP32-CAM web server created by easytarget. This upgraded version of the default web server improves speed and code readability to make it easier to upgrade and configure. You can download the program from: https://github.com/easytarget/esp32-cam-webserver
Select Code - Download ZIP, and then unzip onto your computer. Then, rename it from esp32-cam-webserver-master to esp32-cam-webserver.
Open the esp32-cam-webserver.ino file from the Arduino IDE, which will import the required files for the program.
The program is ready to flash at this point in an access-only mode, but you can also configure WiFi and hardware settings to your liking, if desired.
You can configure the program to connect to your WiFi network by editing the configuration file. To change the program settings, open the myconfig.sample.h file from the program directory and replace my_ssid and my_password with your network name and password. You can also edit various other WiFi and hardware settings in this file.
Once the changes are complete, save the file as myconfig.h, overriding the existing file.
To install the web server, you need to have the ESP32-CAM connected to your computer with the FTDI cable and programming jumper. With the correct COM port selected, upload the program. The Arduino IDE will compile the web server program, along with any configuration changes you made, and attempt to upload it to the ESP32-CAM.
Once the program upload is complete, power down the ESP32-CAM by disconnecting the FTDI power.
Remove the programming jumper cable between GND and IO0 on the ESP32-CAM so that the web server program will be run on boot.
Power on the device with the FTDI cable or other power source, and the web server application will start.
Connect to the ESP32-CAM web server by either joining the access point created by the device or finding the local IP address of the device, depending on how you configured the program. If you left the settings as default, the ESP32-CAM will create a WiFi network called “ESP32-CAM-CONNECT” with the password “InsecurePassword”.
Once connected to the network, you can access the interface from http://192.168.4.1/. If the device has been configured to connect to your home network, the interface will be accessible from the ESP32-CAM's local IP address.
The web interface offers simple and full modes. Simple mode only shows the live feed, while full mode allows you to configure various video settings and capture still images.
You can control the integrated flash brightness and set it to auto using the interface. The resolution can also be selected which will impact the frame rate of the video stream. Various other options are available to adjust the image, including brightness, contrast, saturation, and filters.
You can also directly view the live video stream on port 81, which is useful if you want to use the video in another program such as VLC, OBS, or Home Assistant. You could use the live stream as a baby monitor, webcam, or home security monitor using these applications.
Power
The ESP32-CAM can be powered directly from the FTDI board but if you want to switch to a dedicated power supply, there are several options.
The ESP32-CAM supports both 3.3V and 5V power, however, as was the case with programming, 5V is known to be more stable so we will be using that.
The simplest option is to modify a USB cable and solder the power and ground cables directly to the 5V and GND pins on the ESP32-CAM. This cable can then be attached to any USB port or battery bank to provide power to the device. This solution is the quickest and easiest but can prove difficult for beginners depending on the USB cable and their soldering skills.
Portable Power
In our case, we are using a Lithium (LiPo) rechargeable battery with a battery charge board to create a compact portable power solution for our ESP32-CAM.
We chose the Adafruit PowerBoost 1000 Charger board as it offered all the features we needed in a compact board. The PowerBoost 1000 features LiPo charging and discharging circuitry, along with the ability to load share input power between charging the battery and powering the device.
We chose a 400mAh LiPo battery as it is small enough to fit within our camera footprint, while still providing sufficient battery life.
We soldered cables directly to the PowerBoost module to power our ESP32-CAM, along with the LiPo cables. We soldered the two jumper cables to 5V and G on the PowerBoost and 5V and GND on the ESP32. Then, the battery power and ground cables to Bat and GND on the PowerBoost.
Note: You can connect the LiPo with the included connector to reduce the amount of soldering required.
This setup allows the ESP32-CAM to run off the internal battery, a direct USB connection, or a mix of both. You can attach and remove battery and USB power while the camera is operating without loss of power which is great if one power source fails.
This system could be combined with a USB solar panel attached to the PowerBoost input to create a 24x7 camera.
The PowerBoost module can easily handle higher capacity LiPo batteries too if you want to extend the usable battery life of the camera.
External WiFi Antenna
The ESP32-CAM features an integrated WiFi antenna on its PCB, which is fine for use in areas with a strong WiFi signal but falls short for more remote or congested operation. To improve this situation, the ESP32-CAM includes an IPEX antenna connector to attach an external WiFi antenna.
Note: The ESP32-CAM can only use one antenna, so you will need to disable the internal antenna if you wish to use an external one.
Extended Wireless
To switch to an external WiFi antenna, you need to move the small SMD resistor next to the IPEX connector so that it is in the bottom position. The resistor comes installed in the top position by default, which uses the internal antenna.
Simply apply heat to the resistor using a soldering iron, and using some pliers, move it to the correct position. You can also bridge the correct pads with some solder once the resistor has been removed if you are certain this is the configuration you want.
Note: This process can be very difficult if you don’t have expertise in soldering and the required tools, so we recommend staying with the internal antenna unless necessary.
We selected a small flexible WiFi antenna from Adafruit with a 4DBi gain and a 50Ω impedance but any WiFi antenna with a suitable IPEX connector will work fine.
We tested the network signal strength with the internal and external antennas and, in our case, found a significant improvement.
Object Tracking
We explored how to run a basic web server on the ESP32-CAM but if you want to experiment with colour based object tracking and other advanced features, you will need to use a different program.
There is the OpenCV.js web server created by RuiSantosdotme to run basic OpenCV functions on a web interface that will allow us to track an object by colour.
This advanced web interface will allow us to track objects using the ESP32-CAM by setting the colour values of our target value and filtering the image to determine its position in each frame. This position value of the tracked object is then sent back to the ESP32-CAM for integration.
To install the program on our ESP32-CAM, download the firmware from: https://github.com/RuiSantosdotme/ESP32-CAM-Arduino-IDE
Choose the Code - Download ZIP option. Extract the downloaded file and navigate to ESP32-CAM-OpenCV-js. Open ESP32-CAM-OpenCV-js.ino, which will import the necessary files into the Arduino IDE ready for configuration and flashing.
Configure the network settings for your ESP32-CAM by editing lines 20 and 21 with your network name and password. Save the edited file and connect the ESP32-CAM to your computer with the FTDI cable and programming jumper connected. Select upload the program, which will compile and upload to the board. Once the application has been uploaded, remove the programming jumper and reboot the device.
The ESP32-CAM will boot and connect to your network, where it will create the website interface for you to connect to.
You can navigate to the local device IP address in your web browser where you will be greeted with the OpenCV.js webserver. The interface will load OpenCV.js and display a ready message when you can start tracking.
To begin tracking an object using the web interface, select the COLOR DETECTION button, which will load the camera streams. The interface will display the live video stream along with the image mask and canvas. Once the camera is positioned so that it can see the object you want to track on a plain background, you can set up the tracking parameters.
Use the Color Probe sliders to move the selector on the Image Canvas to your object. Once the selector is over your object, note the detected Rp, Gp, and Bp values. Next, adjust the RGB Color Trackbar to ± 20 of these values, which should cause the board to only detect the object in the Image Mask preview. You can also adjust the Quality, Brightness, and Contrast sliders to adjust the result.
Once you have configured the tracking settings and are getting a reliable Image Mask, you can enable TRACKING, which will draw a bounding box around the object on the Image Canvas. The centre coordinates of the object will also be displayed. You can also enable show contour for a more detailed tracking preview.
If you now move the object, you should see it being tracked on the Image canvas along with the XCM and YCM centre point values updating. These values could be integrated with servos attached to the ESP32-CAM to follow an object as it moves throughout a room or to point a light, etc.
Where to from here?
The examples explored in this article are just the tip of the iceberg when it comes to creating projects with the ESP32-CAM. Some more advanced projects you can create with the ESP32-CAM include:
- Saving photos and videos to a microSD card when motion is detected using a PIR sensor.
- Streaming the video feed remotely to the VLC application on your phone or desktop.
- Configuring a security camera in Home Assistant for integration with other smart home items.
- Creating a DIY webcam for use with Zoom, Teams, and Discord calls.
- Attaching some servo motors to create a pan-tilt remotely controllable camera.
- Setting up a solar panel system with batteries and microSD card saving for an off-grid nature camera.