How to Perform Over-the-Air (OTA) Updates Using the ESP32 Microcontroller and its ESP-IDF
Contributed By Digi-Key's North American Editors
Designers of Internet of Things (IoT) products need to continually evaluate platform and component selection with a view toward decreasing cost and power while improving performance and accelerating the design of connectivity applications. There are currently quite a few solutions to pick from, but designers are faced with the challenge of performing wireless, over-the-air (OTA) updates to keep the device firmware up to date, once deployed.
The key is to look at available platforms to see what additional tools and support they come with to support OTA updates. Such support can greatly simplify the process but may need some attention upfront.
This article discusses OTA fundamentals and why it’s a critical function that nearly every IoT system needs to support, despite the challenges developers face. It then uses Espressif Systems’ ESP32 Bluetooth and Wi-Fi enabled microcontroller, with associated modules, kits, and ESP IoT Development Framework (ESP-IDF), to show how to create an OTA partition and utilize the otatool.py script to perform a firmware update while an application is still running.
Introduction to OTA updates
The core focus for most development teams is implementing their product-specific features, that is, the business logic that differentiates their product. However, every IoT product has a base feature set that needs to be deployed, configured, and maintained throughout the device’s lifetime. Security updates are a good example. Given the need to perform these updates, an important but easily overlooked feature when evaluating a suitable development platform is the bootloader or the firmware OTA (FOTA) update (sometimes just referred to as OTA) capability.
OTA provides engineers with the ability to remotely maintain and upgrade their products in response to both technical and business requirements without the need to send maintenance personnel to the device or have the end client actively do something with the device to update it. Instead, all those costs can be removed by having the devices silently upgrade their firmware in the background, or during operational “downtime” hours like the middle of the night.
OTA architectures can come in many different forms and configurations, from custom-built solutions all the way through cloud-provider supplied standard implementations. A typical architectural example can be seen in Figure 1.
Figure 1: An OTA architectural overview that shows an example process for updating application firmware in the field to deployed devices. (Image source: Beningo Embedded Group)
In this example, an OEM is using Amazon Web Services (AWS) IoT Core to upload new firmware versions and then uses the built-in Job capabilities to deploy updates to devices in the field. This is just one of many examples, and nearly every cloud provider has a similar solution.
There are many microcontroller choices available today that support OTA. One popular microcontroller for both low-cost systems and among makers is the ESP32. There are several reasons why the ESP32 has been so popular, including:
- It has an integrated microcontroller with Wi-Fi/Bluetooth certification modules available
- Low cost
- Open-source development environment and software frameworks such as the ESP-IDF and ESP Audio Development Framework (ESP-ADF)
- Many existing application examples freely available on the web
Selecting an ESP32 module for OTA testing
There are several different ESP32 modules and development boards available that users can purchase to walk through the OTA examples. Take, for example, the Adafruit 3405 ESP32 Huzzah Feather board (Figure 2). This is a low-cost development board that includes all the circuitry to program an ESP32 and power it through a USB connector.
Figure 2: The 3405 Huzzah Feather Board contains an ESP32 WROOM-32D certified Wi-Fi/Bluetooth module with 4 Mbytes of flash. The board includes all the hardware necessary to program and communicate with the module through USB. (Image source: Adafruit)
At the core of the 3405 is an ESP32-WROOM-32D module that comes with 4 Mbytes of flash, Wi-Fi, Bluetooth, and a complete peripheral set for nearly any application.
Figure 3: The ESP32-LYRATD-SYNA board is based on an ESP32 WROVER-B certified Wi-Fi/Bluetooth module with 4 Mbytes of flash. Along with enabling designers to program and communicate with the module through USB, it also has the circuitry needed to develop audio applications. (Image source: Espressif Systems)
The ESP32-LYRATD-SYNA module also has 4 Mbytes of flash, as well as all the circuitry for audio applications. The board includes an audio codec, an audio amplifier, and headphone and speaker jacks to fully test out an audio application.
One last development board that can be used for OTA testing is the Espressif ESP32-S2-SAOLA-1RI development board (Figure 4). When it comes to development boards, this is the least expensive. The board contains an ESP32 Wrover module along with the circuitry to program the chip. There are no frills other than the fact it contains pins that allow it to easily be put into a breadboard for testing.
Figure 4: The ESP32-S2-SAOLA-1RI, based on the Wrover module, is a bare-bones development board that is low-cost but includes enough circuitry to program the on-board module. (Image source: Espressif Systems)
The specific board selected for testing doesn’t matter too much because each ESP32 module leverages the ESP-IDF. This framework is designed to ease software development activities for developers by including drivers, middleware, an RTOS, and—importantly for the purposes of this article—bootloaders, and OTA libraries.
The bootloader allows developers to leverage OTA updates and partition their memory to update firmware while the primary application is still running, which helps minimize downtime. The bootloader setup can appear complicated at first, but it is straightforward if guided properly.
The OTA development workflow
The OTA development workflow for the ESP32 is going to vary slightly based on the business needs and product component selection. For example, a team leveraging AWS will likely use the AWS getting started guides and examples to get their ESP32 OTA solution working. A company that is customizing their own solution on the other hand will likely leverage the ESP32 documentation. In this article, we are going to look at the pieces at the ESP32 level and not in the cloud. The reason is that these pieces are generic and apply to OTA with the ESP32, regardless of which cloud provider or solution is used.
In general, the process to set up an OTA update on the ESP32 involves the following steps:
- Configure the ESP32 partition table
- Download firmware that supports OTA
- Develop a tool to act as a server and push new firmware
- Download the latest firmware onto the ESP32
- Swap to the new application
Obviously, this is the simplified approach. Developers should look at Figure 1 again for a view of the overall firmware update process. This process can be quite involved, so it’s advisable to leverage the existing ESP32 OTA examples located on GitHub. These examples provide several critical examples such as:
- HTTPS OTA
- Native OTA
- Simple OTA
- OTA Tool (python scripts example)
Figure 5 shows the deployment and update process steps. A developer will need to perform the steps in red first to deploy the OTA solution to the ESP32 module. The steps in orange are next and are executed to facilitate an OTA update.
Figure 5: The Espressif Systems OTA update examples located on GitHub provide developers with several simple examples for getting their ESP32 to perform OTA updates. (Image source: Espressif Systems)
Configuring an ESP32 application for OTA
The ESP32 contains a partition table that describes what type of data is located on the microcontroller and where it lives. For example, a standard ESP32 partition table looks something like Table 1:
Table 1: A standard ESP32 partition table showing the type of data and where it is located on the microcontroller. (Table source: Beningo Embedded)
There is the factory application and then a section for the NVS library and the physical layer (PHY) initialization (init) data. In order to use the OTA functionality, this table needs to be updated so that there are memory locations specified for the OTA update firmware, in addition to the primary (factory) application. For OTA, there are typically two partitions that are allocated for updates. One for the active updated firmware, and one for the firmware that is being downloaded and which will become the latest version. This allows the factory application to remain intact. An updated OTA partition table would look something like Table 2.
Table 2: Typical ESP32 updated OTA partition table. (Table source: Beningo Embedded)
As shown, there is now an ota_0 and an ota_1 application section that is 1 Mbyte in size, in addition to a data section (otadata) that is RAM allocated for the update process. This table can be modified and updated by the developer to suit the application.
In order to run the OTA example, there are a simple set of instructions that are listed on GitHub under the “How to use the examples” section. This describes how to build and program the application.
There is also the otatool that can be used to update firmware. This script is typically used to:
- Read, write and erase the OTA partitions
- Switch boot partitions
- Switch to the factory partition
The example script can be executed by simply running the example in a terminal using the command:
Or using Python:
When it comes to configuring the ESP32 for OTA, making sure the partitions are set up is a critical step.
Tips and tricks for using
The EPS32 OTA solution can accelerate and simplify a developer’s firmware update solution. To prevent the solution from becoming a development burden, there are several “tips and tricks” that should be kept in mind:
- If possible, leverage an existing OTA framework included with the company’s cloud provider. This can dramatically simplify development and integration.
- Use a low-cost development board to test OTA capabilities and bootloaders. The ESP32 has several options, and it may take some experimenting to determine which one is best for the application at hand.
- For custom solutions, leverage the ESP32 OTA examples on GitHub.
- For applications where the product acts as the Wi-Fi router or hub, consider downloading the firmware image to external memory and performing an update from a mass storage device.
- Spend some time reviewing the ESP32 documentation on partition tables. This is a little different from the typical microcontroller implementation.
- For security reasons, it is best to disable application rollback. If the application can rollback to previous versions, would-be attackers could potentially push a version with a known exploit and compromise the system.
Developers that follow these “tips and tricks” will find that they save quite a bit of time and grief when attempting to leverage the ESP32, or any other OTA solution, as applicable.
OTA updates are a critical feature for an increasing number of IoT and embedded systems. Developers need to get a good handle on how to do it effectively in order to save time upfront during the design and development process, and after the product has been shipped.
The ESP32 wireless microcontroller has found its way into a wide range of devices, and as shown it has a ready-made OTA solution. By taking advantage of the ESP-IDF and associated modules and platforms, and using some experience-based tips and tricks, developers can dramatically lighten their design time and get their own OTA solution up and running.
Disclaimer: The opinions, beliefs, and viewpoints expressed by the various authors and/or forum participants on this website do not necessarily reflect the opinions, beliefs, and viewpoints of Digi-Key Electronics or official policies of Digi-Key Electronics.