Leveraging DevOps for Streamlined Firmware Delivery

As embedded developers, we’re consistently seeking ways to make our processes more efficient and our teams more collaborative. The magic ingredient? DevOps.Its origins stem from the need to break down silos between development (Dev) and operations (Ops) teams. DevOps fosters greater collaboration and introduces innovative processes and tools to deliver high-quality software and products more efficiently.

In this talk, we will explore how we can bring the benefits of DevOps into the world of IoT. We will focus on using GitHub Actions for continuous integration and delivery (CI/CD) while also touching on how physical device operations like shipping & logistics which can be streamlined using a DevOps approach.

Understanding the importance of DevOps in IoT is crucial to unlocking efficiencies and streamlining processes across any organization that manages connected devices. This talk, originally given at the 2023 Embedded Online Conference (EOC), serves as one of the many specialized talks freely accessible on the EOC site.

GitHub Actions for IoT

To illustrate how to put these concepts into practice, we’re going to look at a demo using an ESP32 with a feather board and Grove sensors for air quality monitoring. It’s important to note that while we utilize GitHub Actions in this instance, other CI tools like Jenkins or CircleCI can also be effectively used in similar contexts based on your team’s needs and preferences.

For this example we use GitHub Actions to automate the build and deployment process.

The two main components of our GitHub Actions workflow are ‘build’ and ‘deploy’ jobs. The ‘build’ job uses the pre-built GitHub Action for ESP-IDF to compile our code, and is triggered when a new tag is pushed or when a pull request is made. The ‘deploy’ job installs the Golioth CLI, authenticates with Golioth, uploads our firmware artifact, and releases that artifact to a set of devices over-the-air (OTA).

Imagine an organization that manages a fleet of remote air quality monitors across multiple cities. This GitHub Actions workflow triggers the build and deployment process automatically when the development team integrates new features or bug fixes into the main branch and tags the version. The updated firmware is then released and deployed to all connected air quality monitors, regardless of their location, with no additional logistics or manual intervention required. This continuous integration and deployment allows the organization to respond rapidly to changes and ensures that the monitors always operate with the latest updates.

Let’s delve into the GitHub Actions workflow and walk through each stage:

1. Trigger: The workflow is activated when a new tag is pushed or a pull request is created.
   Plain text

   Copy to clipboard

   Open code in new window

   EnlighterJS 3 Syntax Highlighter

   on:

   push:

   # Publish semver tags as releases.

   tags: [ 'v*.*.*' ]

   pull_request:

   branches: [ main ]

   on: push: # Publish semver tags as releases. tags: [ 'v*.*.*' ] pull_request: branches: [ main ]

   on:
     push:
       # Publish semver tags as releases.
       tags: [ 'v*.*.*' ]
     pull_request:
       branches: [ main ]

2. Build: The workflow checks out the repository, builds the firmware using the ESP-IDF GitHub Action, and stores the built firmware artifact.
   Plain text

   Copy to clipboard

   Open code in new window

   EnlighterJS 3 Syntax Highlighter

   jobs:

   build:

   runs-on: ubuntu-latest

   steps:

   - name: Checkout repo

   uses: actions/checkout@v3

   with:

   submodules: 'recursive'

   - name: esp-idf build

   uses: espressif/esp-idf-ci-action@v1

   with:

   esp_idf_version: v4.4.4

   target: esp32

   path: './'

   env:

   WIFI_SSID: ${{ secrets.WIFI_SSID }}

   WIFI_PASS: ${{ secrets.WIFI_PASS }}

   PSK_ID: ${{ secrets.PSK_ID }}

   PSK: ${{ secrets.PSK }}

   - name: store built artifact

   uses: actions/upload-artifact@v3

   with:

   name: firmware.bin

   path: build/esp-air-quality-monitor.bin

   jobs: build: runs-on: ubuntu-latest steps: - name: Checkout repo uses: actions/checkout@v3 with: submodules: 'recursive' - name: esp-idf build uses: espressif/esp-idf-ci-action@v1 with: esp_idf_version: v4.4.4 target: esp32 path: './' env: WIFI_SSID: ${{ secrets.WIFI_SSID }} WIFI_PASS: ${{ secrets.WIFI_PASS }} PSK_ID: ${{ secrets.PSK_ID }} PSK: ${{ secrets.PSK }} - name: store built artifact uses: actions/upload-artifact@v3 with: name: firmware.bin path: build/esp-air-quality-monitor.bin

   jobs:
     build:
       runs-on: ubuntu-latest
       steps:
       - name: Checkout repo
         uses: actions/checkout@v3
         with:
           submodules: 'recursive'
       - name: esp-idf build
         uses: espressif/esp-idf-ci-action@v1
         with:
           esp_idf_version: v4.4.4
           target: esp32
           path: './'
         env:
           WIFI_SSID: ${{ secrets.WIFI_SSID }}
           WIFI_PASS: ${{ secrets.WIFI_PASS }}
           PSK_ID: ${{ secrets.PSK_ID }}
           PSK: ${{ secrets.PSK }}
       - name: store built artifact
         uses: actions/upload-artifact@v3
         with:
           name: firmware.bin
           path: build/esp-air-quality-monitor.bin

3. Deploy: The workflow installs the Golioth CLI, authenticates with Golioth, downloads the built firmware artifact, and uploads it to Golioth for OTA updates.
   Plain text

   Copy to clipboard

   Open code in new window

   EnlighterJS 3 Syntax Highlighter

   jobs:

   build:

   runs-on: ubuntu-latest

   steps:

   - name: Checkout repo

   uses: actions/checkout@v3

   with:

   submodules: 'recursive'

   - name: esp-idf build

   uses: espressif/esp-idf-ci-action@v1

   with:

   esp_idf_version: v4.4.4

   target: esp32

   path: './'

   env:

   WIFI_SSID: ${{ secrets.WIFI_SSID }}

   WIFI_PASS: ${{ secrets.WIFI_PASS }}

   PSK_ID: ${{ secrets.PSK_ID }}

   PSK: ${{ secrets.PSK }}

   - name: store built artifact

   uses: actions/upload-artifact@v3

   with:

   name: firmware.bin

   path: build/esp-air-quality-monitor.bin

   jobs: build: runs-on: ubuntu-latest steps: - name: Checkout repo uses: actions/checkout@v3 with: submodules: 'recursive' - name: esp-idf build uses: espressif/esp-idf-ci-action@v1 with: esp_idf_version: v4.4.4 target: esp32 path: './' env: WIFI_SSID: ${{ secrets.WIFI_SSID }} WIFI_PASS: ${{ secrets.WIFI_PASS }} PSK_ID: ${{ secrets.PSK_ID }} PSK: ${{ secrets.PSK }} - name: store built artifact uses: actions/upload-artifact@v3 with: name: firmware.bin path: build/esp-air-quality-monitor.bin

   jobs:
     build:
       runs-on: ubuntu-latest
       steps:
       - name: Checkout repo
         uses: actions/checkout@v3
         with:
           submodules: 'recursive'
       - name: esp-idf build
         uses: espressif/esp-idf-ci-action@v1
         with:
           esp_idf_version: v4.4.4
           target: esp32
           path: './'
         env:
           WIFI_SSID: ${{ secrets.WIFI_SSID }}
           WIFI_PASS: ${{ secrets.WIFI_PASS }}
           PSK_ID: ${{ secrets.PSK_ID }}
           PSK: ${{ secrets.PSK }}
       - name: store built artifact
         uses: actions/upload-artifact@v3
         with:
           name: firmware.bin
           path: build/esp-air-quality-monitor.bin

For those eager to dive in and start implementing DevOps into their own IoT development process, we’ve provided an example Github Actions workflow file on GitHub. Feel free to fork this repository and use it as a starting point for streamlining your own IoT firmware development process. Remember, the best way to learn is by doing. So, get your hands dirty, experiment, iterate, and innovate. If you ever need help or want to share your experiences, please reach out in our our community forum.