Using the ESP32-C3 as an AT modem on the Aludel Elixir

We’re preparing to do some testing around power consumption of different services on the Aludel Elixir, our open source hardware with the Nordic nRF9160 and the Espressif ESP32-C3. More specifically, we’re hoping to benchmark the time and energy it takes to acquire position and balance the energy usage versus the provided accuracy. As with anything in engineering, there are tradeoffs.

As I went to set up a new board to start pulling Wi-Fi access points like we did on the $2 Geofence article, I realized I didn’t document the process of loading that firmware onto the module. Long ago we wrote about loading ESP-AT onto an Espressif devboard, but with a module on a custom board, the process changes slightly.

What is ESP-AT

ESP-AT is a firmware binary offered by our friends at Espressif. It enables a standard set of AT commands to control the capabilities of the underlying ESP32 modems. Many of these are the same across different models of their modems, which is a helpful standardization when upgrading or moving across product lines. The underlying firmware that builds the binary is also available, so you can add custom commands and create your own images if needed.

We have used the ESP-AT image regularly in our hardware-in-the-loop (HIL) testing for the Golioth Firmware SDK, notably to give the nRF52840 access to the internet via Wi-Fi. This was a key consideration when adding the ESP32-C3 module to the Aludel Elixir board (as opposed to another Wi-Fi offering). Since the last post, we have also shown that the Wi-Fi modem can be a fallback when the Cellular capabilities of the nRF9160 are turned off. This was a driving force to adding a secondary Wi-Fi modem to our custom hardware for conference demos. We also benefited from being able to scan Access Points (APs) for our new location service.

How the modem is hooked up

We separated out the pins from initial programming (UART0) from the pins we would use to talk to the modem in AT mode (UART1). The latter in the default image are GPIO6/GPIO7 on the ESP32-C3-MINI module. This Hardware Connection page for the ESP-AT guide was helpful when designing the board.

We decided for Revision B of the Elixir not to use UART0 so that we didn’t need to maintain our own build of ESP-AT, so that it aligned with UART0 and UART1 being separated. For future revisions of the board, we want to ensure we have the ability to jumper over the serial connections or switch back to using UART0 for both functions like we did on RevA and then build a custom version of ESP-AT.

Programming and Testing

When it comes to actually programming the image, it’s pretty simple.

In the photo below, we use press fit connectors and some clamps to get a temporary connection to the broken out pins. We decided not to populate the pin headers because they don’t provide much user value aside from additional GPIO (the nRF9160 is constrained).  In a pinch, I have also used dupont cables held up to these pins to program, but it is not production worthy (nor should I have ever admitted as much in writing). In production, a simple jig with pogo pins would serve this purpose well.

These wires are being sourced from a USB-to-serial dongle, I often use ones based on Silicon Labs parts. You’ll need to know which serial port shows up when the dongle is plugged in, like /dev/ttyUSB0. Windows and Mac will have different formats, but you’ll need to know which port to target in later steps.

Aludel Elixir Rev B being programmed via ESPtool (courtesy of Chris Wilson)

Once your hardware is connected, you’ll need Python 3.7+ and pip installed on your machine. Then to install esptool.py, you type:

pip install esptool

For more detailed instructions and troubleshooting, check out the Espressif docs page

Running esptool.py from the command line should look something like this:

Now cross-connect your wires (TX from dongle wires to RX on the Elixir, and the reverse). Here’s another view of the 8 pin header where the pins will connect:

The red overlays are which wires from your USB-to-serial should plug into which pins

Next is to download the latest image from Espressif, if using the stock image (we are). Go to the Released Firmware page for your version of the ESP-AT image to download the zip. Always note in the URL or on the left side of the page that you’re targeting the correct device (in our case ESP32-C3) as there are significant differences between the versions. While esptool.py will check your image against your device, it’s always important to not risk loading errant firmware. At the time of this post, the latest firmware is v3.3.0.0, so we would replace {{version}} above with that number.

I’m going to open a terminal at the location of the file:

unzip ESP32-C3-MINI-1-AT-{{VERSION}}.zip
cd ESP32-C3-MINI-1-AT-{{VERSION}}/ESP32-C3-MINI-1-AT-{{VERSION}}

This will put us into the directory where we can access the esp-at.bin package.

To put the device in bootloader mode, hold down the MODE and RESET buttons, and then release RESET while still holding down MODE. When the device wakes up from reset and sees IO9 being held, it switches into the flash-based bootloader mode and will accept new images over UART0.

Finally, on your computer run the following lengthy command, replacing {{SERIAL_PORT}}with the address of your USB-to-serial dongle (mine is at /dev/ttyUSB0 when nothing else is plugged in).

esptool.py --chip auto --port /dev/{{SERIAL_PORT}} --baud 115200 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size 4MB 0x0 bootloader/bootloader.bin 0x60000 esp-at.bin 0x8000 partition_table/partition-table.bin 0xd000 ota_data_initial.bin 0x1e000 at_customize.bin 0x1f000 customized_partitions/mfg_nvs.bin

Successful programming will look like this:

Now we are ready to start sending commands from the nRF9160 over the ESP32-C3’s UART1 port, including querying Wi-Fi access points and using it as a secondary modem in the case of lack of cellular coverage.

We will be following up with more power measurement using this offloaded modem to scan for Wi-Fi, and the true costs of powering up a secondary modem for simple tasks. If you’re interested in our private alpha for locations services, please reach out via the announcement blog post. For any questions or troubleshooting with ESP-AT, we can always be found via our forum.

/by
,

Adding The Golioth Firmware SDK to Your ESP-IDF Project

One of the most common questions we hear from our community is how to add Golioth to existing ESP-IDF projects. Until recently, our go-to resource was a blog post that walked users through the process. While the fundamentals of adding Golioth to CMake-based projects remain the same, that guide has become outdated due to significant updates to the Golioth API. Naturally, many users have noticed this and have pointed out areas where the older guide no longer applies.

The most notable change is the evolution of the Golioth Firmware SDK. We’ve made significant improvements, including combining with our previously standalone Zephyr SDK, making it a unified SDK for multiple platforms. These changes make it easier for developers to switch between frameworks, leverage common tools, and maintain consistency across IoT projects. However, they also mean some of the specifics in the original ESP-IDF blog post no longer align with the latest practices.

For example, in step 4 of the original post, users were instructed to include the golioth.h header file to access the API functions. That file no longer exists, so if you’re using the latest version of the Golioth Firmware SDK, following the original guide will lead to confusion and build errors.

Another limitation of the original post is that it doesn’t address the needs of users who are new to CMake. If you’re just starting with ESP-IDF and want to leverage the common example code we provide in our SDK, the older guide won’t be of much help.

We’re here to remedy that. Whether you’re updating an existing project or starting fresh, this updated post will guide you step-by-step through the process of adding Golioth to your ESP-IDF project.

Getting Started with Golioth Integration

This guide focuses on adding Golioth to an existing ESP-IDF application, so we’re assuming you already have the ESP-IDF development environment set up on your system. If not, we recommend following the ESP-IDF Quickstart Guide to get up and running before proceeding.

For this walkthrough, we’ll use the /esp-idf/examples/wifi/getting_started/station example code as our starting point. This example provides a solid foundation for adding Golioth functionality. To begin, copy the example project to a new folder, which we’ll call esp_hello_golioth:

cp -r ~/esp-idf/examples/wifi/getting_started/station esp_hello_golioth

Step 1: Adding Golioth as a Component

To integrate Golioth into your project, we first need to include the Golioth Firmware SDK as a submodule. This allows you to pull in the latest SDK features while keeping your project organized. However, for this step to work, your project must be under version control. If it isn’t already, initialize a Git repository:

cd ~/esp_hello_golioth
git init

Next, navigate to your project directory and add the Golioth Firmware SDK as a submodule. Run the following commands to include the SDK and ensure all its dependencies are updated:

git submodule add https://github.com/golioth/golioth-firmware-sdk.git submodules/golioth-firmware-sdk
git submodule update --init --recursive

After running these commands, you’ll find the Golioth Firmware SDK located in the submodules folder, specifically pulled from the SDK’s main branch.

Working with Specific SDK Releases

While using the main branch ensures you’re working with the latest code, it’s often better to lock your project to a specific SDK release for stability and consistency. To do this, you can checkout a specific tag or release version after adding the submodule:

cd submodules/golioth-firmware-sdk
git checkout <release-tag>
git submodule update --init --recursive

Replace <release-tag> with the version of the SDK you wish to use, such as v0.15.0. This ensures your project always uses a stable and tested version of the Golioth SDK, avoiding potential issues with breaking changes or experimental updates.

With the Golioth SDK installed as a component, we are ready to move on to the project configuration.

Step 2: Configuring CMake for the Golioth SDK

To integrate the Golioth SDK into your ESP-IDF project, we need to modify the CMake configuration files to ensure the SDK is included during the build process. This involves updating the top-level CMakeLists.txt file and the CMakeLists.txt file in the main directory.

Add the Golioth SDK to the Component Directories

First, we need to tell CMake where to find the Golioth SDK components. Open the project’s top-level CMakeLists.txt file and add the following line before the project() directive:

list(APPEND EXTRA_COMPONENT_DIRS submodules/golioth-firmware-sdk/port/esp_idf/components)

This informs CMake to include the Golioth SDK’s ESP-IDF components, which are located in the submodules/golioth-firmware-sdk/port/esp_idf/components folder.

Specify Golioth as a Dependency

Next, we need to declare the Golioth SDK as a dependency in your project’s main/CMakeLists.txt file. For the ESP-IDF example code we’re using, there are no existing dependencies defined. To add Golioth, insert the following at the top of the main/CMakeLists.txt file:

set(deps
"golioth_sdk"
)

This step ensures that CMake properly resolves the Golioth SDK as a required component during the build process.

Why These Changes Matter

By making these adjustments, we’re effectively linking the Golioth SDK into your project’s build system. This allows you to call Golioth APIs and ensures that the SDK is correctly compiled along with the rest of your application.

Now, let’s move on to configuring the application!

Step 3: Configuring Kconfig Settings

To ensure the Golioth SDK functions correctly, we need to enable specific configuration options using ESP-IDF’s Kconfig system. These settings enable essential features like Mbed-TLS and Golioth-specific options. Instead of modifying configurations manually each time, we’ll replace the contents of the originalsdkconfig.defaults file in the top-level project folder with the following:

CONFIG_ESP_WIFI_SOFTAP_SUPPORT=n
CONFIG_MBEDTLS_SSL_PROTO_DTLS=y
CONFIG_MBEDTLS_PSK_MODES=y
CONFIG_MBEDTLS_KEY_EXCHANGE_PSK=y
CONFIG_MBEDTLS_TLS_CLIENT_ONLY=y
CONFIG_LWIP_NETBUF_RECVINFO=y
CONFIG_FREERTOS_USE_TRACE_FACILITY=y
CONFIG_FREERTOS_USE_STATS_FORMATTING_FUNCTIONS=y
CONFIG_COMPILER_OPTIMIZATION_SIZE=y
CONFIG_GOLIOTH_AUTO_LOG_TO_CLOUD=y
CONFIG_GOLIOTH_COAP_REQUEST_QUEUE_MAX_ITEMS=20

Explanation of Key Settings

  • Mbed-TLS Configuration
    • Golioth relies on Mbed-TLS for secure communication. The first four CONFIG_MBEDTLS_ options enable support for DTLS, PSK modes, and client-only TLS.
  • Networking Enhancements
    • CONFIG_LWIP_NETBUF_RECVINFO enables additional networking features needed by Golioth, while the FreeRTOS settings CONFIG_FREERTOS_USE_TRACE_FACILITY and CONFIG_FREERTOS_USE_STATS_FORMATTING_FUNCTIONS help monitor and debug system performance.
  • Wi-Fi Adjustment
    • Since this example focuses on station mode, we disable unnecessary SoftAP support with CONFIG_ESP_WIFI_SOFTAP_SUPPORT=n.
  • Code Optimization
    • The setting CONFIG_COMPILER_OPTIMIZATION_SIZE=y ensures the code is optimized for size, which is often critical for IoT devices.
  • Golioth-Specific Settings
    • CONFIG_GOLIOTH_AUTO_LOG_TO_CLOUD enables automatic uploading of GLTH_LOGX log statements to the Golioth Logs service.
      CONFIG_GOLIOTH_COAP_REQUEST_QUEUE_MAX_ITEMS sets the maximum number of queued CoAP requests for smooth operation.

Why These Settings Matter

The configurations in sdkconfig.defaults ensure that all necessary features are enabled and optimized for using Golioth with ESP-IDF. The final two symbols are particularly useful for automatically uploading log messages to the Golioth Logs service, making debugging and monitoring easier.

With this step complete, your project is now configured to build with Golioth’s essential components.

Step 4: Using Golioth API Calls in Your Code

Now that the Golioth Firmware SDK is configured in your project, you can start incorporating its functionality into your application. The first step is to include the appropriate header file in your code to provide access to Golioth’s API, enabling communication with Golioth services:

#include <golioth/client.h>

For demonstration purposes, we’ll create a simple counter that runs in an infinite loop. Every five seconds, the counter value will be sent to Golioth as a Log message. Below is the modified app_main function, with the new code starting at line 14:

void app_main(void)
{
    //Initialize NVS
    esp_err_t ret = nvs_flash_init();
    if (ret == ESP_ERR_NVS_NO_FREE_PAGES || ret == ESP_ERR_NVS_NEW_VERSION_FOUND) {
        ESP_ERROR_CHECK(nvs_flash_erase());
        ret = nvs_flash_init();
    }
    ESP_ERROR_CHECK(ret);
    ESP_LOGI(TAG, "ESP_WIFI_MODE_STA");
    wifi_init_sta();

    //Everything above this line is unchanged from the ESP-IDF example code
    const char* psk_id = "your-golioth@psk-id";
    const char* psk = "your-golioth-psk";

    const struct golioth_client_config config = {
        .credentials = {
        .auth_type = GOLIOTH_TLS_AUTH_TYPE_PSK,
        .psk = {
            .psk_id = psk_id,
            .psk_id_len = strlen(psk_id),
            .psk = psk,
            .psk_len = strlen(psk),
    }}};

    struct golioth_client *client = golioth_client_create(&config);
    assert(client);

    uint16_t counter = 0;
    while(1) {
        GLTH_LOGI(TAG, "Hello, Golioth! #%d", counter);
        ++counter;
        vTaskDelay(5000 / portTICK_PERIOD_MS);
    }
}

For simplicity, in this example, we’ll hardcode the credentials (PSK-ID and PSK) directly in the application code. While this approach is sufficient for demonstration purposes, it’s never recommended for production use. In the next blog post, we’ll explore how to securely store these credentials using the ESP-IDF common code from the Firmware SDK to set them in the NVS (Non-Volatile Storage) subsystem.

If you’re following along, the next step is to set up your WiFi credentials. You can do this in one of two ways:

  1. Use menuconfig to configure the WiFi settings interactively
  2. Modify the default values directly in the main/Kconfig.projbuild file

Viewing the Results on the Golioth Console

Once your application is running, you can observe its behavior through the serial output. This output confirms that the Golioth client has successfully connected to the platform and is sending log messages as expected. Here’s a sample snippet of what the serial output might look like:

I (896) wifi:AP's beacon interval = 102400 us, DTIM period = 1
I (3316) esp_netif_handlers: sta ip: 192.168.8.113, mask: 255.255.255.0, gw: 192.168.8.1
I (3316) wifi station: got ip:192.168.8.113
I (3316) wifi station: connected to ap SSID:mpuric password:abcdeedcba
I (3326) golioth_mbox: Mbox created, bufsize: 2520, num_items: 20, item_size: 120
I (3336) wifi station: Hello, Golioth! #0
I (3396) golioth_coap_client_libcoap: Start CoAP session with host: coaps://coap.golioth.io
I (3396) golioth_coap_client_libcoap: Session PSK-ID: 20240403080418-esp32@dev
I (3406) libcoap: Setting PSK key

I (3416) golioth_coap_client_libcoap: Entering CoAP I/O loop
I (4376) golioth_coap_client_libcoap: Golioth CoAP client connected
I (8336) wifi station: Hello, Golioth! #1
I (13336) wifi station: Hello, Golioth! #2
I (18336) wifi station: Hello, Golioth! #3
I (23336) wifi station: Hello, Golioth! #4

From the log messages, you’ll see the counter value incrementing every five seconds, just as we implemented in the code. This data is now available in the Golioth Console:

What’s Next?

In this demo, we’ve covered the basics of integrating Golioth into an ESP-IDF project and sending log messages. In the next blog post, we’ll walk through using and adapting the common example code from the Golioth Firmware SDK to speed up your development process!

/by
,

Golioth Firmware SDK v0.16.0

Golioth Firmware SDK v0.16.0

Yesterday we released v0.16.0 of the Golioth Firmware SDK. This release includes a number of improvements, which are described in the following sections. Importantly, this release also introduces the Golioth Root X1 root CA certificate, which Golioth device services will start using 1 year from today. Golioth users are encouraged to update to their firmware to the v0.16.0 release at their earliest convenience.

For the full set of changes in this release, see the changelog.

Golioth Root X1

Note: this update does not impact devices that are using pre-shared keys (PSKs) for authentication. Golioth does not recommend the use of PSKs in production.

Golioth device services have always relied on Let’s Encrypt for server certificates. Let’s Encrypt is a fantastic service that has contributed to making a more secure internet for all. We plan to continue using Let’s Encrypt for the foreseeable future for the Golioth Management API and all of our web properties.

Let’s Encrypt is a public Certificate Authority (CA). The benefit of using certificates issued by a public CA is that its root certificates are widely trusted. That trust is what allows your browser to securely connect to this website. Operating systems, browsers, and other applications include a set of root CA certificates in their root store (see Chromium’s list for example), meaning that they will trust leaf certificates that have a chain of trust rooted in one of those root CA certificates. When someone wants to serve content on the public internet, they can request a leaf certificate from a service like Let’s Encrypt after completing an ACME challenge verifying ownership of their domain. Because Let’s Encrypt’s root CA certificates are included in almost all root stores, clients will automatically trust that site without requiring any manual intervention.

While this system underpins the security of the entire internet, the value of widely trusted certificates is less applicable to embedded devices. Typically, root CA certificates are loaded into a secure element or baked into firmware by the manufacturer of the device or the developer of its firmware. This is in stark contrast to buying a PC with a pre-installed operating system or downloading a web browser.

Embedded devices also differ in their constraints. With limited flash and memory, every root CA certificate and supported cipher suite cuts into precious resources. Furthermore, many of the devices are deployed in hard-to-access locations, meaning that losing remote connectivity can be fatal. Extreme care must be taken to ensure that devices are designed to continue communicating securely in perpetuity. One key aspect of developing a strategy is ensuring robust support for Over-the-Air (OTA) updates, which Golioth makes simple and straightforward.

As part of our ongoing commitment to our users we have evaluated all the ways in which changes in the trust chain for certificates used by Golioth device services could negatively impact devices in the field. For example, a slight change in the cipher suite leveraged by an intermediate certificate could render a device unable to securely connect to the platform if the new cipher suite was not enabled in its firmware. While public CAs like Let’s Encrypt typically communicate changes with ample lead time, by using their certificates we, and in turn our users, are subject to their policies and procedures (as well as any future changes to them). This constitutes a level of risk that we are not comfortable with.

For this reason, the v0.16.0 release of the Golioth Firmware SDK includes the Golioth Root X1 root CA certificate alongside Let’s Encrypt’s ISRG Root X1 root. It also formally starts the transition period for devices connecting to the Golioth platform to move to including Golioth Root X1 in their root store. The simplest way to accomplish this is by upgrading to the v0.16.0 release for your next firmware update. We have set the transition period to 1 year from today, but as always, we will work with all current and future users to ensure that the transition process is seamless.

If you have any questions please do not hesitate to reach out to us on the forum.

Asynchronous Callback Status Handling

All asynchronous callbacks now have both a status member and a coap_rsp_code member to replace the response member. All of the same information remains accessible, but the updated structure supports more granular error handling in callbacks, such as performing operations when a request is unable to be sent. These callbacks are now also invoked when requests are canceled prior to receiving a response, which may be the case if the Golioth client is manually stopped or connection is lost.

Callback function signatures must be updated and accesses to response->status should be updated to status.

OTA Download Resume

golioth_ota_download_component() has a new uint32_t *next_block_idx parameter. This parameter can be used to specify an offset when downloading an artifact. This is particularly useful if an artifact download was interrupted and it is desirable to resume from the last successful transfer.

Set next_block_idx to NULL to use previous functionality in existing code.

Zephyr and NCS Version Updates

This release also includes an update to Zephyr’s new major version release, v4.0.0. There is also corresponding support for v2.8.0 of the Nordic Connect SDK (NCS).

What’s on the Horizon?

Recent updates in the Golioth Firmware SDK have focused on improved OTA stability, and that focus will continue into the coming releases. We are also excited about coming support for new Golioth device services, which will greatly expand the capabilities of devices communicating with the platform.

/by
,

Built on Golioth: The Aludel Lute Relay Smart Locker

When building IoT applications for a simple application space, it pays to build on top of already working systems. That’s the case for the hardware we’re showing today, the Aludel Lute Relay.

This design started as an experiment for a new form factor to plug into our Aludel Elixir Board. The unit has 4 relays capable of switching 20 amps AC or DC. This unit could be useful in remote physical access (i.e. locks), lighting control, or the application that inspired this design: smart lockers. Let’s dive into how this is built and how someone could use this latest Golioth Solutions Marketplace design to build a business.

New form factor

The Aludel platform generally assumes that you will have a somewhat tall case that can fit two different Mikroelektronika Click Boards into the case and then cover it with our Open Source board called the Aludel Ostentus. The Aludel Elixir is the main processing board and contains an nRF9160 primary processor and cellular modem and an ESP32-C3 secondary modem for Wi-Fi and Bluetooth.

If I wanted to build custom hardware to work with the Elixir board, I would probably need to make a Click board of my own. But that is somewhat space constrained. Instead, I decided to take the idea of using a PCB as a cover for this form factor (as is the case for the Ostentus) and instead put the customized components on this new ‘front panel’. The result is the Lute form factor, which can be extended beyond the current implementation. Relays are a great proving ground for large industrial-style components that can control large industrial sized machines.

 

Control Power from Anywhere

This hardware is a great fit for applications where you want to remotely control power. The 4 relays on board and the large 5.08 mm pitch terminal blocks means you can safely pass high currents through the board. The traces on the relay are configured for creepage and clearance required for 240V. The relays being used are marked for 120V, but the datasheet says they are capable of 240V.

This application draws on my own experience building a simple smart locker using Golioth. I built it using an nRF9160-based dev board and then point-to-point wired a bunch of off-the-shelf power controls. It worked great, as the control requirements are low: nothing more than a GPIO is required for output. However, from a manufacturing capability and scaling to a larger fleet of devices, the wiring would have been a horrendous experience. The Lute Relay board codifies the ability to switch power and enables it in a much more compact package. In my case, I was passing 12V through the relays to a set of LED lamps inside the locker and to the solenoid controlling the door latch.

The relays truly are the easy part. The important piece in the system starts from the GPIO (controlling the relays) and goes all the way back to the cloud. The firmware is built on top of Golioth’s Reference Design Template. We trigger the various GPIOs within app_settings.c, which is triggered from the Golioth Settings Service. When we change the relay setting on the console, it gets pushed out from the Golioth service down to the device over CoAP, and then the Golioth Firmware SDK translates the packet and triggers the proper GPIO.

Made for extensibility, not durability

The Lute Form factor, like most of the devices in the Solutions Marketplace, is a starting point, not an off-the-shelf product. We intend for you to take these devices to prove an idea and then modify them for the specifics of your business. You can hire Golioth Solutions Services to help you with that, tap one of our wonderful Design Partners, or take on the task yourself.

What about future Lute boards? Well, the testing of this board has created a great starting point for future designs and applications. If you have one in mind, get in touch via our forum and let us know what you’d like to see us build next.

Watch the Livestream playback

We live streamed the entire creation flow of this board. You can watch it below.

/by
,

Zephyr for Hardware Engineers: Changing Boot Configurations

As a hardware engineer, Zephyr can be a bit confusing at first. I am used to tweaking registers, not layering file upon file. Today we’ll talk about how those layers and overlays can help you to configure your device differently than the default board startup sequence.

Devicetree (Re)Introduction

There are two pieces to the configuration of any board in Zephyr: KConfig and Devicetree. The former I think of like a control panel, toggling switches on and off to tell the West build system whether or not to include something. I recently did this to enable the i2c shell on a project, which pulled in code that might be unnecessary for production but helped me troubleshoot a problem. The latter (Devicetree) describes the various elements that make up your board, SoC, and architecture. As I said above, these are really layered on top of one another, which is a source of confusion for new Zephyr developers.

The topmost level of the devicetree is the board setting, and this is where I find myself making changes most often. If you include a boards/<boardname>.overlay file in your application, the build system will pull in those changes and give priority to them during the build. We often write about how we use this for swapping out a sensor, say on the i2c bus, and then add commands to control that sensor in the application code, like in main.c or app_sensors.c. Super useful!

Another key piece of knowledge for newcomers is that Zephyr has many development boards and products included in the overall repository. I find this to be critical for quickly trying out a new board and having sensible defaults for the built in Zephyr samples. Without these, I would be required to build a sample up from scratch, and the last thing I want to be doing is configuring clock trees and digging through register sheets just to blink an LED for the first time.

But with the good, comes the bad. The defaults of the board, aren’t always what we want. And in today’s example, those defaults were put there by…us!

Why not just change the board file?

When you’re in control of the entire board definition, this can make sense. But it’s not the default I would go for. Much like we won’t dive down into the SDK and change some fundamental aspect of the RTOS or a plugin, I wouldn’t normally go and change the board file. It changes the behavior on your computer for your future self and also it might not be tracked in your application’s repo, which could cause issues when building on different systems.

An Example in Action

For this example we’re looking at the Aludel Elixir board running the Reference Design Template. For the RD Template we employ a separate golioth zephyr boards repository called out in the west manifest of the Template. What this means in practice is we control an ‘official’ copy of the Zephyr board files for this hardware we make (a useful tip for your organization’s needs). In the aludel_elixir_common.dtsi file you can see that we call out the behavior of the 3v3 and 5v regulators onboard. This was the subject of the post about Enabling Power Regulators at Boot.

Well, guess who changed his mind? (me)

Now that I want to not enable the regulators at boot. Normally, this would be a simple overlay for a characteristic, say the delay times:

&reg_mikrobus_3v3 {
    startup-delay-us = <5000>;    /* 5ms - adjust as needed */
    off-on-delay-us = <5000>;     /* 5ms - adjust as needed */
};

But in the case of regulator-boot-on, it’s simply a property that is included or not, so there’s no way to set it to true or false. Instead, I need to delete it entirely. In my overlay, it would look like this:

&reg_mikrobus_3v3 {
    /delete-property/ regulator-boot-on;
};

Now when I build with west, the board won’t enable that regulator automatically, but I’ll still be able to configure it during runtime. This is exactly what we did using the Golioth Settings service in our recent Joulescope demo, which happened to be the genesis of this article/topic.

Deletion can also happen at the node level. This occurs when you want to change the behavior or the existence of some element of your configuration. For instance, say I wanted to change the button configuration for the mode_button in the Aludel Elixir board file:

/* Replace a button with different GPIO configuration */
/delete-node/ &mode_button;
mode_button: mode-button {
    gpios = <&gpio0 12 (GPIO_PULL_DOWN | GPIO_ACTIVE_HIGH)>;
    /* additional properties */
};

This would allow me to keep the tooling in the rest of my code but potentially target a different behavior of that particular button. Say some irresponsible hardware engineer ripped a trace off the board and the only setup that would work after repairing it requires a new pin configuration; that would never happen with me, though, right?

Learning to climb the (Device)tree

Despite my trepidation about configuring the devicetree for a project, it really enables a lot of configuration tweaks, which suits the needs of hardware engineers everywhere. Learning how to modify your devicetree overlays to enable new capabilities will unlock the ability to prototype new parts at a moment’s notice. When you’re ready to send that data to the cloud, simply pop on the Golioth Firmware SDK and send some data through friendly device-side APIs. If you get stuck, we’re always happy to help over on the Golioth Forum. Happy hacking!

/by
,

Always Use The Correct FW Version with Golioth and Zephyr

Firmware versioning is a crucial part of ensuring the binary you’re about to load onto your device is the correct arrangement of bits that will control the hardware how you want. It is fundamental to a system like Golioth’s Over-The-Air update service. Today we’re going to look at different ways that you can tell which version you’re running in Zephyr and how your firmware update is correct the first time and every time. We’ll use Golioth Reference Designs as an approximation of what a product design would do for the devices in their fleet.

How we generate firmware versions

Around these parts, we adhere to Semantic Versioning, or SemVer. It’s a useful way to assign incremental changes to a firmware image. It also matches how Zephyr increments versions, and how the Golioth SDK is versioned. So…it’s a good idea to have any binaries we create for our Reference Designs to also have a SemVer attached. As you can see in the video, we also able to detect versions and show interesting elements when the images adhere to SemVer. We expect our users to follow a similar format (especially if using MCUboot), though Golioth is flexible across different version methods.

For Reference Designs, the firmware version is reported through the bootloader; MCUboot in the case of our Zephyr Designs. That means when the image is being loaded by MCUboot, it is reading the version embedded in the image.

Previously, we needed to do this manually. At some point in the generation of a firmware binary, the firmware engineer needs to decide this is the version that will be released as 1.2.3 (or whatever number is next in your internal process). That’s where things can get tricky, because humans are often in the loop. Perhaps we have a commit in Git like 1a2b3c4dand we want to attach the version 1.2.3 to it. Well, we would do that on the command line… a place I notoriously mess things up! See more in the last section (historical footnote) for more info.

We changed how we do things in Reference Designs starting in v2.4.0 of the RD Template. We now use a controlled VERSION file, following the guidance of the Zephyr Application Version Management system. This has been working out great. We store the file in the repository and it gets incremented (and tracked!) like any other file.

Where can I see it?

OK, so we have the version number embedded into the binary, but how do I actually see it? We can do this in a couple different places: the log output, the MCUboot shell, all the Golioth console.

Application load and boot

When built on top of nRF Connect SDK and Zephyr, you can see logging output for the version in our main application code. This application also shows the firmware version of the modem and the attached Ostentus display firmware (when present).

The MCUboot shell

This shell is not enabled by default on the Reference Design Template, so I turned it on by adding CONFIG_MCUBOOT_SHELL=y to prj.conf in my file. This gives me a new shell menu that shows my regions. Super useful when trying to troubleshoot OTA issues.

On Golioth’s Devices -> Firmware Tab

Part of the OTA code in the Golioth SDK is that it reports back the current version and the current state of OTA, so you can see where you are in the midst of an update. This is a great place to see the last reported version. The same information is available on the Summary tab of the same page.

On Golioth’s Firmware Updates -> Cohorts Tab

This shows the history of packages, which will include the one currently running

On Golioth’s Firmware Updates -> Packages Tab

This is where you upload new images to be included in future deployments and where Golioth extracts the MCUboot version and uses it as another check against existing images on the server. We also point out when you’re trying to upload a similar image you have uploaded before.

Keeping your deployments organized

Attaching a SemVer to your binary won’t solve all your problems, but it will help you organize the range of firmware images you’re sending out to your devices. Golioth helps to check your versions on the cloud and seamlessly deliver the package to eligible devices. If you have questions about how to build or deploy your next firmware image, let us know in the forum!

 

Historical Footnote

When we used to have to call out the image version on the command line, it was a mess. I recall a scenario where I made a change to my firmware and re-used the last build command which also happened to have the version number attached. But that’s because we’re not actually using a controlled document to set the version. Sure, I might have a tag in GitHub that says 1.2.3, but there’s no mandate for the firmware to adhere to that so I needed to do that manually before. Note that there is a --which escapes the west build command and then the next DCONFIG command, which actually sent the version to CMake to include in the build.

west build -p -b nrf9160dk_nrf9160_ns samples/dfu -- -DCONFIG_MCUBOOT_IMAGE_VERSION=\"1.2.3\"

Note: the above code will not work for modern versions of Zephyr

/by
, ,

Improve Electronic Design with Module Abstraction Layers

What if you kept the “core” or the “essence” of a board the same throughout your designs and only changed the peripherals? You could build a range of products that had similar processing capabilities, but might be completely different products. By defining an abstraction layer at the edge of the core module you design, it’s possible to achieve efficiencies for your variety of products, including when those products are manufactured over time.

The above video is a talk given at the Zephyr meetup hosted by NXP at Embedded World North America in October of 2024. I focus on modular hardware design for scalability and adaptability. The Drachm (pronounced “dram”) module is something we have been regularly streaming on YouTube. This talk describes some of the motivations, the challenges of unconstrained designs, and how to build modular systems that take advantage of Zephyr RTOS capabilities.

Many SKUs, One Core

The Drachm module was conceived to be an extension of the Aludel Platform. We currently target a range of different verticals and end-applications with Golioth Reference Designs. But these are somewhat bulky: made to accept MikroBus Click boards, the case measures roughly 80 x 80 x 40 mm. What if we want to go smaller, but not re-design the board every time we have a new product idea?

The Drachm module was meant to achieve this on multiple fronts. The castellated edge design would allow us to have a standard offering that covers a range of peripheral we want to hook up to the core module. But the module itself can change as well. The different versions of the Drachm module offer different capabilities of communication, processing power, and peripheral access on the underlying silicon. As in many modular systems, we trade off flexibility for an overall reduced feature set.

The Challenge of Hardware Changes

Modularity at the module connector edge benefits long term manufacturing as well. Building scalable IoT hardware requires constant adaptability. Component shortages, changing device requirements, and new tech trends demand flexible designs. Traditional methods, such as swapping processors or upgrading communications modules, can be tedious and impractical without a modular system.

How it works in Zephyr 

This talk built on top of the article we published about abstract hardware interfaces in Zephyr. The key change is how we think of each component of the design. The module is being mounted to a carrier board, so it seems like a component of the carrier board’s BOM and software system. But in fact, we need to think about the carrier board as a “shield” (in Zephyr/Arduino) parlance, and instead map pins based on how we define the module edge (castellated pins). See the linked article on how to use predefined node labels and GPIO nexus nodes to achieve those ends.

Another key piece is that each module variant would need to maintain its own board files (Devicetree and Kconfig) to match up the pins on the castellated edge connection to the silicon onboard. The “analog input 1” pin on the drachm-1 variant (nRF9151 + ESP32-C3) would have a different mapping than the drachm-2 variant (nRF52480 + BG77), but to the parts on the shield / carrier board, it’d be all the same, since they only interface via the “analog input 1” nomenclature.

How it fits with Golioth

Golioth works great with customized hardware, which is subject to the harsh realities of sourcing. Engineers who need to redesign their hardware to upgrade or even just keep their production line running don’t want to re-do every aspect of their design, including firmware and cloud software. That’s why engineers are increasingly choosing Zephyr to target many hardware SKUs and why they’re choosing Golioth to help create a seamless experience for those variety of targets.

Using Golioth’s features like Blueprints, Tags, and our new OTA Cohorts capability an engineer could deliver different main firmware updates to boards using drachm-1 or drachm-n modules, but still have a consistent set of AI models being delivered to all eligible units. Golioth can also help to target different cohorts in your fleet to ensure the right updates are going to the right devices in the field.

Want to create a more resilient fleet? We can help! Our Solutions Engineering team can help to introduce more modularity and have a robust OTA strategy for your fleet. If you’re interested in doing it yourself but you’d like to talk through the options with Golioth team members, head over to our forum to start a conversation there.

Slides

Golioth-Module-Abstraction-Layers-Zephyr-Meetup-Oct-9th-2024
/by
, ,

Built on Golioth: A Cellular Data Logger by AL2TECH

Today we are highlighting the AL2LOG, a cellular data logger by Golioth Design Partner AL2TECH. This is the latest entry into the Golioth Solutions Marketplace, announced last week. This device is a great way to capture a wide range of input data from an industrial setting, especially when you don’t know every type of measurement you want to take at the start of your project. You can deploy an AL2LOG to your factory floor or rugged environment and start monitoring signals and processing data on the cloud.

The AL2LOG is a general purpose logger

Let’s take a look at some of the specifications and use cases for the AL2LOG.

The case and PCB of the AL2LOG cellular data logger

A range of input options

The name of the game with the AL2LOG is flexiblility. That’s why you can interface it with the following signals from all types of sensors

  • GPIO
    • Digital Input
    • Dry Contact
    • Pulse Counter
  • CAN
  • RS-485
  • 4-20 mA
  • 0-10 V

Industrial users will recognize many of those signal types. Golioth has some Reference Designs that include more bespoke monitors targeted at specific verticals, like the CAN Asset Tracker and the MODBUS (RS-485) Monitor.

When you need to send control signals out into the world, you can do so with a programmable
power output (5 to 12V) for devices downrange of the logger, open drain outputs, as well as a 220VAC smart switch.

The device is powered off of two massive D-cell batteries for a 10 year battery life, but can also be powered from external sources when you want an even longer lifetime (or higher power utilization).

Built for speed, but not high power drain

While this design is based around the nRF9160 LTE-M / GPS / NB-IoT modem, it also relies on a higher speed ADC for advanced data capture. This Ultra Low Power acquisition and logging subsystem utilizes a true 14 bit ADC, and can efficiently capture high resolution data and cache it before sending it back to the Golioth Cloud. Utilizing Golioth’s new Batch Pipeline Destination, it’s super easy to bundle up a slug of readings and send them to the cloud to be unpacked later. Read about how Mike showcased sending data in batches.

A Case Study on AL2TECH’s Client Work

We recently profiled AL2TECH and their work with Hubwater, also seen in the video above. Hubwater is reducing waste and improving water distribution at commercial locations in Italy and beyond.

Read the full case study of AL2TECH and Hubwater

Some key points from that case study include how quickly they were able to get up and running using Golioth, and how they get to continually offer new services to their clients by utilizing Golioth Pipelines and OTA.

More Solutions Coming Soon

This is only the second of many solutions in our Golioth Solutions Marketplace. We are excited to share additional work of our Design Partners and Golioth Reference Designs. If you have a solution you’d like to share with Golioth and the world, please get in touch!

/by
,

All Golioth Hardware is Now Open Source

Golioth is all about sharing ideas and knowledge. This now includes our hardware, which today has been released as open source under the CERN OHL-P license. The Aludel Elixir and Ostentus are available on GitHub as KiCad projects and you can use them to start building your own designs.

The Aludel Platform

As an IoT platform that doesn’t build or sell commercial hardware, we still need to find ways to highlight the capabilities our platform offers. This often includes building on top of development boards from our various partners. Engineers are used to starting from well known vendor platforms, like some of our favorites which are in the list of Continuously Verified Boards (CVBs) on the Golioth platform:

When you’re ready to start building devices that look more like real-world products, however, it becomes a bit bulky to use the easy-to-access development platforms. We knew we wanted to capture this “magic in a box”, which was the genesis of the name — an “aludel” was a sublimating pot in alchemy.

We started using alternative boards like the nRF9160 Feather from Circuit Dojo and encapsulating the Feather form factor in a box. The early Aludel boxes were much larger and included headers to plug in different boards. Over time, we knew we wanted to more closely approximate designs that will go out into the field, so we downsized to the current ‘Mini’ version that utilizes the Bud Industries CU-1937-MB Utilibox and then we started modifying it and adding 3D printing elements. That formed the basis of the new Aludel platform, which includes two boards we’re releasing today.

Elixir


The Elixir Hardware Repository is now open source on GitHub

The Elixir board is a natural offshoot of the Aludel Mini form factor, including the new case we adopted. We had first created a board called the aludel-mini which you’ll see mentioned in some of our reference design repositories. This was an interposer board between a soldered down nRF9160 Feather and some mikroBUS Click headers. However, we started to experience some limitations with how the board operated and wanting to add other features. This is when we planned to build the Elixir.

The Elixir has many of the same elements from the OSHW Feather board, and adds on new capabilities that are common to many of our designs:

  • nRF9160 – Cat M1 / NB-IOT / GPS chip with dual core Cortex M33 processing
  • ESP32-C3 – Running ESP-AT firmware, interfaced to the nRF9160 over UART
  • BME280 – Weather sensor from Bosch, common to many Golioth designs
  • Power
    • Low quiescent buck circuit, inherited from Feather Board
    • 5V boost to service the 5V output on the mikroBUS
    • Battery charging
    • Fuel gauge
    • High voltage input (5.5V to 48V) for powering from things like OBD-II and industrial supplies
  • LIS2DH – Accelerometer, inherited from Feather board
  • PWM Buzzer – Allow audible indication similar to the Thingy91
  • Real Time Clock
  • Secure Element (ATECC608B)
  • Power Switching – Power switch of 3V3 for sensor nodes to shut down in low power situations
  • QWIIC / STEMMA Headers – External sensors and peripherals including the Aludel Ostentus
  • mikroBUS headers
  • USB C interface and power
  • External SIM connector
  • Multiple programming interfaces
    • 10 pin SWD (02×05 .127mm pitch), accessible from outside the case
    • 10 pin SWD Tag-Connect (accessible from the top side of the board)
    • USB using the serial bootloader / MCUBoot

Ostentus


The Ostentus Hardware Repository is now open source on GitHub

Early Reference Designs were simply aludel-mini boards in the Utilicase. We utilized vinyl stickers to differentiate between designs, but this didn’t have the impact we were looking for when we went to tradeshows or shared photos on our projects site. What’s more, it didn’t represent a product that might be on-site at an industrial facility (more on the ‘field readiness’ in a bit).

If you walk up to a sensor monitor at a plant, you likely don’t want to pull out a phone and require a network in order to interact with the device in front of you. Having a simple, power efficient, persistent display that is visible in high brightness environments felt really useful. In fact, our early design hypothesis is that you’d still want to have readings on the screen, which is why the Ostentus continues displaying data even when the QWIIC header has been powered down from the main board (in low power modes). That trick is handles using ePaper displays.

Over different iterations we added other capabilities to allow user interactivity. Capacitive touch buttons around the bezel of the screen and an accelerometer for sensing double tap actions through the case deliver user input in the field.

  • Raspberry Pi Pico – Originally chosen during the part shortages of 2022-23, the RP2040 on the Pico provides a low cost solution including a programming interface over the USB port + MicroPython ease-of-use
  • ePaper interface – This design was derived from the Pimoroni Badger 2040 and that had a reference circuit for interfacing to a Good Display 1.54in square ePaper display (not cheap!)
  • Accelerometer (LIS2DH12) — Currently unimplemented but targeting “double tap” behaviors in the future
  • 3 button capacitive touch controller (CAP-1203-1)
  • QWIIC header for incoming power and i2c
  • Downward firing LEDs

Two critical pieces of firmware power this part of the platform. We utilized i2c in order to cut down on the number of I/O required on the main board (in this case the Elixir) and also to abstract it for future boards that might differ from the Elixir:

License and certification

These designs are being released under the CERN Open Hardware License – Permissive, version 2. This is one of the most permissive hardware licenses available. This board has not yet been certified by OSHWA, but we believe the hardware and files comply with the OHSWA definition. No FCC, CE, or cellular carrier certification is guaranteed with these boards and users should expect to take any resulting product through the proper regulatory channels.

Field Readiness

One thing to note is that the Ostentus plus the cut-out version of the Bud Industries case results in an unrealistic deployment option for the platform. I would never feel comfortable sending these boards out into a harsh environment, let alone a slightly damp one, due to the fact that there are gaps in the case and there is no IP rating. However, we didn’t design it for high reliability or even production: the Aludel platform is truly to showcase the widest variety of use cases and maximum flexibility.

This is why you’ll often hear us discussing how this is an “80% done design”. We expect our users to take these designs and shrink them, harden them for environments, and extend them for specific use cases. This could also include doing rounds of Design for Manufacturing (DfM), doing cost optimization, and improving testing for high volume production. Also important are taking the designs through FCC Part 15B certification and any required cellular carrier certification (which depends on which carrier you use and their requirements).

All of those things are left to you, dear reader. We followed the best design practices that we know, though we are always open to learning more and receiving feedback on our forum and issues filed on our GitHub repositories. You can start to evaluate whether this custom hardware is the right path by starting from our range of Follow-Along Hardware and open source firmware on the Golioth Reference Design Project site.

/by
, ,

How to Send IoT Data in Batches

IoT is all about data. How you choose to handle sending that data over the network can have a large impact on your bandwidth and power budgets. Golioth includes the ability to batch upload streaming data, which is great for cached readings that allows your device to stay in low power mode for more of the time. Today I’ll detail how to send IoT data in batches.

What is Batch Data?

Batch data simply means one payload that encompasses multiple sensors readings.

[
    {
        "ts": 1719592181,
        "counter": 330
    },
    {
        "ts": 1719592186,
        "counter": 331
    },
    {
        "ts": 1719592191,
        "counter": 332
    }
]

The example above shows three readings, each passing a counter value the represents a sensor reading, along with a timestamp for when that reading was taken.

Sending Batch Data from an IoT Device

The sample firmware can be found at the end of the post, but generally speaking, the device doesn’t need to do anything different to send batch data. The key is to format the data as a list of readings, whether you’re sending JSON or CBOR.

int err = golioth_stream_set_async(client,
                                   "",
                                   GOLIOTH_CONTENT_TYPE_JSON,
                                   buf,
                                   strlen(buf),
                                   async_push_handler,
                                   NULL);

We call the Stream data API above. The client and data type are passed as the first two arguments, then the buffer holding the data and the buffer length are supplied. The last two parameters are a callback function and an optional user data pointer.

Routing Batch Data using a Golioth Pipeline

Batch data will be automatically sorted out by the Golioth servers based on the pipeline you use.

filter:
  path: "*"
  content_type: application/json
steps:
  - name: step0
    destination:
      type: batch
      version: v1
  - name: step1
    destination:
      type: lightdb-stream
      version: v1

This example pipeline listens for JSON data coming in on any stream path. In step0 it “unpacks” the batch data into individual readings. In step1 the individual readings are routed to Golioth’s LightDB stream. Here’s what that looks like:

Note that all three readings are coming in with the same server-side timestamp. The device timestamp is preserved in the data, but you can also use Pipelines to tell Golioth to use the embedded timestamps.

Batch Data with Timestamp Extract

For this example we’re using a very similar pipeline, with one additional transformer to extract the timestamp from the readings and use it as the LightDB Stream timestamp.

filter:
  path: "*"
  content_type: application/json
steps:
  - name: step0
    destination:
      type: batch
      version: v1
  - name: step1
    transformer:
      type: extract-timestamp
      version: v1
    destination:
      type: lightdb-stream
      version: v1

Note that we didn’t even need an additional step, but simply added the transformer to the step that already set lightdb-stream as the destination.

You can see that the Linux epoch formatted timestamp has been popped out of the data and assigned to the LightDB timestamp. Extracting the timestamp is not unique to Golioth’s LightDB Stream service.

Streaming data may be routed anywhere you want it. For instance, if you wanted to send your data to a webhook, just use the webhook destination. If you included the extract-timestamp transformer, you data will arrive at the webhook with the timestamps from your device as part of the metadata instead of nested in the JSON.object.

Using a Special Path for Batch Data

What happens if your app wants to send other types of streaming data beyond batch data? The batch destination will automatically drop data that isn’t a list of data objects. But you might like to be more explicit about where you send data and for that you can easily create a path to receive batch data.

filter:
  path: "/batch/"
  content_type: application/json
steps:
  - name: step0
    destination:
      type: batch
      version: v1
  - name: step1
    transformer:
      type: extract-timestamp
      version: v1
    destination:
      type: lightdb-stream
      version: v1

This pipeline is nearly the same as before with the only change on line 2 where the * wildcard was removed from path and replaced with "/batch/". Now we can update the API call in the device firmware to target that path:

int err = golioth_stream_set_async(client,
                                   "batch",
                                   GOLIOTH_CONTENT_TYPE_JSON,
                                   buf,
                                   strlen(buf),
                                   async_push_handler,
                                   NULL);

Although the result hasn’t changed, this does make the intent of the firmware more clear, and it differentiates the intent of this pipeline from others.

Sample Firmware

This is a quick sample firmware I made to use while writing this post. It targets the nrf9160dk. One major caveat is that the function that pulls time from the cellular network is quite rudimentary and should be replaced on anything that you plan to use in production.

To try it out, start from the Golioth Hello sample and replace the main.c file. This post was written using v0.14.0 of the Golioth Firmware SDK.

Wrapping Up

Batch data upload is a common request in the IoT realm. Golioth has not only the ability to sort out your batch data uploads, but to route them where you want and even to transform that data as needed. If you want to know more about what Pipelines brings to the party, check out the Pipelines announcement post.

/by