Google Summer of Code: Zephyr + Arduino

The Arduino ecosystem is ever-growing, and thanks to the excellent work of Dhruva Gole it’s coming to Zephyr as well.

For the 2022 Google Summer of Code with Golioth, Dhruva took on the challenge integrating the Arduino core with Zephyr RTOS as the base. Now the program is nearing completion and we’ve been excitedly building and running Arduino sketches that can directly call Zephyr libraries and subsystems. And of course, the cross-platform nature of Zephyr means there are suddenly many more boards that can run that Arduino code.

What is GSoC

Google Summer of Code is a program that focuses on bringing new contributions to Open Source. The program matches up participants (often students but not necessarily) with mentors to work on a specific goal previously accepted by the GSoC program.

Dhruva Gole finished his Bachelor of Technology in Electrical Engineering earlier this year. He has been working throughout the summer to build a Zephyr module that includes the Arduino core. This is the second GSoC program for Dhruva, who in 2021 worked on bringing Bela support to the BeagleBone AI platform. We’re also very happy to share the news that Dhruva began a job with Texas Instruments last month, congratulations!

It has been wonderful working with Dhruva, who has made a sizable contribution to the open source ecosystems of Zephyr and Arduino. At the same, as mentors we have been able to share a glimpse of how the Golioth engineering team operates. Finding our way through some hairy issues with the I2C system, mapping the Zephry Arduino header paradigm in DeviceTree, and getting the build system to work without extra command line arguments provided a great opportunity to work collectively.

What has been accomplished

Zephyr is a Real-Time Operating System (RTOS) that focuses on abstracting hardware functions so that the same code may be run on innumerable hardware platforms. This is an excellent underpinning for the Arduino code.

When successfully combined, Arduino sketches can be run on hardware that does not have an Arduino port. And at the same time, Zephyr features like multi-threading and the network stack become available to those Arduino sketches.

The milestones reached in the project thus far include:

  • Arduino Digital GPIO functions (e.g.: pinmode, digitalRead, digitalWrite)
  • Arduino time functions (e.g.: delay, millis)
  • I2C support via the Wire library API (works with external Arduino libraries)
  • A DeviceTree framework for mapping Arduino header pins to any Zephyr boards
  • Serial.println support in progress

How to use it

The project is organized as a Zephyr Module called Arduino-Zephyr-API. Any board that has Zephyr support can be defined in the variants directory of this module. This is accomplished by adding a DeviceTree overlay file (to map the pins of the board to the pin numbers from the Arduino ecosystem) and a pinmap header file which wraps Zephyr pin functions with the familiar Arduino API calls.

Using west, the Zephyr meta tool, Arduino sketches can be compiled and flashed to your target board. Beyond core functions, the addition of an RTOS opens up the ability to use multiple threads in Arduino. Dhruva has includes an excellent threading demonstration by blinking multiple LEDs asynchronously.

Rudimentary instructions for adding the module to your Zephyr installation are available in the project README. Documentation for adding boards is currently in progress and will be complete by the end of the GSoC program in September.

Challenges for future contributors

The heavy lifting of integrating the Arduino core with Zephyr has already been completed and is working well. But to use this, you must already have a Zephyr workspace installed (and know how to use it). Sure, Golioth has a quickstart that helps install Zephyr, but the ability to use this project from the Arduino IDE is desirable for existing Arduino users.

Unfortunately, there isn’t time left to work this level of integration. But contributions to the program are welcome and we hope that future contributors will take on this task.

There is also low-hanging fruit when it comes to implementing the most-used Arduino functions. For instance, random numbers, bits and bytes, and math functions should all be trivial to get working. More ambitious contributors may consider mapping the Zephyr ADC system to analogRead() and analogWrite().

Even people who have been living under a rock for the past couple of years know that there’s a global chip shortage. The correct response from the engineering community should be changes to our design philosophy and that’s the topic of the talk that Chris Gammell and I gave at the 2022 Zephyr Developer Summit back in June. With the right hardware and firmware approach, it’s possible to design truly modular hardware to respond to supply chain woes.

Standardization enabled the industrial revolution, and the electronics field is a direct descendant of those principles. You can rest easy in knowing that myriad parts exist to match your chosen operating voltage and communication scheme. But generally speaking it’s still quite painful to change microcontrollers and other key-components mid-way through product design.

Today’s hardware landscape has uprooted the old ways of doing things. You can’t wait out a 52-week lead time, so plan for the worst and hope for the best. Our talk covers the design philosophies we’ve adopted to make hardware “swapability” possible, while using Zephyr RTOS to manage the firmware side of things.

Meet the Golioth Aludel

Golioth exists to make IoT development straightforward and scalable–that’s not possible if you’re locked into specific hardware, especially these days. So we designed and built a modular platform to showcase this flexibility to our customers.

Golioth Aludel prototyping platform internal view

Called Aludel, Chris Gammell centered the design around a readily available industrial enclosure. The core concept is a PCB base that accepts any microcontroller board that uses the Feather standard. This way, every pin-location and function is standardized. Alongside the Feather you’ll find two headers that use the Click boards standard (PDF), a footprint for expansions boards facilitating i2c, SPI, UART, and more. The base includes Qwiic and Stemma headers for additional sensor connectivity, as well as terminal blocks, room for a battery, and provisions for external power.

The key customization element for Aludel is the faceplate. It’s a circuit board that can deliver a beautiful custom design to match any customer request. But on the inside, each faceplate has the same interface using a flat cable connection to the base board. Current versions of the faceplate feature a a screen, an EEPROM to identify the board, and a port expander that drives buttons, LEDs, and whatever else you need for the hardware UI.

The beauty of this is that electrically, it all just works. Need an nRF52, nRF9160, ESP32, STM32, or RP2040? They all already exist in Feather form factor. Need to change the type of temperature sensor you’re using? Want to add RS485, CANbus, MODBUS, 4-20 ma communication, or some other connectivity protocol? The hardware is ready for it–at most you might need to spin your own adapter board.

Now skeptics may look at the size of this with one abnormally high-arched eyebrow. But remember, this is a proof-of-concept platform. You can set it up for your prototyping, then take the block elements and boil them down into your final design. Prematurely optimizing for space means you will have many many more board runs as the parts change.

When you do a production run and your chip is no longer available, the same concepts will quickly allow you to test replacements and respin your production PCB to accommodate the changes.

Zephyr Alleviates your Firmware Headaches

“But wait!” you cry, “won’t someone think of the firmware?”. Indeed, someone has thought of the firmware. The Linux Foundation shepherds an amazing Real-Time Operating System called Zephyr RTOS that focuses on interoperability across a vast array of processor architectures and families. Even better, it handles the networking stack and has a standardized device model that makes it possible to switch peripherals (ie: sensors) without your C code even noticing.

We use Zephyr to maintain one firmware repository for the Aludel hardware that can be compiled for STM32F40, nRF52840, nRF9160, and ESP32 using your choice of Ethernet, WiFi, or Cellular connections. How is that even possible?

&i2c0 {
	[email protected] {
		compatible = "bosch,bme280";
		reg = <0x76>;
		label = "BME280_I2C";
	};
};

&spi1 {
	compatible = "nordic,nrf-spi";
	status = "okay";
	cs-gpios = <&gpio0 3 GPIO_ACTIVE_LOW>,
	           <&gpio0 27 GPIO_ACTIVE_LOW>,
	           <&gpio1 8 GPIO_ACTIVE_LOW>;
	test_spi_w5500: [email protected] {
		compatible = "wiznet,w5500";
		label = "w5500";
		reg = <0x0>;
		spi-max-frequency = <10000000>;
		int-gpios = <&gpio0 2 GPIO_ACTIVE_LOW>;
		reset-gpios = <&gpio0 30 GPIO_ACTIVE_LOW>;
	};
};

/ {
    aliases {
        aludeli2c = &i2c0;
        pca9539int = &interrupt_pin0;
    };
    gpio_keys {
        compatible = "gpio-keys";
        interrupt_pin0: int_pin_0 {
			gpios = < &gpio0 26 GPIO_ACTIVE_LOW >;
			label = "Port Expander Interrupt Pin";
		};
	};
};

Zephyr uses the DeviceTree standard to map how all of the hardware is connected. A vast amount of work has already been done for you by the manufacturers who maintain drivers for their chips and supply .dts (DeviceTree Syntax) files that specify pin functions and mappings. When writing firmware for your projects you supply DeviceTree overlay files that indicate sensors, buttons, LEDs, and anything else connected to your design. The C code looks up each device to receive all relevant information like device address, GPIO port/pin assignment and function.

const struct device *weather_dev = DEVICE_DT_GET_ANY(bosch_bme280);
sensor_sample_fetch(weather_dev);
sensor_channel_get(weather_dev, SENSOR_CHAN_AMBIENT_TEMP, &temp);
sensor_channel_get(weather_dev, SENSOR_CHAN_PRESS, &press);
sensor_channel_get(weather_dev, SENSOR_CHAN_HUMIDITY, &humidity);

The hardware abstraction doesn’t stop there. Zephyr also specifies abstraction for sensors. For instance, an IMU will have an X, Y, and Z output — Zephyr makes accessing these the same even if you have to move to an IMU from a different manufacturer due to parts shortages. You update the overlay files for the new build, and use a configuration file to turn on any specific libraries for the change, but the code you’re maintaining can continue on as if nothing happened.

Of course there are caveats which we cover in the talk. There is no one-size-fits-all in embedded engineering so you will eventually need to define your own .dts files for custom boards, and add peripherals that are unsupported in Zephyr. This is not a deal breaker, there is a template example for adding boards (and I highly recommend watching Jared Wolff’s video on the subject). And since Zephyr is open source, your customizations can be contributed upstream so others may also benefit.

Hardware will never again be the same

Electronics manufacturing will eventually emerge from the current shortages. But there are no signs of this happening soon, and it’s been ongoing for two years. We shouldn’t be trying to return to “normal”, but planning a better future. That’s what Golioth is doing for the Internet of Things. Choosing an IoT management platform shouldn’t require you to commit to one type of hardware. Your fleets should evolve, and our talk outlines how they can evolve. Golioth helps to manage your diverse and growing hardware fleet. Golioth is designed to make it easy to keep track of and control all of that hardware, whether that’s 100 devices or 100,000. Take Golioth for a test drive today.

NXP RT1060-EVKB Ethernet development board

I get to work with a lot of different hardware here at Golioth, but recently I’ve found my self gravitating to the NXP i.MX RT1060-EVKB evaluation board as my go-to development hardware. This brand new board is a variation on the RT1060-EVK, which (in addition to the extra ‘B’ in the new part number) adds an m.2 slot. But what catches my eye is the Ethernet. While I’ve previously written about using SPI-connected Ethernet modules, this board builds the connectivity right in. It’s stable, it’s fast, and has wonderful support in Zephyr.

The microcontroller that controls everything is the RT1062 which I first heard about three years back when the Teensy 4.0 was released. A 600 MHz powerhouse, it boasts 1024 kB of SRAM and is packed with peripherals. It’s an embarrassment of riches for my day-to-day firmware work, but makes sure I’ll never hit my head on the proverbial “development ceiling”.

Of course it’s not just the system clock that I’m happy about. This board has Ethernet built-in, so I’m not waiting for a WiFi connection (or really waiting for a cell connection) at boot. And it’s programmed via J-Link for a fast flash process and the RTT and debugging features that this programmer brings to the party.

Let’s take a look at how I get this board talking to Golioth, and what we might see in the near future.

An i.MX RT1060-EVKB demo

NXP RT1060-evkb webinar demo

Demonstrating Golioth LightDB Stream for temperature/pressure/humidity sensor data

A couple of weeks ago I did a demo of the i.MX RT1060-EVKB board as part of NXP’s Zephyr webinar series. It gives you a great overview of the board and the ecosystem. I was even able to do a live demo of all the Golioth features like data management, command/control, remote logging, and OTA firmware updates. (Requires a free NXP registration to see the video.)

This board already has great support with the Golioth platform. My hope is to go further than that and add this to our list of Continuously Verified Boards. This would mean that we test and verify all official Golioth code samples with this board during every Golioth release. That’s a tall hill to climb, but it would be a snap if we used automated testing through a process called Hardware in the Loop; CI/CD that automatically runs compiled code on the hardware itself. This is already the case for our recently announced Golioth ESP-IDF SDK. Nick Miller is working on a post about how he pulled that off, so keep your eyes on the Golioth Blog over the next couple of weeks!

Get your tools ready

To build for this board you need a few tools, like the HAL library for NXP, LVGL, and the board files themselves.

Adding the NXP HAL and the LVGL library

Zephyr needs a hardware abstraction layer for NXP builds, and this board also depends on having the LVGL library installed. Add these to your west manifest:

  • use the west manifest --path to help you locate which file to edit (modules/lib/golioth/west-zephyr.yml in my case)
  • Add hal_nxp and lvgl to the name-allow list
  • Run west update

With the edits in place, here’s what the relevant section of my west manifest looks like:

manifest:
  projects:
    - name: zephyr
      revision: v3.1.0
      url: https://github.com/zephyrproject-rtos/zephyr
      west-commands: scripts/west-commands.yml
      import:
        name-allowlist:
          - cmsis
          - hal_espressif
          - hal_nordic
          - mbedtls
          - net-tools
          - segger
          - tinycrypt
          - hal_nxp
          - lvgl

Cherry Pick the EVKB files (if needed)

This is a new board and the devicetree files for it were just added to Zephyr on Jun 5th. My Zephyr is still on v3.1.0 so it’s older than the commit that added support for the “evkb” board. I used git’s cherry-pick feature to add those files to my workspace. From the golioth-zephyr-workspace/zephyr directory run:

git pull
git cherry-pick 9a9aeae00b184cac308c1622676bb91c07dd465b

This will pull in just the commit that adds support for the new board.

Thar ‘b dragons

Here’s the thing about making manual changes to the Zephyr repo in your workspace: the west manifest doesn’t know you’ve done this. That means the next time you run west update, this cherry-picked commit will be lost.

Prep the hardware

Last, but not least, I used a J-Link programmer to flash firmware to this board. There’s just a bit of jumper setup necessary to make this all work. NXP has a guide to setting up an external J-Link programmer. I followed that page, using a USB cable on J1 for power (and a serial terminal connection when running apps).

Saying Hello

With that preamble behind me, I’m excited to get to the “Hello” part of things using the standard Golioth Hello sample. Running Golioth samples is fairly easy with this board, with just one puzzle piece that needs to be added for DHCP to acquire an IP address.

Use DHCP to get an IP address

I added a boards/mimxrt1060_evkb.conf file to create a configuration specific to this board. In this case, it selects the Ethernet and DHCP libraries.

CONFIG_NET_L2_ETHERNET=y
CONFIG_NET_DHCPV4=y

At the top of main.c I include the network interface library. Then in the main() function, we need to instantiate an interface and pass it to the DHCP function. This code requests an IP address from the network’s DHCP server.

#include <net/net_if.h>
struct net_if *iface;
LOG_INF("Run dhcpv4 client");
iface = net_if_get_default();
net_dhcpv4_start(iface);

client->on_message = golioth_on_message;
golioth_system_client_start();

Add credentials, then build and flash

The final step is to add your Golioth credentials before building and flashing the app. Add the credentials to the prj.conf file as outlined in the hello sample README. You can get your credentials by creating a new device on the Golioth Console.

CONFIG_GOLIOTH_SYSTEM_CLIENT_PSK_ID="my-psk-id"
CONFIG_GOLIOTH_SYSTEM_CLIENT_PSK="my-psk"

The build calls out the board name to specifically target our board and the newly added configuration. The flash command will automatically find the J-Link programmer and use it.

west build -b mimxrt1060_evkb samples/hello
west flash

In my testing it only took about three seconds to get an IP address. From there, I was up and connected to Golioth after just another two seconds!

Golioth Hello serial terminal output

Golioth Hello example output

Off to the races

There really isn’t much to getting this up and running. Once I worked through this process I was able to test out all of the Golioth samples, including using the terminal shell to provision the board (ie: adding the Golioth credentials) and performing Over-the-Air (OTA) firmware updates.

As I mentioned earlier, the speed and reliability of this dev board keeps drawing me back. It’s not just the initial connection, but when testing out OTA, the 1024 byte block downloads seem to fly right by. I’m unsure if this is the Ethernet or the chip’s ability to quickly write to flash, but if you’re working with a 250 kB firmware file the difference over, say an ESP32, is obvious.

Give it a try using Golioth’s Dev Tier which includes up to 50 devices for your test fleet. The Golioth forum is a great place for questions on the process, and we’d love to hear about what you’re building over on the Golioth Discord server.

Keep your eye on this space. When Zephyr issues its next release we’ll roll forward the Golioth SDK to include the board files for the RT1060-EVKB, and my hope is that we’ll be able to include the DHCP call into our common samples code for a truly out-of-the-box build experience. We might even see a bit of that Hardware-in-the-Loop build automation I hinted at earlier. See you next time!

In this article, we showcase how to use Wireshark–an open source, free network analysis tool–to troubleshoot wireless mesh networks set up using OpenThread, Zephyr, and Golioth. The tooling shown here can also be used for other Thread-based devices, assuming you understand the layers of the network. We used these tools internally to help us when get Thread devices to connect with Golioth and take advantage of all of the features we have to offer IoT device makers.

Building Thread Networks

Golioth started building out Thread networks when several users approached us about their interest in creating Golioth-managed Thread devices. We created example projects to show our users how to create mesh networks of custom low-power sensors and connect them back to the internet. We benefit from the fact that Thread network devices are IPv6 devices (thanks to 6LoWPAN), and that they talk over the CoAP protocol, all of which aligns very well with Golioth capabilities. We showed this in our most recent blog post about custom Thread nodes connecting through an OpenThread Border Router (OTBR) back to Golioth and transmitting information that can be displayed anywhere on the web.

Hardware and firmware engineers can utilize the Golioth Zephyr SDK to implement a wide range of features on Thread nodes and interact with those nodes like any other internet-connected device. In the Golioth Red Demo showcased at a number of recent live events, we had nodes that could report back sensor data and react to stimulus from the cloud; future versions could also get firmware updates directly from the cloud.

As in any hardware and firmware development process, things didn’t always go according to plan. When we were troubleshooting our early Proof of Concept, we needed to check which part of the chain was not passing packets along. We broke out Wireshark to start sniffing packets and figured out that there was a mismatch in the number of bytes being sent (since fixed). We think this kind of pinpoint accuracy in troubleshooting is a tool that all our users should have in their toolbox.

Good Security is meant to slow you down

Golioth is secure by default, which means all packets going to our Cloud must be encrypted. Normally, this is a feature! You don’t want anyone with a packet sniffer to be able to see plain-text data. However, when you do want to see what’s inside a packet during troubleshooting, you need to make sure you have the keys to unlock everything. You also need to make sure you have the tools properly configured for the various layers involved in Thread networks. These will be the steps we review below and in the video.

Setting up a sniffer

In order to use Wireshark to troubleshoot a Thread network, you’ll need the following:

Pretty simple!

The first step is getting the tools onto the dongle. Much like the dongle was used as a Radio Co-Processor on the Open Thread Border Router, we’ll be using a different set of firmware to sniff radio packets and hand them over USB to the computer. This firmware is specific to 802.15.4, which is the Physical and MAC layer used by Thread. Download the firmware from Nordic Semiconductor and load it onto your dongle and you’re ready to go!

Next you need to be able to interact with the output of the dongle. This includes installing a python script in Wireshark that is located in the Nordic Semiconductor repository. Around the 2:15 mark in the video, Mike shows where and how to install this in Wireshark.

Configuration Settings

Other important parts of the video include things like:

  • 3:45Choosing the correct 802.15.4 channel (channel 15 for most Golioth examples)
  • 5:00Network settings for Wireshark to capture Thread network traffic
  • 7:15Adding a Pre-Shared Key (PSK) to decrypt DTLS packets

Once those configuration steps are done, you can view wireless traffic coming from your Thread node, through each of the layers, up to the Golioth servers, onto specified endpoints like /logs. In the decoded payload area (bottom window), you can also see the messages that are actually being sent, in this example a log message saying “starting connect”.

Tools for when you need them

Wireshark and plugins developed by the community make for a powerful set of tools for troubleshooting network problems. We hope that our examples and tutorials allow you to quickly deploy a Thread network and build out custom Thread nodes; but when you need a bit more insight or are looking to try something new, Wireshark can help.

We’re here to help too! You can reach us on our Discord or Forum, and can always reach us at [email protected].

TL;DR: we’ve enabled people to compile Zephyr programs from a computer with no toolchain installed, almost instantly.

Part of our charter at Golioth is to help people prototype and scale IoT devices faster. That’s why we offer an open source SDK built on top of Zephyr. We think this represents a “fast forward” or “cheat code” for quickly standing up an IoT device prototype. On the cloud side, our servers represent hundreds of hours of customization and testing; you can instantly connect and get access to resources that allow hardware and firmware developers to scale to thousands or millions of devices. But sometimes it can be scary to get started in a new ecosystem or Real Time Operating System (RTOS) like Zephyr, even if it will speed things up later. As such, we do public and private training for companies and individuals.

As part of the resources we offer, we maintain a Training site that walks people through how to get started using Zephyr, normally targeting remote training. You can follow along right now; you’ll need to purchase an Adafruit MagTag board and sign up for a free Dev Tier account, but everything else is covered on the training site. At the end of the training, you should understand how to interact with hardware in Zephyr and send data to and from the Golioth cloud over WiFi. It’s a short jump from there to re-target other hardware, including your custom designs.

The tripping points for the training often revolve around the installation process. This is multi-pronged:

  • The size of a Zephyr install is relatively large, even when you are only targeting a specific platform. Having multiple people in a room, even with good WiFi or network connectivity, means that the shared bandwidth will be a limiting factor. More trainees means slower downloads.
  • Everyone comes to training with a computer in a different state. They might have tried to install Zephyr tools in the past, or they might have a particularly rare Linux distro, or many other possible variations. It would be best if everyone showed up with a fresh OS install…but that is very unrealistic.
  • There are different expectations around how installations should go. Many embedded engineers are “Windows first” and expect a complete IDE for any new platform. Some silicon vendors help to support this in Zephyr, such as Nordic Semiconductor. But Zephyr was originally targeting Linux-based machines, and we have found the smoothest flow for installing tools for all of the platforms that Zephyr can target means you are Linux-first.

In this article, we’re going to talk about our attempt to normalize setups and have pre-installed tools using Kasm and Docker. These are not the only tools in this space; we have previously written about GitPod and are investigating GitHub Codespaces, but this is a look at one of the latest experiments we’re running at Golioth.

Kasm thin client

The concept of a browser based client or a “thin client” is nothing new. They were all the rage back in the day of time share servers (really those were “dumb terminals”) and then again in the 90s as computing was more ubiquitous throughout the office (with a centralized set of servers). The difference is that now things are much more graphical and running completely inside the browser.

Kasm was started in 2017 and includes an open source project run by Kasm Technologies. The company behind Kasm has a per seat licensing model or they will run the servers directly for you (once you’re past 5 trial seats). They specialize in visualizations around containers. Once you log into a Kasm server, you are able to launch a range of containers, normally a desktop view or a single app that will load up in your browser. You can try this for yourself on the Kasm demo page.

The server that we’re running on is a pre-configured image that I pulled from the Digital Ocean marketplace. I was able to install all of the required software on a provisioned server running in some unknown datacenter. All I did was log in the first time to get my credentials for a user and an admin, and the rest of my interaction was on the web interface that the Kasm server presents to me as an admin.

Docker

As a hardware engineer, Docker is one of those things I heard about for a long time and never really “got it”. I’m still not sure I do. But following the tutorial for customizing a Kasm container, I started to understand a bit more. In that set of tutorials I started from a base Operating System image (Ubuntu Focal) that allowed visualization through the browser. Then I was able to start customizing, adding things like custom files on the desktop, custom icons to launch programs I installed, or adding background images pulled in from the web. It was in this customization section that I could add all of the commands from the Golioth Docs for installing Zephyr tools.

My layman explanation of Docker would be “Creating a virtual computer where I can automatically install a bunch of software using shell scripts. Once I have built that virtual computer, I am able to use it over and over again, including different instances of that virtual computer (for this Kasm scenario)”. The analogy would be if I bought a bunch of laptops, had an install CD (remember those?) with all of the required software on them, and then I mailed the freshly installed laptop to everyone who is taking our training. Sound crazy? That’s one of the best solutions we have seen, where a trainer will bring a pelican case with 24 laptops freshly imaged to on-site training. Their training works flawlessly every time!

I don’t have much else to mention about Docker aside from the idea that it’s possible to script a bunch of install commands that match the install instructions we have on our Zephyr getting started guide. In fact, I used those very directions to build the container shown in the video above. So all I’m doing in this case is automating the install process, doing it once, and then deploying the container (with all of the software and dependencies installed) over and over again for different users.

Challenges

We don’t think this is the ultimate solution for our training, so much as an experiment that showcases what we can do with containerized solutions. There are some remaining challenges, and we would love to have some help from our community.

Loading firmware onto the device

Currently our plan (as shown in the video) is to have our users/trainees pull the final built binary to their local computer to run it on the device like the MagTag. This echoes the way the mbed online compiler worked.

If there is a bootloader and a USB to serial connection, it’s possible to directly load onto the embedded device. In the case of some Espressif boards, this would be something like having ESPtool.py installed locally on your machine. There are an increasing amount of tools that make this process easier, such as an ESP tool that allows you to load firmware using WebUSB. Certain specialized bootloaders like the one that comes default on the MagTag loads UF2 files. When the MagTag is plugged in over USB and a sequence of buttons are hit, the device shows up as a mass storage drive. You drop a UF2 formatted binary–which is just an alternative form of compiled format–onto the drive and the device reboots and starts running the code.

If it’s a board without a bootloader, the user would need to have a debugger and local tools to communicate with that debugger, such as a JLink device and JFlash software. This means they would still need some OS specific loader tools to get the binary into the embedded device. The user would not be able to take advantage of the built-in tools in west that allow direct loading onto the device.

You steppin’?

If you would like to do debugging instead of “printf/printk” debugging, you simply need to download a different file from the container. If you download the zephyr.elf file instead of the zephyr.bin file, you can load it into a 3rd party debugger like Segger Ozone (made by the same company as the JLink). We have done some experiments with this in the past, including also analyzing where the device is spending its time using SystemView. This would once again require installing local programs that could talk over the USB port to something like a JLink.

Experimental port forwarding and WebUSB

Some GDB debuggers/servers will host the control of the debugger over a port on the machine’s localhost. We have some experiments we’re trying where we forward this port to the container so we could directly run a debugger from a software debugger inside the container.

We have also heard some whispers of a WebUSB implementation that can tunnel to the container. So we could plug in a board on our host machine (ie. my laptop) and connect to it over WebUSB, and then forward all information along to the container machine (ie. the browser based desktop running on the Kasm server).

We would love to hear about other projects that are trying this.

Shared resources

The final challenge we are dealing with is the fact that we’re basically “renting” a computer to do exactly what we could be doing with the host machine sitting right in front of us. Most developers have access to very powerful machines and we are instead using the resources on a remote machine (the Kasm server). The cost of standardization is the cost of renting server time for each person in the workshop. It might be worth it, but it is a constraint and a challenge.

Containers are another tool

Anyone reading this with a web background is likely thinking, “Yeah, containers, cool, 2010 called and wants their headline back”. But we are excited about it because these tools are finally making their way into the historically sluggish embedded industry. While our use case of containers is mostly around zero-install-time training, others are using containers to automate their testing and implementing best software engineering practices for the range of devices they have on their desk or in the field.

We’d love to hear how you think we can improve our training and make it easier for you to learn more about Golioth, Zephyr, and building code instantly. Check out our forums, our Discord, ping us on Twitter, or send us an email at [email protected]

Golioth is a device management cloud that is easy to add to any Zephyr project. Just list the Golioth Zephyr SDK as a module in your manifest file and the west tool will do the rest. Well, almost. Today I’ll walk through how to add Golioth to an existing Zephyr project. As an example, I’ll be using our hello sample. We chose this because it already has networking, which makes explaining things a bit easier. Generally any board that has led0 defined in the device tree and you can enable networking should be a good fit. What we want to do here is showcase the elements that allow you to add Golioth, so let’s dive in!

0. Understanding the west manifest

A Zephyr workspace is made up of a number of different code repositories all stored in the same tree. Zephyr uses a west manifest file to manage all of these repositories, including information like the origin URL of the repo, the commit hash to use, and where each repository should be placed in your local tree.

Think of the manifest as a code repository shopping list that the west update command uses to fill up your local Zephyr tree. There may be more than one manifest file, but today we’ll just focus on the main manifest.

1. Add Golioth to the west manifest

Start by locating your manifest file and opening it with a code editor.

~/zephyrproject $ west manifest --path
/home/mike/zephyrproject/zephyr/west.yml

Under projects: add an entry for the Golioth Zephyr SDK (highlighted in the abbreviated manifest sample below).

manifest:
  defaults:
    remote: upstream
 
  remotes:
    - name: upstream
      url-base: https://github.com/zephyrproject-rtos
 
  #
  # Please add items below based on alphabetical order
  projects:
      # Golioth repository.
    - name: golioth
      path: modules/lib/golioth
      revision: v0.2.0
      url: https://github.com/golioth/golioth-zephyr-sdk.git
      import:
        west-external.yml
    - name: canopennode
      revision: 53d3415c14d60f8f4bfca54bfbc5d5a667d7e724
...

Note that I have called out the v0.2.0release tag. You can set this to any of our release tags, to main, or to a specific commit.

Now run an update to incorporate the manifest changes:

west update

2. Select libraries to add to the build

We use KConfig to build in the libraries that Golioth needs. These changes are made in the prj.conf file of your application.

The first set of symbols deals with Zephyr’s networking stack. Of course every Internet of Things thing needs a network connection so you likely already have these symbols selected.

# Generic networking options
CONFIG_NETWORKING=y
CONFIG_NET_IPV4=y
CONFIG_NET_IPV6=n

Golioth is secure by default so we want to select the mbedtls libraries to handle the encryption layer.

# TLS configuration
CONFIG_MBEDTLS_ENABLE_HEAP=y
CONFIG_MBEDTLS_HEAP_SIZE=10240
CONFIG_MBEDTLS_SSL_MAX_CONTENT_LEN=2048

Now let’s turn on the Golioth libraries and give them a bit of memory they can dynamically allocate from.

# Golioth Zephyr SDK
CONFIG_GOLIOTH=y
CONFIG_GOLIOTH_SYSTEM_CLIENT=y
 
CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE=256

Every device needs its own credentials because all connections to Golioth require security. There are a few ways to do this, but perhaps the simplest is to add them to the prj.conf file.

# Golioth credentials
CONFIG_GOLIOTH_SYSTEM_CLIENT_PSK_ID="[email protected]"
CONFIG_GOLIOTH_SYSTEM_CLIENT_PSK="cab43a035d4fe4dca327edfff6aa7935"

And finally, I’m going to enable back-end logging. This is not required for connecting to Golioth, but sending the Zephyr logs to the cloud is a really handy feature for remote devices.

# Optional for sending Zephyr logs to the Golioth cloud
CONFIG_NET_LOG=y
CONFIG_LOG_BACKEND_GOLIOTH=y
CONFIG_LOG_PROCESS_THREAD_STACK_SIZE=2048

I separated each step above for the sake of explanation. But the combination of these into one block is the boiler-plate configuration that I use on all of my new projects. See this all as one file in the prj.conf from our hello sample.

3. Instantiate the Golioth system client and say hello

So far we added Golioth as a Zephyr module and enabled the libraries using KConfig. Now it’s time to use the Golioth APIs in the main.c file.

The first step is to include the header files and instantiate a golioth_client object. The client is used to manage the connection with the Golioth servers. Including the coap header file is not strictly required to establish a connection, but it is necessary for processing the responses from Golioth, so I always include it.

#include <net/coap.h>
#include <net/golioth/system_client.h>
static struct golioth_client *client = GOLIOTH_SYSTEM_CLIENT_GET();

I also add a callback function to handle messages coming back from the Golioth cloud. Technically this is optional, but if you want two-way communication with the cloud you need it!

/* Callback for messages received from the Golioth servers */
static void golioth_on_message(struct golioth_client *client,
                   struct coap_packet *rx)
{
    uint16_t payload_len;
    const uint8_t *payload;
    uint8_t type;
 
    type = coap_header_get_type(rx);
    payload = coap_packet_get_payload(rx, &payload_len);
 
    printk("%s\n", payload);
}

In the main function, I register my callback, start the Golioth client, and call the golioth_send_hello() API.

/* Register callback and start Golioth system client */
client->on_message = golioth_on_message;
golioth_system_client_start();
 
while (1) {
    /* Say hello to Golioth */
    int ret = golioth_send_hello(client);
    if (ret) {
        printk("Failed to send hello! %d\n", ret);
    }
    else printk("Hello sent!\n");
}

When successful, the golioth_send_hello() call will prompt for a message back from the server which includes the name of the sending device. This is printed out by the golioth_on_message() callback.

Hello sent!
Hello blinky1060
Hello sent!
Hello blinky1060
Hello sent!
Hello blinky1060

Extra Credit: Back-end Logging

Observant readers have noticed that I enabled back-end logging using KConfig symbols but I didn’t use it in the C code. Here’s the really awesome part: just use Zephyr logging as you normally would and it will automatically be sent to your Golioth console.

At the top of main.c, include the log header file and register a logging module.

#include <logging/log.h>
LOG_MODULE_REGISTER(any_name_you_want, LOG_LEVEL_DBG);

In our C functions, call logs as you normal would.

LOG_ERR("An error message goes here!");
LOG_WRN("Careful, this is a warning!");
LOG_INF("Did you now that this is an info log?");
LOG_DBG("Oh no, why isn't my app working? Maybe this debug log will help.");

Now your log messages will appear on the Golioth console!

Further Reading

This covers the basics of adding Golioth to an existing Zephyr project. You can see the steps all in one place by looking at the Golioth code samples. Here are the examples that you’ll find immediately useful:

  • Hello – basics of connecting to Golioth and sending log messages (what was covered in this post)
  • Lightdb Set – send data to Golioth
  • Lightdb Observe – device will be notified whenever an endpoint on the Golioth cloud is updated

To go even deeper, you can see how we use the Golioth Over-the-Air (OTA) firmware update feature, and how to use the Zephyr settings subsystem for persistent credential storage. And remember, the Dev Tier of Golioth includes the first 50 devices, so you can try all of this out for free.

Image from Todd Lappin on flickr

Marcin Niestrój helped to implement a solution to IoT logging in Zephyr, and that work is the subject of his Connecting Zephyr Logging to the Cloud Over Constrained Channels talk, presented during the 2022 Zephyr Developer’s Summit.

Marcin has been working as an embedded engineer for over 10 years, with the last four of them in Zephyr. He is an active contributor to the Zephyr networking stack, and works on the Zephyr SDK at Golioth, a device management cloud company.

Long-distance logging

Logging is the first line of defense in figuring out what an embedded system is doing. Whether you want to monitor that all is well, get some quick feedback on sensor data, or jump into troubleshooting when something isn’t working, logs are a developer’s best friend. But what about those times when you can’t just plug a programmer or a USB cable into the device sitting on the desk in front of you?

The Internet of Things presents a challenge with device logging as you need to find a way to access logs when you don’t have physical access to the device. The answer, of course, is logging IoT data to the cloud. Marcin guides us through the layers of the existing Zephyr logging system, then shows how Golioth built a backend that allows Zephyr to move messages from the logging core to a remote server.

The Basics of Zephyr Logging

Logs are like fancy printf() statements. They have a subsystem behind them to keep the logs out of the way of more time-sensitive operations. The string messages you would expect to find in each log are joined by some metadata. This includes a timestamp that records the exact timing of the logged event, a log level (Error, Warning, Info, or Debug), and the module/component name where the log originated.

Hundreds of Zephyr logging modules can send messages to the logging core, which then routes those to whichever backend is configured. You’ve probably used the UART, the Shell, and the RTT backends of Zephyr to route your logs to the most convenient place for your development.

To solve the cloud-logging challenge, Golioth implemented a backend that packages each log message and its metadata for transport. Messages are stored, and may be accessed via a web interface or command line tool. This makes those messages persistent for future debugging, and easily filterable/searchable.

Sending messages with constrained devices in mind

When connected over USB, the more data the better! But when you start to think about devices operating from a small battery source, or chirping data over cellular or a thread network, you need your messaging to be as lean and quick as possible.

One simple way to do this is by making sure that the device only sends messages that are needed, based on the logging level. The device monitors a LightDB endpoint on the server, and will only transmit messages within that log level range. This effectively becomes “on-demand logging”. You can leave it off by default, but you always have the option to turn it on remotely to monitor devices in production without breaking the bank on bandwidth charges.

But of course, the way each message is sent also matters greatly. Log messages are transmitted as a UDP datagram, using CBOR for serialization and CoAP as the protocol layer. This has numerous bandwidth-saving benefits (and therefore radio-on time battery benefits) over other options like sending JSON over HTTP. This approach has also shown to be more efficient than the MQTT protocol which itself targets constrained devices.

This is just the preamble

What you’ve read so far is all just set up for the technical dive that Marcin treats us to. Scroll back up and watch his talk which shares the reasoning behind each decision that has been made. He also delves into some areas for future work: advanced message queuing, resend logic, packing multiple messages into each datagram for bandwidth saings, and using a dictionary for module names instead of sending them as text in every message.

Zephyr custom shell commands

The Zephyr shell is a powerful tool that we use all the time here at Golioth for prototyping and troubleshooting. But it’s also a fantastic way to provide user control of your devices. It’s actually quite easy to add your own commands, and that’s what I’m writing about today!

Back in May we showed how to provision devices by adding Golioth encryption keys using the Zephyr shell. One use case is sending unprovisioned devices to customers, enabling them to connect via USB to add their encryption keys when they first receive the hardware. That system used a custom shell command build into the Golioth SDK as the interface. The same technique can be used to provide run-time interactivity. I recently implemented a serial command interface that let me send commands from Node-RED to a Zephyr device for automated control.

The good news is that all you have to do is turn on the shell (if it’s not already enabled), register your commands with Zephyr, and provide a callback function to react to the user input.

Starting a basic project and turning on the Zephyr shell

The first step is to enable the Zephyr shell using KConfig, and include a header file in your main.c. I’m using a virtual device based on QEMU for today’s example (but this will work with any Zephyr-compatible device that has a serial connection). Golioth has a quickstart guide for using QEMU to simulate devices, which is really handy for testing out new development in isolation from existing projects.

To begin this demo I copied the basic/minimal sample from the Zephyr tree. From there I added a prj.conf file with just one KConfig symbol in it to turn on the shell:

CONFIG_SHELL=y

I also included shell.h in the main.c file:

#include <shell/shell.h>

Creating the custom shell command

I will walk through a very basic custom shell command. The topic goes much deeper and it’s worth reading through the official Zephyr documentation on shell commands.

Inside of main we want to set up the command structure, then register the command. Our demo will set a numeric value for how awesome Golioth is. We also want to be able to read back that value. So I’ll set up two different commands, one that takes a value as an argument.

SHELL_STATIC_SUBCMD_SET_CREATE(
    g_awesome_cmds,
    SHELL_CMD_ARG(set, NULL,
        "set the Golioth awesome level\n"
        "usage:\n"
        "$ golioth_awesome set <awesome_value>\n"
        "example:\n"
        "$ golioth_awesome set 1337\n",
        cmd_g_awesome_set, 2, 0),
    SHELL_CMD_ARG(get, NULL,
        "get the Golioth awesome level\n"
        "usage:\n"
        "$ golioth_awesome get",
        cmd_g_awesome_get, 1, 0),
    SHELL_SUBCMD_SET_END
    );
 
SHELL_CMD_REGISTER(golioth_awesome, &g_awesome_cmds, "Set Golioth Awesomeness", NULL);

The SHELL_STATIC_SUBCMD_SET_CREATEdefines our command structure and I’ve given it an arbitrary symbol name (g_awesome_cmds)as an identifier.

The next two commands are established using the SHELL_CMD_ARG. Let’s walk through the arguments for that macro:

  • The first argument is the subcommand set orget (note these are not strings)
  • The second argument is for subcommands of these subcommands, which are needed for this example
  • The third argument is the help string to show in the shell
  • The fourth argument is the callback function the shell will execute
  • The fifth and six arguments are the number of required and optional arguments. Note that we need 2 required arguments for set in order to capture the submitted value, but only one for get (the subcommand itself).

Finally, I need to register the shell command. The SHELL_CMD_REGISTERtakes a series of arguments. First is the command itself (note that this is not a string), the address of the subcommand structure, the help text for this command, and a function handler which we don’t need since we’re using callback functions defined in the subcommand macro.

A custom shell command in action

We’re not quite done coding yet as we need to make the callback functions. See the bottom of this post for the complete code. But let’s skip right to the demo since the meat of the work has already been done.

uart:~$ help
Please press the <Tab> button to see all available commands.
You can also use the <Tab> button to prompt or auto-complete all commands or its subcommands.
You can try to call commands with <-h> or <--help> parameter for more information.
 
Shell supports following meta-keys:
  Ctrl + (a key from: abcdefklnpuw)
  Alt  + (a key from: bf)
Please refer to shell documentation for more details.
 
Available commands:
  clear            :Clear screen.
  device           :Device commands
  devmem           :Read/write physical memory"devmem address [width [value]]
  golioth_awesome  :Set Golioth Awesomeness
  help             :Prints the help message.
  history          :Command history.
  kernel           :Kernel commands
  resize           :Console gets terminal screen size or assumes default in case
                    the readout fails. It must be executed after each terminal
                    width change to ensure correct text display.
  shell            :Useful, not Unix-like shell commands.
uart:~$

The readout above shows that typing help lists our new command in the menu.

uart:~$ golioth_awesome --help
golioth_awesome - Set Golioth Awesomeness
Subcommands:
  set  :set the Golioth awesome level
        usage:
        $ golioth_awesome set <awesome_value>
        example:
        $ golioth_awesome set 1337
 
  get  :get the Golioth awesome level
        usage:
$ golioth_awesome get
uart:~$

If we call our custom golioth_awesome --help command, the help strings we specified are displayed.

Golioth custom shell command in Zephyr

And finally, using the commands shows the expected output. I’ve embedded an image of my terminal so that you can see the different styles of output. This printout is the result of calling `shell_fprintf()` in my callback functions. We haven’t covered that yet, so let’s look at the whole main.c file next.

Putting it all together

The final piece of the puzzle is to define the callback functions for custom shell commands. These functions interact with the Zephyr app on your device, and are responsible for printing messages in the Zephyr shell as feedback for the user.

#include <zephyr/zephyr.h>
#include <stdlib.h>
 
#include <shell/shell.h>
 
uint16_t golioth_awesome = 10000;
 
static int cmd_g_awesome_set(const struct shell *shell, size_t argc,
                char *argv[])
{
    /* Received value is a string so do some test to convert and validate it */
    int32_t desired_awesomeness = -1;
    if ((strlen(argv[1]) == 1) && (argv[1][0] == '0')) {
        desired_awesomeness = 0;
    }
    else {
        desired_awesomeness = strtol(argv[1], NULL, 10);
        if (desired_awesomeness == 0) {
            //There was no number at the beginning of the string
            desired_awesomeness = -1;
        }
    }
 
    /* Reject invalid values */
    if ((desired_awesomeness < 0) || (desired_awesomeness > 65535)) {
        shell_fprintf(shell, SHELL_ERROR, "Invalid value: %s; expected [0..65535]\n", argv[1]);
        return -1;
    }
    /* Otherwise set and report to the user with a shell message */
    else {
        golioth_awesome = (uint16_t)desired_awesomeness;
        shell_fprintf(shell, SHELL_NORMAL, "Golioth awesomeness set to: %d\n", desired_awesomeness);
    }
 
    return 0;
}
 
static int cmd_g_awesome_get(const struct shell *shell, size_t argc,
                char *argv[])
{
    shell_fprintf(shell, SHELL_NORMAL, "Current Golioth awesomeness level: %d\n", golioth_awesome);
    return 0;
}
 
void main(void)
{
    SHELL_STATIC_SUBCMD_SET_CREATE(
        g_awesome_cmds,
        SHELL_CMD_ARG(set, NULL,
            "set the Golioth awesome level\n"
            "usage:\n"
            "$ golioth_awesome set <awesome_value>\n"
            "example:\n"
            "$ golioth_awesome set 1337\n",
            cmd_g_awesome_set, 2, 0),
        SHELL_CMD_ARG(get, NULL,
            "get the Golioth awesome level\n"
            "usage:\n"
            "$ golioth_awesome get",
            cmd_g_awesome_get, 1, 0),
        SHELL_SUBCMD_SET_END
        );
 
    SHELL_CMD_REGISTER(golioth_awesome, &g_awesome_cmds, "Set Golioth Awesomeness", NULL);
}

The callback functions begin on line 8 and 38 of the example above. The cmd_g_awesome_set function looks more complicated than it is. That’s because the value we receive from the shell is a string and needs to be converted to an integer and then validated (confirm it is actually a number, and inside the acceptable bounds).

The thing to focus on are the shell_fprintf() functions which use some constants to select the text decoration. SHELL_NORMAL is used when everything is working correctly, and SHELL_ERROR for out-of-bounds settings. You can see all constants that work for this, as well as the shorthand functions for them, in the official docs.

The utility of custom shell commands

So fare I’ve focused on shell interaction with a human user. But once this is in place, you can leverage it programmatically as well. In Linux, I can set our Golioth value from the command line: echo "golioth_awesome set 10000" > /dev/ttyUSB0.

You can also extrapolate this feature from set/get to a much more robust tool. For instance, I covered the built-in Zephyr i2c and sensor shells in a previous post. These facilitate runtime changes to the menu (what sensors and devices are available changes without needing to compile for that information). This should get your mind running on how deep you can go with a custom shell tool to fit your needs. But that’s a topic for next time!

Hello from ZDS!

This week the Golioth team is at Zephyr Developer Summit. Previously we announced that we’ll be there and shared the talks we are presenting. We will post those shortly after the conference takes place. In the meantime, let’s recap how we got here in the first place and share a little bit more about what we’re showcasing.

Why Zephyr?

In short, because it helps our users. We are members of the Technical Steering Committee (TSC) and have been almost since the inception of the company. We built our first Device SDK on top of Zephyr because of the broad hardware support and high level integration of Golioth features into the Real Time Operating System (RTOS).

The assertion that “Zephyr helps our users” might be extra frustrating to beginners: Zephyr—and RTOSes more broadly—represents a tricky new set of skills that might be foreign to firmware or hardware engineers. For beginners coming from the hobby space, it can be an extra rude introduction into the world of command line compilation and large ecosystem. However, connecting to the internet is a difficult task, especially for custom hardware: we think that Zephyr represents a great first step towards managing those devices over time. We are committed to pushing for more user-friendly code and methods from the Zephyr foundation, and we will continue to publish best practices on our blog and our YouTube channel to help people get connected.

Showcase

One thing we’re excited about is showcasing how Golioth works to members of the community. We have been developing different “color coded” demos to make them a bit more memorable for folks that stop by our booth. Each of these demos feature a hardware (device) component and a dashboard component, in order to visualize the data that is on the Golioth Cloud.

This is the first time we have showcased the “Aludel”, which is our internal platform for prototyping ideas and switching out different development boards and plug-in sensors. We will post more about this in the future, including our talk on the subject.

Red Demo

The Red Demo is our showcase of devices running OpenThread on Zephyr; this is part of our larger interest in Thread, which we see as a very interesting way to connect a large range of sensors to the internet securely. We have been excited to show how we can use low power devices like the Nordic nRF52840 to communicate directly with the Golioth Cloud.

The devices we are using in this case are off-the-shelf multi-sensor nodes from Laird called the BT510. This hardware has additional sensors on the board which we integrated with LightDB Stream to send time-series data back to Golioth. This was fast work, thanks to Laird’s Zephyr support, it was as simple as calling out the board when we compiled the demo firmware.

We then capture the data from these on the Red Demo Dashboard, showing both historical and live data for the sensors.

 

Green Demo

The Green Demo showcases LightDB State, our real-time database that can be used to control a wide range of devices in a deployment. On the device side, it uses the Aludel platform to measure a light sensor, as would happen in a greenhouse. There is also a secondary Zephyr-based device inside a lamp, representing a grow light that might be inside a grow house. The lamp is set up to “listen” to commands from another node, in this case the Aludel.

LightDB State is used to control elements like “update rate” to control regulate flow of information. It also lets us monitor critical device variables on an ongoing basis and set up logic on the web to take actions as a result. Command and control variables can be set from multiple places, including a custom mobile app, the Golioth Console, a visualization platform, a web page, or (as is the case here) even from another device!

Our Green Demo Dashboard (below) again showcases live and historical information, as well as the current status of the connected lamp.

As an added bonus, we control some of the logic on the back end from a Node-RED instance, including control logic. That takes the light intensity sensor output and calculates how bright the lamp should be. Because this is written in Node-RED, we can include an additional input from a mobile app to control the “target intensity”. In this way, people at the booth can adjust the lamp output if the exhibition space is brighter or darker. Plus…it looks cool!

Blue Demo

The Blue Demo helps to showcase how data migrates into and out of Golioth. Using Output Streams, you can export all cloud events to 3rd party providers like AWS, Azure, and Google Cloud. Buttons on the Blue faceplate switch the output being sent back to the cloud. The sensor readings being exported to all 3 clouds can be turned on or off by changing which variables are exported from the device.

On the device side, we capture a sensor using our Aludel platform. The sensor is a BME280 (in-tree sensor in Zephyr), going through a feather form-factor dev board, talking to the network over a WizNet W5500 Ethernet external chip to the network. The Blue Demo Dashboard showcases the live data, and of course the data is being exported simultaneously to the 3 cloud platforms in real-time.

Orange Demo

Golioth is a “middleware” built on top of Zephyr RTOS, which means you can use it to implement new features on top of already-existing hardware. This demo uses the Nordic Semiconductor Thingy91 with custom firmware to send GPS data back over the cellular network to Golioth using LightDB Stream. This demo also has Golioth Logging and Device Firmware Update, which are easy to add to any project as an additional service for troubleshooting or in-field updates.

On the dashboard side, we wanted different ways to showcase this data, including “latest update”. Having access to the raw data is useful for anyone wanting to try asset tracking applications. We’re excited to be able to showcase this data as it dynamically flows into the Golioth Console and back out to the Grafana dashboard.

Future showcases

We’re excited to be showcasing our demos at the Zephyr Developer Summit, but these are moving targets! We will continue to update and pull in new feature for future events. We will be at Embedded World in two weeks (June 20-24th) and will have many of the same demos there.

The Zephyr project uses the KConfig system to select what subsystems and libraries are included in the build of your project. Sometime this is really simple, like adding CONFIG_GPIO=y when you need to use input/output pins. That’s easy enough to add manually. But things get more complex when you don’t know which configuration you need, or you want to fine tune a value like stack size.

In this post, we’re going to look at how to make changes in menuconfig, but also how to check those changes and make them permanent.

Turning on a watchdog timer

If you want to use the watchdog timer subsystem in Zephyr you need to add CONFIG_WATCHDOG=y to your build. That’s not entirely easy to figure out as the Watchdog entry in the Zephyr docs doesn’t mention it. There is some sample code called Task Watchdog and you can see the symbol is enable in the project configuration file, but there are other symbols related to watchdog timers in there as well. Which ones do we need?

I like to use menuconfig to work out these issue. It’s something we’ve talked about a few times before, and Marcin and Chris even filmed walkthrough of menuconfig system. The first step is to build your project before trying to enable the new configuration. This is necessary to generate the initial configuration for the project; if your project has a build error, you can’t load up menuconfig. Here I’m building for an nRF52840 feather board:

west build -b adafruit_feather_nrf52840 my_app_directory -p

Notice I’m using the -p directive for a “pristine” build. This makes sure that anything in the build directory is generated fresh so we are certain that we begin with a clean build.

Next, use west build -t menuconfig to launch the interactive configuration menu:

Zephyr menuconfig

Zephyr menuconfig

My most-used feature is the forward-slash ( / ) which brings up a text search. I searched for “watchdog” and two similar options came up.

I find that the options with helper text (“Watchdog Support” in this example) next to them are the best items to choose. Pressing enter on that entry takes me to the configuration item where I use the space bar to select it, indicated by an asterisk. That’s it, I’m not going any deeper right now. Pressing q brings up the exit menu, don’t forget to save your changes.

Zephyr kind of takes care of the rest, automatically pulling in a bunch of other options based on the top-level option I enabled. But those are actually why I’m writing this article. How can I see what changes other were made by “under the hood” by menuconfig?

Seeing changes made by menuconfig

Running a build command on a Zephyr project pulls in configurations from all over the place (eg: the board’s default configuration from the Zephyr tree, the project’s prj.conf file, and any .conf files in the boards directory). These coalesce into the build/zephyr/.config file, which is what menuconfig loads and saves to. Luckily, before this happens it makes a copy called .config.old which we can compare using the diff command:

(.venv) [email protected] ~/golioth-compile/AgTech-Soil/fw $ diff build/zephyr/.config.old build/zephyr/.config
30c30
< # CONFIG_WATCHDOG is not set
---
> CONFIG_WATCHDOG=y
179c179,180
< # CONFIG_NRFX_WDT0 is not set
---
> CONFIG_NRFX_WDT=y
> CONFIG_NRFX_WDT0=y
813a815
> CONFIG_HAS_DTS_WDT=y
1046a1049,1057
> # CONFIG_WDT_DISABLE_AT_BOOT is not set
> # CONFIG_WDT_LOG_LEVEL_OFF is not set
> # CONFIG_WDT_LOG_LEVEL_ERR is not set
> # CONFIG_WDT_LOG_LEVEL_WRN is not set
> CONFIG_WDT_LOG_LEVEL_INF=y
> # CONFIG_WDT_LOG_LEVEL_DBG is not set
> CONFIG_WDT_LOG_LEVEL=3
> # CONFIG_WDT_COUNTER is not set
> CONFIG_WDT_NRFX=y

Check that out! Enabling CONFIG_WATCHDOG automatically pulls in the CONFIG_NRFX_WDT because of the chip used in my project. Be we also learn some interesting things here. It looks like some Nordic chips have multiple watchdog timers as CONFIG_NRFX_WDT0 is selected. If we search in menuconfig for that symbol and type ? on the entry we can get more information on what that’s all about:

Nordic watchdog timer instance info

The help screen in menuconfig for CONFIG_NRFX_WDT0

You can see here that this symbol was enabled (y-selecting) by the WDT_NRFX symbol. So if you’re cruising through menuconfig and want to know why something is turned on, this is a handy way to track down those answers. Try bringing up the help screen for the CONFIG_WDT_COUNTER which is shown above but not enabled. I certainly learned something about this chip for doing so!

Making menuconfig changes persistent

Remember way back at the top of the article when I mentioned I was using a “pristine” build with the -p flag? If you do that now, you’ll lose your menuconfig changes. So make sure you do an incremental build (just don’t use the pristine flag) to test those changes out. But eventually you will want to make them persistent.

The solution for that is easy. Use the diff trick I showed above to confirm what symbols were changed by your menuconfig selections, and add those changes to the prj.conf or board-specific .conf files. Once you commit these to your revision control, you’ll be certain to compile in the options you need for future projects.

Stop by our Discord or our Forum for more quick tips about Zephyr and your next IoT project!