,

Sipping Celluar IoT Power – A Golioth Webinar with Jared Wolff

Cellular IoT products struggle with battery life. Getting and staying connected to a cell tower takes a good amount of energy. Though we’re past the days of GSM drawing amps of current (!), it is still costly to open sessions to the tower. Understanding how your device is communicating with the Cloud is crucial to building robust devices that will last years in the field.

This webinar will include Golioth team members, alongside Jared Wolff of CircuitDojo. Jared is an early adopter of Golioth and a Golioth Ambassador, as well as an advocate for Zephyr devices. Golioth utilizes Jared’s nRF9160 Feather board designs in all of our current cellular-based Reference Designs.

What you can expect from this Webinar

First and foremost, we hope to make this a more dynamic and interactive session than many technical webinars. (No one will be reading Powerpoint slides in a monotone voice here!) The session will cover:

  • Understanding your connection to cellular towers
  • Understanding your power draw when in a passive or sleep mode
  • Measurement challenges for embedded cellular projects
  • Methods for saving data (and power) when connecting to the cloud
  • Building robust device health metrics for your fleet
  • System architecture decisions for lower power circuit boards

Sign up now!

This webinar is at 1 pm EST / 10 am PST on January 18th, 2023. If you can’t make it the day of, you can still still sign up to access the on-demand content. Those who attend will have an opportunity to ask questions towards the end of the presentation.

/by
,

How to Debounce Button Inputs in a Real Time Operating System

Embedded systems are riddled with complexity, mainly because they are at the intersection of expertise. Problem are often a mixture of software, electrical and mechanical issues. This is the case even for seemingly simple tasks, such as reliably detecting a button-press.

“Wait — a button-press??”

Let’s think for a second, how a button press is detected:

  1. Initial State: a button-press typically consists of a “normally-open switch”, which through a pull-up/down resistor, is normally “high” or normally ‘low’: this is the initial state.
  2. Press Event: then, when the button is pressed, the switch is closed and the signal transitions towards the opposite state (e.g, low for normally “high” state), and it’s sustained for as long as the button is held by the user.
  3. Release Event: Finally, when the button is released, it goes back to its initial state.
diagram showing the busy electrical signal produced at the beginning of a button press

Image: Signal during a button-press/release sequence where the transitions are outlined; and the scoped signal has noise. Source: GeeksforGeeks.

In an ideal world, we could just look at the signal edges to keep track of the transitions and assume a falling edge is a “press-event” and rising edge the “release event”. In our world, these transitions are affected by electrical transients caused by the mechanical properties of the button actuator. The suppression of this noise it’s commonly called “debouncing”.

“Ok, I get it. How can we `debounce` button-presses?”

There are two main ways we can approach this: the hardware-way and the software-way. Today I’ll touch on both, and detail the technique I prefer to use when debouncing button input with the Zephyr RTOS.

The Hardware Way

The hardware-way focuses on the root cause: the electrical noise. It guarantees the digital signal won’t have such noise during transitions; and it does it through the use of low-pass filters (most probably RC-filters). There are some pretty cool articles that detail this approach (see references at the end of the article).

The Software Way

On the other hand, the software-way is about “ignoring” these false positives on the signal transitions to determine which ones are the real events we’re looking for, and which ones aren’t. Even though there are many ways implement software debouncing, there are two main approaches, depending on whether the variable of interest is the signal state or the transitions: periodic sampling and tracking edge interrupts.

A. Periodic Button State Sampling

Button sampling works by periodically acquiring the signal state, which is buffered on a continuously rolling sample-set. Through detection of consecutive states, the signal change event is detected (either pressed or released). The rule is simple: if there are X-number of consecutive samples with a changed state, we assume the transition really happened. This periodic sampling is often in the order of 10 to 25-ms and is commonly paced by a hardware timer to guarantee fixed intervals and to free some CPU usage.

diagram showing a series of 1 and 0 signals sampled to detect a button press

B. Tracking Edge-Interrupts with Minimum Cooldown

This works by coordinating the detection of edge changes with the spacing between these: there must be a minimum duration before legitimate signal transition. This cooldown phase is commonly implemented through a timer, which kicks-off on the edge-interrupts: the timer gets restarted on each transition and only when it expires (after 10-ms to 25-ms of no edge-changes), the firmware handles the transition as an event.

diagram showing the ignored edges of a button press signal

Software Approach B: Tracking Edge-Interrupts with Minimum Cooldown

In multi-threaded systems, we can leverage the use of RTOS primitives to make the code more modular while simplifying the logic to achieve the same purpose: featuring a thread and semaphores to control the state transitions and decide when to notify the user of the module when an event occurred.

Example Code – Zephyr RTOS

The following code presents an example of achieving the software approach B on Zephyr, with the following observations:

  • We’re using Zephyr GPIO interrupt APIs to keep track of the edge-changes.
  • We’re using the system workqueue as the cooldown mechanism for false positives.
  • Our button-detection module, abstracts both of these details, and only notifies the user of the relevant events: pressed or released.
  • Note: the user callback context is the workqueue handler, therefore: actions on this context shall be kept brief to allow proper functioning of other parts of the system.
#ifndef _BUTTON_H_
#define _BUTTON_H_
 
enum button_evt {
   BUTTON_EVT_PRESSED,
   BUTTON_EVT_RELEASED
};
 
typedef void (*button_event_handler_t)(enum button_evt evt);
 
int button_init(button_event_handler_t handler);
 
#endif /* _BUTTON_H_ */

#include <zephyr/kernel.h>
#include <zephyr/drivers/gpio.h>
#include "button.h"
 
#define SW0_NODE    DT_ALIAS(sw0)
 
static const struct gpio_dt_spec button = GPIO_DT_SPEC_GET(SW0_NODE, gpios);
static struct gpio_callback button_cb_data;
 
static button_event_handler_t user_cb;
 
static void cooldown_expired(struct k_work *work)
{
   ARG_UNUSED(work);
 
   int val = gpio_pin_get_dt(&button);
   enum button_evt evt = val ? BUTTON_EVT_PRESSED : BUTTON_EVT_RELEASED;
   if (user_cb) {
       user_cb(evt);
   }
}
 
static K_WORK_DELAYABLE_DEFINE(cooldown_work, cooldown_expired);
 
void button_pressed(const struct device *dev, struct gpio_callback *cb,
           uint32_t pins)
{
   k_work_reschedule(&cooldown_work, K_MSEC(15));
}
 
int button_init(button_event_handler_t handler)
{
   int err = -1;
 
   if (!handler) {
       return -EINVAL;
   }
 
   user_cb = handler;
 
   if (!device_is_ready(button.port)) {
       return -EIO;
   }
 
   err = gpio_pin_configure_dt(&button, GPIO_INPUT);
   if (err) {
       return err;
   }
 
   err = gpio_pin_interrupt_configure_dt(&button, GPIO_INT_EDGE_BOTH);
   if (err) {
       return err;
   }
 
   gpio_init_callback(&button_cb_data, button_pressed, BIT(button.pin));
   err = gpio_add_callback(button.port, &button_cb_data);
   if (err) {
       return err;
   }
 
   return 0;
}

Check out the working sample code on https://github.com/ubieda/zephyr_button_debouncing

Conclusion

The most important part of debouncing inputs is to understand how far you should go and which approach best suits you. Cost sensitivity tends to favor software debouncing, whereas less CPU usage favors offloading it to the hardware approach. Like any engineering problem, there are 1000 ways to solve it: always favor the simplest (yet effective) solution that works for you.

References

/by
,

Zephyr’s New ZBus Feature with Rodrigo Peixoto

As members and contributors to the Zephyr Project, we keep an eye on new developments. A recently published feature of particular interest because it represents a new way to structure programs and messaging between different parts of your program.

ZBus (Zephyr Bus) is a recently merged feature on the Zephyr Project, which brings a standardized version of event driven architecture in the form of a publish and subscribe model inside of your program. The lead author Rodrigo Peixoto spoke with Golioth about the details of this new feature and how it might help Golioth users to make more responsive, flexible programs. In the associated video, Rodrigo walks through the history and capabilities of ZBus.

Why should you consider an event-driven bus architecture?

The decision to take on a new system architecture is not something to do willy nilly. It’s important to understand where an event driven architecture is a good fit.

The first that I think about is scalability. When you have a bus architecture, adding an additional “listener” can be done with much less work.

Consider the alternative to event-driven architecture. When you want to add a new action (say some code that initiates a sensor reading), you need to call that new code from your trigger event (say a timer being finished). That means you need to know where the trigger is located in the code and make changes there to add the call to your new task. Once you add in the required testing, each additional feature can become burdensom. This scales poorly.

With an event-driven system, the trigger is already set up to publish an event. New tasks can be added that look for the event. You don’t need to change any trigger code, you don’t even need to know where that code lives. This performs well as the amount of data increases, which will ultimately depend on how large you think your system will be.

Flexibility is another consideration. An event-driven bus allows the system to be easily adapted to handle new events or changes in the environment. This means that the system can be easily updated and modified without having to completely rewrite large swaths of code. Another type of “flexibility” is how and where you can re-use your code. This makes it easier to develop and test your system, as well as to troubleshoot and fix any problems that may arise.

Finally, if your device needs to meet critical timing, an event-driven system will not only deal with higher levels of complexity, but also respond quickly to new inputs to the system, such as external events. For example, an embedded system might be designed to control a robot, and it would use event-driven architecture to respond to sensor data from the robot’s environment and control its movements accordingly.

How ZBus implements an event-driven architecture

In ZBus, there are “producers” that generate messages and “consumers” that act upon them. There are also “Filters” help to process raw data (such as a sensor output). Each of these are organized into different “channels” to allow listening on a particular lane of data being produced.

Source: Rodrigo’s ZBus presentation (click for link)

Source: Rodrigo’s ZBus presentation (click for link)

Source: Rodrigo’s ZBus presentation (click for link)

Each of these are built into normal scenarios such as listening for timers and taking a reading from a sensor and then alerting other parts of the program that the data is now available. A common scenario is show below:

How will you use ZBus?

Microcontrollers are used in increasingly complex scenarios and are being asked to do more and more. Connecting a low power device to the internet often requires higher levels of complexity that Zephyr helps with. We expect to see more devices using Ecosystems and RTOSes like Zephyr in the future, and implementing ZBus on high complexity devices.

Are you looking at using an event-driven architecture in your system? Let us know on our forum and tell us how we can help!

/by
, ,

Golioth Developer Training: Open to the public on Dec 14th

A key mission at Golioth is to make it easier for hardware and firmware developers to connect devices to the internet. We do that in two ways:

  1. Providing easy-to-use APIs and SDKs for IoT devices to connect to Golioth Cloud endpoints.
  2. Training developers how to use the Device side code.
Sign Up For Dec 14th Training

We have done many successful training sessions so far, showing individuals and companies how to connect their first devices. Along the way, we have learned that there is a large unfulfilled need in the market for training in the IoT space. So we’re doing it again! We’ll show people how to connect devices and get access to things like:

  • Secure Over-The-Air Updates to constrained devices
  • Command and Control over remote devices
  • Learn how to create and modify settings for remote devices
  • Understand how to implement data tracking from your device

We will be running our first training open to the public on December 14th, 2022. Read more below if you’d like to take part.

Training challenges

Once again we’ll be training developers from afar. We did this back in October for a select group of hardware engineers looking to learn more about Zephyr:

Click to learn more about our experience back in October of 2022

The upcoming training will be built upon the lessons we learned during that training, and our last in-person training at the Hackaday Superconference. In both cases, we used Kasm to provide fully remote development environments so that users don’t need to install anything on their local machine (there are directions on how to do that after the training is over). We think this is an important piece to ensure people can get started quickly.

How to use Zephyr

We currently offer 3 SDKs as part of our device support, including an ESP-IDF SDK, a Modus Toolbox SDK, and a Zephyr SDK (including the Nordic Connect SDK variant). These SDKs cover a wide range of embedded hardware from different vendors.

The training includes some segments that detail how to use Zephyr, a Real Time Operating System (RTOS) that covers a wide range of different hardware platforms. We use it on many of our internal hardware reference designs at Golioth, and it was the first platform we launched. Hardware and firmware engineers who are new to Real Time Operating Systems will continue their learning journey by understanding how the RTOS connects to sensors and low level GPIO and how to manipulate different elements of the subsystems. Once a trainee understands how to get the data off of an external component (like a sensor), the Golioth Zephyr SDK makes it a simple task to forward that data along to the Golioth Cloud.

Requirements and background

We have referred to “Hardware and Firmware Engineers” in this article, because we expect that intermediate to expert level engineers will get the most out of this training. If you are brand new to understanding C or if you have never tried programming embedded hardware before, this might be a frustrating experience. If you would like some pointers to starter content that might prepare you for the training, please ask on our Forum and we will try to get you a customized list of resources that will help prepare you for future versions of this training.

Logistics

  • We are not charging for this training
  • We will be capping the training at 30 people
  • All attendees will be on a first-come, first-served basis
  • Those who are accepted for this training will receive an email with more details
  • You will be expected to purchase your own hardware
    • Details will be sent with your acceptance to this training
    • Be sure you leave enough time for shipping from your local distributor
  • Signing up to take part and not attending will disqualify you from future training

Sign up here


If you have clicked “submit” and don’t see any changes, please scroll back up to the top

/by
, ,

A new API just rolled out with Golioth Zephyr SDK v0.4.0

The Golioth Zephyr SDK is now 100% easier to use. The new v0.4.0 release was tagged on Wednesday and delivers all of the Golioth features with less coding work needed to use them in your application. Highlights include:

  • Asynchronous function/callback declaration is greatly simplified
    • User-defined data can now be passed to callbacks
  • Synchronous versions of each function were added
  • API is now CoAP agnostic (reply handling happens transparently)
  • User code no longer needs to register an on_message() callback
  • Verified with the latest Zephyr v3.2.0 and NCS v2.1.0

The release brings with it many other improvements that make your applications better, even without knowing about them. For the curious, check out the changelog for full details.

Update your code for the new API (or not)

The new API includes some breaking changes and to deal with this you have two main options:

  1. Update legacy code to use the new API
  2. Lock your legacy code to a previous Golioth version

1. Updating legacy code

The Golioth Docs have been updated for the new API, and reading through the Firmware section will give you a great handle on how everything works. The source of truth continues to be the Golioth Zephyr SDK reference (Doxygen).

Updating to the new API is not difficult. I’ve just finished working on that for a number of our hardware demos, including the magtag-demo repository we use for our Developer Training sessions. The structure of your program will remain largely the same, with Golioth function names and parameters being the most noticeable change.

Consider the following code that uses the new API to perform an asynchronous get operation for LightDB State data:

/* The callback function */
static int counter_handler(struct golioth_req_rsp *rsp)
{
    if (rsp->err) {
        LOG_ERR("Failed to receive counter value: %d", rsp->err);
        return rsp->err;
    }

    LOG_HEXDUMP_INF(rsp->data, rsp->len, "Counter (async)");

    return 0;
}

/* Register the LightDB Get callback from somewhere in your code */
static int my_function(void)
{
    int err;
    err = golioth_lightdb_get_cb(client, "counter",
                                 GOLIOTH_CONTENT_FORMAT_APP_JSON,
                                 counter_handler, NULL);
}

Previously, the application code would have needed to allocate a coap_reply, pass it as a parameter in the get function call, use the on_message callback to process the reply, then unpack the payload in the reply callback before acting on it. All of that busy work is gone now!

With the move to the v0.4.0 API, we don’t need to worry about anything other than:

  • Registering the callback function
  • Working with the data (or an error message) when we hear back from Golioth.

You can see the response struct makes the data itself, the length of the data, and the error message available in a very straightforward way.

A keen eye already noticed the NULL as the final parameter. This is a void * type that lets you pass your user-defined data to the callback. Any value that’s 4-bytes or less can be passed directly, or you can pass a pointer to a struct packed with information. Just be sure to be mindful of the memory allocation lifespan of what you pass.

All of the asynchronous API function calls follow this same pattern for callbacks and requests. The synchronous calls are equally simple to understand. I found the Golioth sample applications to be a great blueprint for updating the API calls in my application code. The changelog also mentions each API-altering commit which you may find useful for additional migration info.

The Golioth Forum is a great place to ask questions and share your tips and tricks when getting to know the new syntax.

2. Locking older projects to an earlier Golioth

While we recommend updating your applications, if you do have the option to continue using an older version of Golioth instead. For that, we recommend using a west manifest to lock your project to a specific version of Golioth.

Manifest files specify the repository URL and tag/hash/branch that should be checked out. That version is used when running west update, which then imports a version of Zephyr and all supporting modules specified in the Golioth SDK manifest to be sure they can build the project in peace and harmony.

By adding a manifest to your project that references the Golioth Zephyr SDK v0.3.1 (the latest stable version before the API change) you can ensure your application will build in the future without the need to change your code. Please see our forum thread on using a west manifest to set up a “standalone” repository for more information.

A friendlier interface improves your Zephyr experience

Version 0.4.0 of the Golioth Zephyr SDK greatly improves the ease-of-use when adding device management features to your IoT applications. Device credentials and a few easy-to-use APIs are all it takes to build in data handling, command and control, and Over-the-Air updates into your device firmware. With Golioth’s Dev Tier your first 50 devices are free so you can start today, and as always, get in touch with us if you have any questions along the way!

/by

CMake and Kconfig tricks for common code in Zephyr apps

One of the best tools in the Zephyr ecosystem is the ability to include different code modules in the build configuration. This feature leverages CMake and Kconfig, two tools that are core to Zephyr. But these tools aren’t limited to officially approved code modules. Today we’ll cover some CMake and Kconfig tricks for including common code in your own Zephyr applications.

The problem: common code across different variations of your app

Earlier this month we posted about the developer training that Golioth offers. It centers around a hardware development board called the MagTag, for which we’ve put together a code repository with many different examples. This includes code to drive the ePaper display, update four ws2812 LEDs, service button presses, and parse JSON objects.

The problem is that we need to use a common set of code in most, but not all of the training samples.

Luckily we’ve seen this problem before. The samples in the Golioth Zephyr SDK use common code for networking and shell settings features. Each feature from that common code has its own KConfig symbol, like GOLIOTH_SAMPLE_WIFI_SETTINGS, that is selected to include it in the build. By defining these in the board-specific conf files, libraries are only built for the devices that actually need them. For instance, an Ethernet-connected device doesn’t need WiFi settings.

The example from the Golioth SDK is a good one to study, but the MagTag implementation is a bit simpler so let’s walk through that code.

Golioth’s Developer Training is a self-guided experience that you can explore at your own pace. If you are interested in setting up a training with Golioth staff for your organization, please contact our Developer Relations team, we’d love to chat!

Directory Structure

Directory structure for a Zephyr project with multiple appsFrom the directory structure of the training you can see that we have five different Zephyr programs in this repository, each with their own boards and src subdirectories.

Common code for each app is placed in the magtag-common directory. Header files for each library are located in the the magtag-common/include/magtag-common directory. That might seem a bit convoluted, but it makes for sensible include names like #include "magtag-common/buttons.h".

Creating Kconfig symbols

Common code is individually enabled with a Kconfig symbol. There are a few ways to approach this. One of the easiest is to assign one symbol to indicate your app uses the common code, and then one symbol for each specific library in the common folder. This is done with a Kconfig file inside of the magtag-common directory.

menuconfig MAGTAG_COMMON
    bool "Common helper code for Golioth MagTag Demo"
    help
      Build and link common code that is shared across MagTag samples.

if MAGTAG_COMMON

config MAGTAG_ACCELEROMETER
    bool "Handle accelerometer readings"
    help
      Get accel from DeviceTree and write readings to shared sensor struct

config MAGTAG_BUTTONS
    bool "Process button reads"
    help
      Configure buttons for interrupts with callbacks

config MAGTAG_EPAPER
    bool "2.9\" grayscale ePaper driver"
    help
      Hardware driver for ePaper, including text and partial writes

config MAGTAG_WS2812
    bool "ws2812 helper functions"
    help
      Intialize and update LED color and state

endif # MAGTAG_COMMON

From this code snippet you can see the library-specific symbols are only defined if the MAGTAG_COMMON symbol has been selected.

Using CMake to build in the libraries

Now that we’ve created the Kconfig symbols, a CMakeLists.txt file in the magtag-common directory is used to build in the code based.

zephyr_library_sources_ifdef(CONFIG_MAGTAG_ACCELEROMETER accelerometer/accel.c)
zephyr_library_sources_ifdef(CONFIG_MAGTAG_BUTTONS buttons/buttons.c)
zephyr_library_sources_ifdef(CONFIG_MAGTAG_EPAPER epaper/magtag_epaper.c)
zephyr_library_sources_ifdef(CONFIG_MAGTAG_EPAPER epaper/magtag_epaper_hal.c)
zephyr_library_sources_ifdef(CONFIG_MAGTAG_WS2812 ws2812/ws2812_control.c)

zephyr_include_directories(include)

The include directory is added by default, but the source files are only added if the associated symbol is defined.

Using the common code in a Zephyr application

So far, the common code is completely separate from the each of the apps we want to build. Let’s use the golioth-demo app as an example. To tell the app about the common code, the subdirectory needs to be added to the CMakeLists.txt file in the application subfolder.

cmake_minimum_required(VERSION 3.20.0)

list(APPEND OVERLAY_CONFIG "../credentials.conf")

find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(lightdb)

add_subdirectory_ifdef(CONFIG_MAGTAG_COMMON ../magtag-common magtag-common)

target_sources(app PRIVATE src/main.c)

This codes makes sure that the common directory is only added to the build when the MAGTAG_COMMON symbol is selected. This completes the plumbing work. In this example, the app selects the common code in the prj.conf file:

# MagTag Common Files
CONFIG_MAGTAG_COMMON=y
CONFIG_MAGTAG_EPAPER=y
CONFIG_MAGTAG_WS2812=y
CONFIG_MAGTAG_ACCELEROMETER=y
CONFIG_MAGTAG_BUTTONS=y

And of course, you must include the header files to access the library functions. Here’s an excerpt of the main.c from the golioth-demo app:

/* MagTag specific hardware includes */
#include "magtag-common/magtag_epaper.h"
#include "magtag-common/ws2812_control.h"
#include "magtag-common/accel.h"
#include "magtag-common/buttons.h"

Wrapping up

MagTag Kconfig options shown in menuconfig

Abstracting your common code makes it a lot easier to maintain. This approach also adds entries in menuconfig for each of the libraries. This is a user-friendly feature as it allows you to convey more details in the help file. For more complex uses (like the Golioth samples common libs) it also makes it easier to see the symbol dependency hierarchy.

This example shows the libraries as a part of the repository, however this will work just as well if you commit them to their own repo. From there, they can be included as a git submodule, or with a little creativity you can get your west manifest file to pull the repo whenever west update is run. This is the approach that we’ll be taking with future development. Doing so allows us to lock the project to a specific hash of the common code so that future changes don’t break application code.

/by
, ,

Fully Remote Hardware Training: A Recap of Golioth’s Experience

Understanding hardware and firmware is the first step towards building any kind of “thing” that will be part of an “Internet of Things” (IoT) deployment. For new chips and firmware ecosystems, this often means going through training.

Golioth Developer Relations (the group I’m part of) is tasked with making sure our users understand how to get their hardware connected to the Golioth Cloud. We have been doing in-person (as safety protocols allow) and remote training for Golioth users.

Last week, we did our first training for the broader public, a group of hardware engineers interested in learning more about Zephyr and Golioth. I wanted to discuss some of the tools and methods we used to do a fully remote hardware training. If you’re interested in taking part in the training in the future, please let us know in the associated forum post or by email. There are more details on the available options at the end of this post.

How we do fully remote hardware training

The main challenge with virtual gatherings is that trainees are not sitting in the same room as the trainers. This is an easy problem to understand, but a harder one to solve.

Hardware

The first step was standardization on the hardware. To achieve that goal, we created a list of the bare minimum of parts and shipped them out to all attendees, even those who already had some of the hardware. We wanted to make sure that we had exactly the same hardware in hand. Our trainees were in the US and Europe, so we ordered through DigiKey, with DHL shipping at least a week in advance. The average cost was about $60 of materials in the US, and $85 to Europe, both numbers including shipping, VAT, tarriffs, etc.

For the hardware itself, we use the Adafruit MagTag. It contains the Espressif ESP32-S2 as the main processor on board. This time around we focused on Zephyr for this training, but the board also works great with the Golioth ESP-IDF SDK. Remember, Golioth has 3 SDKs currently (vanilla Zephyr, NCS, ESP-IDF), and we are always working to enable other hardware platforms.

Adafruit MagTag

Video meeting

You might think that after 3 years of a global pandemic, the world would have figured out video software. But the key difference between a simple video call and our remote training is that it’s not just 10 people sitting in a single video session. We want to be able to dynamically “move” between different conversations, especially as trainers.

To achieve this, we used Gather.town. It utilizes tiny avatars you can move around the screen with your keyboard like a video game. As your avatar gets closer to another avatar, your video and audio pops up to those in range. The rugs in the image below also allow “private” areas which work kind of like mini conference rooms. We found this allowed us to dynamically help smaller groups of people while also making announcements to the entire group. Trainees could also choose to work together on different components of the training material.

Our Gather.town Training Location for Hardware Developers

Tooling

The biggest challenge with any training is dealing with firmware tooling, specifically with Zephyr. We have been working on solving this issue for a while. The Zephyr installation continues to get simpler overall, thanks to the Zephyr dev team updating what gets installed for each user. But we still have trainees “walking through the door” (figurative in this case) with a wide range of computers where the tools will be installed. A range of different operating systems (Windows, Mac, Linux–with different distributions) means that there is no one standardized setup. What’s more, if someone has tried out Zephyr in the past, it’s possible there are interactions between the old installation and the new one, not to mention a variety of dependencies within each machine. If we could, we would ask everyone to show up with a brand new installation of Windows or Linux on a laptop…but that’s unlikely to happen.

We wrote in the past about using Kasm and Docker to have a standardized install. Last week’s training represents the first time we used it with real users. In short, it worked! We collaborated with the wonderful team at KasmWeb (makers of the Kasm tool) who gave us seats of their Cloud offering to use with our trainees.

Zero install embedded training with Zephyr using Kasm and Docker

The first step was created a dockerfile and a Docker image that contained everything required for the training. This way we were able to spin up a standardized, browser based UI environment for everyone taking the training. The entire Zephyr toolchain and Espressif support tools were pre-installed in that image, in addition to VScode, a terminal, a browser, and text editing tools.

Trainees open a browser window that looks like Ubuntu (it is, under the hood), and immediately start compiling Zephyr code. At past in-person training sessions, we had issues with bandwidth just downloading the tool packages. While we still needed bandwidth to communicate with the browser-based UI, all of the processing and storage is done on the cloud. Our plan is to use Kasm at our next in-person training.

We continue to evolve our strategy for making training a more seamless process and offering a range of different training to our customers.

Kasm instance of a Docker container with Zephyr / Espressif / Golioth tools installed

Training material

The final piece of the puzzle is the training material itself. Again, this is an evolving solution, because we continue to add more material, and because we track changes to the Zephyr SDK over time.

On Training.Golioth.io, we take people through getting signed up for Golioth, creating a first binary for the MagTag using their Golioth credentials, and then launch them into understanding different parts of Zephyr. We take developers all the way through working with the DeviceTree and understanding how to interact with an in-tree sensor in Zephyr. Our goal is to help hardware and firmware engineers understand how to use some of the most critical parts of an RTOS, and then use those features to communicate back to the Cloud, thereby making their device an “IoT” based device.

training.golioth.io

If you’re interested in trying the training without Golioth present, you can do the “asynchronous” version of our training. Training.Golioth.io is free to the public. You won’t have access to our Kasm server, but you will be able to install the Zephyr toolchain using supplemental directions. Please utilize the Golioth Forums for any questions you have about training.

Results

We scheduled a 2 hour training session, as we know that engineers’ time is valuable. In that time, 100% of the trainees were able to get their devices connected to the internet, pass data back and forth between Device and Cloud, and explore the features of Zephyr. As in any training, each person worked at a different pace and was able to get to different checkpoints. Each trainee could use their hardware and training site at home for later study.

An unfortunate side effect of having multiple tools, including training materials, browser based UI, and video software is a lot of tab/window switching to keep track of it all. In-person training will cut down on the video window, but it would be nice to further integrate the training to include all required materials in one place. We had a “browser within a browser” (Firefox installed within the Kasm container), but that consumed a lot of virtual memory and wasn’t an optimal solution.

The other major restriction is that the binaries our trainees compiled were not directly loaded onto the MagTag hardware. Because we worked inside a virtual (browser based) container, there was no direct connection to the hardware. We also talked about this in the past article about Kasm based development. Because virtualization tools are created to be independent from the hardware, it is difficult to then reverse course and say, “We want this container to talk to USB!”.

We are working with the Kasm team and looking at different ways to program binaries directly from inside the container, and would love to hear about other options people have seen. We solved this during the training by having trainees install the Espressif esptooldirectly on their machines. This still involved some amount of install, but it was limited to a small program. When a trainee completed a build, they could download the binary image from the container to their local machine and load it onto the MagTag using esptool. After that, the MagTag was able to talk back to Golioth.

Future training

We are excited to continue iterating on this training in the future and would love to hear from people that are interested in participating. Here are the options you can try:

/3 Comments/by

Zephyr Threads, Work Queues, Message Queues and how we use them

Zephyr threads, workers, and message queues

Zephyr is a Real Time Operating System (RTOS). That means it’s built to let you do multiple things at the same time using a single (or limited number) of processing cores. To be fair, you’re not doing things at the same time; the RTOS shares processing time across all of the tasks, with a priority system for the more important ones.

The trick with an RTOS is to design your applications so that they utilize the real-time-ness and you don’t miss reacting to real-time events like receiving data from a network, taking sensor readings, or reacting to user input.

In this post, we’ll discuss the difference between Zephyr threads and work queues and show you why you might want to use one versus the other. We’ll also discuss how to use message queues to pass around data between running processes.

Zephyr Threads

A thread is a set of commands to be executed by the CPU. The while(1) loop that runs inside of the main function of your application is a thread. But Zephyr allows you to create multiple threads. Each thread operates independently, with Zephyr’s built-in scheduler deciding when to run each thread.

For example, the Golioth Blue Demo run animations on the LEDs to indicate what mode the device was in. Instead of trying to get the loop in the main thread to update the lights on a tight schedule, we use a thread to run the animations. The main thread of the program deals with flow of the things happening on the device, and it can suspend/resume the animation thread as necessary.

I like to think of threads as tasks that need keep running, things that don’t return like a discrete function would. Your thread should have a while(1) and can call other functions just like you would in your main loop. It is up to you to yield processor control back to the scheduler so that it may run other threads. This is pretty easy, just call any of the k_sleep() functions or k_yield().

/* Every thread needs its own stack in RAM */
#define ANIMATE_PING_STACK	1024

/* Define and initialize the new thread */
K_THREAD_DEFINE(animate_ping_tid, ANIMATE_PING_STACK,
            animate_ping_thread, NULL, NULL, NULL,
            K_LOWEST_APPLICATION_THREAD_PRIO, 0, 0);

/* All animations handled by this thread */
extern void animate_ping_thread(void *d0, void *d1, void *d2) {
    uint8_t spinner_idx = 0;
    while(1) {
        led_state &= ~LEDS_LOGO;
        led_state |= leds_logo_order[spinner_idx];
        refresh_leds(led_state);
        if (++spinner_idx > 7) spinner_idx = 0;
        /* Control is yielded back to the scheduler during */
        /* sleep between animation frames                  */
        k_sleep(K_MSEC(100));
    }
}

int main(void) {
    /* Start animation thread running */
    k_thread_start(animate_ping_tid);

    while(1) {
        /* normal program flow */
    }
}

The example shown here runs an animation when an IoT device is trying to connect to a network. When it begins running, the LEDs will update every 100 milliseconds. Because the loop sleeps in between frames, control is given back to the scheduler or other tasks. Elsewhere in the program, we call k_thread_suspend(animate_ping_tid) and k_thread_resume(animate_ping_tid) to stop and start the animation.

Just be careful that you don’t have multiple threads trying to access the same resource at the same time (protect those resources with a mutex).

Zephyr Work Queues

A Zephyr workqueue is a thread with extra features bolted on. It also requires less setup and management than threads.

The work queue has a built-in buffer to store pending work (functions you want to run). Each item you add will be executed in the order you submitted it, like a “to-do list” for your processor. This is a perfect place for something that you want to run once and return from, but don’t want to do it inside of an interrupt service routine.

The “work” you submit to a work queue is a function. So you’re telling the work queue “run this function, then run this other function, then run this other…” you get the idea. Just set it and forget it–the Zephyr scheduler will run take care of popping work out of the queue and running it until the queue is empty. Many of the sensor readings we do in our demos are handled through work queues.

The work queue expects a specific type of function usually referred to as a “work handler”. You don’t actually give the handler any information, you just tell your work queue you want to add your handler to the list of pending work.

/* Work handler function */
void button_action_work_handler(struct k_work *work) {
    do_something_because_a_button_was_pressed();
}

/* Register the work handler */
K_WORK_DEFINE(button_action_work, button_action_work_handler);

/* Add a work item to the system workqueue from a button interrupt function */
void button_pressed(const struct device *dev, struct gpio_callback *cb,
            uint32_t pins)
{
    k_work_submit(&button_action_work);
}

This code snippet demonstrates using a work queue when a button is pressed. You want to spend as little time as possible in the button interrupt function. All this handler does is queue up a button action function that will be run by the scheduler, sometime after the interrupt service routine ends.

The example above uses the system workqueue. You also have the option of declaring your own workqueues which will need its own stack (remember… it’s a thread with extra features). If your application is freezing up when using the system workqueue, check to see if you’ve run our of stack space and set CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE accordingly.

The challenge here is how to get data to the worker since we can’t pass any parameters. How do you know which button was pressed? One approach is to place your work item inside of a struct along with the data you want to pass (button number, etc.). Then in the handler you can use CONTAINER_OF() to access the data in the structure. This is beyond the scope of today’s post, so let’s talk about a bit simpler approach we often take: using a message queue.

Zephyr Message Queues

A Zephyr Message Queue is a collection of fixed-sized data that is safe to access from multiple threads. You decide what the data is (a variable, a struct, a pointer, etc.) and how many of those objects the queue can contain (limited by your available RAM). Zephyr handles all of the logic necessary for adding to and removing from the queue.

#define BUTTON_ACTION_ARRAY_SIZE	16
#define BUTTON_ACTION_LEN		4
K_MSGQ_DEFINE(button_action_msgq,
        BUTTON_ACTION_LEN,
        BUTTON_ACTION_ARRAY_SIZE,
        4);

void button_pressed(const struct device *dev, struct gpio_callback *cb,
            uint32_t pins)
{
    /* add the current button states to the message queue */
    k_msgq_put(&button_action_msgq, &pins, K_NO_WAIT);

    /* This is a good place to queue a worker to process the buttons */
}

/* Work handler to process actions from button presses */
void button_action_work_handler(struct k_work *work) {
    /* Process everything in the message queue */
    while (k_msgq_num_used_get(&button_action_msgq)) {
        uint32_t button_states;
        k_msgq_get(&button_action_msgq, &button_states, K_NO_WAIT);

        /* Run some function based on which button was pressed */
        if (pins & BUTTON_0) {
            do_something_because_a_button_was_pressed();
        }
        /* Give the schedule a chance to run other tasks */
        k_yield();
    }
}

Continuing with our button analogy, the example above uses a message queue to record the button states for processing after the interrupt service routine returns. I’ve highlighted the lines that are crucial to the message queue system. Notice that the work handler uses a while loop to check if the message queue is empty. This is a nice pattern for processing all of the queued message. I’ve used k_yield() between loops to give the scheduler a chance to run other tasks.

Message queues are a spectacular tool for IoT applications. Golioth has used Message Queues to cache GPS readings while the cell modem is turned off on our Orange Demo. Periodically, the modem will turn on, send all of the readings from the queue to the Golioth servers, then turn off the radio once again to save power.

Threads, Workers, and Queues

We often focus on the “ecosystem” aspect of Zephyr, because we like that so many silicon vendors contribute to the code base. But it’s important to remember that Zephyr is a Real Time Operating System and has a fully featured scheduler. We’ve only just scraped the surface of what you can do with it. Hopefully the above explanations helped you to begin thinking in these patterns, understanding how to divide up the application work, and how to pass data between different parts of your code.

/by
, ,

Zephyr <3 Internet: How Zephyr Speeds Implementation for New IoT Devices by Jonathan Beri

It’s no secret that we like Zephyr Real Time Operating System (RTOS) around here. Nor is it a secret that we’re very interested in the Internet of Things (IoT). Golioth Founder and CEO Jonathan Beri gave a talk at the 2022 Zephyr Developer Summit (ZDS) about why those two things are a perfect match, and how Zephyr can help you create an IoT product faster.

A high level overview

Since this talk was on the first day of ZDS, it was focused on people less familiar with the Zephyr ecosystem. So what should a beginner know about Zephyr and its relationship to IoT devices?

Batteries are included

Zephyr has a couple of key features that makes it a “one stop shop” for building an IoT device. Some other RTOSes rely on the engineer to piece together libraries and implementations to get started, which either means more startup time or reliance on hardware vendors to make a ready-to-go solution for you. Instead, Zephyr provides individual elements already configured to work together:

  • Device drivers
  • An OS kernel (including scheduler)
  • Services
  • A build system

Tying together the ecosystem with these tools provides a consistent experience. This also leads to another very important aspect.

Chip vendor buy-in

Because Zephyr has a well defined system, the chip vendors develop code to make their parts work within the ecosystem. With other RTOSes, that script is flipped; the RTOS is made to work with abstraction layers within the vendor specific ecosystem, meaning there is less likelihood of interoperability with other vendor solutions.

In addition to the chipsets from the vendors, a wide range of boards are supported. Targeting specific boards makes it easier to get dev boards working when getting started, and remapping pins to specific functions on a board doesn’t require a configurator tool like those available in many vendor IDEs.

Talking to the Internet

If you choose an RTOS that isn’t a broader ecosystem, you need to either define your own network layers or lean on a vendor implementation to do it for you. In Zephyr, the network layer and the protocol definitions are done as a part of the community. That means that chip, module, and software vendors are all working towards a common implementation. Add in the fact that there are modems and abstraction layers all the way up the stack, and an engineer using Zephyr doesn’t need to think about all of the pieces it takes to connect to the internet. At Golioth, we are able to switch between various networking technologies easily when using Zephyr.

Protocol Variety

Golioth talks to the internet at a very high level, so network access “just works”. This comes into focus as a small embedded device can talk to the variety of internet protocols available. Most people understand HTTP, but also consider the more common IoT favorites like MQTT and CoAP. Golioth favors CoAP but also supports MQTT. This enables implementations in other parts of the network stack. A good example is devices running OpenThread, a Thread network protocol implementation. As we showed in our recent post about Thread, a small Zephyr-based device utilizes 6LoWPAN and talks over CoAP using UDP packets to talk back to the Golioth cloud.

Last but not least: Security

Zephyr enables a secure connection back to the internet through the network stack and with features like DTLS, the basis of a secure connection over UDP. Security is a deeper topic in Zephyr though, all the way down to secure bootloaders and working within things like TrustZone. Talking to secure elements requires drivers that understand how to communicate with particular chip features (internal or external to a microcontroller). At a very high level, Zephyr focuses on things like Software Bill of Materials (SBOM), so your security teams understand the various software you are pulling into the build.

Built for IoT

Zephyr not only ❤️ internet…it’s purpose built for enabling IoT devices. Paired with Golioth, a Zephyr device has a higher likelihood of getting to market and growing to a massive scale. We’d love to hear your thoughts about the video on our Forum, our Discord, or by email.

 

/by

How to use Zephyr Pin Control (pinctrl) for pin multiplexing and configuration

I feel like I begin every Zephyr article talking about how this RTOS’s greatest strength and greatest weakness is the extent of its hardware abstraction. Such is the case with the Zephyr Pin Control (pinctrl) system that became the default approach for pin configuration and multiplexing back in June. This development made great strides in standardizing how to work with pinmux configuration across many vendors, and unified the process of reassigning pin functions at runtime. These gains are at the expense of needing to understand the new syntax. Let’s dig in!

What is pinctrl?

The pinctrl (Pin Control) system is a standardized way of assigning peripheral functions to pins, a concept adopted from Linux. Pin Control lets us define which pins will be used for special functions (SPI/i2c/UART) and how those pins will be configured (pull up/down resistors, slew rate).

In the past we would have remapped an i2c pin inside the i2c node itself using a syntax like sda-pin = <33>;. With pinctrl we don’t necessarily touch the i2c node–it is instead given the address of a pinctrl nodelabel. The pinctrl-state nodes are where actual pins are specified, using a syntax that might look like &i2c1_sda_gpio33. (The exact format of pin addressing varies by vendor. We’ll discuss how to look up that info.)

So what do we gain from this change? One example is reconfiguring a chip for low-power before entering sleep mode. pinctrl automatically looks for two states: default and sleep. When changing from the default state to the sleep state, pinctrl will automatically reconfigure all of your power-sucking pins for low-power, and set them back up again upon wake.

GPIO mapping and usage remains separate for pinctrl. If you’re blinking LEDs or reading buttons, you don’t need to use pinctrl. When you think ‘pinctrl’, think ‘special-function peripherals’.

Example: use pinctrl to change i2c pins

A great exercise when learning pinctrl is to reassign pin locations on an existing board. I’ve been working with the ESP32s2 and there is a “saola” board definition in Zephyr. If we study the pinctrl file for that board we find that i2c1 normally maps GPIO3 and GPIO4:

i2c1_default: i2c1_default {
	group1 {
		pinmux = <I2C1_SDA_GPIO3>,
			 <I2C1_SCL_GPIO4>;
		bias-pull-up;
		drive-open-drain;
		output-high;
	};
};

This pinctrl node uses a nodelabel of i2c1_default which is associated with the i2c1 peripheral in the board’s .dts file:

&i2c1 {
    clock-frequency = <I2C_BITRATE_STANDARD>;
    pinctrl-0 = <&i2c1_default>;
    pinctrl-names = "default";
};

Let’s say we wanted to change these pins to GPIO33 and GPIO34. We could make our own pinctrl node with a different node label and change which nodelabel is used for i2c1. However, the easiest thing is to simply change the pin assignments in the default node, which allows us to inherit all the other settings from the board definition. This is done by adding an esp32s2_saola.overlay in the boards directory of our application. Remember, an “overlay” allows you to tell the build system that the local changes in your build directory are more important than the default settings in Zephyr, and to adopt the local changes.

&pinctrl {
    i2c1_default: i2c1_default {
        group1 {
            pinmux = <I2C1_SDA_GPIO33>,
                     <I2C1_SCL_GPIO34>;
        };
    };
};

Whenever remapping pins, it’s important to check that the new pin assignments are available. In our case, GPIO33 is available, but GPIO34 is mapped to the pinctrl default for SPI3. To avoid a conflict, that pin should be remapped, or if the SPI3 peripheral is not used the SPI3 node can be deactivated or deleted.

Read the bindings pages for vendor-specific pinctrl info

In the example above we used I2C1_SDA_GPIO33 to remap a pin. This syntax is unique to Espressif chips, and will be different for Nordic, Microchip, NXP, Infineon, or any other vendor chips. The same assignment for a Nordic chip would be something like NRF_PSEL(TWIM_SDA, 0, 30). We admit, this is annoying. But Zephyr is an ecosystem, in addition to being an RTOS. Each vendor does the hard work of making their specific chips work with the rest of the code, an this is the cost of them being in control.

Vendors have documented their pinctrl implementation in their device bindings pages. Head over to the Zephyr bindings index and look for the pinctrl entry for your vendor. Espressif’s entry points to a set of macro definitions that lay out every possible function for each pin. Nordic’s entry indicates that rather than define each pin, we can use a macro to call out the nRF pin function along with the port and pin assignments.

/* Nordic pinctrl */
i2c2_default: i2c2_default {
    group1 {
        psels = <NRF_PSEL(TWIM_SDA, 0, 30)>,
            <NRF_PSEL(TWIM_SCL, 0, 31)>;
    };
};

/* Espressif pinctrl */
i2c1_default: i2c1_default {
    group1 {
        pinmux = <I2C1_SDA_GPIO3>,
             <I2C1_SCL_GPIO4>;
        bias-pull-up;
        drive-open-drain;
        output-high;
    };
};

The syntax itself is different from one vendor to another. The snippets above demonstrate that, in addition to pin assignment differences, Nordic uses psels to set the pins while Espressif uses tmux. Always check the bindings index pages, and when in doubt, look to the board definition files as examples.

Dynamic control

A full guide on dynamic control is out of scope for this post. To touch on the topic, choosing to use dynamic pinctrl makes it possible to update pin function at runtime by moving the pin configuration from ROM to RAM. The pinctrl_update_states() function is then used to assign a new state, executing the necessary configuration changes in the process.

Check the pinctrl test app for an example of dynamic control.

The good and the bad of pinctrl

I generally think the move to pinctrl is a good one. Sure, there are syntax differences between vendors, but that was true before the change as well. Now you can expect to find all important pin assignments inside of the pinctrl state nodes, which is a bit easier to reference, and makes it easier to see not just the pin numbers, but the configuration (slew rate, pull up/down, etc.).

i2c1_default: i2c1_default {
    phandle = < 0x5 >;
    group1 {
        pinmux = < 0x301821 >, < 0x2f97e2 >;
        bias-pull-up;
        drive-open-drain;
        output-high;
    };
};

One change that I have not been as happy about is in troubleshooting Devicetree issues. After building a project, it’s often useful to inspect the build/zephyr/zephyr.dts file that is generated. I’ve included an example above, can you tell me if my pins have been properly assigned?

The more you use pinctrl, the more useful it becomes. I’m sure you’ve noticed that some nodes include groupN style groupings. This associates multiple pins with the same needs (ie: for SPI, these three pins are outputs, but this other pins is an input) which is a welcomed shorthand that saves us from assigning pin flags over and over again. Take some time to learn the new system and start taking advantage of its perks!

/by