Debugging nRF9160 Zephyr applications with Ozone

Ozone is a free graphical debugger for embedded firmware from SEGGER. It’s a powerful tool that can give you deep visibility into what’s happening in your embedded system. It’s especially useful when debugging nRF9160 Zephyr apps. Sorting out multiple threads and multi image builds can be tough, but this is the tool you want.

In our previous post Taking the next step: Debugging with SEGGER Ozone and SystemView on Zephyr, Chris Gammell wrote about how to set up a SEGGER Ozone project to debug a Zephyr app running on the i.MX RT1060 Evaluation Kit. It’s a great general introduction to debugging a Zephyr app in Ozone and profiling the RTOS runtime behavior SystemView.

When I was trying to set up a similar Ozone project for debugging the Nordic nRF9160 SIP, I ran into a few snags along the way. Today, I’ll share what I’ve learned!

In this article, I’ll walk through how to:

  • Configure a nRF9160 Zephyr app for thread-aware debugging
  • Create an Ozone project for nRF9160 with the New Project Wizard
  • Modify the Ozone project to support debugging nRF9160 multi-image builds

Hardware Configuration

In the examples that follow, I’ll be using the Nordic nRF9160 DK board. This development kit from Nordic has a SEGGER J-Link OB debugger built into the board, so an external J-Link debugger is not required to follow-along with the examples.

 

Thread-awareness support in Zephyr

In a typical Zephyr app built using the Golioth Zephyr SDK, there will be multiple threads. For instance, one for the app’s main loop, one for the Golioth system client, and others for the UART shell, logging subsystem, network management, etc.

SEGGER provides a Zephyr RTOS plugin for Ozone that can show the status of each thread, but it requires that the Zephyr firmware is built with support for thread-aware debugging. Zephyr provides a CONFIG_DEBUG_THREAD_INFO Kconfig symbol that instructs the kernel to maintain a list of all threads and enables thread names to be visible in Ozone.

While you could simply add CONFIG_DEBUG_THREAD_INFO=y to your app’s prj.conf file, you probably only want to enable this extra debug info when you are building for debugging purposes. Instead, we can create an additional debug.conf Kconfig file that will only get merged in when we pass the -DEXTRA_CONF_FILE=debug.conf argument to the build system.

Since this article is about using Ozone for thread-aware debugging, we’ll use the zephyr/samples/basic/threads/ app from the nRF Connect SDK Zephyr repo as our example app in this article.

If this is your first time building one of the Zephyr sample apps, make sure to complete the nRF Connect SDK Installation Guide first to make sure your dev environment is set up correctly.

How to Enable Thread-Awareness

First, create a zephyr/samples/basic/threads/debug.conf file and add the following lines:

CONFIG_DEBUG_THREAD_INFO=y

# CONFIG_DEBUG_THREAD_INFO needs the heap memory pool to
# be defined for this app
CONFIG_HEAP_MEM_POOL_SIZE=256

Next, build the firmware, specifying the debug.conf file to be merged into the build configuration:

cd <your ncs workspace>/
west build -p -b nrf9160dk_nrf9160_ns zephyr/samples/basic/threads/ -- -DEXTRA_CONF_FILE="debug.conf"

If the build completed successfully, you’ll see the build/zephyr/zephyr.elf file we need to start a debugging session in Ozone.

Create the Ozone project

Now that we’ve built the firmware, you can launch Ozone and use the New Project Wizard to create an Ozone project:

Choose the nRF9160_xxAA device:

Select the J-Link device you want to use:

Select the build/zephyr/zephyr.elf ELF file we generated in the previous section:

Leave these fields the default values for now (we’ll update them later on):

After clicking “Finish”, you’ll see the Ozone project window appear.

In the “Console” window, run the following command to load the Zephyr RTOS plugin:

Project.SetOSPlugin("ZephyrPlugin.js");

You should now see a new “Zephyr” window in the Ozone project (if not, click on “View” → “Zephyr” to show the window):

Finally, save the project file by clicking on “File” → “Save Project as…”:

Start the debug session

Now that we’ve configured the Ozone project, we can start the debug session.

Click on “Debug” → “Start Debug Session” → “Download & Reset Program”:

Surprise! When the firmware starts to run, you’ll see a pop-up window indicating that the target has stopped in a HardFault exception state!

At this point, you might be wondering what’s going on here…

We’ve followed the same basic steps as outlined in our previous article, so why isn’t this working for the nRF9160?

Here’s a hint: the answer has to do with multi-image builds.

The missing step: flashing the merged image

You may have noticed that the board argument we passed to west build (nrf9160dk_nrf9160_ns) ends in _ns. This suffix is an indicator that the firmware will be built with Trusted Firmware-M (TF-M). This is the reference implementation of ARM’s IoT Security Framework called Platform Security Architecture (PSA).

TFM uses the ARM TrustZone security features of the nRF9160’s Cortex-M33 MCU to partition the MCU into a Secure Processing Environment (SPE) and Non-Secure Processing Environment (NSPE).

Here’s how the boot process works in a nutshell:

  1. When the MCU boots up, it starts executing in the secure environment (SPE).
  2. The boot process can optionally start with a secure bootloader chain using NSIB and/or MCUboot.
  3. If used, the bootloader starts TF-M, which configures a part of the MCU memory and peripherals to be non-secure.
  4. TF-M starts your Zephyr application which runs in the non-secure environment (NSPE).

When we build for _ns build targets, the TF-M image is automatically built and linked with the Zephyr app. If you look in the build/zephyr/ output directory, you’ll see a file named merged.hex, which is a single merged file containing the MCUboot bootloader (optional), the TF-M secure image, and the non-secure Zephyr app.

west flash knows to flash the full merged image, but Ozone doesn’t do this by default!

We need to configure Ozone to load the full merged image and start execution in the secure environment.

Fixing the Ozone project file

We’ll make a couple changes directly in the Ozone project file, which can be opened within Ozone by clicking “File” → “Edit Project File”:

Flash the merged image

Navigate to the TargetDownload section of the Ozone project file and add the following to configure Ozone to flash the merged image (changing the path to match the merged image file in your project):

/*********************************************************************
*
* TargetDownload
*
* Function description
* Replaces the default program download routine. Optional.
*
**********************************************************************
*/
void TargetDownload(void)
{
  Exec.Download("$(ProjectDir)/build/zephyr/merged.hex");
}

Fix the Vector Table & PC addresses

Navigate to the _SetupTarget section of the Ozone project file and make the following changes:

  1. Set the vector table address to 0
  2. Read the entry point program counter address from the vector table
/*********************************************************************
*
*       _SetupTarget
*
* Function description
*   Setup the target.
*   Called by AfterTargetReset() and AfterTargetDownload().
*
*   Auto-generated function. May be overridden by Ozone.
*
**********************************************************************
*/
void _SetupTarget(void) {
  unsigned int SP;
  unsigned int PC;
  unsigned int VectorTableAddr;

  VectorTableAddr = 0;
  //
  // Set up initial stack pointer
  //
  SP = Target.ReadU32(VectorTableAddr);
  if (SP != 0xFFFFFFFF) {
    Target.SetReg("SP", SP);
  }
  //
  // Set up entry point PC
  //
  PC = Target.ReadU32(VectorTableAddr + 4);
  if (PC != 0xFFFFFFFF) {
    Target.SetReg("PC", PC);
  } else {
    Util.Error("Project script error: failed to set up entry point PC", 1);
  }
}

When you save the project file, you should get a modal pop-up asking if you want to reload the project.

Choose “Yes”:

Restart the debug session:

After the image has been flashed to the device, you should see the debugger halted at main:

Click “Debug” → “Continue”:

The firmware should now run without exceptions!

Summary

Hopefully this helped you get started debugging the nRF9160 with Ozone.

The nRF9160 is fully supported in Zephyr, and has the highest level of support in the Golioth IoT device management platform (Continuously Verified). With Golioth, you can connect and secure your devices, send sensor data to the web, update firmware over-the-air, and scale your fleet with an instant IoT cloud.

Try it today—with Golioth’s Dev tier your first 50 devices are free!

/by

How to turn Helper Code into a Zephyr Module

Embedded developers always maintain sets of helper code that get used across multiple projects. With Zephyr RTOS, you can easily turn your helper code into a portable Zephyr module.

Creating a Zephyr module means you can version control your code, making changes in one centralized place, while targeting a specific git commit, tag, or branch of that code in a project. You only need to add two or three additional files to qualify as a Zephyr module. And once the module is published you can make the code available to your Zephyr projects simply by adding it to your manifest file.

A Working Example

Ostentus faceplate installed on Aludel-mini reference design

You may remember reading about Ostentus, Golioth’s custom faceplate for conference demos. This I2C display can be added to any Zephyr project by leveraging some helper code that simplifies sending data from the device to the faceplate. As this code will change in the future, it isn’t maintainable to copy it between projects, so we turned it into a Zephyr module. Let’s use this as an example. Here are the steps:

  1. Create a new repository to store your helper code
  2. Add a CMake file and an optional Kconfig file
  3. Add a module.yml file that tells Zephyr where to find files in the module repo

That creates the module, and the final step is to add it to your project’s West manifest.

1. Create a new repository

For our example, we have just one C file and one header file. I created a new repository to store these files. Place the header file in a directory named include; you may add subdirectories if you want there be a category-like prefix when the header is included (e.g. #include <yoursubdirectory/yourfile.h>).

Here are the important parts of my tree for this module:

➜ tree
.
├── include
│   └── libostentus.h
└── libostentus.c

2. Add CMake and Kconfig files

Next we need to add a CMake file to include our source code in the build. You also have the option of including a Kconfig file. I’m going to do this optional step because it allows me to create a symbol that will turn on or off this library in the project build.

First, I create the Kconfig file in the root of the repo that adds a unique symbol (LIB_OSTENTUS) for this library:

config LIB_OSTENTUS
    bool "Enable the helper library for the Golioth Ostentus faceplate"
    default n
    depends on I2C
    help
      Helper functions for controlling the Golioth Ostentus faceplate.
      Features include controlling LEDs, adding slides and slide data,
      enabling slideshows, etc.

Next, I add the C file and the header file to the build using a CMakeLists.txt file in the root directory of the repository. Note that this uses the Kconfig symbol created in the last step to decide whether or not to build with these files.

if (CONFIG_LIB_OSTENTUS)
  zephyr_include_directories(include)
  zephyr_library_sources(libostentus.c)
endif()

3. Add module.yml

The glue that holds this together is the module.yml file, which must be placed in a zephyr subdirectory of the repository. Here I tell it where to look for the CMake and Kconfig files, although there are other options that can be added to this file.

build:
  cmake: .
  kconfig: Kconfig

I find the relative paths of this file to be a bit confusing. But the gist of it is that this file will be located at zephyr/module.yml and the paths in the file are based on the parent of that zephyr subdirectory.

Congratulations, we now have a Zephyr module made up of the following files:

➜ tree
.
├── CMakeLists.txt
├── include
│   └── libostentus.h
├── Kconfig
├── libostentus.c
└── zephyr
    └── module.yml

Using the Module in a Zephyr Project

Using the newly created module follows the familiar Zephyr pattern of adding it to the projects section of your West manifest file. (You can find your manifest file by running west manifest --path.)

manifest:
  projects:
    - name: libostentus
      path: modules/lib/libostentus
      revision: v1.0.0
      url: https://github.com/golioth/libostentus

self:
  path: app

Calling west update will now checkout the helper code and place it in modules/lib/libostentus. Remember that I added a Kconfig symbol in my example, so I need to add that to the project prj.conf file or a <board>.conf file to include this code in the build.

CONFIG_LIB_OSTENTUS=y

Using the module in your C files is a simple matter of adding the #include and then calling the functions.

#include <libostentus.h>

int main(void)
{
    clear_memory();
    show_splash();

    k_sleep(K_FOREVER);
}

Summary

Golioth supports a wide variety of hardware and use cases, and part of our strategy to make that scalable is to write and maintain modular code. The Golioth Zephyr SDK is a Zephyr module, which makes it easy to include in your project and easy to lock to a known version until you’re ready to upgrade to a newer version. We’ve used this same Zephyr module approach for helper code when building our numerous reference designs. It works well and I encourage you to adopt modular practices for your own work.

This process of creating a Zephyr module is a nice improvement over copying and pasting code between projects because it implements reliable revision control. It is also an intermediary step between implementing a project-specific driver, and creating a modular Zephyr driver. The Ostentus faceplate should eventually be converted to a full Zephyr driver, and enabled directly from Devicetree. But that’s a post for another day!

In the meantime, if you find yourself in need of a device management service with robust OTA, data handling, fleet settings, RPC, and more, give us a try. With Golioth’s Dev tier your first 50 devices are free!

/by

Troubleshooting IoT Cellular Connections with Nordic nRF9160

Troubleshooting IoT Cellular

Cellular connected IoT can be intimidating, especially when the cellular connection doesn’t work, or only works intermittently. Today we will explore Nordic’s LTE Link Monitor and Cellular Monitor applications to show how you can troubleshoot cellular connection using the nRF9160 DK as the development board.

Programming the modem firmware

The modem allows applications to send and receive data on the cellular network, and the application talks to the modem via the AT commands. Every nRF9160 SiP has modem firmware (separate from your application code) that is provided as a pre-compiled binary file, signed and encrypted by Nordic Semiconductor.

It’s always beneficial to update the modem firmware to the latest version. In fact, this is the first thing you should try. Make sure you have the nRF Connect for Desktop tool installed and follow these steps to do so:

  1.  Download the latest modem firmware (at the moment of writing, it’s v1.3.5),
  2. Open the Programmer Application from nRF Connect for Desktop
  3. Connect the nRF9160 DK to your computer and select it in the Programmer Application
  4. Click the Add file button, add the mfw_nrf9160_1.3.5.zip file
  5. Flash modem firmware with the Write button

LTE Link Monitor

If updating the firmware didn’t work, it’s time to look at the status of the cellular connection. LTE Link Monitor is part of the nRF Connect for Desktop tool. I works alongside a modem client application that monitors the modem status and activity using AT commands.

For this demo, we’ll use Nordic’s at_client sample for the nRF9160 DK, which can be found at nrf/samples/nrf9160/at_client (Nordic changed folder names over the summer so this may be in samples/cellular/at_client for you).

The AT Client sample demonstrates the asynchronous serial communication taking place over UART to the nRF9160 modem, enabling you to use an external computer to send AT commands to the LTE-M/NB-IoT modem. This facilitates the reading of responses or analyzing of events related to the nRF9160 modem.

Note that not all commands are supported in all modem firmware versions; that is why the first step was to update the modem firmware to the latest version.

Switch to NB-IoT standard with AT commands

In our recent blog post, we showed how to switch the preferred cellular connection standard from LTE-M to NB-IoT with configuration files. Now, we are going to do the same thing with AT commands and later show how the two are connected with Zephyr’s Kconfig Configuration System.

To switch from the default LTE-M connection standard to NB-IoT, do the following:

  • Flash the at_client example built for the nRF9160 DK,
  • open the LTE Link Monitor Application and connect to the development board,
  • check the modem system mode with AT%XSYSTEMMODE? command.

XSYSTEMMODE AT command is used for enabling system modes, and the command response syntax is:
%XSYSTEMMODE: <LTE_M_support>,<NB_IoT_support>,<GNSS_support>,<LTE_preference>.

The response we got was: 1,0,1,0, which means our device is set for LTE-M and GNSS support.
Now, let’s change it to NB-IoT.

  • Send AT%XSYSTEMMODE=0,1,0,0,
  • Send AT+CFUN=1 to set the device to full functionality
  • Send AT+CFUN?

After sending the AT+CFUN? command, the nRF9160 DK has connected to the cellular network T-Hrvatski Telekom using the NB-IoT standard and has obtained an IP address 178.160.19.177.


With the at_client sample, you can test and manually send all AT commands to the modem.

Cellular Monitor

Cellular Monitor is another nRF Connect for Desktop application used for capturing and analyzing modem traces to evaluate communication and view network parameters. Let’s use it with Golioth’s hello sample, which can be found at https://github.com/golioth/golioth-zephyr-sdk/tree/main/samples/hellogolioth/samples/hello.

But before we start, we need to add CONFIG_NRF_MODEM_LIB_TRACE=y to prj.conf file in the sample directory, PSK and PSK-ID as stated in the README file, build the sample, and flash the nRF9160 DK.

After that, open the Cellular Monitor Application, click the Start button in the upper left corner to capture a trace and reset the nRF9160 DK. When capturing a trace, the data is saved to the .mtrace binary file so you can view previously collected traces in the Cellular Monitor Application.

The trace data is categorized into the following 6 dashboard panels:

  • LTE Network
  • Device
  • Power Saving Mode
  • SIM
  • Connectivity Statistics
  • PDN (Packet Data Network)

The Packet Event Viewer visualizes communication at the AT command, Radio Resource Control (RRC), Non-access Stratum (NAS), and Internet Protocol (IP) levels.

For this example, I have enabled the CONFIG_LTE_NETWORK_MODE_NBIOT symbol in the configuration file, and we can see in the Packet Event Viewer the XSYSTEMMODE AT command is sent to the modem automatically during the configuration.

Conclusion

In this demo, we have shown how to start troubleshooting Cellular Network problems with LTE Link Monitor, how to use the Cellular Monitor Application, and how the modem can be configured with Zephyr’s Kconfig Configuration System. You should consider using these tools to observe your modem configuration when it is correctly functioning. This provides much of the intuition you will need when you are called upon to troubleshoot a misbehaving cellular modem!

Want to see what happens when your device is sending and receiving data over the network? Give Golioth a try, our Zephyr SDK has a number of ready-to-use samples and your first 50 devices are free on our Dev Tier.

/by

Displaying Firmware Version and Battery State with Zephyr

We recently open-sourced the Golioth Reference Design Template that we have been using internally as the starting point for our growing library of reference designs. Out of the box, the template provides an end-to-end working firmware example showcasing all of Golioth’s key features. You can read more about what’s included in the Reference Design Template in the announcement post Open Sourcing Golioth Reference Designs and Template.

The Reference Design Template firmware currently supports two boards: the Nordic nRF9160-DK and the Golioth Aludel Mini. The Aludel Mini is an internal prototyping platform designed by Golioth for rapidly building and testing proof-of-concept ideas using widely available off-the-shelf modules. The Aludel Mini integrates a nice ePaper display (Ostentus) into the enclosure, and we have been using it to display custom sensor readings specific to each reference design—for example, on the DC Power Monitor it displays current & voltage, and on the Air Quality Monitor it displays particulate matter, CO₂, and weather data.

As I was working on some of the Golioth reference designs, I found myself occasionally checking the Golioth Console to confirm the firmware version was correct (for example after an OTA update) or to check the remaining battery level. The Golioth Console makes it easy to monitor values like this once the device has connected to the network, but it would be helpful to display this info immediately on the ePaper display when the device boots up (and without having to connect to the device’s UART console).

Luckily, this turned out to be a really easy addition!

Displaying the firmware version

The Golioth Reference Design Template is built with support for the MCUboot bootloader. The firmware version is defined and later made available to the Zephyr application firmware via the KConfig symbol CONFIG_MCUBOOT_IMAGE_VERSION.

When the app boots up, the firmware logs the current firmware version:

LOG_INF("Firmware version: %s", CONFIG_MCUBOOT_IMAGE_VERSION);

Now, it also displays a “Firmware” slide that shows the currently running firmware version on the Ostentus ePaper display:

Displaying the battery voltage

The Aludel Mini board uses a SparkFun nRF9160 Thing Plus module that has the ability to run off a Li-Poly battery. The battery voltage can be sampled using the ADC on the nRF9160 using the onboard voltage-divider circuit:

We can enable support for this voltage-divider circuit in Zephyr by adding a voltage-divider-compatible node to the board’s device tree definition. This specifies the ADC channel to use, the values of the resistors in the voltage divider circuit, and which GPIO is used to enable power to the divider:

/ {
  vbatt {
    compatible = "voltage-divider";
    io-channels = <&adc 7>;
    output-ohms = <100000>;
    full-ohms = <(100000 + 100000)>;
    power-gpios = <&gpio0 25 0>;
  };
};

The Zephyr tree provides an example of how to read this voltage divider circuit in zephyr/samples/boards/nrf/battery. I was able to use this sample as a starting point for quickly adding some simple battery monitoring to the Reference Design Template.

Now, whenever the template firmware is built with our custom  CONFIG_ALUDEL_BATTERY_MONITOR Kconfig symbol enabled, the firmware displays two slides: one that shows the current battery voltage and another that displays the estimated remaining battery level.

Golioth Reference Designs

Firmware version and battery status display slides are now included in all the Golioth reference designs we’ve published. You can check out the full set of reference designs on the Project page at https://projects.golioth.io. And as always, we’d love to hear about what you’re building. Show off your projects and ask questions over on the Golioth Forum.

/by

Manifests: Project Sanity in the Ever-Changing Zephyr World

If you take one thing away from this [talk], it should be this: Manifest files are a great way to manage revision control in your Zephyr applications.

Mike Szczys is my teammate in the Developer Relations group and the primary firmware engineer creating Golioth Reference Designs. We build on top of the Golioth Zephyr SDK to create showcase examples of how you can use Golioth firmware and cloud capabilities to create real-world applications quickly.

To keep everything straight, we rely heavily on Zephyr manifest files to ensure we are always building the correct code each time. We include things like the Zephyr codebase, the Golioth Zephyr SDK codebase, the Nordic Connect SDK codebase, and any custom code we write, all of which might be at different points in their lifecycles. It’s not a stretch to say that it’d be nearly impossible to manage all the code we use on a daily basis without manifest files. Mike used the knowledge he has built to give the above talk at the Zephyr Developer Summit 2023, which was part of the larger Embedded Open Source Summit in Prague in June of this year.

What is a manifest?

Zephyr utilizes a manifest file, which is part of the west meta-tool. These are a tough topic at first because they have multiple levels of inheritance possible. If you’re using a vendor-provided manifest file (or using one of the vendor SDK tools that hides the complexity), you likely will have an easy time following along their path. But once you want to start customizing your own project, you need to dig in and learn how they work.

A visualization of the west meta tool, showing how confusing things can get (image from Zephyr West page)

Mike points out that at their core, manifest files manage hierarchy and provide a revision-controlled record of what should be included in a project. You can directly call out which version of a piece of code you want to include. As the subsystems you utilize in your project continue to upgrade over time, you can choose to lock to the older version. More importantly, you can make a concerted effort to change the version and then test the upgrade has not broken your build or introduced unknown behaviors in your program.

Structuring your Zephyr application

While we normally avoid being overly prescriptive on this blog, we have a few opinions about how to find success with Zephyr applications. Some of this comes from seeing Zephyr projects go wrong in the past. Some examples of this are:

  • Cloning the Zephyr tree and putting it into your project repository
  • Placing your code among Zephyr samples
  • Having a standalone application repository (good) that references the Zephyr tree somewhere else on your machine (bad, because you don’t have a guarantee that the tree isn’t being used by some other Zephyr repo)

The Right Way™ (at least how we see it!) is to put a manifest file in your application and then specifically call out the versions of all dependencies. Asgeir Stavik Hustad guest-wrote about this method on the Golioth blog last year and we have taken those ideas and extended them even further (and changed a few things). The downside is that you will have many independent copies of the Zephyr tree on your hard drive…but hard drive space is cheap and making mistakes in a repo is very expensive. 

Examples and more!

I could continue to summarize the talk here, but it’s best to go and watch Mike’s examples as part of the talk. If you’re not a video person, feel free to scroll through the slides, embedded below. We love talking about Zephyr and would love to hear about your challenges and successes using manifests over on our forum.

https://www.youtube.com/watch?v=PVhu5rg_SGY

/by
,

How to Configure VS Code for Zephyr Development

Visual Studio Code, colloquially known as VS Code, has become the de-facto Swiss Army Knife of Integrated Development Environments (IDEs). It is already configured for a lot of different languages and ecosystems when first installed. It’s also great for developing Zephyr RTOS projects, but not out of the box. Jonathan Beri presented at talk at the 2023 Embedded Open Source Summit detailing how to configure VS Code for Zephyr development.

To follow along, check out Jonathan’s example repository for us Zephyr in Visual Studio Code.

New to Golioth? Sign up for our newsletter to keep learning more about IoT development or create your free Golioth account to start building now.

VS Code for Embedded Development

Most of the pre-configuration rolled into the stock installation of VS Code centers around languages used in web development. But increasingly we see the IDE called upon for embedded system development–moving beyond merely working with source code to include native debugging and flashing. These more specialized features tend to be the areas requiring custom configuration. Notably, Nordic and NXP are both actively developing VS Code environments for Zephyr.

For those who prefer to maintain their own workspace configuration, selecting the right extensions is a good place to start. The Microsoft’s extensions for C/C++ and Cmake are table stakes for embedded development. Jonathan also recommends the Cortex Debug plugin and suggests that Microsoft’s Embedded Tools is a newer plugin you may find useful.

Diving into Configuration Files

The meat of the talk comes about sixteen minutes in as Jonathan walks through the settings files he’s using to configure his workspace: settings.json, tasks.json, launch.json, and extensions.json. You can view these files in the example code repository.

Telling VS Code where to find GCC will get syntax highlighting and linting working well. Preventing CMake from trying to auto-configure Zephyr projects will silence a lot of the popup window noise when opening a new VS Code window. And setting up the task runner is akin to configuring aliases for your oft-used commands, accessible from the VS Code command palette. Finally, debugging configuration and recommended extensions round out the workspace-specific configurations using the built-in “profile” features.

All the Things You’ve Been Missing from VS Code

If you’ve already tried developing for Zephyr in VS Code it’s easy to forget what you’ve been missing. The live demo begins at about 20:30 and immediately shows the most basic annoyances have been solved. There are no longer random red squigglies sprinkled throughout your code, because VS Code now knows where to find all of the header files. You can jump to OS-specific function declarations as expected and access the function syntax hinting because the entire Zephyr tree is accessible for the extensions. These features are not novel, they’re just not fully configured out-of-the-box.

Building and flashing works as expected, because your target board and programmer have now been specified in the configuration. They’re triggered via simple to remember commands like west build that Jonathan added to the command palette via those JSON files. The serial terminal connection to the device is available in the IDE without having to juggle external console windows. The general noise floor of the IDE is almost non-existent at this point, all thanks to a comprehensive configuration that makes the developer experience worlds better.

Modern Tools for Modern IoT

Zephyr RTOS is paving the way for the next generation of embedded devices. As the Founder and CEO of Golioth, Jonathan Beri recognized the power of Zephyr to support cross-vendor hardware in the IoT space, which makes it possible for Golioth to support the hardware that you the choose (and not a narrow set of hardware required by your IoT Cloud). As this talk shows, he spends a lot of hands-on time with the RTOS and is sharing the workflow he has developed over the last several years.

Set aside an afternoon to prototype your IoT Fleet. Your first 50 devices are free with Golioth’s Dev Tier. Follow Jonathan’s examples for using VS Code, and you’ll be sending sensor data to the cloud in a matter of hours!

/by
,

Switching Between LTE-M and NB-IoT Cellular Standards in Zephyr

It’s a tale as old as IoT. You get a project with a cellular connection working, but when you move to a different physical location, all of a sudden you cannot connect to the internet. You know your network provider has cellular coverage in the area, and your SIM card is paid up. So why is data not working?

While there are a number of carriers that offer nearly global coverage, not all coverage is the same. IoT devices generally use the NB-IoT or the LTE-M standard. Each is different and has different use cases. Golioth is your instant IoT cloud, and works with devices using either standard. Let’s learn more about what the differences are and how to use them in your IoT projects.

NB-IoT vs. LTE-M

NB-IoT stands for Narrowband IoT. It is a low-power wide-area network (LPWAN) radio technology standard developed by 3GPP for cellular network devices. NB-IoT uses a subset of the LTE standard, but limits the bandwidth to a single narrow-band of 200kHz. It’s suitable for applications that require more frequent communication with the backend (cloud), and it doesn’t support tower handoff. NB-IoT is suitable for stationary devices, such as smart metering devices.

On the other hand, LTE-M standard is designed to transfer low-to-medium amounts of data (200-400 kbps) across a wide geographical range and supports cellular tower handoff. LTE-M is suitable for mobile applications such as asset tracking and fleet management.

There are a couple of older standards you may remember hearing about. Largely these have been grouped into the two mentioned above. LTE Cat-M1 is part of the LTE-M standard. LTE Cat-NB1 and Cat-NB2 are part of the NB-IoT standard.

Cellular World Coverage

Screenshot of an interactive world map of NB-Iot and LTE-M coverage from gmsa.com

Click the image to open an interactive map of NB-Iot/LTE-M coverage at gmsa.com

Depending on your location, you might have NB-IoT or LTE-M coverage. Some areas of the world have coverage for both standards, so choosing which one to use can be challenging. Check the interactive map at GSMA to see how your country is adopting for the future.

Switch Between Cellular Standard in Zephyr

Nordic’s nRF Connect SDK (based on Zephyr RTOS) offers an elegant and simple way of prioritising the connection standard. A pair of network mode Kconfig symbols are available when building for the Nordic nRF9160 cellular modem.

By defining the CONFIG_LTE_NETWORK_MODE_NBIOT symbol in board-specific conf files, the NB-IoT cellular standard will be preferred. On the other hand, CONFIG_LTE_NETWORK_MODE_LTE_M will prioritise the LTE-M standard.

Try including one of these KConfig symbols in your application, and compare the differences in connection time when connecting to Golioth.

# Prioritise NB-IoT
CONFIG_LTE_NETWORK_MODE_NBIOT=y

# Prioritise LTE-M
CONFIG_LTE_NETWORK_MODE_LTE_M=y

Conclusion

In this blog post we talked about differences between NB-IoT and LTE-M standards and how to prefer one over the other in the Zephyr ecosystem. In the next blog post, we’ll investigate automatic switching between NB-IoT and LTE-M, connection time-out, and how Zephyr RTOS handles all of that.

Get your IoT fleet started today. With the Golioth Dev Tier your first 50 devices are free, so try Golioth now!

/by

Zephyr Devicetree Troubleshooting: Generated Files

Learning Devicetree is one of the more difficult parts of getting comfortable with Zephyr. I find the error messages can be extremely long and hard to decipher. One tool that has helped me along the way is the ability to look at the header files that are being generated when the Devicetree files are combined at build time. Every project has a build/include/generated/devicetree_generated.h file that you can reference against error messages.

Let’s look an example of a common Devicetree error message and how I might troubleshoot it.

Building zephyr/samples/sensor/bme280

The Bosche BME280 sensor is one of our favorites here at Golioth. Let’s build the Zephyr sample application for that sensor, using a Nordic nRF9160-DK. The only change we need to make is to add an overlay file for this board:

/* Warning: we've made an error in this file for the demo */
&i2c3 {
    pinctrl-0 = < &i2c2_default >;
    pinctrl-1 = < &i2c2_sleep >;
    pinctrl-names = "default", "sleep";

    bme280@77 {
        compatible = "bosch,bme280";
        reg = <0x77>;
    };
};

&pinctrl {
    i2c2_default: i2c2_default {
        group1 {
            psels = <NRF_PSEL(TWIM_SDA, 0, 12)>,
                <NRF_PSEL(TWIM_SCL, 0, 13)>;
        };
    };

    i2c2_sleep: i2c2_sleep {
        group1 {
            psels = <NRF_PSEL(TWIM_SDA, 0, 12)>,
                <NRF_PSEL(TWIM_SCL, 0, 13)>;
            low-power-enable;
        };
    };
};

Now if you try to build this application:

$ west build -b nrf9160dk_nrf9160_ns . -p

You will eventually be greeted by dozens of lines of error output. The part of the error I usually look at most closely is the line that actually says error in it:

/home/mike/golioth-ncs-workspace/zephyr/include/zephyr/device.h:83:41:
 error: '__device_dts_ord_109' undeclared here (not in a function);
 did you mean '__device_dts_ord_19'?                                                                                                     
   83 | #define DEVICE_NAME_GET(dev_id) _CONCAT(__device_, dev_id)                                                                       
      |                                         ^~~~~~~~~

Now __device_dts_ord_109 is certainly not part of my code. But I recognize the format as belonging to the Devicetree build process. Let’s see if we can make more sens of that identifier.

Troubleshooting Devicetree with generated header files

Look in the build/include/generated/devicetree_generated.h file that was generated during the build process. Near the top, in comments for this file, you will find the Node dependency ordering list.

* Node dependency ordering (ordinal and path):
*   0   /
*   1   /aliases
*   2   /analog-connector
*   3   /chosen
*   4   /connector
*   5   /entropy_bt_hci
*   6   /gpio-interface
*   7   /soc
*   8   /soc/peripheral@40000000
*   9   /soc/peripheral@40000000/gpio@842500
*   10  /gpio-reset
 
... 96 lines removed for blog post brevity ...
 
*   107 /soc/peripheral@40000000/flash-controller@39000/flash@0/partitions/partition@f0000
*   108 /soc/peripheral@40000000/flash-controller@39000/flash@0/partitions/partition@fa000
*   109 /soc/peripheral@40000000/i2c@b000
*   110 /soc/peripheral@40000000/i2c@b000/bme280@77

Okay, now we’re getting somewhere! I can see in the list above that node 109 (the number that appeared in the error message) is an i2c bus, and node 110 is our BME280 node on that i2c bus. So the error we’re getting relates in some way to this node being undeclared.

The easiest way to look at the declaration of the nodes is to view the build/zephyr/zephyr.dts file that is the combination of all Devicetree files during the build process:

i2c3: i2c@b000 {
    compatible = "nordic,nrf-twim";
    #address-cells = < 0x1 >;
    #size-cells = < 0x0 >;
    reg = < 0xb000 0x1000 >;
    clock-frequency = < 0x186a0 >;
    interrupts = < 0xb 0x1 >;
    status = "disabled";
    pinctrl-0 = < &i2c2_default >;
    pinctrl-1 = < &i2c2_sleep >;
    pinctrl-names = "default", "sleep";
    bme280@77 {
        compatible = "bosch,bme280";
        reg = < 0x77 >;
    };
};

I was able to find i2c@b000 in this file. It corresponds to the i2c3 node I want to use for my sensor. And indeed, you can see the sensor node is present. So why can’t the build system locate this node? The answer is in line 264: status = "disabled“.

Disabled nodes are not included in the build. So even though we see information here, the preprocessor will not include symbols for this node. If we want to use this peripheral, we need to enable it. That is the mistake I made in my overlay file.

Correcting the Overlay File

Correcting the overlay file is a simple matter of enabling our target node. If you’re like me, you might assume the opposite of disabled is enabled, but you would be wrong. Zephyr wants enabled nodes to use the okay keyword:

&i2c3 {
    status = "okay";
    pinctrl-0 = < &i2c2_default >;
    pinctrl-1 = < &i2c2_sleep >;
    pinctrl-names = "default", "sleep";
    bme280@77 {
        compatible = "bosch,bme280";
        reg = <0x77>;
    };
};

&spi3 {
    /* The nRF9160 cannot have both
     * i2c3 and spi3 enabled concurrently */

    status = "disabled";
};

&pinctrl {
    i2c2_default: i2c2_default {
        group1 {
            psels = <NRF_PSEL(TWIM_SDA, 0, 12)>,
                <NRF_PSEL(TWIM_SCL, 0, 13)>;
        };
    };

    i2c2_sleep: i2c2_sleep {
        group1 {
            psels = <NRF_PSEL(TWIM_SDA, 0, 12)>,
                <NRF_PSEL(TWIM_SCL, 0, 13)>;
            low-power-enable;
        };
    };
};

When solving this issue I also received an error after enabling i2c3 because spi3 was already enabled by default. This device can only have one of those enabled at a time, which explains the additional node above that disables the unused SPI bus.

Conclusion

Understanding Devicetree errors is a bit like playing jazz. There’s a pattern to it, but you do need to develop a bit of a feel for it. That begins with developing a sense for what the error output is telling you. I hope this tidbit will make things a bit easier.

If you have other Zephyr troubleshooting tricks we should know about, we’d love to hear it! Please share your experiences on the Golioth Forum!

/by
,

Golioth trains future Zephyr engineers

Golioth just wrapped another Zephyr training session that was open to the public. This was a group of 30 trainees, all remote. There continue to be challenges with remote training, but we are always refining how we train engineers and will detail some of the learnings below. We appreciate everyone who took part in the training and are energized to do another one. In fact, you can sign up for our July session at the end of this post!

Why does Golioth focus on training?

Golioth’s main business is creating the web infrastructure that IoT devices connect to and enhance IoT offerings. The devices have a seamless connection experience by using our various device SDKs. Supporting customers that use Zephyr means we have a large amount of hardware that we can support “out of the box”. Our Zephyr SDK helps us advance our goal to support the maximum number of hardware platforms. We also have a Golioth Firmware SDK that extends that goal by supporting additional ecosystems like ESP-IDF and ModusToolbox™. We have plans for other ecosystems in the future.

Golioth focuses on Zephyr training because we think it gives customers a great opportunity to build hardware that is tightly coupled to Cloud services that Golioth provides.

Zephyr is unique because it is both an ecosystem and a Real Time Operating System (RTOS). This is in contrast to something like FreeRTOS, which is an RTOS (obviously), but then vendors like Espressif or Infineon (and a range of others) maintain a vendor specific ecosystem on top of the core RTOS. The net result is that Zephyr has many vendors contributing, but ultimately the core members of the open source Zephyr project drive the direction of the ecosystem and the RTOS. I believe this is what is driving a lot of the popularity of Zephyr. As the popularity continues to increase, we see more people looking to learn about how to use Zephyr in their projects.

There is a steep learning curve when getting started with Zephyr. This, of course, depends on your experience. To hardware and firmware engineers who are used to bare metal programming, it might be difficult to learn about an RTOS for the first time. Understanding how KConfig, Devicetree, and the west build systems can work for your project is another layer to unfold. Device vendor specific implementations of Zephyr APIs and interacting with real hardware takes the problem off of the code editor and into the real world. And throughout the entire process, understanding error messages and building troubleshooting capabilities is a necessary skill-building exercise.

This is why we want to help train people around Zephyr. We know the power of the platform and we want to unblock developers from getting their device talking to the internet using the Golioth Zephyr SDK.

Things we learned

In preparation for our June training, we revamped multiple aspects of our Zephyr program. We also learned more about what works, and what doesn’t

New training site

Training.golioth.io is where we maintain all of our training documentation. Our training programs are focused around written material, instead of being dependent on presentations from the trainers. This unlocks users to move ahead during the training or go back and review a section that is difficult for them to understand.

We also broke up the training into some new sections. We have a new REST API training section, that is separate from our Zephyr training. We will hopefully have other types of training on there in the future. We broke out the “Intro to Golioth” into its own section, because this is the same regardless of which training you’ll be going through.

Having well written documentation continues to be a positive aspect to our training, though we found additional areas we can clarify.

A focus on binaries

In the Intro to Golioth section, we encourage users to download and flash a binary onto their hardware. This matches a recent experiment we did with pre-compiled binaries on the Thingy91, since we think this is the fastest way to try out the capabilities of Golioth. We want trainees to also be able to explore the Golioth Console, which is made easier by a binary that can be put onto the supported hardware during the training. An added benefit is that this tests that users are capable of programming their hardware with a known-working application, which means they’re ready for subsequent training sections.

Previously we had users compiling a program immediately after starting the training. The binary seems to be a better solution to accelerate users’ progress.

Switching to Nordic hardware

Another change for this training is focusing on development hardware from our partner Nordic Semiconductor. We wrote about our excitement around the nRF7002-DK for training, since it provides an affordable board with a built-in programmer. We also enabled a second piece of hardware, the nRF9160-DK. That board is cellular based, which gave users more flexibility to take the training in places where they didn’t have access to the Wi-Fi credentials; it also happens to be one of our best supported boards. Finally, this showcases how Golioth, Zephyr, and Nordic Semiconductor hardware enable cross platform solutions (even different connectivity types), with almost the same underlying code.

Another reason we were excited to switch to Nordic-based hardware is the cross platform support of the nRF Connect for Desktop tool suite. This gives a graphical programmer and reliable serial terminal, which we integrated into our training. Supporting engineers across the range of computing hardware (laptops, desktops) is a surprising challenge. Even though Windows, Mac, and Linux based options are the norm, there is a huge variability between different types of machines. This is also why we continue to be excited about Kasm.

Kasm Environments

Kasm is a streaming container platform that we have standardized on in our training. This allows us to create an entire desktop computing environment that gets streamed to a user’s browser. The best part is that as soon as the user logs in, they have access to all of the toolchains, IDEs, downloaded firmware projects, and any helper tools needed on the virtual remote machine. As the user goes through training and compiles binaries for themselves, they can download the binary to their host machine. That binary can then be programmed using the nRF Connect for Desktop programmer tool. The net result is that our users are ready to compile Zephyr programs within minutes of starting the training. See below for a look “inside” our training.

Where our training still falls short is a direct connection from the Kasm remote environment to the users’ development boards. This is a tough nut to crack.

You might be thinking, “Why not have the user install tools locally? Or have them install tools before they show up to training?”. Past training experiments where we asked users to install the Zephyr tooling onto their host machine took a significant amount of time, and sometimes didn’t work at all because users show up with a wide range of laptops to a training. Don’t believe us? At our in-person training at Supercon 2022, we had a user show up with a Valve Steamdeck handheld Linux gaming computer (his laptop had broken the night before). He was able to successfully use our Kasm container to compile and program the hardware!

For now, we will continue to experiment with connections to pre-configured containers, as we think this unified development experience is the best case scenario for training users.

Sign up for the next one

We continue to refine our capabilities on these training sessions and decided to hold another one on July 12th! We also changed our signup policy: if you meet the criteria, you will be automatically enrolled into training (see form for more details). We hope to see you there!

Fill out the signup form at https://forms.gle/3yk5WrWJ3Dunds9CA

Many Thanks

Though I had the opportunity to help administer this training, the majority of credit goes to my co-worker Mike Szczys. He wrote, tested, and built all of the firmware images and training materials and the training would not be possible without him.

Thanks also go to Golioth team members like Chris W, Dan, Dylan, Marko, and Sam for being on the training and walking users through Golioth.

/by
,

Adding PWM sound in Zephyr

It’s easier than ever to create a system that plays audio. Greeting cards do it, watches do it, just about any consumer system is capable of playing sound. Yes, the quality is variable, but that abides by the old maxim of, “you get what you pay for”.

So what happens if we have a very low cost piezoelectric element (shortened to piezo for the rest of this article), an NPN transistor, and a microcontroller with PWM capabilities…can we make that do anything fun from Zephyr? It turns out, yes, we can. Let’s take a look.

Simplicity

The hardware components on the schematic are pretty bare bones. The output pin from the microcontroller drives an NPN transistor, which then draws current down through a piezo. If we vary the on-off nature through the piezo, we’ll get a tone that matches the frequency of the AC waveform we are using. Using a frequency of 440 Hz should result in an A4 tone. In fact, it does!

Click to download Thingy91 schematic files from Nordic Semiconductor

The on-off nature of a PWM signal creates a square wave. This creates harmonics that produce a recognizable, if not harsh, tone. Songs with tonality like this are sometimes referred to as “chiptune“, because of its association with early video games and their limited capabilities to generate more complex waveforms. Personally, I have a lot of positive associations with this style of sound and music, since I grew up with early games that employed this type of music; it has also developed as a musical subgenre. But for today’s article, all that’s necessary to know is that we will be generating simple square wave tones, which produces a particular sound (examples in a video below).

Another important element is that there is only one generator of sound being used. This means the sound will be monophonic (“one voice”), as opposed to more complex polyphonic sounds, even within the chiptune genre. So like we said, this will be a simplistic output.

Zephyr challenges

Now that we know the hardware, how do we actually get the PWM generator inside a microcontroller to output the waveform we want?

For this exercise, we will be using the Nordic Semiconductor nRF9160, based on a dual core Cortex-M33 part with some standard PWM peripherals built in. However, because we are using Zephyr, the actual register map of the Nordic-specific PWM are not important. Instead, we will be interacting with the standard Zephyr PWM peripheral and APIs. This level of abstraction is implemented in the Zephyr ecosystem by Nordic Semiconductor and the community, and is also implemented by other chip vendors who participate. The net result is that code interacting with the Zephyr PWM element should work with any device that is supported in the Zephyr ecosystem.

We had trouble finding code that was already written for driving the piezo, so we decided to dig into the PWM peripherals. This is where things got confusing.

PWMs only for LEDs?

The first confusing fact is that all of the references to PWM in existing code seemed to apply only to LEDs. PWM is a great use case for LEDs and we implemented this in a thread on our Thingy91 code and pre-compiled binary to help show device status. But what about an alternative PWM device that’s not an LED?

The issue exists in the “compatible” keyword in the devicetree overlay. We only found pwm-leds as a compatible type. We spent a bunch of time chasing other types and seeing if we should use it, but at the end of the day we shrugged and decided our buzzer wouldn’t be bothered being called an ‘led’ in this way. Below is what our thingy91 overlay file looks like.

From our file thingy91_nrf9160.overlay

The pwms field is all we’re really going to be messing with in the program, so we set it to something that made sense for a getting started value and moved on.

From the included thingy91_nrf9160_common.dts files in the nRF Connect SDK

You’ll also note that it refers to &pwm1 in that line. It is calling out a PWM element in the hardware that is also called out in the thingy91_nrf9160_common.dts file as a placeholder to be used with the buzzer. This could be used for other features as well, but the Thingy91 has a limited set of peripherals and not many ways to break out signals, so it makes sense that it was “reserved” for the buzzer, despite not being plumbed in otherwise.

We call out the devicetree entry in the overlay file using the following line in app_work.c (view the entire thingy91_golioth project here)

const struct pwm_dt_spec sBuzzer = PWM_DT_SPEC_GET(DT_ALIAS(buzzer_pwm));

This gives us a struct to work with. When we want to turn the buzzer on, the call looks like

pwm_set_dt(&sBuzzer, PWM_HZ(1000), PWM_HZ(1000) / 2);

We’re passing in the handle sBuzzer and then telling it to play a tone at 1 kHz with a cycle time of 500 Hz–a 50% duty cycle. Since we’re creating a square wave, we can keep the duty cycle at 50% and then vary the frequency to change tones. For things like LED fading, we’d actually change the duty cycle but not the frequency (see other parts of the thingy91_golioth repo for example of LED fades).

Framework

So now we can play a 1 kHz tone, which works great for a buzzer. But what about when we want to easily add a bunch of different notes? To get recognizable notes into the code, we set up a struct that ties together a frequency and a duration:

struct note_duration
{
    int note; // hz
    int duration; // msec
};

We also put a bunch of lookup values into the app_work.h header file:

The note durations were hard-coded in here, but could also be set to be a multiplier of a song tempo, if you wanted a faster or slower song; each note is just a fraction of the overall tempo. The notes are simple frequency lookups. If you don’t have a music background, check out how each increasing octave of a particular note is double the frequency of the note an octave below (ie. A5 is 880 while A4 is 440). Music is math!

Songs

Now that we have a framework for creating songs, it’s all about translating sheet music into an array of notes/durations. This is what our version of “funkytown” looks like:

The REST note sets the frequency to 1 Hz. Since the small piezo is not capable of playing low frequency notes (think about how large most speaker woofers need to be for playing bass loudly), we treat anything less than 10 Hz as a “skip” or “rest”.

All of this is done in an RTOS thread dedicated to playing the song. We pass in which song we want to play, cycle through a for loop playing all the notes in the array and then the thread goes back to sleep. The nice thing is that the buzzer is set as an extremely low priority, so if anything else important is happening in the system, the RTOS scheduler will switch to that task. Even when it’s interrupted to go handle something like an incoming message on the cellular modem, it’s not a noticeable change in how the song sounds. See the app_work.c code to see how this thread operates.

See it in action

This standalone video shows the songs being triggered using the Golioth Remote Procedure Call (RPC) service.

Since we’re running the piezo music in a separate thread, we can receive the RPC, process which song we want to play, and then wake up the thread with the requested song that was passed over the internet.

What will you build?

It takes a lot of tech to play a song over a cellular network, but it helps showcase all the things you can do with Zephyr, Golioth, and Nordic Semiconductor hardware. Of course you can always modify our thingy91_golioth repo and play any other song you’d like…or you can expand on these capabilities and build a wide range of other IoT applications. Our Reference Designs give a good idea of real-world applications you can achieve with the Golioth platform. We’d love to hear about what you’re building on our Forum!

/by