How to use Zephyr Pin Control (pinctrl) for pin multiplexing and configuration

I feel like I begin every Zephyr article talking about how this RTOS’s greatest strength and greatest weakness is the extent of its hardware abstraction. Such is the case with the Zephyr Pin Control (pinctrl) system that became the default approach for pin configuration and multiplexing back in June. This development made great strides in standardizing how to work with pinmux configuration across many vendors, and unified the process of reassigning pin functions at runtime. These gains are at the expense of needing to understand the new syntax. Let’s dig in!

What is pinctrl?

The pinctrl (Pin Control) system is a standardized way of assigning peripheral functions to pins, a concept adopted from Linux. Pin Control lets us define which pins will be used for special functions (SPI/i2c/UART) and how those pins will be configured (pull up/down resistors, slew rate).

In the past we would have remapped an i2c pin inside the i2c node itself using a syntax like sda-pin = <33>;. With pinctrl we don’t necessarily touch the i2c node–it is instead given the address of a pinctrl nodelabel. The pinctrl-state nodes are where actual pins are specified, using a syntax that might look like &i2c1_sda_gpio33. (The exact format of pin addressing varies by vendor. We’ll discuss how to look up that info.)

So what do we gain from this change? One example is reconfiguring a chip for low-power before entering sleep mode. pinctrl automatically looks for two states: default and sleep. When changing from the default state to the sleep state, pinctrl will automatically reconfigure all of your power-sucking pins for low-power, and set them back up again upon wake.

GPIO mapping and usage remains separate for pinctrl. If you’re blinking LEDs or reading buttons, you don’t need to use pinctrl. When you think ‘pinctrl’, think ‘special-function peripherals’.

Example: use pinctrl to change i2c pins

A great exercise when learning pinctrl is to reassign pin locations on an existing board. I’ve been working with the ESP32s2 and there is a “saola” board definition in Zephyr. If we study the pinctrl file for that board we find that i2c1 normally maps GPIO3 and GPIO4:

i2c1_default: i2c1_default {
	group1 {
		pinmux = <I2C1_SDA_GPIO3>,
			 <I2C1_SCL_GPIO4>;
		bias-pull-up;
		drive-open-drain;
		output-high;
	};
};

This pinctrl node uses a nodelabel of i2c1_default which is associated with the i2c1 peripheral in the board’s .dts file:

&i2c1 {
    clock-frequency = <I2C_BITRATE_STANDARD>;
    pinctrl-0 = <&i2c1_default>;
    pinctrl-names = "default";
};

Let’s say we wanted to change these pins to GPIO33 and GPIO34. We could make our own pinctrl node with a different node label and change which nodelabel is used for i2c1. However, the easiest thing is to simply change the pin assignments in the default node, which allows us to inherit all the other settings from the board definition. This is done by adding an esp32s2_saola.overlay in the boards directory of our application. Remember, an “overlay” allows you to tell the build system that the local changes in your build directory are more important than the default settings in Zephyr, and to adopt the local changes.

&pinctrl {
    i2c1_default: i2c1_default {
        group1 {
            pinmux = <I2C1_SDA_GPIO33>,
                     <I2C1_SCL_GPIO34>;
        };
    };
};

Whenever remapping pins, it’s important to check that the new pin assignments are available. In our case, GPIO33 is available, but GPIO34 is mapped to the pinctrl default for SPI3. To avoid a conflict, that pin should be remapped, or if the SPI3 peripheral is not used the SPI3 node can be deactivated or deleted.

Read the bindings pages for vendor-specific pinctrl info

In the example above we used I2C1_SDA_GPIO33 to remap a pin. This syntax is unique to Espressif chips, and will be different for Nordic, Microchip, NXP, Infineon, or any other vendor chips. The same assignment for a Nordic chip would be something like NRF_PSEL(TWIM_SDA, 0, 30). We admit, this is annoying. But Zephyr is an ecosystem, in addition to being an RTOS. Each vendor does the hard work of making their specific chips work with the rest of the code, an this is the cost of them being in control.

Vendors have documented their pinctrl implementation in their device bindings pages. Head over to the Zephyr bindings index and look for the pinctrl entry for your vendor. Espressif’s entry points to a set of macro definitions that lay out every possible function for each pin. Nordic’s entry indicates that rather than define each pin, we can use a macro to call out the nRF pin function along with the port and pin assignments.

/* Nordic pinctrl */
i2c2_default: i2c2_default {
    group1 {
        psels = <NRF_PSEL(TWIM_SDA, 0, 30)>,
            <NRF_PSEL(TWIM_SCL, 0, 31)>;
    };
};

/* Espressif pinctrl */
i2c1_default: i2c1_default {
    group1 {
        pinmux = <I2C1_SDA_GPIO3>,
             <I2C1_SCL_GPIO4>;
        bias-pull-up;
        drive-open-drain;
        output-high;
    };
};

The syntax itself is different from one vendor to another. The snippets above demonstrate that, in addition to pin assignment differences, Nordic uses psels to set the pins while Espressif uses tmux. Always check the bindings index pages, and when in doubt, look to the board definition files as examples.

Dynamic control

A full guide on dynamic control is out of scope for this post. To touch on the topic, choosing to use dynamic pinctrl makes it possible to update pin function at runtime by moving the pin configuration from ROM to RAM. The pinctrl_update_states() function is then used to assign a new state, executing the necessary configuration changes in the process.

Check the pinctrl test app for an example of dynamic control.

The good and the bad of pinctrl

I generally think the move to pinctrl is a good one. Sure, there are syntax differences between vendors, but that was true before the change as well. Now you can expect to find all important pin assignments inside of the pinctrl state nodes, which is a bit easier to reference, and makes it easier to see not just the pin numbers, but the configuration (slew rate, pull up/down, etc.).

i2c1_default: i2c1_default {
    phandle = < 0x5 >;
    group1 {
        pinmux = < 0x301821 >, < 0x2f97e2 >;
        bias-pull-up;
        drive-open-drain;
        output-high;
    };
};

One change that I have not been as happy about is in troubleshooting Devicetree issues. After building a project, it’s often useful to inspect the build/zephyr/zephyr.dts file that is generated. I’ve included an example above, can you tell me if my pins have been properly assigned?

The more you use pinctrl, the more useful it becomes. I’m sure you’ve noticed that some nodes include groupN style groupings. This associates multiple pins with the same needs (ie: for SPI, these three pins are outputs, but this other pins is an input) which is a welcomed shorthand that saves us from assigning pin flags over and over again. Take some time to learn the new system and start taking advantage of its perks!

/by

How to Use Generic SPI Devices with Zephyr

If there’s a driver built into Zephyr, controlling a part over Serial Peripheral Interface (SPI) is a snap. But there’s an ocean of parts out there and only so many built-in drivers in existence. Today I’m going to show you how to use generic SPI devices with Zephyr so you can try writing your own test code in userspace, to be used for the long term or in preparation for creating a driver.

New to Golioth? Sign up for our newsletter to keep learning more about IoT development or create your free Golioth account to start building now.

SPI is simple, right?

In practice, communicating with SPI devices is simple…but only after you get the hardware peripheral configured for your chip. Zephyr is built for cross-compatibility, but that is sometimes its biggest usability flaw. The SPI bus and the devices themselves are abstracted so much that it’s hard to know where to begin if you need to do it all yourself.

I spent far too long searching the internet for a guide before I remembered that all Zephyr systems have tests. Don’t be me, check the test files before you head to Google.

It turns out that the Zephyr SPI driver tests tell us how to register a generic SPI device, and then how to talk to it.

Zephyr SPI Step-by-Step

For this experiment I have chosen perhaps the easiest SPI chip ever, the Microchip MCP3201 12-bit ADC. No need to send register settings to that device, you simply enable the chip-select pin and toggle the clock signal to start shifting out ADC readings. In short: all I need is to read 4-bytes. That’s it!

1. Enable SPI in KConfig

First things first, we need to tell Zephyr to build SPI support into the app. We also need the ability to control GPIO pins. Add these to your prf.conf file:

CONFIG_GPIO=y
CONFIG_SPI=y

Why enable a library unless you’re actually going to use it? Let’s add these line to main.c to ensure we can access all of the Zephyr SPI goodness:

#include <drivers/gpio.h>                                                                                                                                                     
#include <drivers/spi.h>

2. Add a SPI node in the overlay file

Next, we need to tell Zephyr where to find our SPI device. The idiomatic way of doing this is to use the DeviceTree. For this project I’m using a Sparkfun Thing Plus nrf9160 so I needed to remap some pins for this device.

&spi2 {
        status = "okay";
        cs-gpios = <&gpio0 18 GPIO_ACTIVE_LOW>;
        pinctrl-0 = <&spi2_default>;
        pinctrl-1 = <&spi2_sleep>;
        pinctrl-names = "default", "sleep";
        mcp3201: mcp3201@0 {
                compatible = "vnd,spi-device";
                reg = <0>;
                spi-max-frequency = <1600000>;
                label = "mcp3201";
        };
};

&pinctrl {
        spi2_default: spi2_default {
                group1 {
                        psels = <NRF_PSEL(SPIM_SCK, 0, 19)>,
                                <NRF_PSEL(SPIM_MOSI, 0, 21)>,
                                <NRF_PSEL(SPIM_MISO, 0, 22)>;
                };
        };

        spi2_sleep: spi2_sleep {
                group1 {
                        psels = <NRF_PSEL(SPIM_SCK, 0, 19)>,
                                <NRF_PSEL(SPIM_MOSI, 0, 21)>,
                                <NRF_PSEL(SPIM_MISO, 0, 22)>;
                        low-power-enable;
                };
        };
};

Don’t be scared away, I had to include all of this in order to remap which pins were being used for SPI on this device. Your mileage will vary based on your chip, just make sure you choose the correct SPI node (spi2, in my case) and include the highlighted lines above as the added node.

Note that SPI requires a chip-select (CS) pin. Zephyr will handle this for you automatically. Study the example above. You will see one entry called cs-gpios. This is a comma-separated list inside of arrow-brackets. If you have multiple SPI devices, just add each CS pin inside those brackets. It’s a zero-indexed list, and the CS for the mcp3201 is the zeroth entry in that list.  I use an @0 in the path and reg = <0> to tell Zephyr which CS pin to use.

I’ve identified this device as compatible with vnd,spi-device. Technically I should be defining my own DeviceTree binding instead of leaning on this “vendor” definition that is meant for testing purposes only. To learn the recommended way, here’s a 2-hour Zephyr driver talk from ZDS 2022. Let’s not get bogged down in the voodoo of DeviceTree; shall we move along and leave the bindings creation process for a future post?

3. Set up the memory structure

The memory structure is a bit quirky for SPI reads. Generally speaking, you need to set up the memory structure to perfectly mirror how much data you want to read or write. I’m only going to be reading in this example.

I want to read four bytes so I need a four-byte array. That array needs to be a member of a spi_buf, and that spi_buf needs to be a member of a spi_buf_set:

uint8_t my_buffer[4] = {0};
struct spi_buf my_spi_buffer[1];
my_spi_buffer[0].buf = my_buffer;
my_spi_buffer[0].len = 4;
const struct spi_buf_set rx_buff = { my_spi_buffer, 1 };

Honestly this was very confusing for me to figure out. It might be this way for DMA-based SPI peripherals. Or maybe this schema enables you to kind of pre-parse data of different lengths. I haven’t found documentation on why this this structure is the way it is. If you have any insight, please let us know on the Golioth forum thread for this post.

Basically the SPI peripheral is going to shift bits until it runs out of room in your memory structure. So format this for exactly the length of data you need to read/write and Zephyr takes care of the rest.

4. Get the device from the DeviceTree

It’s time to get the SPI device from the DeviceTree. This is a great time to include the configuration for your SPI bus. I followed the Zephyr test file examples and first defined my config values:

#define SPI_OP  SPI_OP_MODE_MASTER |SPI_MODE_CPOL | SPI_MODE_CPHA | SPI_WORD_SET(8) | SPI_LINES_SINGLE

Once we get to the main function, it’s now a much simpler call to get your SPI device:

const struct spi_dt_spec mcp3201_dev =
                SPI_DT_SPEC_GET(DT_NODELABEL(mcp3201), SPI_OP, 0);

5. Issue read command

My standard advice for Zephyr applies here: always check the return codes. When working through this issue, the return codes and enabling debug-level logs in the SPI libraries helped me work out a problem with my memory structure.

ret = spi_read_dt(&mcp3201_dev, &rx_buff);
if (ret) { LOG_INF("spi_read status: %d", ret); }

What we have here is a SPI read operation which immediately prints out the return code if there were any errors. If not, the data you wanted to get from your device is now ready and waiting inside of the buffer that was passed in during the read command.

Further reading

It was quite a task for me to figure all of this out. It would have been much easier had I first looked to the test files (as I mentioned previously). You may want to write to a SPI device, or read and write at the same time. This same process can be used! You just need to also have a memory structure for the data you want to transmit, and you need to use the write, or the “transceive” versions of the SPI functions.

I suggest you start by reading through the SPI API reference. You should also make time to work through all of the functions in the spi_loopback test app which uses SPI in just about every way possible on Zephyr.

Beyond that, if you do have any questions, hit up the Zephyr discord or start a thread on the Golioth forums.

/by
,

ZDS: End-to-end IoT Development with Zephyr by Alvaro Viebrantz

Founding Engineer of Golioth, Alvaro Viebrantz went to the Zephyr Developer Summit…and said the hardware isn’t the only part of an IoT system that matters. That’s bold! This was a room full of hardware and firmware designers!

In reality, this talk is about everything else you should consider when building a device using the Zephyr Real Time Operating System (RTOS). Alvaro goes through the many layers and considerations when architecting large scale Internet of Things (IoT) deployment, from the servers, to the apps, to how devices are communicating back to the cloud, and yes, all the way down to the code put onto the end device.

Top Down Design

Most IoT users first experience a device not from holding the hardware in their hand and seeing it do some particular action. Instead, users rely on things like Dashboards and Applications. The physical device is often hidden from the end user, at least inside an enclosure, and sometimes entirely (as in the case of something like a home monitor in your attic).

It’s important to start by mapping your data model, often before you even start your first schematic. What will your device do? How will it be accessed hierarchically? What other elements are part of the system or will be in the future?

Also important, you should follow the ownership of both the physical device and that device’s output data all the way up the chain back to the user. How will a user claim a device belongs to them? How will you organize the various levels of permission required? What kind of questions will your users seek to answer with the end device or with the entire fleet? (ie. “Do any vehicles in the fleet have low battery?”)

Who is in charge here?

After you have considered how you will configure your database and what the end user will see when they are interacting with data, you should move on to access levels and which parts of the system are accessible by different individuals. This may include access to the end application, but should also include access control to the configuration components of the system. You don’t want your end users to be able to access your device management console, even if they are the ultimate owner of the device. Instead, you will want to provision the devices so you know the unique identifiers of the end-device, and then assign that and the access level to the user.

Additional layers required for a realistic IoT deployment

As you continue to move “down the stack” from Applications all the way to the hardware, there is still a good chunk of Cloud in between. You’ll want to make sure you at least understand all of the elements you’ll likely want in your design, including:

  • Device management
  • Device state and configuration
  • Bidirectional command and control
  • Time-series data and events

Connecting to the network

We’re almost down at the device itself, but the connection to the network is another key consideration before we get there. Not just the method of connection (which ultimately relies on the hardware), but also the protocol and the pathways of each packet coming back. In Golioth’s case, we default to CoAP, which is a lower overhead, UDP based protocol. All of our device SDKs include support to make it easy to talk to CoAP endpoints on the Golioth servers. As Alvaro mentioned in his talk, and as our SDKs also feature, encoding can help to shrink messages even further, using things like protobuf, JSON, and CBOR.

Finally the hardware

By this point in the room, we were squarely back into the realm of expertise of Zephyr developers, though many now had a greater appreciation of all the other things required for IoT systems. Alvaro mentioned how some of the interaction of the connectivity on-device in Zephyr could affect your decision making further up the stack. This talk to a packed room showed developers how to plot out a successful IoT deployment beyond the limits of the hardware, and to think about IoT systems from the hardware, all the way up to the end user.

 

/by
, ,

How to use the Golioth Settings Service with Zephyr

Controlling 10 devices is easy, controlling 10,000 is a different story. The trick is to plan for scale, which is what we specialize in here at Golioth.

A few weeks ago we announced the Golioth Device Settings Service that enables you to change settings for your entire fleet at the click of a button. Of course, you can drill down to groups of devices and single devices too. While the announcement post showed the Settings Service using our ESP-IDF SDK, today we’ll look at this setting service from the device point of view with Zephyr RTOS.

Device-side building blocks

The good news is that the Golioth Zephyr SDK has done all the hard work for you. We just need to do three things to start using the settings service in any Zephyr firmware project:

  1. Enable the settings service in KConfig
  2. Register a callback function when the device connects to Golioth
  3. Validate the data and do something useful when a new setting arrives

Code walk-through

Golioth’s settings sample code is a great starting point. It watches for a settings key named LOOP_DELAY_S to remotely adjust the delay between sending messages back to the server. Let’s walk through the important parts of that code for a better understanding of what is involved.

1. Enable settings in KConfig

CONFIG_GOLIOTH_SETTINGS=y

To turn the settings service on, the CONFIG_GOLIOTH_SETTINGS symbol needs to be selected. Add the code above to your prj.conf file.

2. Register a callback for settings updates

static void golioth_on_connect(struct golioth_client *client)
{
    if (IS_ENABLED(CONFIG_GOLIOTH_SETTINGS)) {
        int err = golioth_settings_register_callback(client, on_setting);

        if (err) {
            LOG_ERR("Failed to register settings callback: %d", err);
        }
    }
}

To do anything useful with the settings service, we need to be able to execute a function when new settings are received. Register the callback function each time the device connects to Golioth. Your app may already have a golioth_on_connect() function declared, look for the following code at the beginning of main and add it if it’s not already there.

	client->on_connect = golioth_on_connect;
	client->on_message = golioth_on_message;
	golioth_system_client_start();

Each time the Golioth Client detects a new connection, the golioth_on_connect() function will run, which in turn registers your on_setting() function to run.

3. Validate and process the received settings

enum golioth_settings_status on_setting(
        const char *key,
        const struct golioth_settings_value *value)
{
    LOG_DBG("Received setting: key = %s, type = %d", key, value->type);
    if (strcmp(key, "LOOP_DELAY_S") == 0) {
        /* This setting is expected to be numeric, return an error if it's not */
        if (value->type != GOLIOTH_SETTINGS_VALUE_TYPE_INT64) {
            return GOLIOTH_SETTINGS_VALUE_FORMAT_NOT_VALID;
        }

        /* This setting must be in range [1, 100], return an error if it's not */
        if (value->i64 < 1 || value->i64 > 100) {
            return GOLIOTH_SETTINGS_VALUE_OUTSIDE_RANGE;
        }

        /* Setting has passed all checks, so apply it to the loop delay */
        _loop_delay_s = (int32_t)value->i64;
        LOG_INF("Set loop delay to %d seconds", _loop_delay_s);

        return GOLIOTH_SETTINGS_SUCCESS;
    }

    /* If the setting is not recognized, we should return an error */
    return GOLIOTH_SETTINGS_KEY_NOT_RECOGNIZED;
}

The callback function will provide a key (the name of the settings) that is a string type, and a golioth_settings_value that includes the value, as well as an indicate of the value type (bool, float, string, or int64). The callback function should validate that the expected key was received, and test that the received value is the proper type and within your expected bounds before acting upon the data.

The final piece of the device settings puzzle is to return an enum value indicating the device status after receiving a new setting. This is used by the Golioth Cloud to indicate sync status. Our example demonstrates the GOLIOTH_SETTINGS_SUCCESS and GOLIOTH_SETTINGS_KEY_NOT_RECOGNIZED status values, but it’s worth checking out the doxygen reference for golioth_settings_status to see how different invalid key and value messages can be sent to Golioth for display in the web console.

Coordinating with the Golioth Cloud

The device settings key format is tightly specified by Golioth, allowing only “A-Z (upper case), 0-9 and underscore (_) characters”. We recommend that you add a key to the Golioth web console while writing your firmware to ensure the device knows what to expect.

Each device page includes a status indicator to show if the settings are in sync with the Golioth cloud.

Additional information is available when a device is out of sync. Here you can see that hovering over the information icon indicates the key is not valid. For this example I created a VERY_IMPORTANT_BOOL_7 but didn’t add support for that to the firmware. This is the result of our callback function returning GOLIOTH_SETTINGS_KEY_NOT_RECOGNIZED. In cases like this, Golioth’s Over-the-Air firmwmare update (OTA) service can help you to push a new firmware to your devices in the field that will recognize the newly created Settings variable.

IoT Fleet control on day one

The pain of IoT often comes when you need to scale. Golioth’s mission is to make sure you avoid that pain and are ready to grow starting from day one.

Our device settings service gives you the power to remotely change settings on your entire fleet, on a group of devices, and all the way down to a single device. You are able to verify your devices are in sync, and receive helpful information when they are not. Take Golioth for a test drive today, our dev tier is free for the first 50 devices.

/by
,

Zephyr + Arduino: a Google Summer of Code story

Google Summer of Code: Zephyr + Arduino

The Arduino ecosystem is ever-growing, and thanks to the excellent work of Dhruva Gole it’s coming to Zephyr as well.

For the 2022 Google Summer of Code with Golioth, Dhruva took on the challenge integrating the Arduino core with Zephyr RTOS as the base. Now the program is nearing completion and we’ve been excitedly building and running Arduino sketches that can directly call Zephyr libraries and subsystems. And of course, the cross-platform nature of Zephyr means there are suddenly many more boards that can run that Arduino code.

What is GSoC

Google Summer of Code is a program that focuses on bringing new contributions to Open Source. The program matches up participants (often students but not necessarily) with mentors to work on a specific goal previously accepted by the GSoC program.

Dhruva Gole finished his Bachelor of Technology in Electrical Engineering earlier this year. He has been working throughout the summer to build a Zephyr module that includes the Arduino core. This is the second GSoC program for Dhruva, who in 2021 worked on bringing Bela support to the BeagleBone AI platform. We’re also very happy to share the news that Dhruva began a job with Texas Instruments last month, congratulations!

It has been wonderful working with Dhruva, who has made a sizable contribution to the open source ecosystems of Zephyr and Arduino. At the same, as mentors we have been able to share a glimpse of how the Golioth engineering team operates. Finding our way through some hairy issues with the I2C system, mapping the Zephry Arduino header paradigm in DeviceTree, and getting the build system to work without extra command line arguments provided a great opportunity to work collectively.

What has been accomplished

Zephyr is a Real-Time Operating System (RTOS) that focuses on abstracting hardware functions so that the same code may be run on innumerable hardware platforms. This is an excellent underpinning for the Arduino code.

When successfully combined, Arduino sketches can be run on hardware that does not have an Arduino port. And at the same time, Zephyr features like multi-threading and the network stack become available to those Arduino sketches.

The milestones reached in the project thus far include:

  • Arduino Digital GPIO functions (e.g.: pinmode, digitalRead, digitalWrite)
  • Arduino time functions (e.g.: delay, millis)
  • I2C support via the Wire library API (works with external Arduino libraries)
  • A DeviceTree framework for mapping Arduino header pins to any Zephyr boards
  • Serial.println support in progress

How to use it

The project is organized as a Zephyr Module called Arduino-Zephyr-API. Any board that has Zephyr support can be defined in the variants directory of this module. This is accomplished by adding a DeviceTree overlay file (to map the pins of the board to the pin numbers from the Arduino ecosystem) and a pinmap header file which wraps Zephyr pin functions with the familiar Arduino API calls.

Using west, the Zephyr meta tool, Arduino sketches can be compiled and flashed to your target board. Beyond core functions, the addition of an RTOS opens up the ability to use multiple threads in Arduino. Dhruva has includes an excellent threading demonstration by blinking multiple LEDs asynchronously.

Rudimentary instructions for adding the module to your Zephyr installation are available in the project README. Documentation for adding boards is currently in progress and will be complete by the end of the GSoC program in September.

Challenges for future contributors

The heavy lifting of integrating the Arduino core with Zephyr has already been completed and is working well. But to use this, you must already have a Zephyr workspace installed (and know how to use it). Sure, Golioth has a quickstart that helps install Zephyr, but the ability to use this project from the Arduino IDE is desirable for existing Arduino users.

Unfortunately, there isn’t time left to work this level of integration. But contributions to the program are welcome and we hope that future contributors will take on this task.

There is also low-hanging fruit when it comes to implementing the most-used Arduino functions. For instance, random numbers, bits and bytes, and math functions should all be trivial to get working. More ambitious contributors may consider mapping the Zephyr ADC system to analogRead() and analogWrite().

/by
, ,

ZDS: What Chip Shortage? How We Use Zephyr for Truly Modular Hardware

Even people who have been living under a rock for the past couple of years know that there’s a global chip shortage. The correct response from the engineering community should be changes to our design philosophy and that’s the topic of the talk that Chris Gammell and I gave at the 2022 Zephyr Developer Summit back in June. With the right hardware and firmware approach, it’s possible to design truly modular hardware to respond to supply chain woes.

Standardization enabled the industrial revolution, and the electronics field is a direct descendant of those principles. You can rest easy in knowing that myriad parts exist to match your chosen operating voltage and communication scheme. But generally speaking it’s still quite painful to change microcontrollers and other key-components mid-way through product design.

Today’s hardware landscape has uprooted the old ways of doing things. You can’t wait out a 52-week lead time, so plan for the worst and hope for the best. Our talk covers the design philosophies we’ve adopted to make hardware “swapability” possible, while using Zephyr RTOS to manage the firmware side of things.

Meet the Golioth Aludel

Golioth exists to make IoT development straightforward and scalable–that’s not possible if you’re locked into specific hardware, especially these days. So we designed and built a modular platform to showcase this flexibility to our customers.

Golioth Aludel prototyping platform internal view

Called Aludel, Chris Gammell centered the design around a readily available industrial enclosure. The core concept is a PCB base that accepts any microcontroller board that uses the Feather standard. This way, every pin-location and function is standardized. Alongside the Feather you’ll find two headers that use the Click boards standard (PDF), a footprint for expansions boards facilitating i2c, SPI, UART, and more. The base includes Qwiic and Stemma headers for additional sensor connectivity, as well as terminal blocks, room for a battery, and provisions for external power.

The key customization element for Aludel is the faceplate. It’s a circuit board that can deliver a beautiful custom design to match any customer request. But on the inside, each faceplate has the same interface using a flat cable connection to the base board. Current versions of the faceplate feature a a screen, an EEPROM to identify the board, and a port expander that drives buttons, LEDs, and whatever else you need for the hardware UI.

The beauty of this is that electrically, it all just works. Need an nRF52, nRF9160, ESP32, STM32, or RP2040? They all already exist in Feather form factor. Need to change the type of temperature sensor you’re using? Want to add RS485, CANbus, MODBUS, 4-20 ma communication, or some other connectivity protocol? The hardware is ready for it–at most you might need to spin your own adapter board.

Now skeptics may look at the size of this with one abnormally high-arched eyebrow. But remember, this is a proof-of-concept platform. You can set it up for your prototyping, then take the block elements and boil them down into your final design. Prematurely optimizing for space means you will have many many more board runs as the parts change.

When you do a production run and your chip is no longer available, the same concepts will quickly allow you to test replacements and respin your production PCB to accommodate the changes.

Zephyr Alleviates your Firmware Headaches

“But wait!” you cry, “won’t someone think of the firmware?”. Indeed, someone has thought of the firmware. The Linux Foundation shepherds an amazing Real-Time Operating System called Zephyr RTOS that focuses on interoperability across a vast array of processor architectures and families. Even better, it handles the networking stack and has a standardized device model that makes it possible to switch peripherals (ie: sensors) without your C code even noticing.

We use Zephyr to maintain one firmware repository for the Aludel hardware that can be compiled for STM32F40, nRF52840, nRF9160, and ESP32 using your choice of Ethernet, WiFi, or Cellular connections. How is that even possible?

&i2c0 {
	bme280@76 {
		compatible = "bosch,bme280";
		reg = <0x76>;
		label = "BME280_I2C";
	};
};

&spi1 {
	compatible = "nordic,nrf-spi";
	status = "okay";
	cs-gpios = <&gpio0 3 GPIO_ACTIVE_LOW>,
	           <&gpio0 27 GPIO_ACTIVE_LOW>,
	           <&gpio1 8 GPIO_ACTIVE_LOW>;
	test_spi_w5500: w5500@0 {
		compatible = "wiznet,w5500";
		label = "w5500";
		reg = <0x0>;
		spi-max-frequency = <10000000>;
		int-gpios = <&gpio0 2 GPIO_ACTIVE_LOW>;
		reset-gpios = <&gpio0 30 GPIO_ACTIVE_LOW>;
	};
};

/ {
    aliases {
        aludeli2c = &i2c0;
        pca9539int = &interrupt_pin0;
    };
    gpio_keys {
        compatible = "gpio-keys";
        interrupt_pin0: int_pin_0 {
			gpios = < &gpio0 26 GPIO_ACTIVE_LOW >;
			label = "Port Expander Interrupt Pin";
		};
	};
};

Zephyr uses the DeviceTree standard to map how all of the hardware is connected. A vast amount of work has already been done for you by the manufacturers who maintain drivers for their chips and supply .dts (DeviceTree Syntax) files that specify pin functions and mappings. When writing firmware for your projects you supply DeviceTree overlay files that indicate sensors, buttons, LEDs, and anything else connected to your design. The C code looks up each device to receive all relevant information like device address, GPIO port/pin assignment and function.

const struct device *weather_dev = DEVICE_DT_GET_ANY(bosch_bme280);
sensor_sample_fetch(weather_dev);
sensor_channel_get(weather_dev, SENSOR_CHAN_AMBIENT_TEMP, &temp);
sensor_channel_get(weather_dev, SENSOR_CHAN_PRESS, &press);
sensor_channel_get(weather_dev, SENSOR_CHAN_HUMIDITY, &humidity);

The hardware abstraction doesn’t stop there. Zephyr also specifies abstraction for sensors. For instance, an IMU will have an X, Y, and Z output — Zephyr makes accessing these the same even if you have to move to an IMU from a different manufacturer due to parts shortages. You update the overlay files for the new build, and use a configuration file to turn on any specific libraries for the change, but the code you’re maintaining can continue on as if nothing happened.

Of course there are caveats which we cover in the talk. There is no one-size-fits-all in embedded engineering so you will eventually need to define your own .dts files for custom boards, and add peripherals that are unsupported in Zephyr. This is not a deal breaker, there is a template example for adding boards (and I highly recommend watching Jared Wolff’s video on the subject). And since Zephyr is open source, your customizations can be contributed upstream so others may also benefit.

Hardware will never again be the same

Electronics manufacturing will eventually emerge from the current shortages. But there are no signs of this happening soon, and it’s been ongoing for two years. We shouldn’t be trying to return to “normal”, but planning a better future. That’s what Golioth is doing for the Internet of Things. Choosing an IoT management platform shouldn’t require you to commit to one type of hardware. Your fleets should evolve, and our talk outlines how they can evolve. Golioth helps to manage your diverse and growing hardware fleet. Golioth is designed to make it easy to keep track of and control all of that hardware, whether that’s 100 devices or 100,000. Take Golioth for a test drive today.

/by
,

Using the NXP RT1060-EVKB Ethernet board with Golioth

NXP RT1060-EVKB Ethernet development board

I get to work with a lot of different hardware here at Golioth, but recently I’ve found my self gravitating to the NXP i.MX RT1060-EVKB evaluation board as my go-to development hardware. This brand new board is a variation on the RT1060-EVK, which (in addition to the extra ‘B’ in the new part number) adds an m.2 slot. But what catches my eye is the Ethernet. While I’ve previously written about using SPI-connected Ethernet modules, this board builds the connectivity right in. It’s stable, it’s fast, and has wonderful support in Zephyr.

The microcontroller that controls everything is the RT1062 which I first heard about three years back when the Teensy 4.0 was released. A 600 MHz powerhouse, it boasts 1024 kB of SRAM and is packed with peripherals. It’s an embarrassment of riches for my day-to-day firmware work, but makes sure I’ll never hit my head on the proverbial “development ceiling”.

Of course it’s not just the system clock that I’m happy about. This board has Ethernet built-in, so I’m not waiting for a WiFi connection (or really waiting for a cell connection) at boot. And it’s programmed via J-Link for a fast flash process and the RTT and debugging features that this programmer brings to the party.

Let’s take a look at how I get this board talking to Golioth, and what we might see in the near future.

An i.MX RT1060-EVKB demo

NXP RT1060-evkb webinar demo

Demonstrating Golioth LightDB Stream for temperature/pressure/humidity sensor data

A couple of weeks ago I did a demo of the i.MX RT1060-EVKB board as part of NXP’s Zephyr webinar series. It gives you a great overview of the board and the ecosystem. I was even able to do a live demo of all the Golioth features like data management, command/control, remote logging, and OTA firmware updates. (Requires a free NXP registration to see the video.)

This board already has great support with the Golioth platform. My hope is to go further than that and add this to our list of Continuously Verified Boards. This would mean that we test and verify all official Golioth code samples with this board during every Golioth release. That’s a tall hill to climb, but it would be a snap if we used automated testing through a process called Hardware in the Loop; CI/CD that automatically runs compiled code on the hardware itself. This is already the case for our recently announced Golioth ESP-IDF SDK. Nick Miller is working on a post about how he pulled that off, so keep your eyes on the Golioth Blog over the next couple of weeks!

Get your tools ready

To build for this board you need a few tools, like the HAL library for NXP, LVGL, and the board files themselves.

Adding the NXP HAL and the LVGL library

Zephyr needs a hardware abstraction layer for NXP builds, and this board also depends on having the LVGL library installed. Add these to your west manifest:

  • use the west manifest --path to help you locate which file to edit (modules/lib/golioth/west-zephyr.yml in my case)
  • Add hal_nxp and lvgl to the name-allow list
  • Run west update

With the edits in place, here’s what the relevant section of my west manifest looks like:

manifest:
  projects:
    - name: zephyr
      revision: v3.1.0
      url: https://github.com/zephyrproject-rtos/zephyr
      west-commands: scripts/west-commands.yml
      import:
        name-allowlist:
          - cmsis
          - hal_espressif
          - hal_nordic
          - mbedtls
          - net-tools
          - segger
          - tinycrypt
          - hal_nxp
          - lvgl

Cherry Pick the EVKB files (if needed)

This is a new board and the devicetree files for it were just added to Zephyr on Jun 5th. My Zephyr is still on v3.1.0 so it’s older than the commit that added support for the “evkb” board. I used git’s cherry-pick feature to add those files to my workspace. From the golioth-zephyr-workspace/zephyr directory run:

git pull
git cherry-pick 9a9aeae00b184cac308c1622676bb91c07dd465b

This will pull in just the commit that adds support for the new board.

Thar ‘b dragons

Here’s the thing about making manual changes to the Zephyr repo in your workspace: the west manifest doesn’t know you’ve done this. That means the next time you run west update, this cherry-picked commit will be lost.

Prep the hardware

Last, but not least, I used a J-Link programmer to flash firmware to this board. There’s just a bit of jumper setup necessary to make this all work. NXP has a guide to setting up an external J-Link programmer. I followed that page, using a USB cable on J1 for power (and a serial terminal connection when running apps).

Saying Hello

With that preamble behind me, I’m excited to get to the “Hello” part of things using the standard Golioth Hello sample. Running Golioth samples is fairly easy with this board, with just one puzzle piece that needs to be added for DHCP to acquire an IP address.

Use DHCP to get an IP address

I added a boards/mimxrt1060_evkb.conf file to create a configuration specific to this board. In this case, it selects the Ethernet and DHCP libraries.

CONFIG_NET_L2_ETHERNET=y
CONFIG_NET_DHCPV4=y

At the top of main.c I include the network interface library. Then in the main() function, we need to instantiate an interface and pass it to the DHCP function. This code requests an IP address from the network’s DHCP server.

#include <net/net_if.h>

struct net_if *iface;
LOG_INF("Run dhcpv4 client");
iface = net_if_get_default();
net_dhcpv4_start(iface);

client->on_message = golioth_on_message;
golioth_system_client_start();

Add credentials, then build and flash

The final step is to add your Golioth credentials before building and flashing the app. Add the credentials to the prj.conf file as outlined in the hello sample README. You can get your credentials by creating a new device on the Golioth Console.

CONFIG_GOLIOTH_SYSTEM_CLIENT_PSK_ID="my-psk-id"
CONFIG_GOLIOTH_SYSTEM_CLIENT_PSK="my-psk"

The build calls out the board name to specifically target our board and the newly added configuration. The flash command will automatically find the J-Link programmer and use it.

west build -b mimxrt1060_evkb samples/hello
west flash

In my testing it only took about three seconds to get an IP address. From there, I was up and connected to Golioth after just another two seconds!

Golioth Hello serial terminal output

Golioth Hello example output

Off to the races

There really isn’t much to getting this up and running. Once I worked through this process I was able to test out all of the Golioth samples, including using the terminal shell to provision the board (ie: adding the Golioth credentials) and performing Over-the-Air (OTA) firmware updates.

As I mentioned earlier, the speed and reliability of this dev board keeps drawing me back. It’s not just the initial connection, but when testing out OTA, the 1024 byte block downloads seem to fly right by. I’m unsure if this is the Ethernet or the chip’s ability to quickly write to flash, but if you’re working with a 250 kB firmware file the difference over, say an ESP32, is obvious.

Give it a try using Golioth’s Dev Tier which includes up to 50 devices for your test fleet. The Golioth forum is a great place for questions on the process, and we’d love to hear about what you’re building over on the Golioth Discord server.

Keep your eye on this space. When Zephyr issues its next release we’ll roll forward the Golioth SDK to include the board files for the RT1060-EVKB, and my hope is that we’ll be able to include the DHCP call into our common samples code for a truly out-of-the-box build experience. We might even see a bit of that Hardware-in-the-Loop build automation I hinted at earlier. See you next time!

/by
, ,

Using Wireshark to troubleshoot Thread networks

In this article, we showcase how to use Wireshark–an open source, free network analysis tool–to troubleshoot wireless mesh networks set up using OpenThread, Zephyr, and Golioth. The tooling shown here can also be used for other Thread-based devices, assuming you understand the layers of the network. We used these tools internally to help us when get Thread devices to connect with Golioth and take advantage of all of the features we have to offer IoT device makers.

Building Thread Networks

Golioth started building out Thread networks when several users approached us about their interest in creating Golioth-managed Thread devices. We created example projects to show our users how to create mesh networks of custom low-power sensors and connect them back to the internet. We benefit from the fact that Thread network devices are IPv6 devices (thanks to 6LoWPAN), and that they talk over the CoAP protocol, all of which aligns very well with Golioth capabilities. We showed this in our most recent blog post about custom Thread nodes connecting through an OpenThread Border Router (OTBR) back to Golioth and transmitting information that can be displayed anywhere on the web.

Hardware and firmware engineers can utilize the Golioth Zephyr SDK to implement a wide range of features on Thread nodes and interact with those nodes like any other internet-connected device. In the Golioth Red Demo showcased at a number of recent live events, we had nodes that could report back sensor data and react to stimulus from the cloud; future versions could also get firmware updates directly from the cloud.

As in any hardware and firmware development process, things didn’t always go according to plan. When we were troubleshooting our early Proof of Concept, we needed to check which part of the chain was not passing packets along. We broke out Wireshark to start sniffing packets and figured out that there was a mismatch in the number of bytes being sent (since fixed). We think this kind of pinpoint accuracy in troubleshooting is a tool that all our users should have in their toolbox.

Good Security is meant to slow you down

Golioth is secure by default, which means all packets going to our Cloud must be encrypted. Normally, this is a feature! You don’t want anyone with a packet sniffer to be able to see plain-text data. However, when you do want to see what’s inside a packet during troubleshooting, you need to make sure you have the keys to unlock everything. You also need to make sure you have the tools properly configured for the various layers involved in Thread networks. These will be the steps we review below and in the video.

Setting up a sniffer

In order to use Wireshark to troubleshoot a Thread network, you’ll need the following:

Pretty simple!

The first step is getting the tools onto the dongle. Much like the dongle was used as a Radio Co-Processor on the Open Thread Border Router, we’ll be using a different set of firmware to sniff radio packets and hand them over USB to the computer. This firmware is specific to 802.15.4, which is the Physical and MAC layer used by Thread. Download the firmware from Nordic Semiconductor and load it onto your dongle and you’re ready to go!

Next you need to be able to interact with the output of the dongle. This includes installing a python script in Wireshark that is located in the Nordic Semiconductor repository. Around the 2:15 mark in the video, Mike shows where and how to install this in Wireshark.

Configuration Settings

Other important parts of the video include things like:

  • 3:45Choosing the correct 802.15.4 channel (channel 15 for most Golioth examples)
  • 5:00Network settings for Wireshark to capture Thread network traffic
  • 7:15Adding a Pre-Shared Key (PSK) to decrypt DTLS packets

Once those configuration steps are done, you can view wireless traffic coming from your Thread node, through each of the layers, up to the Golioth servers, onto specified endpoints like /logs. In the decoded payload area (bottom window), you can also see the messages that are actually being sent, in this example a log message saying “starting connect”.

Tools for when you need them

Wireshark and plugins developed by the community make for a powerful set of tools for troubleshooting network problems. We hope that our examples and tutorials allow you to quickly deploy a Thread network and build out custom Thread nodes; but when you need a bit more insight or are looking to try something new, Wireshark can help.

We’re here to help too! You can reach us on our Discord or Forum, and can always reach us at [email protected].

/by
, , , ,

Zero-install embedded training with Zephyr using Kasm and Docker

TL;DR: we’ve enabled people to compile Zephyr programs from a computer with no toolchain installed, almost instantly.

Part of our charter at Golioth is to help people prototype and scale IoT devices faster. That’s why we offer an open source SDK built on top of Zephyr. We think this represents a “fast forward” or “cheat code” for quickly standing up an IoT device prototype. On the cloud side, our servers represent hundreds of hours of customization and testing; you can instantly connect and get access to resources that allow hardware and firmware developers to scale to thousands or millions of devices. But sometimes it can be scary to get started in a new ecosystem or Real Time Operating System (RTOS) like Zephyr, even if it will speed things up later. As such, we do public and private training for companies and individuals.

As part of the resources we offer, we maintain a Training site that walks people through how to get started using Zephyr, normally targeting remote training. You can follow along right now; you’ll need to purchase an Adafruit MagTag board and sign up for a free Dev Tier account, but everything else is covered on the training site. At the end of the training, you should understand how to interact with hardware in Zephyr and send data to and from the Golioth cloud over WiFi. It’s a short jump from there to re-target other hardware, including your custom designs.

The tripping points for the training often revolve around the installation process. This is multi-pronged:

  • The size of a Zephyr install is relatively large, even when you are only targeting a specific platform. Having multiple people in a room, even with good WiFi or network connectivity, means that the shared bandwidth will be a limiting factor. More trainees means slower downloads.
  • Everyone comes to training with a computer in a different state. They might have tried to install Zephyr tools in the past, or they might have a particularly rare Linux distro, or many other possible variations. It would be best if everyone showed up with a fresh OS install…but that is very unrealistic.
  • There are different expectations around how installations should go. Many embedded engineers are “Windows first” and expect a complete IDE for any new platform. Some silicon vendors help to support this in Zephyr, such as Nordic Semiconductor. But Zephyr was originally targeting Linux-based machines, and we have found the smoothest flow for installing tools for all of the platforms that Zephyr can target means you are Linux-first.

In this article, we’re going to talk about our attempt to normalize setups and have pre-installed tools using Kasm and Docker. These are not the only tools in this space; we have previously written about GitPod and are investigating GitHub Codespaces, but this is a look at one of the latest experiments we’re running at Golioth.

Kasm thin client

The concept of a browser based client or a “thin client” is nothing new. They were all the rage back in the day of time share servers (really those were “dumb terminals”) and then again in the 90s as computing was more ubiquitous throughout the office (with a centralized set of servers). The difference is that now things are much more graphical and running completely inside the browser.

Kasm was started in 2017 and includes an open source project run by Kasm Technologies. The company behind Kasm has a per seat licensing model or they will run the servers directly for you (once you’re past 5 trial seats). They specialize in visualizations around containers. Once you log into a Kasm server, you are able to launch a range of containers, normally a desktop view or a single app that will load up in your browser. You can try this for yourself on the Kasm demo page.

The server that we’re running on is a pre-configured image that I pulled from the Digital Ocean marketplace. I was able to install all of the required software on a provisioned server running in some unknown datacenter. All I did was log in the first time to get my credentials for a user and an admin, and the rest of my interaction was on the web interface that the Kasm server presents to me as an admin.

Docker

As a hardware engineer, Docker is one of those things I heard about for a long time and never really “got it”. I’m still not sure I do. But following the tutorial for customizing a Kasm container, I started to understand a bit more. In that set of tutorials I started from a base Operating System image (Ubuntu Focal) that allowed visualization through the browser. Then I was able to start customizing, adding things like custom files on the desktop, custom icons to launch programs I installed, or adding background images pulled in from the web. It was in this customization section that I could add all of the commands from the Golioth Docs for installing Zephyr tools.

My layman explanation of Docker would be “Creating a virtual computer where I can automatically install a bunch of software using shell scripts. Once I have built that virtual computer, I am able to use it over and over again, including different instances of that virtual computer (for this Kasm scenario)”. The analogy would be if I bought a bunch of laptops, had an install CD (remember those?) with all of the required software on them, and then I mailed the freshly installed laptop to everyone who is taking our training. Sound crazy? That’s one of the best solutions we have seen, where a trainer will bring a pelican case with 24 laptops freshly imaged to on-site training. Their training works flawlessly every time!

I don’t have much else to mention about Docker aside from the idea that it’s possible to script a bunch of install commands that match the install instructions we have on our Zephyr getting started guide. In fact, I used those very directions to build the container shown in the video above. So all I’m doing in this case is automating the install process, doing it once, and then deploying the container (with all of the software and dependencies installed) over and over again for different users.

Challenges

We don’t think this is the ultimate solution for our training, so much as an experiment that showcases what we can do with containerized solutions. There are some remaining challenges, and we would love to have some help from our community.

Loading firmware onto the device

Currently our plan (as shown in the video) is to have our users/trainees pull the final built binary to their local computer to run it on the device like the MagTag. This echoes the way the mbed online compiler worked.

If there is a bootloader and a USB to serial connection, it’s possible to directly load onto the embedded device. In the case of some Espressif boards, this would be something like having ESPtool.py installed locally on your machine. There are an increasing amount of tools that make this process easier, such as an ESP tool that allows you to load firmware using WebUSB. Certain specialized bootloaders like the one that comes default on the MagTag loads UF2 files. When the MagTag is plugged in over USB and a sequence of buttons are hit, the device shows up as a mass storage drive. You drop a UF2 formatted binary–which is just an alternative form of compiled format–onto the drive and the device reboots and starts running the code.

If it’s a board without a bootloader, the user would need to have a debugger and local tools to communicate with that debugger, such as a JLink device and JFlash software. This means they would still need some OS specific loader tools to get the binary into the embedded device. The user would not be able to take advantage of the built-in tools in west that allow direct loading onto the device.

You steppin’?

If you would like to do debugging instead of “printf/printk” debugging, you simply need to download a different file from the container. If you download the zephyr.elf file instead of the zephyr.bin file, you can load it into a 3rd party debugger like Segger Ozone (made by the same company as the JLink). We have done some experiments with this in the past, including also analyzing where the device is spending its time using SystemView. This would once again require installing local programs that could talk over the USB port to something like a JLink.

Experimental port forwarding and WebUSB

Some GDB debuggers/servers will host the control of the debugger over a port on the machine’s localhost. We have some experiments we’re trying where we forward this port to the container so we could directly run a debugger from a software debugger inside the container.

We have also heard some whispers of a WebUSB implementation that can tunnel to the container. So we could plug in a board on our host machine (ie. my laptop) and connect to it over WebUSB, and then forward all information along to the container machine (ie. the browser based desktop running on the Kasm server).

We would love to hear about other projects that are trying this.

Shared resources

The final challenge we are dealing with is the fact that we’re basically “renting” a computer to do exactly what we could be doing with the host machine sitting right in front of us. Most developers have access to very powerful machines and we are instead using the resources on a remote machine (the Kasm server). The cost of standardization is the cost of renting server time for each person in the workshop. It might be worth it, but it is a constraint and a challenge.

Containers are another tool

Anyone reading this with a web background is likely thinking, “Yeah, containers, cool, 2010 called and wants their headline back”. But we are excited about it because these tools are finally making their way into the historically sluggish embedded industry. While our use case of containers is mostly around zero-install-time training, others are using containers to automate their testing and implementing best software engineering practices for the range of devices they have on their desk or in the field.

We’d love to hear how you think we can improve our training and make it easier for you to learn more about Golioth, Zephyr, and building code instantly. Check out our forums, our Discord, ping us on Twitter, or send us an email at [email protected]

/by
, ,

How to add Golioth to an existing Zephyr project

Golioth is a device management cloud that is easy to add to any Zephyr project. Just list the Golioth Zephyr SDK as a module in your manifest file and the west tool will do the rest. Well, almost. Today I’ll walk through how to add Golioth to an existing Zephyr project. As an example, I’ll be using our hello sample. We chose this because it already has networking, which makes explaining things a bit easier. Generally any board that has led0 defined in the device tree and you can enable networking should be a good fit. What we want to do here is showcase the elements that allow you to add Golioth, so let’s dive in!

0. Understanding the west manifest

A Zephyr workspace is made up of a number of different code repositories all stored in the same tree. Zephyr uses a west manifest file to manage all of these repositories, including information like the origin URL of the repo, the commit hash to use, and where each repository should be placed in your local tree.

Think of the manifest as a code repository shopping list that the west update command uses to fill up your local Zephyr tree. There may be more than one manifest file, but today we’ll just focus on the main manifest.

1. Add Golioth to the west manifest

Start by locating your manifest file and opening it with a code editor.

~/zephyrproject $ west manifest --path
/home/mike/zephyrproject/zephyr/west.yml

Under projects: add an entry for the Golioth Zephyr SDK (highlighted in the abbreviated manifest sample below).

manifest:
  defaults:
    remote: upstream
 
  remotes:
    - name: upstream
      url-base: https://github.com/zephyrproject-rtos
 
  #
  # Please add items below based on alphabetical order
  projects:
      # Golioth repository.
    - name: golioth
      path: modules/lib/golioth
      revision: v0.2.0
      url: https://github.com/golioth/golioth-zephyr-sdk.git
      import:
        west-external.yml
    - name: canopennode
      revision: 53d3415c14d60f8f4bfca54bfbc5d5a667d7e724
...

Note that I have called out the v0.2.0release tag. You can set this to any of our release tags, to main, or to a specific commit.

Now run an update to incorporate the manifest changes:

west update

2. Select libraries to add to the build

We use KConfig to build in the libraries that Golioth needs. These changes are made in the prj.conf file of your application.

The first set of symbols deals with Zephyr’s networking stack. Of course every Internet of Things thing needs a network connection so you likely already have these symbols selected.

# Generic networking options
CONFIG_NETWORKING=y
CONFIG_NET_IPV4=y
CONFIG_NET_IPV6=n

Golioth is secure by default so we want to select the mbedtls libraries to handle the encryption layer.

# TLS configuration
CONFIG_MBEDTLS_ENABLE_HEAP=y
CONFIG_MBEDTLS_HEAP_SIZE=10240
CONFIG_MBEDTLS_SSL_MAX_CONTENT_LEN=2048

Now let’s turn on the Golioth libraries and give them a bit of memory they can dynamically allocate from.

# Golioth Zephyr SDK
CONFIG_GOLIOTH=y
CONFIG_GOLIOTH_SYSTEM_CLIENT=y
 
CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE=256

Every device needs its own credentials because all connections to Golioth require security. There are a few ways to do this, but perhaps the simplest is to add them to the prj.conf file.

# Golioth credentials
CONFIG_GOLIOTH_SYSTEM_CLIENT_PSK_ID="220220711145215-blinky1060@golioth-settings-demo"
CONFIG_GOLIOTH_SYSTEM_CLIENT_PSK="cab43a035d4fe4dca327edfff6aa7935"

And finally, I’m going to enable back-end logging. This is not required for connecting to Golioth, but sending the Zephyr logs to the cloud is a really handy feature for remote devices.

# Optional for sending Zephyr logs to the Golioth cloud
CONFIG_NET_LOG=y
CONFIG_LOG_BACKEND_GOLIOTH=y
CONFIG_LOG_PROCESS_THREAD_STACK_SIZE=2048

I separated each step above for the sake of explanation. But the combination of these into one block is the boiler-plate configuration that I use on all of my new projects. See this all as one file in the prj.conf from our hello sample.

3. Instantiate the Golioth system client and say hello

So far we added Golioth as a Zephyr module and enabled the libraries using KConfig. Now it’s time to use the Golioth APIs in the main.c file.

The first step is to include the header files and instantiate a golioth_client object. The client is used to manage the connection with the Golioth servers. Including the coap header file is not strictly required to establish a connection, but it is necessary for processing the responses from Golioth, so I always include it.

#include <net/coap.h>
#include <net/golioth/system_client.h>
static struct golioth_client *client = GOLIOTH_SYSTEM_CLIENT_GET();

I also add a callback function to handle messages coming back from the Golioth cloud. Technically this is optional, but if you want two-way communication with the cloud you need it!

/* Callback for messages received from the Golioth servers */
static void golioth_on_message(struct golioth_client *client,
                   struct coap_packet *rx)
{
    uint16_t payload_len;
    const uint8_t *payload;
    uint8_t type;
 
    type = coap_header_get_type(rx);
    payload = coap_packet_get_payload(rx, &payload_len);
 
    printk("%s\n", payload);
}

In the main function, I register my callback, start the Golioth client, and call the golioth_send_hello() API.

/* Register callback and start Golioth system client */
client->on_message = golioth_on_message;
golioth_system_client_start();
 
while (1) {
    /* Say hello to Golioth */
    int ret = golioth_send_hello(client);
    if (ret) {
        printk("Failed to send hello! %d\n", ret);
    }
    else printk("Hello sent!\n");
}

When successful, the golioth_send_hello() call will prompt for a message back from the server which includes the name of the sending device. This is printed out by the golioth_on_message() callback.

Hello sent!
Hello blinky1060
Hello sent!
Hello blinky1060
Hello sent!
Hello blinky1060

Extra Credit: Back-end Logging

Observant readers have noticed that I enabled back-end logging using KConfig symbols but I didn’t use it in the C code. Here’s the really awesome part: just use Zephyr logging as you normally would and it will automatically be sent to your Golioth console.

At the top of main.c, include the log header file and register a logging module.

#include <logging/log.h>
LOG_MODULE_REGISTER(any_name_you_want, LOG_LEVEL_DBG);

In our C functions, call logs as you normal would.

LOG_ERR("An error message goes here!");
LOG_WRN("Careful, this is a warning!");
LOG_INF("Did you now that this is an info log?");
LOG_DBG("Oh no, why isn't my app working? Maybe this debug log will help.");

Now your log messages will appear on the Golioth console!

Further Reading

This covers the basics of adding Golioth to an existing Zephyr project. You can see the steps all in one place by looking at the Golioth code samples. Here are the examples that you’ll find immediately useful:

  • Hello – basics of connecting to Golioth and sending log messages (what was covered in this post)
  • Lightdb Set – send data to Golioth
  • Lightdb Observe – device will be notified whenever an endpoint on the Golioth cloud is updated

To go even deeper, you can see how we use the Golioth Over-the-Air (OTA) firmware update feature, and how to use the Zephyr settings subsystem for persistent credential storage. And remember, the Dev Tier of Golioth includes the first 50 devices, so you can try all of this out for free.

Image from Todd Lappin on flickr

/by