Black text on yellow background that reads "Understanding Your Golioth Usage".

Today we are launching a new feature in the Golioth console: usage visualization. All Golioth users can now access real-time usage data, broken down by project and service, on the settings page for organizations in which they are an admin.

Screenshot of Golioth console with usage metrics shown for an organization with three projects.

This feature follows a series of console updates, including the launch of the Golioth Simulator, and a restructuring of project and organization navigation. With each new release, Golioth is becoming more conducive to managing large fleets with greater visibility, while also enabling more seamless collaboration between users in an organization.

Usage data is currently displayed for the following services:

By default, the current month’s usage is displayed, but the selector in the top right can be used to change the range. Additional usage metrics, as well as more advanced filtering and visualization functionality will be introduced over the next few weeks.

Sign in to the Golioth console to view your usage data, and reach out on the forum with any feature requests or feedback!

Nordic’s nRF9160 cellular modem includes a great peripheral called the Key Management Unit (KMU). This secure key storage mechanism lets you write keys to it which cannot be read back. However, they can still be used for DTLS authentication. In this video and blog post I’ll walk you through how to use the feature with the newest release of the Golioth Firmware SDK.

Overview of secure storage and TLS Tag authentication

With the v0.10.0 release of the Golioth Firmware SDK, credentials may be selected using the TLS secure tag. One example of hardware that embraces this is the Nordic nRF9160, which implements Zephyr’s TLS credential management API. Credentials (either x509 certificates or pre-shared keys) are stored on the device using a security tag. Pass that tag to the Golioth Firmware SDK and enable offloaded DTLS sockets in order to utilize those securely stored secrets.

Since these credentials are stored separately from firmware, they are persistent and you can store multiple different credentials. At runtime, pass the security tag as a parameter when creating the Golioth client and you’re all set.

How to store credentials on the nRF9160

Storing credentials on the nRF9160 is accomplished in two steps: first generate and prepare the credentials, then place them on the device using AT commands. (Don’t worry, there are helper tools to generate the AT commands for you.)

Generating Certificates

Golioth has long supported certificate authentication. You can follow the Certificate Authentication guide in our Docs. You will need to use the .pem version of the device certificate and key, and you’ll also need the root certificate from the CA that issued Golioth’s server certificate (we use LetsEncrypt.org).

  1. Generate the root certificate and device certificate by following the docs page.
  2. Download the CA certificate from Let’s Encrypt

Now upload the public root certificate you generated (golioth.crt.pem) to Golioth so it may be used to authenticate your device. From the left sidebar choose Project Settings, select the Certificates tab, and click Add CA Certificate:

Upload public root certificate to Golioth

Generating PSK Credentials

You should not use PSK credentials for production purposes, certificates are far more secure. However, for demonstration purposes, here’s what you need to do to prepare your PSK:

  1. Use the Golioth console to add a new device.
  2. Copy the PSK-ID and PSK from your newly generated device.
  3. Use the following command to format your PSK as a HEX string—the format required by the nRF9160 (note this is only for the PSK, not for the PSK-ID).
echo "your-psk-here" | tr -d '\n' | xxd -ps -c 200

Loading Credentials onto the nRF9160

  1. Build and flash the Nordic at_client sample onto your device.
  2. Use the Nordic nRF Connect for Desktop tools to write credentials to the device.
    1. Older versions of this tool will use the LTE Link Monitor, newer versions will use the Cellular Monitor.
    2. Choose the security tag you wish to use.
    3. Add your credentials to the interface and use the Update certificates button to write to the device. I found that I wasn’t able to write all three certificate artifacts at the same time and instead needed to enter them one at a time.

Nordic Cellular Monitor used to store device credentials on nRF9160

The credentials are stored using AT commands over a serial connection. You do not need to use the Nordic desktop tools if you don’t care to as the commands can be sent from any serial terminal connection.

Configure Golioth to locate credentials by tag

Now that our credentials are stored on the device, it is a trivial process to adapt the Golioth client to use them. In the video, I stored my certificates at security tag 24. I simply pass this to my application by adding the following Kconfig symbols to the prj.conf file:

CONFIG_GOLIOTH_AUTH_METHOD_CERT=y
CONFIG_GOLIOTH_SAMPLE_AUTH_TYPE_TAG=y
CONFIG_GOLIOTH_COAP_CLIENT_CREDENTIALS_TAG=24

(If you are testing PSK authentication instead of certificates, do not use the Kconfig symbol that sets CERT as the auth method. Without it, PSK will be selected by default.)

You must also remove one symbol from the boards/nrf9160dk_nrf9160_ns.conf file. This symbol tells the application to skip the offloaded DTLS sockets, however, that’s exactly the feature we want when using tag-based credential selection. Remove this line to re-enable socket offloading:

# Remove this line from nrf9160dk_nrf9160_ns.conf
CONFIG_NET_SOCKETS_TLS_PRIORITY=35

With that in place, compile the application and flash it to your nRF9160. It will now use the stored credentials to authenticate with Golioth.

Understanding the Golioth client config

The Golioth samples are configured to use a common library that configures the Golioth client on your behalf. However, the configuration process is very simple as I demonstrated it in the video.

Create a config struct that chooses TLS tags. Use the struct to pass the security number where the credentials are stored. Here’s what that would look like for my credential authentication example:

/* Config for credential tag authentication */
struct golioth_client_config client_config = {
    .credentials =
        {
            .auth_type = GOLIOTH_TLS_AUTH_TYPE_TAG,
            .tag = 24,
        },
};

client = golioth_client_create(&client_config);

Device Credentials with Golioth

The new APIs present in the Golioth Firmware SDK make it really easy to select credentials using tags. We’d love to hear how you plan to utilize this functionality. Show off your project on the Golioth Forum, or hit us up at the DevRel email!

Back in September, we released beta support for Zephyr RTOS in the Golioth Firmware SDK, as an additional option alongside our existing standalone Zephyr SDK. Today, we’re excited to remove the beta tag and announce full support for Zephyr in our cross-platform Firmware SDK. 

 

Over the last several months we’ve been improving the efficiency of the Firmware SDK and it is now on par with the standalone Zephyr SDK. We’ve expanded our automated test coverage, taken a fine-tooth comb through our APIs and documentation, and listened to and incorporated your feedback (thanks!). All of this means that the version of the SDK we’re releasing today is our best yet – and all of our customers will get these improvements at the same time, whether using Zephyr, ESP-IDF, or any other platform.

 

That’s why we chose to incorporate Zephyr support into our Firmware SDK and deprecate our standalone Zephyr SDK. We’re big fans of Zephyr, but our goal is to support the IoT community at large as a universal IoT connector, and that includes folks who choose to use other platforms. In order to provide the highest quality support to all of our users while moving as fast as possible to deliver new features and value, it made the most sense to invest in a single unified Firmware SDK.

 

Of course, our commitment to Zephyr itself hasn’t changed. We continue to be financial sponsors of the project, sit on both the Technical and Marketing Steering Committees, and provide free Zephyr training. And you’ll be able to find us at the Zephyr Developer Summit in Seattle in April.

Ready to get started?

You’ll find the same features and quality you’ve come to expect from Golioth in our Firmware SDK, but we’ve taken this opportunity to rework some of our APIs. Migrating from the Zephyr SDK is easy, and we’ve prepared a migration guide to help you along the way. For those beginning a new project, our getting started documentation will get you up and running in no time.

What does this mean for the Zephyr SDK?

As of today, our standalone Zephyr SDK is deprecated and not recommended for new designs. We may make periodic bug fixes and security updates to the SDK, but we won’t be adding new features or actively developing the standalone Zephyr SDK. This will result in accelerated delivery of features and support in our cross-platform Firmware SDK, as our team is now singularly focused.

 

We will reach formal end-of-life for the standalone Zephyr SDK on July 31, 2024, at which point we will no longer make any updates to the SDK. If you have any questions, please reach out to your Golioth account rep or contact us at [email protected].



Over-the-Air (OTA) firmware updates are table stakes for Internet of Things (IoT) devices. Once a device is in the field, OTA means you can make changes by updating the firmware remotely.

The ability to update devices remotely is great, but that alone is the bare minimum of functionality. Options for targeting specific devices for each firmware update are crucial to building a fleet that scales.

For example, at Golioth we use a small subset of our deployed fleet to test releases before rolling them out to all devices. In some cases we have different hardware in the same fleet that each need different binaries. And when adding new features we conditionally roll out release candidates to certain team members for testing.

All of these features are built into Golioth and ready for you to use. Recently one of our customers opened a forum thread asking how firmware version, device blueprints, and device tags work together to determine the OTA each device receives. It’s a great question that we’ll dive into today!

What are OTA Firmware Updates?

Over-the-Air (OTA) firmware updates are a method of using a network connection to send a device a new firmware version that it then validates and runs.

At Golioth, we’ve used multiple types of network connections to accomplish this, including cellular, WiFi, Ethernet, and Thread. Our Golioth Firmware SDK demonstrates the feature, using MCUboot to store two copies of firmware (the currently running version, and the newly received update). Each image is cryptographically signed so it can be verified for authenticity and integrity before the device uses the new image.

Golioth applies firmware updates on the device side using the semantic version number. The Golioth servers make these update versions available based on their sequence number. Let’s unpack the differences.

Demystifying Semantic Versions and Sequence Numbers

Remember this rule of thumb: devices will always match the semantic version of a firmware release, while the Golioth servers will always advertise the most recent sequence number.

Devices match semantic version

Devices have no awareness of “newer” or “older” firmware releases. They merely check if the version of firmware currently running is an exact match for the semantic version being advertised by the server. It they do not match, the device will download the binary and update itself to the version available from the server. This might be a newer semantic version, or an older one, the only thing that matters is that the device sees a different version is available.

Golioth OTA device serial output

Device serial output indicates version 1.0.100 is currently running. The manifest received from Golioth shows main-1.0.99 as the latest version. The device begins downloading. We call this a roll back, the device is only aware that the versions don’t match and the server is the source of truth.

This is crucial in delivering the ability to “roll back” a firmware update.  The Golioth web console has a rollout button that facilitates automatic roll back when you unselect the most recently uploaded firmware.

The server advertises the most recent sequence number

The Golioth server separates OTA into two distinct parts: the artifact and the release. The artifact is the binary itself which has a package name (the default name is main) and a semantic version number. The release is created using an existing artifact, adding a time-based sequence number (new releases have higher sequence numbers) and controlling whether or not the release is rolled out to devices.

Golioth OTA Artifacts

Each of the artifacts is a unique binary with its own package name and semantic version. This fleet uses all the same hardware so no Blueprint has been assigned to these artifacts

The Golioth servers will check to ensure artifacts have unique name & version combos — you can only upload main - 1.2.3 once. The next artifact will need a different version number or package name. (The only way around this is to delete the existing artifact so you may reuse the package/version number.)

Assigning a blueprint to an Artifact makes it unique. The rules from the previous paragraph still apply, but artifacts with a blueprint will only be compared to other artifacts with the same blueprint.

Finally, the file hash uploaded with each artifact must be unique from all other artifacts. The Golioth web console will check the file, and issue an error if the same binary is uploaded more than once. This is a safety feature to help ensure that the wrong artifact isn’t uploaded by mistake.

Golioth OTA Releases

Each release requires one artifact to be assigned. Notice that the main-1.0.1 artifact has been used multiple times, targeting devices with different tags assigned. The releases are made available to devices by enabling the Rollout toggle.

Golioth releases require an artifact, but the semantic version number will not be used to decide which release is advertised to devices. Instead, the release with the newest sequence number (meaning the most recently created release) will be advertised to devices.

There are many conditions to determining which release is advertised

It is important to understand that there are several ways to target devices with a release. The most obvious is the Rollout setting—a release will only be advertised if the rollout is enabled. Package names, device blueprints, and device tags are also used to determine which releases will be advertised to any given device in your fleet.

Applying Blueprints to Firmware Updates

If all devices in your fleet use the exact same hardware, they may all be able to run the same firmware. But in many cases, a fleet will have more than one hardware variant and need more than one compiled version of the same firmware release. For instance, if you have some devices that use an nRF9160 (cellular) and others that use an NXP i.MX RT1024 (Ethernet) you must run different firmware compiled specifically for those two distinct devices.

Golioth uses device blueprints to account for this issue. When you create devices on the Golioth cloud, you can choose one device blueprint to assign to the device. The same blueprint may be selected when uploading an artifact or creating a release.

When a device has a blueprint assigned, it will receive notification of releases with the newest sequence number and the matching blueprint.

Using Tags to Target OTA

Device tags can be assigned in a similar way to device blueprints but you may assign multiple tags (blueprints are limited to a single assignment per device/artifact/release).

Device tags must match exactly to receive a release notification. This means if you have a device with two tags and a release with only one tag, the device will not be notified. That said, if you roll out a release with zero tags selected, all devices in your fleet will be notified of the release no matter what tags are assigned to those devices.

Golioth Device Summary

This summary page for a single device in our fleet shows that this is a member of the “release-candidate” tag group. The device reports which version of firmware it is currently running, as shown in this view.

Multiple releases may be created using the same artifact. As an example, at Golioth we will roll out a release that targets the release candidate tag, so that devices that we have previously identified for testing new features will receive the update but the larger fleet will not. When testing is complete and the firmware artifact is determined to behave as we expected, a second release using the same artifact will be made without any tags, prompting the rest of the fleet to download and apply the OTA update.

How Does Golioth OTA Fit Your Needs?

Wow, this is a lot. Thanks for sticking with me through this post. I hope you will agree that there is quite a bit more to OTA than just providing an update file.

We’d love to hear your feedback. We’re especially interested to know your thoughts on how our system approaches tags. The last thing a fleet manager wants is a “surprise” update to a device they weren’t expecting. This is why we’ve gone to great lengths to implement granular control, and a multitude of options. But we’re always keen on using customer feedback to improve, so let us know on the Golioth Forum or hit us up at the DevRel email address.

Until next time, happy OTA updating!

At Golioth we are on a quest to improve the security, reliability, and efficiency of IoT products. Today we are launching the Golioth Simulator as part of an increasing focus on addressing the organizational challenges that come with building those products. The Simulator enables cloud engineering teams to generate data that mimics what devices deployed in the field will eventually send to Golioth, prior to the devices being ready. This eliminates bottlenecks caused by long hardware and firmware development cycles, and allows for the services that interact with device data to be developed in parallel.

Screenshot of Golioth Simulator with many instances of Golioth logo in background.

The Development Cycle Mismatch

Building connected products is challenging for a number of reasons, but one of the most prominent is the range of engineering work that is required. This is illustrated by comparing to the development of products that are either only software, or non-networked hardware. In the former, engineering work ranges from distributed systems and backend development to frontend and graphic design. In the latter, the range may be from electrical engineering and PCB design to firmware development.

IoT products essentially take those two spectra and join them.

Three boxes, one describing hardware products as a spectrum between hardware and firmware, one describing software products as a spectrum between backend and frontend, and one combining them to describe IoT products.

While this clearly increases the breadth of skills that are necessary for a team to possess, it also introduces a more subtle challenge: wildly different iteration cycles and development timelines. In the worst case scenario, the longest development cycle, which is typically hardware, will dictate the velocity of an entire project. A common fallacy is to compare the relative effort of software development to hardware development and conclude that the former can be delayed until the latter is in its final stages. In reality, any aspect of the engineering spectrum can derail a product.

However, even if a team properly prioritizes all aspects of developing the product, it is easy to fall into the trap of creating both implicit and explicit dependencies between components of the system. Mitigating these organizational hazards can be a force multiplier.

Innovation at the Interface

We frequently describe Golioth as the interface between firmware and cloud engineers. Sitting between these two groups comes with a responsibility to present a familiar set of tools to both parties, but also an opportunity to introduce new functionality that closes the gap between them. While the Golioth platform presents separate APIs for devices and servers, each designed to be familiar to the respective consumer, the commonality between them is data.

Ultimately, connected products are about producing data in one location, and consuming it, or some transformation of it in another location. Typically, the majority of data for our customers today is being produced by devices, and consumed by servers. All of this data flows through Golioth, meaning that customers have a consistent interface for both sending and receiving data. But what about the shape of the data itself?

Abstract diagram of a device and a server being connected by a common understanding of the shape of data.

In the end, it will depend on the behavior of devices, but software iteration that depends on managing physical devices, or even firmware running in an emulated environment, to produce data is sacrificing efficiency. In the ideal case, firmware and cloud teams use the shape of data as a contract between systems, and software engineers are able to easily produce data that adheres to that contract. If that contract changes in the future, the tooling should make testing and updating the software that interacts with it seamless. The Golioth Simulator is a first step in that direction.

How it Works & Getting Started

As of today, the Simulator is available in the Golioth console for all users. Navigating to any device in your fleet should result in the Simulator launcher appearing in the bottom right corner of your screen. Clicking on it will open a floating window.

Navigate to the Credentials tab of the chosen device, and copy your preferred PSK ID and PSK into the Simulator. Use the default JSON payload or enter your own, then click Send. If you navigate to the LightDB Stream tab, you should see a new message with your payload.

To view new messages as you send them, be sure to enable Auto-Refresh. Because the Simulator is actually interacting with Golioth device APIs, messages will also flow through to any Output Streams you may have configured in your project.

Next Steps & Feedback

The initial functionality of the Golioth Simulator is intentionally basic as we are interested in seeing how users leverage the new interface to increase their pace of development. If you have feedback about how you would like to see the Simulator evolve to support more use cases, please reach out to us on the forum.

Around these parts, we refer to Golioth as “the Universal Connector”. Our firmware SDK works seamlessly with your firmware project, our cloud APIs are directly mapped to device APIs within the SDK, and that singular connection from device to cloud allows your data to flow unhampered to the rest of the internet. As a result, we are always looking for the most useful services to route your device data to, while also offering you device management and Over The Air updates.

Memfault is a device diagnostics and telemetry platform that we have been friends with for a long time. If you want deep insight into what’s happening with your device in the field, Memfault is a great way to visualize what’s going right and wrong. We know them from their integrations into the Zephyr ecosystem, as well as their great writing on their Interrupt blog. Last month, we co-announced our partnership with them, working with NXP to integrate using the Zephyr Real-Time Operating System.

Golioth enables connections over CoAP to field devices and routes the data out to services like Memfault

Today, Memfault published a joint piece of content with our Cloud Lead, Dan Mangum, that explains more about how data flows through Golioth to the Memfault platform.

See it in action

If you’re anything like me, a live demo really solidifies how a thing works. NXP’s RT10xx family provides a great platform for showcasing Zephyr applications that can route data up to the cloud. We have been partners with NXP since last year and continue to find their software capabilities and seriously powerful processors a great addition to our Continuously Verified Boards, as well as excellent targets for all kinds of applications.

If you like demos, you’re in luck! We’re doing a joint webinar with Memfault and NXP on November 16th at 9 am PST / 12 pm EST / 6 pm CET. Register now to watch it live, or sign up and get access to the on-demand recording if you can’t make it.

Join our live webinar with NXP and Memfault on Thursday Nov 16th

Use this new integration on your project

The capabilities shown in the blog post linked above and during the webinar this Thursday include some functionality that is still in early preview. If you’re interested in having access to Golioth data routing that instantly enables the Memfault, sign up on our page here. You’ll be among the first to know when this capability is available for your projects. In the meantime, you can attend the webinar, try out other projects on Golioth, and discuss your latest IoT experiments on our forum.

Golioth Reference Designs now include purchasable hardware setups that mirror all functionality on our custom hardware solution. Utilize Golioth firmware and cloud together to showcase complex IoT applications in minutes!

Compact designs have custom elements

Golioth Reference Designs help to showcase end-to-end IoT applications using quasi-custom hardware. If you’ve seen any of our articles or our guides on projects.golioth.io, you have probably seen a setup that looks something like this:

The Cold Chain Asset Tracker with an open case

This is a stackup of a custom board (the “Aludel Mini”) that ties together a battery, ClickBus based sensor boards, a custom display made out of a PCB (the “Ostentus”), switches, buttons, antennas, and off-the-shelf devboards with processing and connectivity (the “Sparkfun ThingPlus nRF9160”). These are all placed inside a case with cutouts where we can insert custom 3D printed panels to fit whichever Click boards we’re using. This creates a relatively compact package, at least given that it can be reused across a range of different ideas:

The Golioth Cold Chain Asset Tracker with an open case: It’s not the smallest form factor possible, but it’s also not a mess of wires and devboards glued together.

Golioth is well tailored to users who make their own custom hardware. Unlike other cloud platforms, we don’t lock you into using a module you can only buy from us. Instead, we publish an SDK that can be used across a range of supported hardware and connectivity types. You can put just about any type of connected embedded device onto the Golioth Cloud.

However, there’s a problem if you want to re-create the Reference Designs pictured above: You can’t buy the custom hardware setup that we use!

What if you want to follow along?

Keen readers of this blog will note that we just open-sourced the firmware for our Reference Designs, including the template we use to create our designs. That’s only really useful if you have hardware to run it on. Since you can’t buy all of the hardware that we showcase, how can you benefit from the open source firmware? We now have an accessible, alternative option for you.

The Hardware

Follow-Along Hardware is the full set of development boards, interface boards, sensor modules, and helper items to replicate the setup that we use at Golioth.

We take advantage of the fact that many vendor development boards (still?) utilize an Arduino Uno pinout on their larger development boards. There is a conversion board that goes from Arduino Uno to 2 Click headers, which is the form factor of the majority of sensors and peripherals we integrate into our Reference Designs.

The OBD-II / CAN Asset Tracker Design, assembled on top of the Nordic Semiconductor nRF9160-DK

The Firmware

In order to support both the custom hardware shown in the photos above, we lean on the portability of the Zephyr Real Time Operating System (RTOS) and ecosystem. Because the Aludel Mini and the nRF9160-DK both have the same chipset on them, we can write a new set of board files that target the pin differences between these two boards. The flexible “fabric” of the nRF9160 (the SIP on both of these boards) allows the pins to be reassigned, as needed. The difference in build commands in Zephyr is as simple as:

# Build for Nordic nRF9160 Development Kit:
$ west build -p -b nrf9160dk_nrf9160_ns app

# Build for custom Golioth hardware:
$ west build -p -b aludel_mini_v1_sparkfun9160_ns app

These two commands will target their respective hardware.

For the “optional” modules that are part of the quasi-custom hardware solution–like the Ostentus display with an eInk screen and touch buttons–we check at runtime whether the module is present. If you’re running on an nRF9160-DK without the display (maybe you’re looking at data output on the serial terminal), the program will disengage the code that runs the Ostentus until the device is reset.

Try before compiling

We want to make it so easy to try out these Reference Designs that have a Follow-Along Hardware component, that we don’t make you compile anything. Each supported piece of hardware (on a per-project basis) also includes a pre-compiled image in the associated firmware repository. If you follow the directions for your hardware, you will have a fully functional project on your bench in a few minutes. Then you can start to customize to your heart’s desire.

More Reference Designs

We are dedicated to supporting hardware and firmware engineering teams building real-world applications. A custom piece of hardware has a ton of challenges, but Golioth makes sure that cloud connectivity and device management are an easy checkbox on your list. If you are working on an application you don’t see covered on our Projects site, drop us a line and we’ll see if we can spin up a new design. You’re always welcome to stop by our forum to discuss your IoT application, as well.

Announcing Beta Support for Zephyr in the Golioth Firmware SDK

We are excited to announce the availability of a public Beta for support for the Zephyr RTOS in the Golioth Firmware SDK! But wait, you say, doesn’t Golioth already support Zephyr? We do! Goloith has always been, and will continue to be, a big proponent of the Zephyr RTOS and its community. We built the first version of our product specifically to work with the Zephyr RTOS. But our goal is to serve the IoT community at large, and that includes folks who choose to use an operating system other than Zephyr.

To that end, we created the Golioth Firmware SDK, which is designed from the ground up to be portable to any RTOS. It already has full support for ESP-IDF and Modus Toolbox, limited support for Linux, and a pathway for porting to any other operating system. It also incorporates all of our learnings over the last few years building IoT firmware, with the result being a cleaner internal architecture and more flexible external APIs.

This left us with two SDKs to support, meaning every bugfix or new feature required twice as much work. Like many of you, we have finite resources, and doing everything twice didn’t feel like the right way to scale. So, we chose to invest in bringing our same first-class support for Zephyr to the Golioth Firmware SDK.

What does this mean for Zephyr support?

Golioth’s support for Zephyr remains as strong as ever. We continue to be financial sponsors of the project, sit on the Technical Steering Committee, and provide free Zephyr training. In short, we love Zephyr!

What’s next?

We are already hard at work on the next set of improvements to Zephyr support in the Golioth Firmware SDK. Our focus now is on driving down the resource usage – RAM, ROM, and CPU cycles – of the SDK when running on Zephyr (with some of those improvements applying to our other ports as well). We will incorporate your feedback on our APIs, functionality, and reliability, and we expect Zephyr support in the Golioth Firmware SDK to be generally available by early next year .

How can I help?

You can help by test-driving Zephyr RTOS on the Golioth Firmware SDK. Follow our guide for setting up your build environment. The Firmware SDK includes a collection of sample code that demonstrates API calls for all Golioth services. And of course, we have an SDK Reference if you really want to dig in. Most importantly, don’t forget to share your thoughts on the experience with us: .

What if I’m already using the Zephyr SDK?

You can continue to use the Zephyr SDK as you do today and devices in the field running the Zephyr SDK (or earlier versions of the Golioth Firmware SDK) will continue to function.  We will continue to support you with bugfixes and new features for the near term.

Of course, if you are curious about where we are headed, we would love to have you try out the Golioth Firmware SDK! We are here to help you with the transition to the new SDK – please reach out to [email protected] or post on the Golioth Forum with any questions.

I’m starting a new project with Zephyr, which Golioth SDK should I choose?

For devices going to production in 2023, we recommend using the Golioth Zephyr SDK. The Firmware SDK supports all the features of the Golioth platform today, but is not yet as optimized as the Golioth Zephyr SDK.

We expect to reach general availability of Zephyr support in the Golioth Firmware SDK in early 2024. If you are building a Zephyr based device with plans to go to production in 2024 or later, you may want to consider beginning development in the Golioth Firmware SDK to reduce the burden of transitioning later. 

At Golioth, we pride ourselves on doing both the deep technical work—as well as the routine maintenance—required to ensure we back up our claim of being the most efficient way that devices in the field can securely talk to services in the cloud.

In service of this mission, last week we turned on DTLS 1.2 Connection ID support in Golioth Cloud. While most users will not see any visible differences in how their devices connect to Golioth, behind the scenes many devices are now enabled to perform more efficiently. This is especially true for those connecting via cellular networks.

Devices using Connection ID can see a meaningful impact on bandwidth costs, power usage, and battery life. This not only benefits users of Golioth, but also means that more devices are able to operate for longer, reducing waste and energy use. Because we believe deeply in this potential widespread impact, we have been and will continue to do all work in this area in the open.

What are Connection IDs?

We have written previously about how we use CoAP over DTLS / UDP to establish secure connections between Golioth and devices. Because UDP is a connection-less protocol, attributes of each datagram that arrive at a local endpoint must be used to identify the remote endpoint on the other side.

In short, DTLS uses the IP address and port of the remote endpoint to identify connections. This is a reasonable mechanism, but some devices change IP address and port frequently, resulting in the need to constantly perform handshakes. In some scenarios, a full handshake may be required just to send a single packet. Performing these handshakes can have a negative impact on battery life, drive up bandwidth costs, and increase communication latency. For some devices, this cost makes the end product at best more expensive, and at worst, infeasible.

Connection IDs provide an alternative solution, allowing for clients and servers to negotiate an identifier that can be used in lieu of the IP address and port. Once negotiated, the endpoint(s) that are sending a Connection ID — typically just the client but sometimes both the client and the server — are still able to have their DTLS connection state associated with incoming records when their IP address and port changes. The result is that a device could travel from one side of the world to the other, continuing to communicate with the same server, while only performing the single initial handshake.

How was Connection ID support implemented?

Efficient communication between devices and the cloud requires compatible software on both sides of the connection.

Cloud Changes

On the cloud side, we are beneficiaries of the work done by folks in the Pion community, which includes a set of libraries for real-time web-based communication. Conventional use of these libraries is enabling video streaming applications on the internet, such as Twitch. However, the protocols they implement are useful in many constrained environments where the network is lossy or unreliable.

The Golioth team contributed Connection ID support in the pion/dtls library. This consisted of both the implementation of Connection ID extension handling and modifications to UDP datagram routing. The former involved careful updates to the parsing of DTLS records; Connection IDs change record headers from being fixed length to variable length. As part of this work, we are also adding a very small new API in the Golang crypto library.

Previously, pion/dtls utilized an underlying net.Listener, which was conventionally supplied by the pion/transport library. This UDP net.Listener handed net.Conn connections to the pion/dtls listener, which would in turn establish a DTLS connection by performing a handshake with the client on the other side of the connection. However, the net.Conn interface does not allow for consumers to change the remote IP address and port to which packets are sent. When not using Connection IDs, this is not an issue because the IP address and port are what identifies the connection. However, when Connection IDs are in use, the ID itself is used to identify the connection, and the remote IP address and port may change over time. Thus, a new interface, net.PacketListener, was added to pion/dtls, which enables the changing of the remote address of a connection, and an implementation of the interface that routes based on Connection IDs when present was supplied.

Device changes

On the device side, most users leverage the Golioth Firmware SDK to communicate with Golioth services, such as OTA, Settings, Stream, and more. The SDK is meant to work with any hardware, which is why we integrate with platforms such as Zephyr, Espressif ESP-IDF, Infineon Modus Toolbox, and Linux. Many of these platforms utilize DTLS support offered by mbedTLS, which added support for the IETF draft of Connection IDs in 2019, then included official support in the 3.3.0 release in 2022. The SDK uses libcoap, which implements CoAP support on top of a variety of DTLS implementations, including mbedTLS. libcoap started consuming mbedTLS’s Connection ID API in July of this year. We have been assisting in ensuring that new versions of libcoap are able to build on the platforms with which we integrate.

However, these platforms frequently maintain their own forks of dependencies in order to integrate with the rest of their ecosystem. We have both been contributing and supporting others contributions wherever possible in order to expedite the use of Connection IDs in the Golioth Firmware SDK. With Connection ID support already in place in ESP-IDF and Nordic’s sdk-nrf, and coming soon in the next Zephyr release, we hope to turn them on by default for all platforms in upcoming Golioth Firmware SDK releases.

How do I use Connection IDs?

Using the Golioth Firmware SDK is the only requirement to utilize Connection IDs. As support is added across all embedded platforms, update to the latest version in your device side code and your connection will automatically be enabled. The video at the top of this post shows how you can inspect your own device traffic to see the functionality in action.

What comes next?

In total, we have made contributions across many open source projects as part of our effort to make Connection IDs widely available, and we look forward to continuing to lend a hand wherever possible. While this post has provided a brief overview of DTLS, Connection IDs, and the ecosystem of libraries we integrate with, ongoing use of the functionality will allow us to provide tangible measures of its impact. We’ll make sure to make announcements as support continues to be added across platforms and consumed in the Golioth Firmware SDK.

Until then, go create a Golioth account and start connecting your devices to the cloud!

 

The Golioth Reference Design Template is now publicly available to review and use with your IoT products. Our reference designs (and the template they are all based on) are now using the Apache 2.0 open source license. Over the last nine months of beta testing and development we’ve been sharing the source code only upon request. Starting today, we’re moving to stable releases and we’re making the repositories public for each design.

We began developing reference designs at the end of 2022 as a way to help customers jumpstart their fleet deployments. This goes far past hello-world, and gets your hardware up and running with an encrypted connection to every device, time-series and stateful data pipelines with the cloud, remote logging, remote procedure call, fleet settings, and of course over-the-air (OTA) firmware updates.

All of our specialized designs begin with the Golioth Reference Design Template. This is a firmware jumping off point that lights the way to customizing an implementation for your unique hardware needs.

You can try it right now using off-the-shelf parts (all that’s needed is a Nordic nRF9160 Development Kit) or on your own hardware using Zephyr RTOS.

What’s Included in a Reference Design?

Golioth Reference Design Template sources list

Our reference design template demonstrates how to use all of the Golioth services, broken out into bite-sized files.

  • dfu: all of our reference designs implement over-the-air (OTA) firmware updates. The files in this folder handle all the details for you.
  • app_work: the core work for the application happens here. Sensor readings, data processing, and storing time-series data is found in this file.
  • app_state: bi-directional stateful data is handled in this file. The device will write information about its actual state using functions in this file.
  • app_settings: update your entire fleet at the same time, or drill down to specific devices. This file registers the Golioth Settings service and takes action when new settings are received.
  • app_rpc: command and control gets truly powerful with the ability to call functions remotely, optionally supplying arguments that weren’t available at compile time.
  • battery_monitor: an example of how to monitor and report device battery state back to the cloud.

Each of these files include API calls using the Golioth Zephyr SDK. Within that SDK, you’ll find samples of each of the above services as well; this is actually how we built the reference design template. An easy way to think about these layers:

  • The Golioth SDK defines the service and assists in connecting to the Cloud-side capability
  • The SDK samples are a simple implementation of a standalone service, such as LightDB Stream
  • The Reference Design Template puts together multiple SDK samples in a format that approximates how product teams will build an all-inclusive firmware design. See below for a real-world example we built on top of the Reference Design Template.

Code Walkthrough

Mike and Chris walk through the code that’s part of the Reference Design Template

DC Power Monitor Reference Design

DC Power Monitor on top of graphed data

Today we are also releasing the DC Power Monitor Reference Design which illustrates voltage, current, and power monitoring for direct current applications like battery monitoring for micro-mobility fleets. This design was built on top of the Reference Design Template.

As with the template, this reference design can be run with off-the shelf parts. Support is included for the Nordic nRF9160 Developer Kit, along with an adapter board and two INA260 sensors from MikroElektronika. Full details on the features and hardware used in this reference design are available from the Golioth Projects page.

More Reference Designs Coming Weekly

Golioth Projects shares reference design details

The Reference Design Template and the DC Power Monitor are both live right now. We plan to publish one or two more each week until all of our reference designs are public. Keep an eye out for those weekly announcements, with the OBD-II / CAN Asset Tracker and the Air Quality Monitor as the next planned releases.

Sign up for the Golioth newsletter to make sure you don’t miss any of these announcements!

What Will You Build?

IoT shouldn’t be so hard, that’s why we’re here! Use Golioth as your instant IoT Cloud and finish your proof of concept quickly.

We’d love to know what you’re building. Post about your projects on the Golioth Forum and get in touch with our developer relations team to help with any questions you have along the way.