Devices that connect to the Golioth Cloud communicate securely thanks to a pre-shared key (PSK) that encrypts all messages. But how do you get a unique set of credentials onto every device? The options for that just got a whole lot more interesting thanks to some new Golioth features. There are now two ways to set the credentials using either the device shell, or our command line tool that also fetches those credentials automatically!

Zephyr settings using the device shell

Zephyr has a shell option that runs on the device itself. This is great for things like network or i2c debugging (there are specialized commands for both of those and much more). You send your credentials over a serial connection and Golioth leverages the Zephyr settings subsystem to store the device credentials (PSK-ID and PSK) in flash memory.

As part of the getting started guide, you already set up a device in the Golioth Console. Use the Devices sidebar option to find that device again (or create a new one) and click on the Credentials tab to access your PSK-ID and PSK:

Golioth Device Credentals

Now we need some code to run on the device. The Golioth settings example already has this feature built in. For this example I’m using a Nordic nRF9160dk. You can build and flash the example right away. Normally we’d put credentials into the prj.conf file first, but this time we’ll just assign those from the shell!

cd ~/golioth-ncs-workspace/modules/lib/golioth/
west build -b nrf9160dk_nrf9160_ns samples/settings/
west flash

Once programming has completed, load up your serial terminal tool of choice. I like to use minicom -D /dev/ttyACM0 but you should have the same success with screen /dev/ttyACM0 115200 or any other similar tools.

  • You’ll be greeted by the uart:~$ command prompt
  • The syntax that we need is settings <settings_key> <setting_value>
    • PSK-ID needs to be assigned to golioth/psk-id
    • PSK needs to be assigned to golioth/psk
  • Reboot the device after changing the settings

Here’s what that looks like in action (important lines highlighted):

uart:~$ *** Booting Zephyr OS build v2.6.99-ncs1-1 ***
[00:00:00.209,716] &lt;inf&gt; golioth_system: Initializing
[00:00:00.216,278] &lt;inf&gt; fs_nvs: 2 Sectors of 4096 bytes
[00:00:00.216,278] &lt;inf&gt; fs_nvs: alloc wra: 0, fa8
[00:00:00.216,278] &lt;inf&gt; fs_nvs: data wra: 0, 90
uart:~$ settings set golioth/psk-id [email protected]
Setting golioth/psk-id to [email protected]
Setting golioth/psk-id saved as [email protected]
uart:~$ settings set golioth/psk my_complex_password
Setting golioth/psk to my_complex_password
Setting golioth/psk saved as my_complex_password
uart:~$ kernel reboot cold

After rebooting, the board connects to a cell tower and the connection to Golioth is successfully established!!

uart:~$ *** Booting Zephyr OS build v2.6.99-ncs1-1 ***
[00:00:00.215,850] &lt;inf&gt; golioth_system: Initializing
[00:00:00.222,381] &lt;inf&gt; fs_nvs: 2 Sectors of 4096 bytes
[00:00:00.222,412] &lt;inf&gt; fs_nvs: alloc wra: 0, fa8
[00:00:00.222,412] &lt;inf&gt; fs_nvs: data wra: 0, 90
[00:01:06.672,241] &lt;dbg&gt; golioth_hello.main: Start Hello sample
[00:01:06.672,485] &lt;inf&gt; golioth_hello: Sending hello! 0
[00:01:06.673,004] &lt;wrn&gt; golioth_hello: Failed to send hello!
[00:01:06.673,095] &lt;inf&gt; golioth_system: Starting connect
[00:01:06.967,102] &lt;inf&gt; golioth_system: Client connected!
[00:01:11.673,065] &lt;inf&gt; golioth_hello: Sending hello! 1
[00:01:16.674,316] &lt;inf&gt; golioth_hello: Sending hello! 2

Golioth Credentials automatically set from the command line

What if I told you that a one-line command could look up your device credentials from the Golioth Cloud and automatically send them to the device? This is literally the feature we’ve implemented. Now, I’m excited about the shell settings above, but this new command line feature is absolutely legendary!

Golioth device name

  1. Look up your device name from the Golioth Console
  2. Issue the command, using your device name and the correct port:
    1. goliothctl device config --name <device name> --port <serial port>

Here’s what it looks like in action:

$ goliothctl device config --name nrf91-settings-demo --port /dev/ttyACM0
failed to get golioth/psk-id from device: setting not found
device success setting golioth/psk-id saved as [email protected]
failed to get golioth/psk from device: setting not found
device success setting golioth/psk saved as my_complex_password
closing serial read

And check this out, it’s a quick way to make sure you have the device credentials correct. Since you’re not copy/pasting or typing the credentials, you know you have it right as long as you get the name of the device right. Running the command a second time confirms those settings are correct:

$ goliothctl device config --name nrf91-settings-demo --port /dev/ttyACM0
golioth/psk-id in the device is already set to [email protected]
golioth/psk in the device is already set to my_complex_password
closing serial read

Visions of End Users and Bulk Provisioning

Two really easy ways to see the new features put to use are end users and manufacturing. Imaging sending devices with “stock” firmware out to customers and having them add their own credentials (we have a snazzy web-based demo in the works so stay tuned). The other thought is toward bulk-provisioning where a script can be used to register the new device on Golioth, generate credentials, and send them to the device all in the same step.

We’d love to heard about your experiences with these new tools. Catch up with us on the Golioth Discord so we can have a chat!

The line between the hardware and software worlds continues to blur. One advantage is the range of software tools that are coming into the hardware and firmware space, such as “Continuous Integration, Continuous Deployment” (CI/CD). This ensures each commit of code to a repository is immediate run against a slate of tests, compiled using a standardized compiler, and deployed to eligible devices for testing. What if your IoT firmware deployments happened automatically just by typing git push?

In this post and the associated video, Lead Engineer Alvaro Viebrantz talks about a sample project that compiles and delivers firmware to eligible devices automatically using GitHub Actions. Follow along in our repository on the Golioth Labs page.

Compiling in the cloud?

Most embedded engineers hear “cloud compilation” and start shaking their head no. Having an IDE or a local toolchain is the expectation for fast iteration and direct debugging of code using tried and true methods. But being “local only” also runs into dependencies that might be on your machine versus your co-worker’s machine. What’s worse, you might be lulled into a false sense of security and never venture past having a single machine that is capable of compiling critical device firmware. (Don’t believe me? Ask a long time firmware engineer about the steps they had to go through to keep that old computer running to compile code for the legacy project.)

Moving your final (production) builds to the cloud is beneficial because it removes localized dependencies. Distributed teams are increasingly common. Now when an engineer in Brazil, the US, Poland–or any other place your teammates might be–commit code to a repo, it receives the same treatment. If you have GitHub actions (or equivalent hooks) set up on your repository, it kicks off a compilation on a container on the cloud. The output firmware is then uploaded to Golioth. As we see in the video, it’s still possible to build on a local machine, but the standardization is very helpful as you move towards scaling your product and making production grade firmware.

Walking through the demo

Let’s recap the steps taking place in the video and discuss what is relevant about each step.

  • Build firmware locally
    • This showcases that while the toolchain is available on the cloud, it’s not only on the cloud. Developers can still create a local image for testing.
  • Load initial firmware to the device
    • Since the firmware is responsible for reacting to new firmware on the Golioth Cloud, we need to load an initial image to start checking whether an update is available. As Alvaro mentions, the firmware recognizes that the same version loaded on the device initially is also the latest on the cloud, so no action is taken.
  • Creating device credentials using the hardware ID
    • Golioth allows you to set a range of different IDs to help identify devices in the field. This is especially important as your fleet grows. As part of the provisioning process, Alvaro shows how the code on the firmware image can also extract a unique identifier that is programmed into the silicon at the factory, so it is possible to always verify which device is being identified, all the way down to the silicon.
  • Generate device credentials on Golioth
  • Set WiFi/Golioth credentials over the Console
    • Setting device credentials allows the Golioth Cloud to validate a device is able to talk to the network. Alvaro demonstrates using a serial connection with the device to add these credentials, a recent improvement that will enhance device programming in the factory or as users provision a new device.
  • Device connects and begins to send logs and LightDB State data
  • Make a small change and commit to the repo. Push a git tag.
    • Pushing the change to the repository and adding a tag is what alerts the GitHub actions logic to start compiling a new firmware image.
  • The firmware is built in the cloud as part of GitHub actions.
    • For this build, Alvaro is using our Arduino SDK along with PlatformIO. This is in a container that the GitHub action boots up and uses to standardize the firmware build.
  • The artifact is pushed to the Golioth cloud
    • As part of the setup process, the user will need to generate an API key to place on GitHub in order to allow GitHub to push the new firmware issue as an Artifact on Golioth.
  • Create a release using the Console and roll out the firmware to the eligible device.
    • This is a manual process in the video, but a user could also utilize the REST API in order to create a Release and set the release to be eligible for rollout to a set of devices that are tagged on Golioth (different than GitHub tags). Remember: Any action possible on the Console can be scripted using the REST API.

Extending the demo

One theme that is obvious throughout this video is the focus on moving your product to production. Programming devices as they come off the line is no small task and something we will continue to make content about here at Golioth. And future videos will describe methods for running real-world tests on hardware, something we are interested in given our support of hundreds of boards.

In the current demo, Alvaro shows loading credentials over serial. Past examples have also shown Bluetooth credentialing using MCUmgr. We’d love to hear more about how you’re creating your devices and how you want to program things as you move towards production. Please join us on our Discord or on our Forums to discuss more.

Looking for direct help getting your devices into production? Reach out at

Golioth is a flexible device management solution that allows you to manage devices that have IP addresses. This includes devices connecting over Ethernet, cellular, and Wi-Fi. Today we’re showcasing a demo of a Thread-based device. Utilizing the Golioth Console and API endpoints, it’s possible to manage a wide array of devices provisioned on the internet through a Thread Border Router. This enables even more low-power devices to reliably push data back to the internet for processing on the cloud.

In this post and the associated video, we show a demo of how developers can try out using Golioth to manage Thread based devices.

What is Thread?

Thread is a network protocol for low power IoT devices. It uses 6LoWPAN for mesh communications, in our case on 2.4GHz. In the examples above, we showcase Thread using OpenThread on an nRF52840 from Nordic Semiconductor. Multiple hardware vendors have Thread-based solutions, using Zephyr or other firmware solutions. Vit Prajzler shows how he is using the OpenThread documentation to build different types of devices that form a Thread mesh network, including a Radio Co-Processor (RCP) and a Full Thread Device (FTD).

Set up a Border Router

A Border Router is an element on a Thread Mesh network that allows the entire network to communicate with the broader internet. Each individual device (Node) has an IPv6 address, which allows Golioth to manage specific devices on a mesh.

While it’s possible to buy a few off-the-shelf Border Routers, we showcase building one using commonly available components. Vit is using a Linux computer (a Raspberry Pi), along with an nRF52840 dongle configured as an RCP to route network traffic from the mesh to the rest of the internet. In the case of his Thread network, Vit needs to include a translation layer (called NAT64) to talk from his IPv4-based router to the IPv6-based Thread network. In the future, when Vit gets an IPv6 address assigned to his home network from his ISP, no conversion will be required.

OpenThread software on a Linux device like a Raspberry Pi also has a web interface for managing connected devices. This is a useful way to visualize your network and the location of the Border Router within that network.

Using the Golioth interface in the Zephyr SDK

Once your Border Router has been set up, you can use the Golioth Zephyr SDK to compile an image for your device using OpenThread. In the video above, Vit is compiling for an nRF52840 (dongle). Once this device is loaded with Zephyr-based firmware and added to the mesh network–in this example, connecting directly to the Border Router–the device acts like any other IP connected device. It has an IPv6 address and can ping the address of the Border Router. It can also connect to various Golioth endpoints.

Vit demonstrates button presses and boot commands being logged in Zephyr, which then display over the UART. However, Zephyr logs are also tied to the Golioth logging end point (from the Thread network connection), so they can be viewed on the Golioth web console. As Vit presses a button on the Zephyr based device, the log message almost instantaneously shows up on the cloud.

Extending the examples

Towards the end of the video, Chris and Vit discuss how (Open)Thread-based devices built with Zephyr work very similarly to any other IP-based devices connecting to Golioth. This means that the other features that Golioth provides–such as control using LightDB State, or easy time series data tracking using LightDB Stream–is already in there. And yes, that means Firmware Updates as well. We’ll be showcasing these more in future videos.

We’re excited to bring Golioth and internet connectivity to even more devices, including the kind of teeny tiny low power ones that Thread enables. Please stop by the Golioth Discord or the Golioth Forums to ask questions and discuss the next device you’re working on!

Interested in Thread in a commercial application? Contact us at

 

Implementing remote firmware updates is one of the most critical steps towards building a resilient IoT deployment. In short: it allows you to change your device firmware in the field without being physically present with the device. This can be critical for security fixes, for feature upgrades, and for extending the lifetime of your IoT implementation. Today we’re going to be looking at Over-The-Air (OTA) firmware updates (DFU) for the Nordic Semiconductor nRF9160 cellular module (SIP). The associated video explanation and walkthrough (below) was published on our YouTube channel a couple months ago, but we’re reviewing it on our blog today.

Why are IoT firmware updates difficult?

There are many difficult components to OTA DFU, including on the system design level, at the device level, and on the hosting service. At a system level view, the hard part of a firmware update is knowing which device should be receiving the update. From the device perspective, it’s critical to be able to recover from any issues during the update and then verify the complete update was received; using cellular as the method of connectivity means there may be intermittent connectivity. With embedded devices, there are fewer ways to have a device intelligently recover, as may be the case with a full Linux system.  On the cloud side, having a reliable hosting and delivery system for firmware packages is also important. Simply sticking files on a FTP server represents a security risk and puts an outsized burden on the logic of the device in the field.

Firmware updates are easier with Golioth

Golioth provides a data pipe to cellular devices and other devices in the Zephyr ecosystem as a one stop shop for all data connectivity. This includes state data handling (digital twin), data streams coming back to the cloud, and logging.  Once that data pipe is established, we can also push firmware updates down to devices utilizing transport layers like CoAP.

We also benefit from the fact that the Zephyr project utilizes MCUBoot as the default bootloader. This open source bootloader (to which we are contributing members!) makes it easy to interact with from within an application. It’s as simple as pushing an updated firmware binary into memory and then calling an API to use that code upon reboot. Our recent blog posts around the ESP32 showcased how the newly-published port of MCUboot to the ESP32 platform enabled DFU on those parts. The Nordic Semiconductor nRF9160 has had this capability from the beginning because of its deep integration in the Zephyr ecosystem.

A key point shown in the video below is the signing of images using MCUboot.  Having a signed firmware binary means there is a way to check that the entire package is there. From a security perspective, the signing allows the device to validate that the firmware image that has arrive is secure and unchanged in transit.

Managing firmware rollout

The Golioth Console enables our users to manage a wide variety of devices in their IoT fleet. A deployment with 10,000+ devices would be difficult to manage using scripts alone. Withouth the ability to visualize which devices have been updated or not means there is higher likelihood of costly mistakes.

To aid in partitioning device deployments, users can tag devices to create logical representations of different groupings. You might use a tag to identify some devices as early recipients of a test firmware image. Blueprints can be used to group devices that match a certain hardware profile. In this way, Golioth users can target key portions of their device fleet to ensure that only those specific devices are being updated at any given time.

Artifacts and Releases

Golioth implements two distinct features that will allow for more complex firmware packages. The first is an Artifact. this is simply a binary that is uploaded to the Golioth cloud. For your security, Golioth has no knowledge of what is inside that image, only how it has been signed. As such, this might be a firmware image, a JPEG that will be displayed on a smart screen. An artifact can also be a binary blob being delivered to a device modem, or even firmware that’s being pushed down the line to other microcontrollers in the system. Each of these artifacts can be versioned and assigned a blueprint, so that they can only be deployed onto the eligible systems.

The second feature is the idea of a Release. Users can take individual Artifacts and bundle them together as a Release. For example, a Release might have an image to display on the screen, the base application firmware for the device we’re targeting (the nRF9160), and a firmware image for an auxiliary processor on board (e.g., an nRF52). Once the user is ready to deploy, they create a Release with matching tags and blueprints and an assigned Release version. Devices in the field that match the criteria set by the user on the console (tags, blueprints) will then be notified that a new device firmware release is available for them. The device in the field will start the firmware update process, downloading chunks of data.

Once the image is verified as complete, the device will utilize the MCUBoot API on the device and restart using that new image. In the event the Golioth user wants to roll back changes, they simply slide a switch on the Golioth Console and the device is alerted to download/restart the previous version of firmware. In this way, Golioth users have the ultimate flexibility for delivering firmware devices to a laser focused set of devices in their fleet.

Watch the demo

See this all happening in real time, including compiling a new firmware image, uploading it to the Golioth Console and watching the OTA DFU happen on the nRF9160.

Scaling makes things more difficult

As your deployments grow, things start to get messy.

At 100, or maybe even just below 1000 devices…you can do it however you’d like. You might have a spreadsheet listing the location of devices and all of the relevant information about them. But even then, it is a full-time job to manage devices. What’s more, as new people come in to take action on particular devices, they need to slice and dice the data about where the device is located, its status, the capabilities that the device has.  According to Satistia in 2021, there are more than 13.8 billion active IoT devices. It’s estimated that the number of active IoT devices will surpass 30 billion by the end of 2025.

A good organizational system will reduce mistakes

Planning and organizing will help you get your work done accurately while avoiding costly mistakes. Managing this huge number of devices requires a good organizational system. Golioth provides a simple solution to manage devices: tags.

Tags provide a flexible, flat hierarchical structure in which you can group multiple devices. We are going to explore the usefulness of tagging in the Golioth platform.

Why use tags?

A tag is a simple text, generally, one to three words, that provides details about a device. The tag is assigned to a unique device on the Golioth network. Later, when searching, it is a simple click or search term in order to locate similar devices with the same tag.

You can easily separate your devices with any criteria you choose. For example, a separation between development devices and production devices. This can be useful when you want to change configurations for a specific set of devices. By using tags, you can group these devices and push the needed configuration using something like the OTA DFU on the nRF91.

You can also group different devices depending on your deployments or regions. Maybe you’re building an asset tracker application and you want to separate your devices that are on the east coast of the US vs the west coast. Or maybe you want to only update devices that are in a particular time zone. Or even separate when you push an update based on timezone, preferring to push firmware when no one is actively using the device. With tags, it’s possible to target firmware update deployment where and when you want it.

Flat structure

Right now, Golioth tags are a flat structure. You can assign multiple tags to a device without the concept of hierarchy. Even with this flat structure, it’s possible to do some powerful things like we just described. For example, using tags with LightDB allows us to pull or send data to specifically tagged devices. If you want to toggle an LED on a group of devices, you can send the new state to that group by sending the command to the associated tag, as shown in the video below. This is practical for test purposes. You can manipulate a device’s tags graphically on the console, or you can use the Golioth Command Lines Interface (CLI) to remove, add, or modify certain tags. The CLI in particular can be really useful as you start to have a deployment that’s much bigger (hundreds, thousands, millions).

When you’re ready to retrieve data from groups of devices, you can also query data and even do aggregations using tags. For example, if you want to see the average temperature of devices located on the 3rd floor in a factory, you could filter the logs for “factory”, “3rd floor”, “temperature” and then aggregate the data. Extending this idea, pulling data from tagged devices in a region could even lead to a sort of ad-hoc mapping.

Watch the demo

 

On the scale of “none at all” to “X.509 certificates“, there is a wide range of security implementations possible. In this article, we discuss how a Pre-Shared Key (PSK) scheme is “secure enough” to get your prototype off the ground and why your first prototype on Golioth will be “secure by default”.

This article isn’t about production-level security

Getting devices connected to the internet is non-trivial. Security protocols and methods exist to keep unwanted devices off of networks, and put processes in place that require devices to verify that they are allowed onto a network. In bootstrapping a device to securely connect to the internet, many people choose plaintext connections instead of taking the time and headache of utilizing good security practices.

At the other end of the spectrum, we have strict security practices and the engineers charged with maintaining networks. Devices that seek to join those networks must implement the layers of encryption required to certify that a device is on a network. Security professionals stake their careers on solid methods and highly complex processes.

We are taking a middle ground.

Enter: the password

First, let’s define what a Pre-Shared Key (PSK) system looks like. With PSK, your device and your backend will share the same secret. If you ever used passwords, you are likely familiar with the concept. You can think of PSK as passwords not for human users, but for devices. Imagine you are a user of a service, to which you log in with a password and a username. When you signed up for your account, you told the service (pre-shared) the username and password you want to use for future log-ins. PSK is essentially the same if you substitute a user for a device. Each device is assigned (pre-shared) a key (“password”) and a PSK-ID (“username”) by the backend, which it then can use to authenticate itself to the backend.

What about the security of PSK usage at scale? To draw a security analogy, let’s start with considering how WiFi PSK works, and how much you trust it. In a PSK WiFi network, you configure a “password” on the WiFi access point, and then give the same password to anyone who you want to let connect to that network. Most home WiFi networks use PSK authentication, even though you might not call it PSK, but a “WiFi password”. Many corporate visitor networks work the same. TLS PSK in IoT is more robust. For authorization purposes, giving the same password (“PSK key”) wouldn’t help much. It would merely tell the device is authorized to use the service, but it would not prove anything about its identity. That’s why when using TLS PSK in IoT, you shouldn’t give the same password to multiple devices. Every device should have its own “password” (and “username” – a PSK identity).

By assigning a unique PSK to each device, compromising the password of one device will not compromise other devices. In this aspect, TLS PSK is more secure than your average WiFi connection. In many smart home deployments, the same WiFi password is shared by all devices, mostly because of its convenience. However, that also means that if any of those devices get compromised, your WiFi network is no longer secure. Anyone with that WiFi password can connect new devices to it.

PSK is good enough for prototypes

PSK has its use case, and it’s convenience motivated. You likely don’t want to be bothered with procuring a Hardware Security Module (HSM) and setting up a secure process to generate and distribute certificates. You likely need to get some level of security as quickly as possible. That’s the “good enough”. Certificates are going to give you great security, but are going to make everything way more complex. This is where PSK is a great candidate.

Adding a layer: PSK key rotation

To make it more difficult for a potential attacker to compromise your device, it is worth changing (rotating) your device’s keys once in a while. The process doesn’t have to be complicated, but it could be if you don’t design for it.

Start by building your device with two sets of keys. One main key (that is actively being used), and one backup key (only used under special conditions). That way, if you need to update your key, you can keep your existing main key in place and only update the backup key. After the update, if you have experienced a successful authentication for that device, all you need to do is swap the roles of the keys, and you’re good to go. On an unsuccessful start, just keep things as they are and try the key update process again.

When you head towards production

So if PSK is so great, why don’t we just drop certificates and use PSK instead? From the single-device perspective, it might seem like a good idea. But once you start thinking about hundreds, thousands, or millions of devices, things become more complicated and more expensive.

If you are preparing to manufacture millions of devices securely, you are likely considering many aspects of not only technical / hardware / software security, but also physical security of your operations. Who can build authentic devices for me? Who do I trust to build authentic devices?

With PSK, the naive approach could be just setting up a manufacturing station that generates random keys and upon producing a device with a specific key, it also registers the key “in the cloud”. This will make your production depend on internet availability, and availability of the servers that register the keys. Outage of any of these will bring your production to halt.

You might work around this problem by having a local database of keys on the manufacturing station, but that leaves you with a synchronization challenge, and you’d have to rely on a local storage of the manufacturing station to persist the information. Going down this path, it’s likely you’ll find more new problems than solutions.

Ultimately, PSK is not a great solution for manufacturing at scale. It might work well for small batches in your PoC, but make sure to consider how it will impact your product at scale.

“Secure by default” with Golioth

At Golioth, we want to make sure your devices can seamlessly go through all phases of an IoT project – from a concept, through a prototype, to production at scale. We want to set you and your project up for success from day one.

That’s why we decided to make TLS PSK authentication required for all devices connecting to Golioth. Using PSK during development requires very little overhead, and meets convenience requirements for fast iteration, fast onboarding, and simple debugging. Your prototype will have security that is based on battle-proven Internet standards and meets compliance and certification requirements of most IoT use-cases. You can assign multiple keys to any one device to try out PSK key rotation. Your team and end customers won’t need to worry about the prototype being safe. It will also pave the way for easy migration to more advanced security once you are ready to produce devices at scale.

Check out the video below to see PSK assignment in action and to hear more about PSK use at Golioth.

 

Troubleshooting high complexity systems like Zephyr requires more thorough tools. Menuconfig allows users to see the layers of their system and adjust settings without requiring a complete system recompilation.

The troubleshoot loop

Modify, compile, test.

Modify, compile, test.

Modify, compile, test.

Modify, compile, test.

How do we break out of this loop of trying to change different settings in a program, recompiling the entire thing, and then waiting for a build to finish? Sure, there are some tools to modify things if you’re step debugging, such as changing parameters in memory. But you can’t go and allocate new memory after compiling normally. So what happens when you need to change things? You find the #define in the code, change the parameter, and recompile. What a slow process!

Moving up the complexity stack

We move up the “complexity stack” from a bare-metal device to running a Real Time Operating System (RTOS) in order to get access to higher level functions. Not only does this allow us to abstract things like network interfaces and target different types of hardware, but it also allows us to add layers of software that would be untenable when running bare-metal firmware. The downside, of course, is that it’s more complex.

When you’re trying to figure out what is going wrong in a complex system like Zephyr, it can mean chasing problems through many layers of functions and threads. It’s hard to keep track of where things are and what is “in charge” when it comes time to change things.

Enter Menuconfig

Menuconfig is a tool borrowed from Linux development that works in a similar way: a high complexity system that needs some level of organization. Obviously, in full Linux systems, the complexity often will be even higher than in an RTOS. In the video below, Marcin shows how he uses Menuconfig to turn features on and off during debugging, including with the Golioth “hello” example. As recommended in the video, new Zephyr users can also utilize Menuconfig to explore the system and which characteristics are enabled and available.

 

 

Every IoT project needs to provision devices that are going to be available in the field. Leveraging open standards, Golioth cuts down on the required time and hassle for IoT development teams.

Provisioning is a critical step in IoT projects when they go to production. Unfortunately, this process remains a mystery for many engineers due to lack of information about the process. At a high level, provisioning is passing configurations and credentials to an IoT device so it can connect securely to the cloud. Once provisioned, the device can send telemetry, receive commands, or be updated (by OTA DFU) when it’s out in the field. How you provision a device depends a lot on the use case. 

(click the image above to see the full diagram)

Example use cases

First, let’s examine a customer-facing product like a smart light bulb. In this scenario, the first step would be for the user to provide WiFi credentials to connect to the user’s home network. On the platform side, the device would obtain a new set of credentials to connect to the backend services. These credentials would be specific to that particular user and device. Later, the user might decide to clean up the device to sell it, so the ability to remove device configurations and deleting a given set of credentials is important. This is a perfect example for using BLE provisioning like shown in the video below.  The user experience is seamless with any existing mobile app used for controlling the bulb and reporting data back from the end device.

Next, we’ll consider factory-level provisioning. An example device like a cellular asset tracker would be pre-provisioned at the factory before being used by your customer. Later the user will only associate that device with their account, but the credentials to talk to the cloud are already set on the device. This can be done as part of the manufacturing process, probing the device via Serial/UART to get the device hardware ID, provisioning it to the cloud, and sending credentials back to the device via the same transport. We can even have different firmware that will only provision in the factory. The device accepts the initial device configuration and saves the credentials to flash. Subsequent firmware that doesn’t have that initial feature enabled, making sure external parties can’t change or reverse engineer the initial configuration.

There are myriad ways that provisioning can be done. Each instance will depend on the factory environment, the capabilities of the user, and on the end application. The video below is a setup similar to the first example explained above, using a Bluetooth application to read and then program the end device, all while working with the Golioth cloud.

Our demo application

As you can see in the video, we developed an end-to-end sample that shows a practical scenario of provisioning IoT devices with a native mobile app, talking with an IoT device over Bluetooth, and provisioning device/credentials in Golioth Cloud. We leverage different tools for doing so:

  • MCUmgr as the device management subsystem and protocol.
  • Zephyr as the real-time operating system, that implements MCUmgr.
  • Open-source mobile SDK to integrate MCUmgr on an app
  • Golioth’s API and the Device/Credentials Management capabilities. 

The MCUmgr community developed multiple types of transports to interact with devices, a benefit of MCUmgr being an open standard and having a vibrant community. One option is to communicate with the device over serial UART using the `mcumgr` cli or even integrate that into your own set of provisioning tools. Another option is to use a mobile SDK that implements MCUmgr protocols over BLE to talk with devices.

We took the Bluetooth approach and forked Nordic’s MCmgr Example application, adding communication with Golioth APIs to manage devices. Once we discover the name of the device, we assign credentials via the REST API and securely send them over Bluetooth to the end device. The device is running one of Golioth’s samples that accepts dynamic configuration for WiFi and DTLS Pre Shared Keys to talk securely with our cloud. The device uses a different Golioth service called LightDB. Using this configuration engine, we can publish the on/off state of the light bulb using LightDB,show that data on a UI, and even send commands to change the state on the device. 

Source code for the mobile app:

More details on how to use our REST API and how to generate API Keys can be checked on our docs website.

References

A possible solution

Let’s pretend you’re in the middle of a global chip shortage.

Surprise! There’s no need to pretend, as we are all currently in the middle of a global chip shortage. Right now it’s very difficult to source certain components.

“Why don’t hardware makers just switch out the components when they can’t source them during the chip shortage? In fact, why don’t people switch chips on a regular basis?”

As a generalization, switching costs for embedded devices are very high. If we were able to magically solve all of the switching costs for the hardware, you’d still need to deal with the switching costs of firmware and cloud platforms. This often is even more dire than than the hardware switching costs. It’s significant at both an individual level (rewriting firmware to target different architectures and board setups) and at an institutional level (maintaining different platforms and interoperability).

Operating systems and Real Time Operating Systems (RTOS) help by abstracting away a lot of the individual hardware details. When a new device is added to an RTOS, it needs to fit within the constraints of the system. If your board has an i2c sensor on it, you need to ensure your supporting firmware for that board or chipset capable of working with the elements of the RTOS. Then you can take advantage of the drivers already written for other boards/chipsets on the platform. Assuming you are willing to work within that system, you can start to supercharge your development. It’s possible to switch out components quickly and confidently, helping to alleviate the woes of the chip shortage currently underway.

Making the switch

Let’s say you have a board using an ESP32 module. Due to sourcing problems, you can no longer source a particular LED on your board, but you don’t want to change your PCB. Instead, you ask a technician wire in an extra LED to a spare pin you have on your production board that has a larger landing area. You need to build a firmware image to drive a different pin on the microcontroller than you previously were using. Now “LED2” (as it’s called in your program) is not driving pin 18, but is instead driving pin 22. With Zephyr overlays, the switch will take 5 minutes. As Marcin shows in the video below, the device tree overlay is where we map the signals internal to the firmware to the physical pins being used.

Now let’s say you cannot source the ESP32 at all, for some reason. You could create a new overlay file for a different target that works with Zephyr, assign the pins to target the functions you need on a new PCB containing a different chip, and then target that device. The time consuming aspect would be checking all of the functions are performing the same as your previous platform. But once you have decided on a new platform, assigning pins and functions to your new device would occur through overlay files.

How to use Zephyr Overlays

In the video below, we  walk through the location and function of overlays in Zephyr. Marcin explains that customization of firmware images for particular hardware targets can be as simple as a different flag on the command line. In this particular example, we are showing how to change the pins for the ESP32 demo. Previously the ESP32 overlay was shown as part of our our LightDB Sample code (docs), which targeted an ESP32-DevKitC in that video.

About The Zephyr Project

The Zephyr Project is a popular and open source Real Time Operating System (RTOS) that enables complex features and easy connectivity to embedded devices.  The project is focused on vendor participation, long-term support, and in-depth security development life cycles for products.

About Golioth

Golioth helps users to speed up development and increase the chances that pilots will be put into production with a commercial IoT development platform built for scale. We offer standardized interfaces for connecting embedded devices to the cloud and build out software ecosystems that allow your projects to get to market faster. Golioth uses Zephyr as part of the Golioth SDK to bootstrap application examples and show how to utilize the range of networking features Golioth enables via APIs.

“I’m sorry boss, I am working as fast as I can here. I reprogrammed about 36 out of the total 50 units, but this is slow going. I only have one programming cable and I need to disassemble the deployed units so I can get to the header on the boards first.”

A bad firmware image on your deployed IoT devices can mean ruined weekends, upset customers, and lost time. Many businesses pursue a network based firmware update so that they can push new versions to their devices. In fact, this is a critical part of the firmware development process, often a very early one. Developing or implementing a bootloader allows engineers to ship new control software to their devices. A straw poll on Twitter showed that some engineers spend a significant amount of time putting this tooling in place.

While the “barely any time” group seems large, it also includes those who aren’t doing a custom bootloader, nor a bootloader that is networked:

In the past, networked firmware updates took a significant amount of planning and coordination between hardware, firmware, software, and web teams. Golioth has collapsed this down to a simple process.

Update all the devices in your fleet with the click of a button

Golioth Device Firmware Update (DFU) is possible because the Golioth SDK is built on top of the Zephyr Project. Part of that implementation includes MCUboot, an open source bootloader. Using open source software up and down the stack, Golioth enables quick, secure deployment of firmware packages to IoT devices throughout the world. The Golioth Console enables easy management of firmware releases, including multi-part binary bundles, enabling updates for devices as diverse as smart speakers, digital signage, machine learning enabled sensor systems, multiple processor embedded devices, and more.

In the video linked below, Lead Engineer Alvaro Viebrantz demonstrates with Chris Gammell how to update the firmware of an nRF52 based device over Ethernet. The video includes code snippets in Zephyr and walking through the build process using the command line tool West. Once the firmware image is built, Alvaro showcases how to push the image to the Golioth cloud, package it for delivery, and then deploying to Golioth enabled devices.

No more fussing with programming cables out in the field, Golioth allows engineers to update their devices with new features, requested fixes, and efficiency improvements. Try it out today!

About Golioth

Golioth is a cloud company dedicated to making it easier to prototype, deploy, and manage IoT systems. Learn more by joining the Golioth Beta program and reading through Golioth Documentation.