Golioth’s web console has a super easy and intuitive way to send Over-The-Air (OTA) updates to devices in the field. But for ultimate flexibility, you can choose to use the Golioth REST API for managing OTA firmware updates.

Every IoT device should have the ability to update firmware remotely. OTA is a crucial feature for fixing bugs and keeping fleet security up-to-date. But the device itself is only one half of the equation, you also need a management tool to get the new firmware to the device. Golioth is that management tool, and every one of our features are available using our REST API. Today we’ll take a look at how firmware update artifacts and releases work, using Python for the demonstration.

Overview of OTA from the Golioth Console

Before jumping into the REST API, let’s take a quick look at the Golioth Console OTA flow to help better understand the abstract concepts. There are two parts to rolling out an OTA update: artifacts and releases.

Golioth OTA artifact

The artifacts screen shown here facilitates the upload of the firmware binary that will be served to the devices in your fleet. During the upload process you assign a package name (usually main) and version number (e.g. 1.0.1), with an optional setting for Blueprint (a way of grouping device types on Golioth)

Golioth OTA release

Creating a release is the second part of creating an OTA update. You can select Blueprint and Device Tags to narrow down which devices are being targeted. The release is associated with an Artifact (uploaded in the previous step). Optional “release flags” are a way to associate some additional information about the release.

Releases include a rollout toggle button. This is a way to stage a release and devices will only be notified when the rollout toggle is turned on. If you ever need to go back to previous firmware version, this button serves as a one-click rollback!

Creating OTA Artifacts and Releases with the Golioth REST API

Now that we’ve reviewed Artifacts and Releases, let’s use the REST API to create some!

The Open API page from the Golioth Doc site

First off, the Open API section of our Docs is the best place to test out your REST API calls. Just click the Authorize button at the top and give it an API key from your Golioth Project. From there you can try out the commands live on that page.

Python API calls

import requests
import base64

For this demo I’ll use Python 3.10.6 to make the API calls. This is pretty easy to do using the requests and base64 packages.

Uploading an Artifact

def upload_artifact(api_key, proj_id, version, package, filename):
    artifact_upload_url = "https://api.golioth.io/v1/artifacts"
    headers = {'x-api-key': api_key}
    with open(filename, 'rb') as f:

        data = {
            "projectId": proj_id,
            "content": base64.standard_b64encode(f.read()).decode(),
            "version": version,
            #"blueprintId": "string",
            "package": package
            }

    jsonData = requests.post(artifact_upload_url, json=data, headers=headers).json()
    if 'data' in jsonData:
        if jsonData['data']['version'] == version:
            print("Artifact {}-{} created successfully for project {}".format(package, version, proj_id))
    elif 'code' in jsonData:
        print("Error code {}: {}".format(jsonData['code'], jsonData['message']))
    return jsonData

The artifact upload requires the API key and project name. The package name is usually main and semantic versioning is used for the version number. I have not assigned a Blueprint in this example but I did leave a commented line if you choose to do so.

>>> result = upload_artifact("V010sGuWtXXM1htPCHBjLGfrW6GlsKDt", "developer-training", "1.2.3", "main", "new_firmware.bin")
Artifact main-1.2.3 created successfully for project developer-training
>>> import json
>>> print(json.dumps(result, indent=4))
{
    "data": {
        "id": "63ceb040c345ce2e0256ac30",
        "version": "1.2.3",
        "package": "main",
        "createdAt": "2023-01-23T16:05:20.235Z",
        "updatedAt": "2023-01-23T16:05:20.235Z",
        "binaryInfo": {
            "digests": {
                "sha256": {
                    "digest": "0b101d7b0ad1330ec49471d6feb0debc02e022a99e839c7951a446c1539802e6",
                    "size": 32,
                    "type": "sha256"
                }
            },
            "headerSize": 0,
            "imageSize": 1219152,
            "tlvTotalSize": 0,
            "type": "default",
            "version": ""
        },
        "size": "1219152"
    }
}

A JSON packet is returned when the upload is successful. It includes all pertinent information for your newly created artifact. This includes the id of the artifact which is needed to create a release.

Creating a Release

def create_release(api_key, proj_id, artifact_id):
    release_url = "https://api.golioth.io/v1/projects/{}/releases".format(proj_id)
    headers = {'x-api-key': api_key}
    data = {
        #"deviceTagIds": [ tagId ],
        "artifactIds": [ artifact_id ],
        "rollout": True
        }

    jsonData = requests.post(release_url, json=data, headers=headers).json()
    if 'code' in jsonData:
        print("Error code {}: {}".format(jsonData['code'], jsonData['message']))
    return jsonData

The release is created using the API key, project name, and artifact ID. In this example I’ve chosen to rollout the release at the same time as it is created, so the devices will be notified immediately that there is a new OTA firmware version available.

>>> result = create_release("V010sGuWtXXM1htPCHBjLGfrW6GlsKDt", "developer-training", "63ceb31ac345ce2e0256ac31")
>>> import json
>>> print(json.dumps(result, indent=4))
{
    "data": {
        "id": "63ceb3b9c345ce2e0256ac32",
        "createdAt": "2023-01-23T16:20:09.563Z",
        "updatedAt": "2023-01-23T16:20:09.563Z",
        "releaseTags": [],
        "deviceTagIds": [],
        "suitManifest": {
            "authentication-wrapper": [
                {
                    "algorithm-id": "sha256",
                    "digest-bytes": "2a3efb45029dc5d23cf0adf5e260ba53c9a78266b9ee28bdbf4ef20b43a2d6c7"
                }
            ],
            "manifest": {
                "common": {
                    "common-sequence": [
                        {
                            "id": "set-component-index",
                            "value": 0
                        },
                        {
                            "arg": {
                                "class-id": "53f7395e-0825-5970-badb-cc7158e49eaa",
                                "image-digest": {
                                    "algorithm-id": "sha256",
                                    "digest-bytes": "0b101d7b0ad1330ec49471d6feb0debc02e022a99e839c7951a446c1539802e6"
                                },
                                "image-size": 1219152,
                                "vendor-id": "36323535-6565-3863-6430-346536653332"
                            },
                            "id": "override-parameters"
                        },
                        {
                            "id": "vendor-identifier",
                            "value": 15
                        },
                        {
                            "id": "class-identifier",
                            "value": 15
                        }
                    ],
                    "components": [
                        [
                            "[email protected]"
                        ]
                    ]
                },
                "install": [
                    {
                        "id": "set-component-index",
                        "value": 0
                    },
                    {
                        "arg": {
                            "uri": "/.u/c/[email protected]"
                        },
                        "id": "set-parameters"
                    },
                    {
                        "id": "fetch",
                        "value": 2
                    },
                    {
                        "id": "image-match",
                        "value": 15
                    }
                ],
                "manifest-sequence-number": 1674490809,
                "manifest-version": 1,
                "run": [
                    {
                        "id": "set-component-index",
                        "value": 0
                    },
                    {
                        "id": "run",
                        "value": 2
                    }
                ],
                "validate": [
                    {
                        "id": "set-component-index",
                        "value": 0
                    },
                    {
                        "id": "image-match",
                        "value": 15
                    }
                ]
            }
        },
        "artifactIds": [
            "63ceb31ac345ce2e0256ac31"
        ],
        "rollout": true,
        "sequenceNumber": "1674490809563381303"
    }
}

Upon success, all details of the release are returned by the REST API.

Get the Artifact ID (or Release ID)

def get_artifact_id(api_key, proj_id, version, package):
    artifact_url = "https://api.golioth.io/v1/projects/{}/artifacts".format(proj_id)
    headers = {'x-api-key': api_key}
    
    jsonData = requests.get(artifact_url, headers=headers).json()

    for i in jsonData['list']:
        a_ver = i['version']
        a_package = i['package']
        if version == a_ver and package == a_package:
            return i['id']
    return None

The artifact ID is returned when uploading a new binary. However, if you need to get it after the fact there is a REST API call for that as well.

>>> result = get_artifact_id("V010sGuWtXXM1htPCHBjLGfrW6GlsKDt", "developer-training", "1.2.3", "main")
>>> print(result)
63ceb31ac345ce2e0256ac31

Artifacts are identified by the version number and package name, but it’s the ID that is needed when creating a release using the REST API. The same approach is used to query the Release ID, just change the URL for the API call:

release_url = "https://api.golioth.io/v1/projects/{}/releases".format(proj_id)

Rollout, rollback

def release_set_rollout(api_key, proj_id, release_id, rollout_state):
    release_url = "https://api.golioth.io/v1/projects/{}/releases".format(proj_id)
    rollout_url = release_url + "/" + release_id
    headers = {'x-api-key': api_key}
    data = { "rollout": rollout_state }

    jsonData = requests.patch(rollout_url, json=data, headers=headers).json()

    return jsonData

Finally, you can rollout and rollback releases using the Release ID.

>>> result = release_set_rollout("V010sGuWtXXM1htPCHBjLGfrW6GlsKDt", "developer-training", "63ceb3b9c345ce2e0256ac32", False)
>>> print(result['data']['rollout'])
False

When rollout change is successful, you will receive the complete release manifest. Here I’ve printed just the state of the rollout.

IoT any way you need it

We’ve gone to great lengths to make it easy to build your IoT fleet. For most users this means a straightforward GUI experience on the web console. But every one of those features are available programmatically for those who need it. Today we showed OTA functionality, but you can just as easily create a custom UI for your fleet.

We’d love to hear about how are you using the REST API. Start a thread in the Golioth Forum to share your story!

Once your first device is online and talking back to the Cloud, your problems immediately start to multiply. As you go from one to ten to one hundred devices, you want to understand some basic information about all the devices in your fleet. This week we showcased how to visualize this data by creating a “Fleet View” in Grafana. Let’s look an how you can build one of your own.

Moving from a Device View to a Fleet View

The example in the video uses the devices in a project related to the Golioth IoT Trashcan Reference Design. In that setup, we have a single device view so that we can visualize the depth of data coming off of the device:

Fig 1. “Device View” Dashboard

In the fleet view we take that individual device data and stretch it horizontally across the page for each individual device. From there, replicate each of the chosen boxes to get the same view for every device in a fleet, in this case it’s 6 devices. The resulting dashboard looks like this:

Fig 2. “Fleet View” Dashboard

The ability to replicate the setup across all the device in a fleet is built upon two elements of Grafana: creating a variable and using the ‘repeat’ option for each visualization element.

Creating a variable

We have posted about creating Grafana dashboards in the past. The first thing you need to do is set up a “data source” to configure Grafana to talk to the Golioth REST API. This is how you will extract the data that your devices are sending back to the Golioth Cloud. Each device has a unique ID that Golioth uses to coordinate data moving through the Golioth platform.

When we want to visualize more than one device in our system, we need to make it possible for Grafana to dynamically switch between the different Golioth Unique Identifiers for all the devices on your project. The first step is to create a variable in Grafana that represents all of the devices.

Click on the gear icon to access the “Dashboard Settings Menu” on the upper right of your dashboard (hit ‘esc’ if you don’t see anything on the upper right of your dashboard, your menu may be hidden).

This will take you to the following screen, click on Point A to get into the variables menu and then click ‘+New Variable’ or click on the variable you want to modify at Point B.

Fig 3. Dashboard Settings Menu

In the variable creation/editing menu, you can set a name (Point 2 in the image below). You will need to select your JSON API data source that connects you to the Golioth REST API (Point 3).

Under the ‘Path’ menu, you will need to do a GET request at the /devices/ end point (not shown). Then replicate the variables shown (Point 4) to pull back data from your project that includes an array of all device IDs in a project ($..id / value) and the associated names of all devices in your project ($..name / text).

Finally you will want to toggle Multi-Select and Include All Option (Point 5). This will give your dashboard a pulldown menu, as is showcased in the video above.

 

Fig 4. Variable creation menu

Finally, you need to set these newly created fields in the Experimental tab (Point 6) and set the “Variable text” and “Variable value” (Point 7).

Fig. 5 Variable Experimental Tab

You now have a variable you can work with in your project.

Creating a repeating element on your Fleet Dashboard

Each individual box shown in Fig. 2 is a Grafana “panel”, which means you can connect it to a data source and do some kind of visualization with it. Normally this is a chart, a dial, a displayed value, or an icon/graphic you assign to a state.

When you edit one of these panels, you need to change two things to enable “fleet view”. The first is to change the target on the Path tab (Point C) to point to the /devices/${device}/stream endpoint on the REST API. This will automatically change which set of device data we’re looking at when we change the pulldown in the upper left corner. As shown, it is looking up the Golioth Device ID for “Trashcan A5” and inserting that into the path in place of ${device}. See below for the code I have in the “Body” field that you can use as a model when doing a POST request to the REST API.

Next we want to modify the “Repeat options” on the right sidebar (Point D). I selected that we repeat based on all of the devices we have selected. I also selected that the “Repeat direction” is vertical so it stacks up the repeated panels on top of one another to get a view like in Fig. 2. See this in action with the associated YouTube video.

Fig. 6 Edit Panel View

{
    "start" : "${__from:date}",
    "end" : "${__to:date}",
	"perPage" : 999,
	"query": {
		"fields" : [
			{ "path": "time" },
			{ "path": "deviceId"},
			{ "path": "*"}
		],
		"timeBucket" : "5s"
	}
}

Build your next fleet with Golioth

In addition to the flexibility that a Grafana dashboard can provide, the Golioth Console acts as a more text-focused fleet visualization and control plane for your devices out in the field. We think you can understand a good amount about your devices simply from glancing at your project Console and seeing the state of each device.

We always want to hear about how we can improve the experience for our users. Please try out the Console with our free Dev tier and then stop by our Forum to discuss your fleet deployment!

DevCon22 was a 2 day event that Espressif held last month to showcase upcoming products from Espressif as well as external partner offerings. Mike and I (Chris), gave a talk about how  the Espressif IoT Development Framework (ESP-IDF) makes it really easy to add Golioth to your IoT projects.

The ESP32 line of Wi-Fi based devices deliver the internet connection, while the Golioth ESP-IDF SDK provides access to things like:

Golioth covers your hardware

Golioth supports multiple hardware ecosystems, and we expect to enable even more in the future! We’ve built everything firmware engineers need to seamlessly connect their devices to the Golioth Cloud. Our goal is to meet engineers where they work, and this often means working directly inside the ecosystems their projects are built upon.

We were excited to support the ESP-IDF, which is based off the popular FreeRTOS core. This open source Real Time Operating System (RTOS) is prevalent throughout the electronics industry, with ports into many different chip ecosystems. The Espressif team has taken the core capabilities and ported it to work across their growing catalog of components. They have also bundled a range of meta tools that includes the build system (idf.py), the programmer (esptool.py), and more.

The Golioth ESP-IDF SDK leverages the necessary components to immediately access the Golioth Cloud on Espressif devices. We use CoAP as a transport layer, so we include a CoAP library to allow your device to “talk” CoAP. Golioth is secure by default, so we also include an mbedtls library, to allow your packets to be encrypted (using your PSK-ID and PSK). The net result is that firmware and hardware engineers don’t need to worry about the low level details of connecting to our servers, instead you can utilize high level APIs shown in the presentation video. For anyone using ESP32, Golioth just works!

The challenges of scale

The core of the talk is to showcase Cloud technologies that make it easier to scale your fleet. If you’re the classic example of a startup in a garage, you probably aren’t planning to scale out to millions of devices. Even if the future is uncertain, it’s good to understand the capabilities that will make it easier to control your fleet, from 10 devices to 10 million devices.

We also want to make sure that firmware and hardware engineers don’t need to become Cloud engineers to enable these capabilities. The features described above and in the video are available by default, without any additional Cloud-side code required. This means engineers can start sending data to the Cloud immediately and focus on the higher value tasks of pushing the data to those that need it, like their internal software or data science teams.

Mike shows many of these services in action during the talk, including showing how users can interact with the data that is sent to the Cloud using the ESP-IDF SDK. Each of these features enables users to grow their Cloud capabilities immediately:

Are you ready to scale?

Golioth is ready to take your fleet to the next level, using the ESP-IDF SDK or any of our other supported hardware ecosystems. You can immediately pull the SDK into your next project and start sending data to the Cloud. Reach out to us on our Forums or the Golioth Discord to discuss how we can help you to build out your fleet.

Slides

If you’d like to preview the slides used the video presentation, check them out below. If you’d like a PDF copy of the slides, please contact [email protected].

What is Golioth LightDB State

The Internet of Things is all about transferring data between devices and the cloud. Of course not all data is equal, but with just a few core services you can cover all the bases. One example is the need for persistent data that is easily accessible.

With the Golioth LightDB State service, data can be stored and retrieved from the both the Device and the Cloud side. This is useful for reporting status information, restoring configurations after a power cycle/network outage, and indicating desired changes.

How Golioth enables persistent data

Recording sensor readings might be the first use that comes to mind for IoT devices (our LightDB Stream service is built for that!) but many times we also need stateful data.

State data has no concept of time but instead answers the question: “what was the last status we received?” It acts as a single point of truth that’s quick and easy to access from both the device side and the cloud side. Every time you request a state endpoint you can be sure you’re getting the most recently updated value.

Devices in the field can use state data to report their actual state, things like:

  • What is the timestamp of the last reboot?
  • How full is that SD card used for offline storage
  • What units has the user set for sensor data?

From the Cloud side, state data may be used for command and control. By setting a “desired state” in the Golioth LightDB State, devices with infrequent connects will receive their marching orders the next time they reconnect. Of course this use-case is also solved by the Golioth Device Settings Service which allows project-wide settings adjustments.

LightDB State data can be as simple as a single key/value pair, or as complex as a nested data structure. Devices registered to observe values will be notified every time there is new state data available.

LightDB State using Zephyr RTOS

Your device chooses a path to store and access the LightDB State data. The cloud doesn’t need to be prepared in advance. If your device sends data to a new state endpoint it will automatically be created. There are number of examples of LightDB State API in the Golioth Zephyr SDK.

/* Setting a value on the "counter" endpoint */
err = golioth_lightdb_set_cb(client, "counter",
                 GOLIOTH_CONTENT_FORMAT_APP_JSON,
                 sbuf, strlen(sbuf),
                 counter_set_handler, NULL);

/* Observing data changes on the "desired/my_config" endpoint */
err = golioth_lightdb_observe_cb(client, "desired/my_config",
                 GOLIOTH_CONTENT_FORMAT_APP_JSON,
                 desired_state_handler, NULL);

Here we see asynchronous API calls to set a counter endpoint and to observe data changes on a desired/my_config endpoint. In both cases a “handler” callback function is registered to process the received data (in the case of the observe operation) or handle error messages received from the server.

The Golioth Zephyr SDK also includes the option to send your sensor data using CBOR encoding. This is an alternative to JSON which serializes the data stream to a more compact form to save battery (less radio-on time) and bandwidth (fewer bits being transmitted).

LightDB State using ESP-IDF

The ESP-IDF SDK shares a very similar set of API calls. The device again chooses the endpoint to use on the Golioth LightDB State service. The golioth_basics sample code demonstrates how to use the set/get/observe functions.

/* Setting a value on the "counter" endpoint */
golioth_lightdb_set_int_async(client, "counter", counter, NULL, NULL);

/* Observing data changes on the "desired/my_config" endpoint */
golioth_lightdb_observe_async(client, "desired/my_config", on_my_config, NULL);

Here we see asynchronous API calls used to set the counter endpoint and observe the desired/my_config endpoint.

LightDB State using your own solution

For the extra ambitious out there, you could roll your own device side code that allows you to connect to Golioth. This is the kind of work our wonderful firmware team does whenever they spin up a new SDK. Some of the required elements include:

  • mbedtls in order to create a DTLS connection to the Golioth Cloud
  • A CoAP library
  • (optional) a CBOR library to encode the data

We hardly ever recommend doing this unless you are a very advanced user. Instead, our existing SDKs cover a wide swath of parts, ecosystems, and Real Time Operating Systems. If you are interested in one we don’t already cover, please let us know.

Formatting data and access from the cloud

The Golioth LightDB State service will accept and store any properly-formatted data encoded in either JSON or CBOR. If the endpoint doesn’t yet exist, it will be created when data is received. null will be received when requesting a non-existent endpoint.

As an example, your device might send the following JSON string to LightDB state:

{ "status": { "last_reboot": 1666586559, "location": {"lat":42.988814,"lon":-90.139254,"alt":369.7,"accuracy":2.3,"speed":0.0,"heading":0.0,"time":"2022-10-31T22:49:03.239Z"}, "debug_mode": 0 } }

This imaginary device is recording the last time it rebooted, saving a location from its GPS unit, and reporting on whether or not the debug mode is activated. Whether this data is sent as JSON (which is the case here) or CBOR, the Golioth Console will unpack the information and display it as human-readable JSON:

Golioth LightDB State

What’s really great is that once the data arrives on the Golioth Servers it’s incredibly easy to work with. You can get real-time updates using WebSockets, connect your data to your favorite cloud provider (e.g. AWS, Azure, or GCP) using our Output Streams, and access/update the data using the Golioth REST API.

Start using LightDB State Today!

All of our Device SDKs show how to use LightDB State with minimal code writing from the user. Just pass your state data up to Golioth and you’ll be on your way. Stop by our Forums or send us an email if you get stuck.

How can security improve when manufacturing large volumes of devices? That’s the question I ended on in my last article about Golioth Pre-Shared Keys (PSK).

Securing a large population of devices (10k or more) in a way that’s scalable and meets your project’s budgets is not trivial, but it is solvable. It takes a combination of modern technologies, and even not so modern technologies.

Certificates have been around for decades. Today the most recognizable certificate-based security protocol is TLS, which started all the way back in the mid-90s as SSL. But certificates haven’t found their way to all IoT applications because of their complexity and infrastructure requirements.

Many connected devices work around the complexity by eschewing good security practices:

  • Completely ignoring security (authentication or encryption)
  • Relying on “security by obscurity” (proprietary solutions that are not secure, but would need to be intentionally exploited by an attacker)
  • Use product-wide passwords
  • Are produced in small enough quantities to not feel the pain (and cost) of handling per-device keys.

None of these are ideal at scale.

The Limits of Pre-Shared Key (PSK) Scaling

When using PSK, your device and your cloud need to pre-share a secret. That means you need to store and distribute a sensitive per-device key for every device (or a system to derive the key). For prototyping, this involves copying and pasting a key from the Cloud to your Device over serial or similar. There are challenges getting those keys to your devices on the production line, but the ultimate risk to your deployments are when those devices are out in the field

PSK symmetry

From a security perspective, the limitation of PSK is that it uses symmetric encryption. Both sides of the communication use the same key. That means that a potential attacker could extract the keys from the cloud (in bulk), and use them to impersonate authentic devices.

Instead, we could use an asymmetric key. With asymmetric encryption, there is no pre-shared secret, but rather a pair of keys: a public key, and a private key. In our case, the public key is shared between the cloud and the device. The private key is kept secret by the device, and can be used to decipher information encrypted with the public key. Only the holder of the private key can decipher information that is encrypted by the public key. That’s why it is called asymmetric. You cannot use the public key to decipher the information enciphered by it. The burden is again on the device maker to load the keys onto the constrained devices at the time of manufacture.

If we want to upgrade our experience even more, Certificates are more “usable” primitives that are based on asymmetric cryptography. But they offer even more trust.

Certificate signing

One extra added value of asymmetric cryptography (and public keys) is that the public keys are well suited for chains of signatures. In other words, they can be used to prove the authenticity of a chain of identity claims. A device claiming it has a specific serial number can prove its authenticity, and not just someone who copied the serial number and pretends to be the device.

How is that possible? The signatures can be independently verified by anyone out there using public keys. Even if you don’t have the public key for a specific device, you can decide you trust the manufacturer that produced the device. By extension, you trust that devices that can prove they were produced by that manufacturer.

But that also means that as an author (or producer) of a secure device, you have to choose wisely who you trust.

Chains of trust enable scaling of secure manufacturing

Let’s imagine a scenario where you design a super-secure device, powered by private / public keys in the form of certificates.

Next, you need to find someone to produce those devices, such as a contract manufacturer. Once the devices start to roll off the production line, they need to be loaded with the private-public key pairs. Someone needs to generate those pairs and help to get them on the device in a secure way.

From a simple perspective, you can use certificate chains to enable manufacturers to produce authentic devices on your behalf. When the devices first connect, they will be able to prove to the cloud that they were produced through a trust chain.

The advantage of that is that you no longer need to store all per-device keys or certificates. All you need to decide is what the trusted “root” certificates are for the devices that are authorized to interact with the cloud.

Downsides of certificates

If the security advantages of certificates are so clear, why don’t we just use them everywhere? If you come from the web world, you’ve likely never encountered PSK because certificates are everywhere. Your browser doesn’t ask for the password to access the SSL version of Google.com, they issue a certificate that can be traced back to a signing authority.

In the Internet of Things, not all use-cases have budgets left to make certificates commercially feasible.

Some of the major challenges you will need to consider:

  • You need to decide who you trust, and how much you trust them. If you can’t trust your manufacturing chain, it’s going to be very difficult (costly) to produce trustable devices
  • Secure storage of private keys for the root certificates can be costly, if you want to do it right. When compromised, private keys allow the attacker to produce certificates that are indistinguishable from authentic certificates
  • Setting up the end-to-end process of certificate provisioning can be costly, and the subsequent maintenance and rotation of certificates can carry a lot of cost, too, if not planned properly in advance

But mostly, when IoT projects are in the planning stage, security tends to be overlooked or be taken for granted. Teams scramble to get something in place just before starting a first large batch of devices; it’s often this time that companies discover the full extent of their security needs and the associated costs. At that point, budgets have already been spent and deadlines are tight, and using proper certificate authentication becomes commercially unfeasible.

Certificate authentication for Internet of Things

Certificates offer the ideal combination of security, usability, and scalability. The manufacturing chain is what ultimately defines how trustable your devices actually are, so you will need to choose who you trust wisely. The overhead of building and maintaining certificate chains and large volumes of certificates is not free, but is cheaper than the alternatives, especially if you account for risk.

At Golioth, we only allow secure device connections, at a bare minimum PSK. Our users have a choice in the degree of security that they need for their application, and that is commercially feasible for their use-case. If certificates are your choice, we have the supporting end-to-end tooling and infrastructure.

Ultimately, each company needs to consider the security sensitivity of their IoT application. Your challenge will be balancing the risks you are willing to take and the budget your application has for security overhead. Hardest of all is understanding where you should draw the line. If you’d like to talk to someone about your security needs at your company, please reach out to our DevRel team for a personalized conversation.

Golioth helps you get your data to and from your constrained IoT device in the field. We make it really easy to ship data up to the Cloud where you can do more interesting things than merely interacting with the device out the field.

But what happens when the data gets to the Cloud?

Once your data is in an environment with many more resources, you probably want it to pass between different services. For the average Device creator, we make this easy with our Output Streams feature. Output Streams immediately export data to 3rd party Cloud providers like AWS, Azure, and GCP. When we built that service (and our other “Cloud-to-Cloud” integrations), we wanted to build on top of an open standard that would reduce the friction of getting your data from the Golioth cloud to your own systems. We decided to reuse a specification we adopted on our internal infrastructure called CloudEvents.

What are CloudEvents?

First off, we need to ask “what are events”? It sounds like a generic term but for cloud developers it means something quite specific. “Event Data” vs “Message Data” is a pattern in Cloud-to-Cloud communication.

CloudEvents are a standardized way of formatting event data so that it’s easier for one Cloud provider to understand incoming data from another Cloud provider. The standard includes things like the Type of event, the Source of the event, and the formatting of the event. If you’re trying to figure out where an event came from (ie: change the value of LED from ‘true’ to ‘false’), you’d need to know if that update request came from an app, the Golioth Console, a 3rd party source via the REST API, or some other previously validated entity talking to our Cloud.

The History of CloudEvents

People have been doing event-driven design (architecture) for a long time. But there was no standard, which resulted in a lot of reinventing of “templates” for events. When you encounter an HTTP event of JSON, the payload is not the important part for processing that event. Without a format for producers and consumers to agree on, it means wasted time and lack of interoperability. Cloud engineers and members of the Cloud Native Computing Foundation in the Linux Foundation started CloudEvents to develop a standard.

An Event Driven Architecture

Cloud software has gotten extremely complex. Services stream all sorts of a data, often times between internal and external and external teams. Event-driven designs have been increasingly popular among modern cloud solutions to manage this growing complexity. Think of a large system with lots of micro-services and serverless functions talking to each other. Getting agreement up front on the message design would slow down innovation and be rather difficult. Events allow for the the “componentization” of the these API contracts. It decouples them. And as long as the agreed upon format is written down somewhere, both the servers (producers) and clients (consumers) can move faster and the process of designing very complex systems becomes easier.

Golioth is a modern cloud native software too, and our architecture internally has an event-driven design. We use CloudEvents internally. For example, when a device sends a LightDB Stream value over CoAP, it is processed by our CoAP API Gateway. The gateway then converts that message into a CloudEvent and publishes to a message broker. The broker and that entire part of the cloud ensures that CloudEvent data makes it to our time-series database securely and reliably. That piece in the middle is performant, scalable, and secure thanks to our design that leans on modern event broker software.

If we didn’t use the event pattern, we’d have to develop a bespoke system to handle CoAP messages in the middle of our stack, which would likely be far more difficult to build and more fragile to maintain.

Why should you care?

If you’re a device maker, do you need to care about CloudEvents? Generally, no. Golioth makes it really easy to connect to the Cloud and handles everything you’ll need to do in order to interact with the rest of the Internet.

But you’ll benefit from our use of CloudEvents because of how easy it is to connect to Golioth in a standard way. When you are working with your Cloud team (or a customer’s Cloud team, if you do design for hire), it will be really easy to interact with their Cloud. Even bespoke Cloud infrastructure will be able to easily connect to the Golioth Cloud. Thanks to the standardization, there are a range of libraries to make it easier to interact with CloudEvent data to match how your Cloud team is operating.

The clearest way to see that in action is to look at our WebHooks. These are primed to hook into and leverage the CloudEvents ecosystem. For example, you can use any of the dozen or so SDKs in your favorite language (we’re fans of Go!) or a bunch of integrations to process the data on your cloud. If you’re using one of our native output streams with AWS, GCP, or Azure, you don’t have to worry about developing the integration. Under the covers? It’s using CloudEvents too.

What are the future of CloudEvents?

CloudEvents have a specification that addressed the most common use cases, like HTTP + JSON or Kafka + Protobuf. But it’s also an extensible spec, so they added IoT protocols like MQTT. Anyone can define their own event format and publish it.

At Golioth, we continue to add functionality to our Output Streams capability, and we’re able to integrate new ones with lower overhead because of CloudEvent standardization. If you’re interested in extending Golioth capability or need help passing data between our Cloud and another Cloud, please reach out on our Forum, our Discord, or by email.

It’s no secret that we like Zephyr Real Time Operating System (RTOS) around here. Nor is it a secret that we’re very interested in the Internet of Things (IoT). Golioth Founder and CEO Jonathan Beri gave a talk at the 2022 Zephyr Developer Summit (ZDS) about why those two things are a perfect match, and how Zephyr can help you create an IoT product faster.

A high level overview

Since this talk was on the first day of ZDS, it was focused on people less familiar with the Zephyr ecosystem. So what should a beginner know about Zephyr and its relationship to IoT devices?

Batteries are included

Zephyr has a couple of key features that makes it a “one stop shop” for building an IoT device. Some other RTOSes rely on the engineer to piece together libraries and implementations to get started, which either means more startup time or reliance on hardware vendors to make a ready-to-go solution for you. Instead, Zephyr provides individual elements already configured to work together:

  • Device drivers
  • An OS kernel (including scheduler)
  • Services
  • A build system

Tying together the ecosystem with these tools provides a consistent experience. This also leads to another very important aspect.

Chip vendor buy-in

Because Zephyr has a well defined system, the chip vendors develop code to make their parts work within the ecosystem. With other RTOSes, that script is flipped; the RTOS is made to work with abstraction layers within the vendor specific ecosystem, meaning there is less likelihood of interoperability with other vendor solutions.

In addition to the chipsets from the vendors, a wide range of boards are supported. Targeting specific boards makes it easier to get dev boards working when getting started, and remapping pins to specific functions on a board doesn’t require a configurator tool like those available in many vendor IDEs.

Talking to the Internet

If you choose an RTOS that isn’t a broader ecosystem, you need to either define your own network layers or lean on a vendor implementation to do it for you. In Zephyr, the network layer and the protocol definitions are done as a part of the community. That means that chip, module, and software vendors are all working towards a common implementation. Add in the fact that there are modems and abstraction layers all the way up the stack, and an engineer using Zephyr doesn’t need to think about all of the pieces it takes to connect to the internet. At Golioth, we are able to switch between various networking technologies easily when using Zephyr.

Protocol Variety

Golioth talks to the internet at a very high level, so network access “just works”. This comes into focus as a small embedded device can talk to the variety of internet protocols available. Most people understand HTTP, but also consider the more common IoT favorites like MQTT and CoAP. Golioth favors CoAP but also supports MQTT. This enables implementations in other parts of the network stack. A good example is devices running OpenThread, a Thread network protocol implementation. As we showed in our recent post about Thread, a small Zephyr-based device utilizes 6LoWPAN and talks over CoAP using UDP packets to talk back to the Golioth cloud.

Last but not least: Security

Zephyr enables a secure connection back to the internet through the network stack and with features like DTLS, the basis of a secure connection over UDP. Security is a deeper topic in Zephyr though, all the way down to secure bootloaders and working within things like TrustZone. Talking to secure elements requires drivers that understand how to communicate with particular chip features (internal or external to a microcontroller). At a very high level, Zephyr focuses on things like Software Bill of Materials (SBOM), so your security teams understand the various software you are pulling into the build.

Built for IoT

Zephyr not only ❤️ internet…it’s purpose built for enabling IoT devices. Paired with Golioth, a Zephyr device has a higher likelihood of getting to market and growing to a massive scale. We’d love to hear your thoughts about the video on our Forum, our Discord, or by email.

 

Founding Engineer of Golioth, Alvaro Viebrantz went to the Zephyr Developer Summit…and said the hardware isn’t the only part of an IoT system that matters. That’s bold! This was a room full of hardware and firmware designers!

In reality, this talk is about everything else you should consider when building a device using the Zephyr Real Time Operating System (RTOS). Alvaro goes through the many layers and considerations when architecting large scale Internet of Things (IoT) deployment, from the servers, to the apps, to how devices are communicating back to the cloud, and yes, all the way down to the code put onto the end device.

Top Down Design

Most IoT users first experience a device not from holding the hardware in their hand and seeing it do some particular action. Instead, users rely on things like Dashboards and Applications. The physical device is often hidden from the end user, at least inside an enclosure, and sometimes entirely (as in the case of something like a home monitor in your attic).

It’s important to start by mapping your data model, often before you even start your first schematic. What will your device do? How will it be accessed hierarchically? What other elements are part of the system or will be in the future?

Also important, you should follow the ownership of both the physical device and that device’s output data all the way up the chain back to the user. How will a user claim a device belongs to them? How will you organize the various levels of permission required? What kind of questions will your users seek to answer with the end device or with the entire fleet? (ie. “Do any vehicles in the fleet have low battery?”)

Who is in charge here?

After you have considered how you will configure your database and what the end user will see when they are interacting with data, you should move on to access levels and which parts of the system are accessible by different individuals. This may include access to the end application, but should also include access control to the configuration components of the system. You don’t want your end users to be able to access your device management console, even if they are the ultimate owner of the device. Instead, you will want to provision the devices so you know the unique identifiers of the end-device, and then assign that and the access level to the user.

Additional layers required for a realistic IoT deployment

As you continue to move “down the stack” from Applications all the way to the hardware, there is still a good chunk of Cloud in between. You’ll want to make sure you at least understand all of the elements you’ll likely want in your design, including:

  • Device management
  • Device state and configuration
  • Bidirectional command and control
  • Time-series data and events

Connecting to the network

We’re almost down at the device itself, but the connection to the network is another key consideration before we get there. Not just the method of connection (which ultimately relies on the hardware), but also the protocol and the pathways of each packet coming back. In Golioth’s case, we default to CoAP, which is a lower overhead, UDP based protocol. All of our device SDKs include support to make it easy to talk to CoAP endpoints on the Golioth servers. As Alvaro mentioned in his talk, and as our SDKs also feature, encoding can help to shrink messages even further, using things like protobuf, JSON, and CBOR.

Finally the hardware

By this point in the room, we were squarely back into the realm of expertise of Zephyr developers, though many now had a greater appreciation of all the other things required for IoT systems. Alvaro mentioned how some of the interaction of the connectivity on-device in Zephyr could affect your decision making further up the stack. This talk to a packed room showed developers how to plot out a successful IoT deployment beyond the limits of the hardware, and to think about IoT systems from the hardware, all the way up to the end user.

 

Controlling 10 devices is easy, controlling 10,000 is a different story. The trick is to plan for scale, which is what we specialize in here at Golioth.

A few weeks ago we announced the Golioth Device Settings Service that enables you to change settings for your entire fleet at the click of a button. Of course, you can drill down to groups of devices and single devices too. While the announcement post showed the Settings Service using our ESP-IDF SDK, today we’ll look at this setting service from the device point of view with Zephyr RTOS.

Device-side building blocks

The good news is that the Golioth Zephyr SDK has done all the hard work for you. We just need to do three things to start using the settings service in any Zephyr firmware project:

  1. Enable the settings service in KConfig
  2. Register a callback function when the device connects to Golioth
  3. Validate the data and do something useful when a new setting arrives

Code walk-through

Golioth’s settings sample code is a great starting point. It watches for a settings key named LOOP_DELAY_S to remotely adjust the delay between sending messages back to the server. Let’s walk through the important parts of that code for a better understanding of what is involved.

1. Enable settings in KConfig

CONFIG_GOLIOTH_SETTINGS=y

To turn the settings service on, the CONFIG_GOLIOTH_SETTINGS symbol needs to be selected. Add the code above to your prj.conf file.

2. Register a callback for settings updates

static void golioth_on_connect(struct golioth_client *client)
{
    if (IS_ENABLED(CONFIG_GOLIOTH_SETTINGS)) {
        int err = golioth_settings_register_callback(client, on_setting);

        if (err) {
            LOG_ERR("Failed to register settings callback: %d", err);
        }
    }
}

To do anything useful with the settings service, we need to be able to execute a function when new settings are received. Register the callback function each time the device connects to Golioth. Your app may already have a golioth_on_connect() function declared, look for the following code at the beginning of main and add it if it’s not already there.

	client->on_connect = golioth_on_connect;
	client->on_message = golioth_on_message;
	golioth_system_client_start();

Each time the Golioth Client detects a new connection, the golioth_on_connect() function will run, which in turn registers your on_setting() function to run.

3. Validate and process the received settings

enum golioth_settings_status on_setting(
        const char *key,
        const struct golioth_settings_value *value)
{
    LOG_DBG("Received setting: key = %s, type = %d", key, value->type);
    if (strcmp(key, "LOOP_DELAY_S") == 0) {
        /* This setting is expected to be numeric, return an error if it's not */
        if (value->type != GOLIOTH_SETTINGS_VALUE_TYPE_INT64) {
            return GOLIOTH_SETTINGS_VALUE_FORMAT_NOT_VALID;
        }

        /* This setting must be in range [1, 100], return an error if it's not */
        if (value->i64 < 1 || value->i64 > 100) {
            return GOLIOTH_SETTINGS_VALUE_OUTSIDE_RANGE;
        }

        /* Setting has passed all checks, so apply it to the loop delay */
        _loop_delay_s = (int32_t)value->i64;
        LOG_INF("Set loop delay to %d seconds", _loop_delay_s);

        return GOLIOTH_SETTINGS_SUCCESS;
    }

    /* If the setting is not recognized, we should return an error */
    return GOLIOTH_SETTINGS_KEY_NOT_RECOGNIZED;
}

The callback function will provide a key (the name of the settings) that is a string type, and a golioth_settings_value that includes the value, as well as an indicate of the value type (bool, float, string, or int64). The callback function should validate that the expected key was received, and test that the received value is the proper type and within your expected bounds before acting upon the data.

The final piece of the device settings puzzle is to return an enum value indicating the device status after receiving a new setting. This is used by the Golioth Cloud to indicate sync status. Our example demonstrates the GOLIOTH_SETTINGS_SUCCESS and GOLIOTH_SETTINGS_KEY_NOT_RECOGNIZED status values, but it’s worth checking out the doxygen reference for golioth_settings_status to see how different invalid key and value messages can be sent to Golioth for display in the web console.

Coordinating with the Golioth Cloud

The device settings key format is tightly specified by Golioth, allowing only “A-Z (upper case), 0-9 and underscore (_) characters”. We recommend that you add a key to the Golioth web console while writing your firmware to ensure the device knows what to expect.

Each device page includes a status indicator to show if the settings are in sync with the Golioth cloud.

Additional information is available when a device is out of sync. Here you can see that hovering over the information icon indicates the key is not valid. For this example I created a VERY_IMPORTANT_BOOL_7 but didn’t add support for that to the firmware. This is the result of our callback function returning GOLIOTH_SETTINGS_KEY_NOT_RECOGNIZED. In cases like this, Golioth’s Over-the-Air firmwmare update (OTA) service can help you to push a new firmware to your devices in the field that will recognize the newly created Settings variable.

IoT Fleet control on day one

The pain of IoT often comes when you need to scale. Golioth’s mission is to make sure you avoid that pain and are ready to grow starting from day one.

Our device settings service gives you the power to remotely change settings on your entire fleet, on a group of devices, and all the way down to a single device. You are able to verify your devices are in sync, and receive helpful information when they are not. Take Golioth for a test drive today, our dev tier is free for the first 50 devices.

Today we’re announcing a new feature on the Golioth Console and on our Device SDKs that enables Remote Procedure Calls (RPCs) for all users on the platform. From the cloud, you can initiate a function on your constrained device in the field, ensure the device received and executed the command, and receive a response from the device back to the Cloud.

What is a Remote Procedure Call (RPC)?

A Remote Procedure Call allows you to call a function on a remote computing device and optionally receive a result. An easy way to think about it is you’re calling a function, like you would in any other program…you’re just doing it from another computer. In this case, you’re triggering actions from the Golioth Cloud.

RPC from the Golioth Cloud (Console)

Each device in your project has a page where you can view details about things like LightDB State, LightDB Stream, Settings, and now RPC. Our Console includes an interface to directly send RPCs to the remote device. The URL will look something like:

https://console.golioth.io/devices/<YOUR_DEVICE_ID>/management/rpc

In all of our examples, we are sending an RPC to single devices. However, they can also be triggered from the REST API. As a reminder, any function you see on the Golioth Console is available on the REST API.

One critical function of RPCs is a confirmation that the remote function has actually run. The device firmware needs to send back a success message that the function has completed, and optionally a returned value. When there is a problem connecting with your device and the RPC does not complete successfully, you will see a screen that looks like this:

An RPC sent to a device that was disconnected from Wi-Fi

Also note that round trip time is measured for all RPCs, including successful ones. Transit times will depend upon your connectivity medium, in addition to the processing time of the function on the remote device.

When an RPC successfully completes, you can click the button with 3 dots to receive the returned value. In the example and in the video, we were using a method called double that takes an integer input, multiplies it by two, and then returns the value to the Cloud. Below, you can see the result when we sent “double” method with a parameter of “37”.

RPC from the Device SDK perspective

Any new feature on Golioth has Device SDK support, in addition to the new APIs and UIs on our Web Console. Earlier this week, Nick wrote about how we test hardware and firmware at Golioth, especially when a new feature is released across the platform. Now that we support 3 SDKs (Zephyr, NCS, ESP-IDF), the testing area has increased.

In the video, the focus is on ESP-IDF, which has a simple way to set up and respond to new RPCs. First, we register the new method, so we’ll recognize the command coming from the Golioth Cloud:

The function that we tie to that newly registered RPC needs to return the RPC_OK variable for the Cloud to be alerted that the function has processed properly.

If you have satisfied these requirements in the ESP-IDF SDK and copied the format, you can customize logic to do whatever task you’d like on the remote device. Let’s look at some examples.

Use cases for an RPC

The double() example is a simple showcase of the minimum requirements to create an RPC in the ESP-IDF SDK. We send a command and a value, we return a modified value.

The remote_reset command we created and showcased in the video is more like a critical function you would want to add to your project. When you want to trigger a remote function like a reset, you want to ensure that the command was properly received, that the function executed, and then that there was output data that validated the reset has happened. In the final point, that includes inferring that the device has restarted from the log messages also being sent back to the Golioth Console. Put all together, it’s a reliable way to tell the device has been reset.

Other use cases could be as simple as sending arbitrary text to a display. You would still want to know the text has been received and properly sent to the physical display. Or perhaps you have a valve and you want to be able to send an arbitrary value to the valve, but you also want to take a reading on an encoder that measures the distance the valve has moved.

Many of these functions could also be achieved with LightDB State (which the RPC service is built upon), but the context for creating an RPC is more targeted at situations like the examples above.

What will you build?

RPCs are another way for you to communicate with your constrained IoT devices from the Golioth Cloud and to get useful information back from your devices. You can start testing out this feature today.

For more questions or assistance, check out our Forums, our Discord, or drop us a note at [email protected].