Demo Archives - Golioth

How to Send IoT Data in Batches

IoT is all about data. How you choose to handle sending that data over the network can have a large impact on your bandwidth and power budgets. Golioth includes the ability to batch upload streaming data, which is great for cached readings that allows your device to stay in low power mode for more of the time. Today I’ll detail how to send IoT data in batches.

What is Batch Data?

Batch data simply means one payload that encompasses multiple sensors readings.

[
    {
        "ts": 1719592181,
        "counter": 330
    },
    {
        "ts": 1719592186,
        "counter": 331
    },
    {
        "ts": 1719592191,
        "counter": 332
    }
]

The example above shows three readings, each passing a counter value the represents a sensor reading, along with a timestamp for when that reading was taken.

Sending Batch Data from an IoT Device

The sample firmware can be found at the end of the post, but generally speaking, the device doesn’t need to do anything different to send batch data. The key is to format the data as a list of readings, whether you’re sending JSON or CBOR.

int err = golioth_stream_set_async(client,
                                   "",
                                   GOLIOTH_CONTENT_TYPE_JSON,
                                   buf,
                                   strlen(buf),
                                   async_push_handler,
                                   NULL);

We call the Stream data API above. The client and data type are passed as the first two arguments, then the buffer holding the data and the buffer length are supplied. The last two parameters are a callback function and an optional user data pointer.

Routing Batch Data using a Golioth Pipeline

Batch data will be automatically sorted out by the Golioth servers based on the pipeline you use.

filter:
  path: "*"
  content_type: application/json
steps:
  - name: step0
    destination:
      type: batch
      version: v1
  - name: step1
    destination:
      type: lightdb-stream
      version: v1

This example pipeline listens for JSON data coming in on any stream path. In step0 it “unpacks” the batch data into individual readings. In step1 the individual readings are routed to Golioth’s LightDB stream. Here’s what that looks like:

Note that all three readings are coming in with the same server-side timestamp. The device timestamp is preserved in the data, but you can also use Pipelines to tell Golioth to use the embedded timestamps.

Batch Data with Timestamp Extract

For this example we’re using a very similar pipeline, with one additional transformer to extract the timestamp from the readings and use it as the LightDB Stream timestamp.

filter:
  path: "*"
  content_type: application/json
steps:
  - name: step0
    destination:
      type: batch
      version: v1
  - name: step1
    transformer:
      type: extract-timestamp
      version: v1
    destination:
      type: lightdb-stream
      version: v1

Note that we didn’t even need an additional step, but simply added the transformer to the step that already set lightdb-stream as the destination.

You can see that the Linux epoch formatted timestamp has been popped out of the data and assigned to the LightDB timestamp. Extracting the timestamp is not unique to Golioth’s LightDB Stream service.

Streaming data may be routed anywhere you want it. For instance, if you wanted to send your data to a webhook, just use the webhook destination. If you included the extract-timestamp transformer, you data will arrive at the webhook with the timestamps from your device as part of the metadata instead of nested in the JSON.object.

Using a Special Path for Batch Data

What happens if your app wants to send other types of streaming data beyond batch data? The batch destination will automatically drop data that isn’t a list of data objects. But you might like to be more explicit about where you send data and for that you can easily create a path to receive batch data.

filter:
  path: "/batch/"
  content_type: application/json
steps:
  - name: step0
    destination:
      type: batch
      version: v1
  - name: step1
    transformer:
      type: extract-timestamp
      version: v1
    destination:
      type: lightdb-stream
      version: v1

This pipeline is nearly the same as before with the only change on line 2 where the * wildcard was removed from path and replaced with "/batch/". Now we can update the API call in the device firmware to target that path:

int err = golioth_stream_set_async(client,
                                   "batch",
                                   GOLIOTH_CONTENT_TYPE_JSON,
                                   buf,
                                   strlen(buf),
                                   async_push_handler,
                                   NULL);

Although the result hasn’t changed, this does make the intent of the firmware more clear, and it differentiates the intent of this pipeline from others.

Sample Firmware

This is a quick sample firmware I made to use while writing this post. It targets the nrf9160dk. One major caveat is that the function that pulls time from the cellular network is quite rudimentary and should be replaced on anything that you plan to use in production.

To try it out, start from the Golioth Hello sample and replace the main.c file. This post was written using v0.14.0 of the Golioth Firmware SDK.

Wrapping Up

Batch data upload is a common request in the IoT realm. Golioth has not only the ability to sort out your batch data uploads, but to route them where you want and even to transform that data as needed. If you want to know more about what Pipelines brings to the party, check out the Pipelines announcement post.

July 9, 2024/by Mike Szczys

Demo, Protocol, Reference Design, Zephyr

OpenThread Demo: From Cellular to Thread with Zephyr

Golioth is expanding its Reference Design portfolio by adding an OpenThread Demo, a Reference Design based on our known and well-tested Reference Design Template. The purpose of the OpenThread Demo is to add Thread networking capability to the RD Template so anyone using Thread and Golioth can start development immediately, use it as a basis for their project, and take full advantage of Golioth’s Device Management, Data Routing, and Application Service capabilities.

Thread Recap

Thread is an IPv6-based networking protocol designed for low-power Internet of Things devices. It uses the IEEE 802.15.4 mesh network as the foundation for providing reliable message transmission between individual Thread Devices at the link level. The 6LoWPAN network layer sits on top of 802.15.4, created to apply Internet Protocol (IP) to smaller devices. In almost all cases, it’s used to transmit IPv6 Packets.

If you need a network of devices that can communicate with each other and connect to the Internet securely, Thread might be the solution you’re looking for.

Built it yourself

The follow-along guide shows how to build your own OpenThread Demo using widely available off-the-shelf components from our partners. We call this Follow-Along Hardware, and we think it’s one of the quickest and easiest ways to start building an IoT proof-of-concept with Golioth.

Hardware

Every mesh network needs some hardware, and for the OpenThread Demo, you will need a Thread Border Router and a Thread node. This demo doesn’t need additional sensors or an actuator, as there are generated values created by the code in the Reference Design Template (ie simulated values). Later you can modify our other Reference Designs and their hardware to get to a prototype or production device that is more specific to a vertical like Air Quality Monitoring or DC Power Monitoring.

Border Router

A Thread Border Router connects a Thread network to other IP-based networks, such as Wi-Fi or Ethernet, and it configures a Thread network for external connectivity. It also forwards information between a Thread network and a non-Thread network (from Thread nodes to the Internet). The Border Router should be completely invisible to Thread Devices, much like a Wi-Fi router is in a home or corporate network.

In this demo, we use a commercially available GL-S200 Thread Border Router designed for users to host and manage low-power and reliable IoT mesh networks.

GL-S200 provides a simple Admin Panel UI to configure the Border Router and a Topology Graph to see all the end node devices and their relationship. As a bonus, it also does NAT64 translation between IPv6 and IPv4, making it a real plug-and-play solution.

Thread Node

Now that the centerpiece of our Thread network is sorted, the next part is a Thread node. In the follow-along guide, we built a Thread node based on the nRF52840 DK. The node is built using Zephyr, and the OpenThread stack will be compiled into it. The GitHub repository used in the guide is open source, so you can build the application yourself, or you can use the pre-built images for the nRF52840 DK or Adafruit Feather nRF52840.

Firmware

Thread node firmware is based on the Reference Design Template, a starting point for all our Reference Designs. With all Golioth features implemented in their basic form, you can now use Device Management, Data Routing, and Application Services with Thread network connectivity.

OTA Updates

Adding Thread support to a device is not cheap, memory-wise. The firmware image is larger than 500kB, and the on-chip flash of the nRF52840 DK has a size of 1MB. Luckily, both the nRF52840 DK and the Adafurit Feather have an external flash chip, making the OTA updates possible. Any custom hardware you create in the future should also follow this model of having external flash mapped to the nRF52840.

To create a secondary partition for MCUBoot in an external flash, we must first enable it in the nrf52840dk_nrf52840.overlay file:

/ { 
    chosen { 
        nordic,pm-ext-flash = &mx25r64; 
    };
};

The CONFIG_PM_EXTERNAL_FLASH_MCUBOOT_SECONDARYKconfig option is set by default to place the secondary partition of MCUboot in the external flash instead of the internal flash (this option should only be enabled in the parent image).

To pass the image-specific variables (device-tree overlay file and Kconfig symbols) to the MCUBoot child image, we need to create a child-image folder in which we need to update the CONFIG_BOOT_MAX_IMG_SECTORS Kconfig option. This option defines the maximum number of image sectors MCUboot can handle, as MCUboot typically increases slot sizes when external flash is enabled. Otherwise, it defaults to the value used for internal flash, and the application may not boot if the value is set too low. In our case, we updated it to 256in the child_image/mcuboot/boards/nrf52840dk_nrf52840.conf file.

CONFIG_BOOT_MAX_IMG_SECTORS=256

Connecting to Golioth Cloud

Thread nodes utilize IPv6 address space, and the question is how to communicate with IPv4 hosts, such as Golioth Cloud.

Golioth Cloud has an IPv4 address, and the Thread node needs to synthesize the server’s IPv6 address in order to connect to it. OpenThread doesn’t use the NAT64 well-known prefix 64:ff9b::/96; instead, Thread Border Routers publish their dynamically generated NAT64 prefix used by the NAT64 translator in the Thread Network Data. Thread nodes must obtain this NAT64 prefix and synthesize the IPv6 addresses.

While the process of synthesizing IPv6 addresses is automatically handled in the OpenThread CLI when using the Zephyr shell and pinging an IPv4 address (e.g. ot ping 8.8.8.8), it’s important to note that this process needs to be specifically implemented in applications.

As part of the Firmware SDK, the Golioth IPv6 address is automatically synthesized from the CONFIG_GOLIOTH_COAP_HOST_URI Kconfig symbol using the advertised NAT64 prefix by leveraging the OpenThread DNS. Even if the Golioth host URI changes within the SDK, you won’t need to change your application.

Learn more

For detailed information about the OpenThread Demo, check out more details the project page! Additionally, you can drop us a note on our Forum if you have questions about this design. If you would like a demo of this reference design, contact [email protected].

July 2, 2024/by Marko Puric

Demo, Pipelines

A $2 Geofence: Wi-Fi Location (Here.com), ESP32-C3, Golioth Pipelines, and n8n

Piecing together different pieces of technology can have a multiplicative effect. I think that’s what happened with this demo: we paired Wi-Fi locationing, low cost hardware, Golioth Pipelines, and n8n (an API workflow tool) to create a “geofence”.

A geofence is a virtual perimeter used to set up alerts or take actions once a device moves outside that virtual perimeter. The example we gave in the video is if you had a tracker on your cat and you wanted to take an action once the device was outside a particular area.

Hardware

The reason we’re calling this a “$2 geofence” is because it’s enabled by the ESP32-C3, a low cost module from Espressif. We put this on the Aludel Elixir as a backup connectivity method if we were again at a conference with no LTE-M coverage.

The ESP-AT firmware does what it sounds like it should do: it responds to AT commands from other microcontrollers talking to it over serial (as many cellular modules also do). One key enhancement is that the ESP-AT mode already works as a connectivity method; in fact, we utilize the ESP-AT firmware as an offloaded Wi-Fi modem when we build and test for the nRF52840 in our Continuously Verified Hardware. In Zephyr, there is an option for utilizing the ESP-AT modem as the main offloaded Wi-Fi modem. This makes it ‘invisible’ to the Zephyr program and acts like any other network interface, since it is built on top of the Wi-Fi subsystem in Zephyr.

One change that was required is we had to re-write how we pulled the information off the ESP-AT modem. Normally the wifi scan shell command returns the (human readable) names and signal strengths of all the access points (APs) visible to the modem. Instead, we want mac address and signal strength, as that’s what’s expected by the API service we’ll describe below.

Golioth Pipeline

We start by scanning Wi-Fi APs and the tower that the cell modem is connected to. Then we publish that on the Stream service up to the Golioth cloud. Because we’re publishing to a specific topic (instead of my normal, generic default of “sensor”), we can start to peel off that data and send it somewhere interesting. How? With pipelines, of course!

I set up the pipeline to watch on the path wifi_lte_loc_req (a name of my own making, this could be any arbitrary name). That data gets sent out to a webhook going to n8n. Webhooks more broadly are a generic way to interface between a lot of cloud services, but we use it to send data into the api platform.

n8n

Now that the data is being sent into n8n (a self hosted instance, no less!) we can start doing interesting things with it. This is an area that is full of similar offerings, sometimes specifically targeted at IoT, and other time targeted a business workflows:

If you’re newer to working with APIs and tying stuff together, it might take a bit of time to figure out how queries should be structured and how your setup should respond when there are errors.

API service

We send data from the device to Golioth already formatted for what the location service API service expects. This is not required in the slightest, as Golioth’s Pipelines can morph and transform data to meet the needs of the endpoint. But…why not? It kind of makes sense to have the device publish data in a format that matches the target API service. Then later if we decide to re-target an alternative service, we can use transformations to mold the incoming data to what that new service expects.

For this demo, I’m using the here.com API service. I like that it combines LTE tower + WiFi AP for its API, which means it will lean on whichever provides a more accurate reading (normally Wi-Fi). Again, this service is one of many! There are a range of API services because this is something that phones are often using to determine location from apps.

Once we receive the lat, lon, and accuracy, we actually pass the data back to the device using LightDB State. This two-sided database is a good defacto way to send arbitrary data from the cloud to the device. In the case of n8n, we’re pulling through the original project name, device identifier, and then publishing to the Golioth REST API. This makes it a data “round trip” from device to cloud and back down to device.

Logic and alerts

Since the data is already on the cloud in an API marketplace like n8n…why not use that data to do some cloud side processing? In this case, I wanted to set up a geofence to show that we can trigger logic and alerts on the cloud and even call 3rd party APIs like Slack and Twilio.

Geofence alert messages being sent into Slack

I asked ChatGPT to help me out with some javascript that would help calculate a true/false output so that I could use that to trigger downstream logic. We insert the lat/lon data that was returned from here.com into this algorithm and it pops out whether or not we are inside the “fence”. As of this writing, I am still using a fixed location for where the center of the “fence” is located, as well as the radius of said “fence”. I’m certain it’s possible in n8n or other tools, perhaps as another Webhook or a configurable variable.

Future demos

Hopefully one thing you noticed from this demo is just how much can be enabled with Golioth’s pipelines. Since Golioth takes care of reliably delivering your data to the cloud, the rest is really a matter of configuration. It’s also difficult to know all the different APIs that could be utilized out in the world. Pulling these elements together shows how a hardware or firmware engineer could enact complex device and business logic to create interesting applications out in the real world. If you need any help getting your next project off the ground, stop by our forum!

June 26, 2024/by Chris Gammell

Bluetooth, Demo

The Inside Story of the Golioth Bluetooth® Mesh Demo

This is a guest post by Sandra Capri, CTO at Ambient Sensors, a Golioth Design Partner who regularly designs and deploys Bluetooth Solutions.

My previous article described a demo that used the Golioth console to send RPCs (Remote Procedure Calls) and device settings to a Bluetooth Mesh. The purpose was to show how easy it is to control a Bluetooth Mesh from an internet connected device. This article is to give some details as to how I created the demo.

But first, let me give you a quick tutorial on the Bluetooth Mesh LC server (the following diagram represents the information from the Bluetooth® SIG specification MshMDL_v1.1):

The Y-axis is the brightness of a luminaire (“the device that is lighting up”). LuxLevel/Lightness is defined for three levels: On, Prolong, and Standby. The general rule is that On is the brightest setting, Prolong is dimmer than On, and Standby is the dimmest (or completely off). The X-axis is the time axis during which the luminaire maintains each of those lightness levels.

Each bbc:microbit in the demo implements an LC Server which while following the above diagram:

Transitions the LEDs to the “On” lightness level (LuxLevel/Lightness On)
Stays at that lightness level for the “Run” time (Light LC Time Run)
Dims the LEDs to the “Prolong” lightness level (LuxLevel/Lightness Prolong)
Stays at that lightness level for the “Prolong” time (Light LC Time Prolong)
Dims the LEDs to the “Standby” level (LuxLevel/Lightness Standby)

The Device Settings

The demo needed a way of setting these five parameters (the three lightness levels and two times). The Device Settings page of the Golioth console was ideal for this. I mapped the settings as follows:

Device setting name
TIME_RUN
TIME_PROLONG
LVL_RUN
LVL_PROLONG
LVL_STANDBY

LC server value
Light LC Time Run
Light LC Time Prolong
LuxLevel/Lightness On
LuxLevel/Lightness Prolong
LuxLevel/Lightness Standby

And of course, the Mesh firmware specifies default values for each of these. But when doing demos (or development), there are many times you want to change these values. If you don’t have a handy LC client around, you often resort to changing the defaults in the firmware and pushing over the air updates. And if you have 8 (or more) LC servers to update in a mesh, that gets painful quickly.

In the demo, you can set TIME_RUN to 5 seconds, so when the LEDs first turn on, they stay “On” for 5 seconds. If that time is too short for you, set it to 20 seconds, and the next time the LEDs turn “On”, they stay there for 20 seconds. Similarly with TIME_PROLONG – you easily define how long the lights stay at “Prolong.”

And then comes the individual light levels – do you want the Run level to be 100%, Prolong at 50%, and Standby at 0% (off)? Then all you need to do is change a setting. Voilà, it’s done. This is much easier than modifying every LC server firmware on each device, every time you want to view a new set of levels and times.

The astute reader will notice that there are other values in the diagram above (and there are several other values in the LC Server model as well), but only a subset is implemented for simplicity’s sake.

The Remote Procedure Calls

Now what about those RPCs? Several were written specifically for this demo, turning the LEDs on and off in different ways. Some turn off the control of the LC server, and some turn it back on. The following functions were implemented:

set_light
- This is often the first RPC sent – it demonstrates the basic LC model functionality. A set_light on RPC tells the microbits to follow the LC server diagram (shown above). Each microbit ramps its LEDs to the “On” lightness level, then starts dimming, eventually ramping back to “Standby”. A set_light off RPC tells the microbits to immediately set the LEDs to “Standby”, bypassing the ramping and the “Prolong” level. Note that pushing the button on the Thingy91 sends the same command as the set_light RPC.
lightness
- This RPC overrides the lightness level, turning off the LC server. The lightness will stay at the specified percentage level until another command is received.
lc_mode
- This RPC turns the LC server on or off. This sends the LC model command to turn on/off the server. Anytime the LC server is turned off (e.g., with a lightness command), this is the only command that can turn it back on.
gen_on_off
- This RPC is specific to the structure of an LC server. Every LC server node has at least two elements – one with a Lightness Server, and one with an LC server. The gen_on_off RPC sends a generic on/off command to the specified element. The first parameter is the on/off command (1 == on, 0 == off), and the second parameter specifies the element (0 == LC server, 1 == Lightness server).

How Does the Code Know about Changes to the Device Settings?

Golioth provides a registration function for device settings. The firmware passes a callback function to the register function. Anytime a device setting changes, the callback function is invoked, and it receives:

The string name of the setting (e.g., “TIME_RUN”)
A structure containing the value.

The firmware then does a simple name string comparison, and then we send that value on to the Mesh (more about that later).

int err = golioth_settings_register_callback(settings_client, on_setting);
if (err) {
    LOG_ERR("Failed to register settings callback: %d", err);
}

And the definition of the on_setting() callback:

enum golioth_settings_status on_setting(const char *key, const struct golioth_settings_value *value)
{
…
    if (strcmp(key, "TIME_RUN") == 0) {
        /* time in seconds - numeric */
        if (value->type != GOLIOTH_SETTINGS_VALUE_TYPE_INT64) {
            LOG_DBG("Received TIME_RUN is not an integer type.");
            return GOLIOTH_SETTINGS_VALUE_FORMAT_NOT_VALID;
        }
        /* Only update if value has changed */
        if (_time_run_sec == (int32_t)value->i64) {
            LOG_DBG("Received TIME_RUN already matching local value.");
        } else {
            _time_run_sec = (int32_t)value->i64;
            _time_run_changed = true;
            // tell system thread to send the property data across the UART to
            // the BLE chip
            wake_system_thread();
        }
        return GOLIOTH_SETTINGS_SUCCESS;
    }
…
}

How Does the Code Know About RPCs?

It’s a slightly different mechanism, but straightforward: Golioth provides an RPC registration function. For every RPC you create, provide it to the registration function along with a string (linking your RPC function to the string name in the console). When the string name is invoked in the console, the local RPC function is called, passing any provided parameters. In the demo, we send that information to the Mesh (more on that later). Code snippet:

err = golioth_rpc_register(rpc_client, "lightness", on_lightness, NULL);
rpc_log_if_register_failure(err);

And earlier in the file we defined RPC function on_lightness() as:

static enum golioth_rpc_status on_lightness(zcbor_state_t *request_params_array,
                                            zcbor_state_t *response_detail_map,
                                            void *callback_arg)
{
    bool ok;
    double valuef1;
    uint8_t value1;

    ok = zcbor_float_decode(request_params_array, &valuef1);
    if (!ok)
    {
        LOG_ERR("Failed to decode RPC int1 argument");
        return GOLIOTH_RPC_INVALID_ARGUMENT;
    }

    value1 = (uint8_t) valuef1;
    LOG_DBG("Received argument '%d' from 'lightness' RPC", value1);
    send_lightness(value1);

    return GOLIOTH_RPC_OK;
}

Passing commands between nRF9160 and nRF52840

Up to now, everything has been communication between the nRF9160 and Golioth (via LTE-M or NB-IoT). So now the nRF9160 is going to send this information to the nRF52840 (the BLE chip configured as an LC client). The UART is a convenient communication path provided by the Thingy91. When any device setting changes or an RPC is called, the nRF9160 maps that to a series of bytes to send to the BLE chip.

For example, when the nRF9160 calls send_lightness(value1), it executes:

bytebuf[0] = '3'; // lightness cmd – character
bytebuf[1] = v1;  // integer val, not character - percentage
err = uart_tx(Uart, bytebuf, sizeof(bytebuf), SYS_FOREVER_US);

which sends those two bytes to the BLE chip. Upon receiving the UART data, the BLE chip parses the lightness command, grabs the lightness value, and then sends a Bluetooth Mesh lighting command to tell the micro:bits to set the lightness value.

How does the BLE chip (the nRF52840) send Mesh commands?

The Bluetooth Mesh has a different communication method from the traditional BLE central/peripheral characteristic write/read method. The Mesh doesn’t really have the idea of a connection, nor of a central or peripheral. Every device on the Mesh is called a node, and Mesh nodes send messages that every node on the Mesh can hear (more details in a future blog).

For the demo, the nRF52840 has been programmed as a full Bluetooth Mesh node – specifically as a client (it sends commands to servers). Each LC server (micro:bit) listens for commands from the nRF52840 clients. Yes, that is plural – clients…The nRF52840 node has

One LC client
Two Generic On/Off clients
One Lightness client.

When the node wants to send the LC mode command, it uses the LC client function:

int bt_mesh_light_ctrl_cli_mode_set(…,…,bool enabled,…);

where the “enabled” parameter will set the LC Mode on or off.

When it wants to send a Lightness command, it uses the Lightness client function:

int bt_mesh_lightness_cli_light_set(…,…,… *set,…);

where the “set” parameter points to a structure containing the lightness level.

In general, when a node needs to send a Mesh command, it calls the appropriate client model function to send it. So, when an LC model command needs to be sent, we call an LC client model function. When a Generic On/Off model command needs to be sent, we call a Generic On/Off client model function (and similarly with Lightness commands, or any of the other commands that have been defined in the Bluetooth Mesh Model Specification).

And then the micro:bits?

The final piece of the puzzle is the micro:bits – they are programmed as mesh nodes, implementing the LC server. Each one “hears” the commands from the client node, and independently acts upon those commands. Note that each server node can return status(es) as a result of these commands, and the client node can act upon these statuses (this was not implemented in the demo, for simplicity).

The full signal chain, described

So now you have an overview of the process – specifically how:

The nRF9160 receives the device settings and RPCs from the Golioth console,
The nRF9160 sends that info over the UART to the nRF52840,
The nRF52840 sends that info, translated into Bluetooth Mesh commands, to the rest of the nodes in the Mesh (the micro:bits).

Bluetooth Mesh and the LC server nodes are simple in concept, but often difficult in execution. With the Thingy91 running the Golioth Firmware SDK, it has gotten a lot easier to control and extend Bluetooth Mesh demos to show to clients.

June 11, 2024/by Sandra Capri

Bluetooth, Demo, Device

Connecting a Bluetooth® Mesh Demo to the Cloud

This is a guest post by Sandra Capri, CTO at Ambient Sensors, a Golioth Design Partner who regularly designs and deploys Bluetooth Solutions.

We have been building Bluetooth® Mesh (hereafter referred to as “BT Mesh”) products for a long time. In fact, we helped write some of the BT Mesh models for Nordic, so we are very familiar with it. However, we’ve always been dependent upon applications written for phones to control the BT Mesh, which can be limiting while doing Mesh development or putting together quick prototypes for potential customers. This particular demo allowed us to add network connectivity by building on top of Golioth’s Thingy91 demo, which unlocks a faster design iteration cycle when it comes to BT Mesh. We asked Chris to help record the demo video above.

Controlling from afar

This demo shows that Golioth can control a BT Mesh-based lighting system where each of the individual components (nodes in the mesh) of the lighting system, (e.g., switches, luminaires, sensors), contain a BLE SoC. Normally a user must be within a few tens of meters of a Bluetooth device to interact with it using (for example) a mobile app on a smartphone. But instead, we show Golioth connecting via LTE-M cellular service to a Nordic Semiconductor Thingy91 (combined nRF9160 cellular modem and nRF52840 BLE SoC) which then communicates to the rest of the Bluetooth mesh, thus allowing an authorized user from anywhere on Earth to control the lights (represented in the demo by a series of bbc:microbits). This gives the user control over all aspects of the Mesh servers – sending commands and updating settings without needing to be physically present.

How it works

The button on the Thingy91 is programmed to function as a simple on/off light switch. A much greater set of control functions for the BT Mesh comes from the Golioth web interface (Console). The demo shows a small sample of the kinds of control that can be accomplished using Golioth:

Ramping up the LED level on the lights
Defining the LED levels
Setting the time to keep the LEDs at those levels before ramping down
Overriding the LED level – preventing ramping

The Thingy91 contains an nRF9160 LTE-M chip and an nRF52840 BLE chip – this represents the bridge between LTE-M and the BT Mesh. The bbc:microbits contain an nRF52833 BLE chip, and represent the luminaires (Mesh lighting nodes). The Golioth SDK has been integrated into the code for the nRF9160, connecting the Thingy91 to Golioth. When a user executes a Remote Procedure Call (RPC) or sets a device setting, this information is sent via LTE-M to the nRF9160. That chip converts that information into opcode values sending it via UART to the nRF52840 chip. The nRF52840 then builds an appropriate Bluetooth mesh command and sends it to the Mesh network (wirelessly). Each lighting node on the network receives and executes the Bluetooth Mesh command.

In this demo, each lighting node (bbc:microbit) implements the Bluetooth Mesh LC server: an intelligent Lighting Controller. If the LC element on the lighting node receives a Generic On command, it ramps up its LED level to the configured lightness on value and maintains that level for the period of the configured run on time. After that time has expired, it will fade the light level to the configured prolong lightness level and maintain that for the configured prolong time. Then it fades the LEDs to the configured standby lightness level.

Each of these configurable lightness levels and times can be changed in the Device Settings.

Once these are set, an RPC can initiate the LC server control, or it can override the LC server and set an exact light level preventing the LC server from controlling the lights. Additionally, there is an RPC that will re-enable the LC server.

Other applications

This demo gives just a taste of the control that Golioth can provide with the Bluetooth Mesh. In addition to lights and switches, many different classes of devices can be on a Mesh:

Burglar and fire alarms
Smart locks
Thermostats
All kinds of sensors (e.g., temperature/humidity, photometry, motion, power, environmental, …)
More!

Commands can flow from the Cloud to each device, and status can be returned to the Cloud (e.g., alerts, alarms, etc.) Are the warehouse doors locked and the burglar alarm set? Well let’s check Golioth – oops, still unlocked! Let’s fix that. OK, Golioth just locked the front door, verified the locked status of the other doors, and turned on the burglar alarm. And just for good measure, we checked that the current temperature and humidity are within acceptable parameters. We didn’t even have to leave home to do that.

Now that we have the ability to control all this from the Cloud, we’ve unlocked new designs for our customers, and we’re able to spin up test deployments faster than ever.

May 14, 2024/by Sandra Capri

Demo, Device, Zephyr

How to localize your devices with GNSS, NMEA, and Zephyr

Jerónimo Agulló is an open-source projects enthusiast and a Zephyr RTOS contributor. He has worked on different IoT projects across different industries, from research centers such as the University of Sevilla, to large companies like Ferrovial Construction. Check out his work on his GitHub profile.

Location is a basic part of the most interesting IoT systems. It’s not just about loss prevention; geolocation opens the door to several applications and enhances the management of devices. In many cases, this involves Global Navigation Satellite System (GNSS).

Position is so important in IoT that the latest Zephyr release (v3.6.0) has added a new API for GNSS based on the NMEA0183 standard. Today I’m going to explain the basics of GNSS, what NMEA means, and dive into this awesome new Zephyr feature.

The basics of the NMEA standard

NMEA is the acronym for the National Marine Electronics Association, an organization created before GPS was invented. Its aim is creating better communications with manufacturers. I can assure you that they achieved their goal with GNSS.

Their most extended and well-known norm is the NMEA0183 protocol, which has become the GNSS standard for almost all manufacturers. It facilitates module integration and new applications development.

NMEA data is transmitted from a source such as a GPS module (known as a “Talker”) to equipment, such as our running Zephyr device (known as a “Listener”). One important aspect is that a single talker can communicate to many listeners.

From the electronic perspective, NMEA0183 originally used RS-422, but now uses several interfaces such as UART, USB, RS-232, WIFI, Bluetooth, and more. We can consider NMEA protocol as a common message structure standard. NMEA0183 protocol makes software developer life easier due to the standardization between GNSS devices.

NMEA message structure

Each NMEA sentence contains only ASCII characters, starting with the dollar symbol “$” and ending with the <CR> (Carriage return) and <LF> (Line feed) characters. The content of the message is a tuple, separated by commas. This starts with an NMEA identifier, followed by data, and ending by a checksum preceded by an asterisk “*”.

For a better understanding, let’s examine a popular NMEA sentence from a Quectel module. “GGA” sentence contains the Global Positioning System fix data, time, position, and fix related data for a GNSS receiver.

$GPGGA,102744.00,6155.393269,N,00848.433734,E,1,03,1.6,821.5,M,52.0,M,,*70

$GPGGA is the sentence identifier which can be split into “GP” which means the GNNS type, in this case GPS, and “GGA” which is the sentence identifier.
102744.00 is the time of fix (hhmmss).
6155.393269,N is the latitude (ddmm.mmm format, N for North).
00848.433734,E is the longitude (dddmm.mmm format, E for East).
1 is the fix quality indicator.
03 is the number of satellites being tracked.
1.6 is the horizontal dilution of precision.
821.5,M is the altitude in meters above mean sea level.
52.0,M is the height in meters of geoid separation.
*70 is the checksum (Note: this is not the correct checksum for this payload, we moved the location manually)

All talker devices don’t rely on GPS or on different constellations simultaneously. The most common constellations are GLONASS (GL), BEIDOU(DB or GB) and GALILEO (GA).

Besides GGA, other popular NMEA sentences are:

RMC: Recommended Minimum Navigation Information
- This contains information similar to GGA such as the latitude and longitude and also the speed over ground (in knots), the date and the magnetic variation (in degrees).

GSV: Satellites in View
- This NMEA message prints information such as the total number of satellites in view for each constellation and for each satellite its number (PRN), elevation in degrees, azimuth in degrees and Signal to Noise Ratio (SNR) in dB. Each GSV sentence contains the information of more than one satellite.

The previous NMEA sentences are named as “talker sentences”. In addition to that, there are “Proprietary sentences” and “query sentences”. On the one hand, the proprietary sentences start with “$P” and allow manufacturers to define custom NMEA sentences format for custom functions such as power management. On the other hand, the query sentences are the means for listener to request a particular sentence from a talker

Zephyr GNSS API

If you’ve reached this point, it means that you already know and like Zephyr RTOS. If you’re not familiar with Zephyr, I encourage you to familiarize yourself with it by using this getting started guide.

Zephyr RTOS is not just a simple scheduler for simple applications. Zephyr is a whole operating system with drivers, services and common APIs which facilitates the new developments and sensors and boards exchange. I would go beyond and say that Zephyr is even a complete ecosystem with integration in awesome clouds such as Golioth.

A great example of this affirmation is the new Zephyr GNSS support in version v3.6.0. The GNSS API is built upon the modem subsystem, which provides the necessary modules to communicate with modems. The GNSS subsystem covers everything from sending and receiving commands to and from the modem, to parsing, creating and processing NMEA0183 messages.

The source code can be found under the path zephyr/drivers/gnss, which is divided into specific GNSS module drivers with custom features such as power management, some files with utils and a generic NMEA0183 driver. Covering in detail each of those utils would require a whole new post. However, I will provide a brief overview of each of them:

gnss_dump
- A set of utilities to get, convert and print GNSS information into readable and useful data.

gnss_nmea0183 and gnss_parse
- NMEA0183 utilities such as checksum calculation, parsing a ddmm.mmmm formatted coordinates to nano degrees or parsing the content of GGA, RMC and GSV NMEA messages.

gnss_nmea0183_match
- This code is based on “modem_chat” match handlers, a Zephyr utility to process messages from modems. The callbacks to parse GGA, RMC and GSV messages are defined here.

Zephyr has created a generic NMEA driver which can be used for any NMEA talker. As any other driver, it is instantiated by the DEVICE_DT_INST_DEFINE macro. It includes the following code to call the callbacks defined in the gnss_nmea0183_match.c for corresponding NMEA identifier:

MODEM_CHAT_MATCHES_DEFINE(unsol_matches,
MODEM_CHAT_MATCH_WILDCARD("$??GGA,", ",*", gnss_nmea0183_match_gga_callback),
MODEM_CHAT_MATCH_WILDCARD("$??RMC,", ",*", gnss_nmea0183_match_rmc_callback),
#if CONFIG_GNSS_SATELLITES
MODEM_CHAT_MATCH_WILDCARD("$??GSV,", ",*", gnss_nmea0183_match_gsv_callback),
#endif
);

GNSS example in Zephyr

Now you know about NMEA0183 and the Zephyr GNSS API, so get your hands dirty and write some code! The full code can be found in my github repository:

https://github.com/jeronimoagullo/Zephyr-GNSS-Sample

KConfig Changes

In this example, I will show you how to configure a microcontroller in Zephyr to use a GNSS module which supports NMEA0183. For this aim, I will use an ESP32S3-Mini-1 development board along with an Air530z GNSS module. We’re going to modify our prj.conf file

Add the CONFIG_GNSS=y kconfig variable to add the GNSS support.
Add the variables CONFIG_GNSS_SATELLITES=y to display satellites’ information
Add CONFIG_GNSS_DUMP_TO_LOG=y to print GNSS information like the fixation status, coordinates and time.

Devicetree Changes

Next, we’ll configure the device tree correctly for our GNSS module. In this case, the Air530z GNSS module uses an UART interface. By default, ESP32S3 has the UART0 configured in pins 43 and 44. However, this UART is configured at 115200 bps and used for both programming the board and by Zephyr to print messages into the serial terminal. This is not a good idea.

What is the solution? The use of another UART, for example, the UART1. However, it is not defined in esp32s3_devkitm.dts. So let ‘s do it!

According to GPIO, the UART1 uses the pins 17 and 18 (you can use others too). I have chosen these default pins and defined them in the device tree overlay using the pin control subsystem. The macro of the pins can be found in the file zephyr/include/zephyr/dt-bidings/pinctrl/esp32s3-pinctrl.h you can find your board pins in the corresponding header file of your board.

&pinctrl {

  uart1_pins: uart1_pins {
    group1 {
      pinmux = <UART1_TX_GPIO17>;
      output-high;
    };

    group2 {
      pinmux = <UART1_RX_GPIO18>;
        bias-pull-up;
      };
    };

};

Then, in the UART node, I have added our above pin definition and a “gnss-nmea-generic” compatible device. I have created an alias to this GNSS device to facilitate the access from the main code.

&uart1 {

  status = "okay";
  current-speed = <9600>;
  pinctrl-0 = <&uart1_pins>;
  pinctrl-names = "default";
  gnssdev: gnss-nmea-generic {
    compatible = "gnss-nmea-generic";
  };

};

Finally, the main code uses the GNSS_DATA_CALLBACK_DEFINE macro to define a callback for getting the information about satellites in view and another for GNSS information such as time, coordinates and tracking satellites. The following screenshot depicts the application output:

We managed to get the GNSS data!

Final thoughts

We have covered in this post what NMEA means and its importance in GNSS as well as the main NMEA messages, how the new Zephyr GNSS API relies on NMEA, and a sample of how to localize our device using Zephyr. Now, it is your turn to send the GNSS data to Golioth cloud!

May 2, 2024/by Jerónimo Agulló

Bluetooth, Demo, Device

Bluetooth Gateways with Golioth and Zephyr

We always seem to meet people at conferences who are looking for a Bluetooth gateway solution. Golioth is the universal connector for IoT, and today we’re going to take a look at one way to extend that to your Bluetooth devices.

Luckily, Bluetooth is well supported by Zephyr. We’re going to run some sample code, then customize it to use Golioth as the cloud connection for a Bluetooth gateway device. Our target hardware is an nRF9160-DK as the gateway (we’ll refer to this as the central), and an nRF52840-DK as the sensor (which we’ll call the peripheral).

Two blue Nordic development boards on a wooden desk

Upper: nRF52840-DK as a Bluetooth health temperature sensor
Lower: nRF9160-DK as a Bluetooth-to-Cellular gateway

We’re using the nRF Connect SDK version of Zephyr. You can follow our NCS with Golioth documentation to install it locally if you have not already done so.

This guide uses Golioth Firmware SDK v0.12.0 with NCS v2.5.2

HCI low power UART

We chose the nRF9160-DK because it has both an nRF9160 to provide a cellular connection, and an nRF52840 as a Bluetooth radio. The nRF9160 will run the show, communicating with the nRF52840 over UART. Nordic has an HCI low power UART sample that we will compile and run on the nRF52840 (the one that’s on the nRF9160-DK board) before we work on the central code for the nRF9160.

1. Build the HCI low power UART

Note that this build command compiles code for the nRF52840 chip that is on the nRF9160-DK development board.

cd ~/golioth-ncs-workspace/nrf/samples/bluetooth/hci_lpuart
west build -b nrf9160dk_nrf52840 .

2. Move the Prog/Debug switch and flash

Locate the PROG/DEBUG switch on the nRF9160-DK and move it to the nRF52 position. This controls a mux that points at the Bluetooth chip. Use west to flash the firmware:

➜ west flash
-- west flash: rebuilding
ninja: no work to do.
-- west flash: using runner nrfjprog
Using board 960088581
-- runners.nrfjprog: Flashing file: /home/mike/golioth-compile/golioth-firmware-sdk/nrf/samples/bluetooth/hci_lpuart/build/zephyr/zephyr.hex
[ #################### ]   4.820s | Erase file - Done erasing
[ #################### ]   1.207s | Program file - Done programming
[ #################### ]   1.227s | Verify file - Done verifying
Enabling pin reset.
Applying pin reset.

When done, move the programming switch back to nRF9160 so it’s ready for the next step.

Load the central_ht sample on the nRF9160-DK

Zephyr includes a Health Temperature Service sample application which we’ll use as our hello world. Let’s compile and flash the “central” part of that sample.

1. Add board files to the central_ht sample code

Navigate to the central_ht sample code in the Zephry tree:

cd ~/golioth-ncs-workspace/zephyr/samples/bluetooth/central_ht/

We need to add a boards directory and create a conf file and and two overlay files for the nRF9160 in that directory. These configure the board to use the HCI low power UART firmware we flashed in the previous section as the Bluetooth radio (think of it as a secondary modem controlled by the application processor on the nRF9160).

In total we add 3 files:

nrf9160dk_nrf9160_ns.conf
nrf9160dk_nrf9160_ns.overlay
nrf9160dk_nrf9160_ns_0_14_0.overlay

# HCI low power UART
CONFIG_NRF_SW_LPUART=y
CONFIG_NRF_SW_LPUART_INT_DRIVEN=y

CONFIG_UART_2_ASYNC=y
CONFIG_UART_2_INTERRUPT_DRIVEN=n
CONFIG_UART_2_NRF_HW_ASYNC=y
CONFIG_UART_2_NRF_HW_ASYNC_TIMER=2

#include <nrf9160dk_nrf52840_reset_on_if5.dtsi>

/ {
    chosen {
        zephyr,bt-uart=&lpuart;
    };
};

&gpiote {
    interrupts = <49 NRF_DEFAULT_IRQ_PRIORITY>;
};

&uart2 {
    current-speed = <1000000>;
    status = "okay";
    /delete-property/ hw-flow-control;

    pinctrl-0 = <&uart2_default_alt>;
    pinctrl-1 = <&uart2_sleep_alt>;
    pinctrl-names = "default", "sleep";
    lpuart: nrf-sw-lpuart {
        compatible = "nordic,nrf-sw-lpuart";
        status = "okay";
        req-pin = <21>; /* <&interface_to_nrf52840 3 0>; */
        rdy-pin = <19>; /* <&interface_to_nrf52840 2 0>; */
    };
};

&pinctrl {
    uart2_default_alt: uart2_default_alt {
        group1 {
            psels = <NRF_PSEL(UART_TX, 0, 18)>,
                <NRF_PSEL(UART_RX, 0, 17)>;
        };
    };

    uart2_sleep_alt: uart2_sleep_alt {
        group1 {
            psels = <NRF_PSEL(UART_TX, 0, 18)>,
                <NRF_PSEL(UART_RX, 0, 17)>;
            low-power-enable;
        };
    };

};

It’s important to add this second overlay file that properly maps the reset line for newer nRF9160-DK boards:

/* Use the reset line that is available starting from v0.14.0 of the DK. */
#include <nrf9160dk_nrf52840_reset_on_if9.dtsi>

2. Build and flash the central_ht sample code

west build -b nrf9160dk_nrf9160_ns .
west flash

If you get an error when trying to flash the board, ensure you have the PROG/DEBUG switch in the nRF91 position.

The nRF9160-DK will immediately begin scanning for compatible Bluetooth human temperature services. Let’s set up one of those next.

Load the peripheral_ht sample on the nRF52840-DK

The previous steps complete the “central” part of the Bluetooth equation which will scan for available sensors. Now we need to create a “peripheral”, which is the sensor that will advertise itself and serve temperature readings from a sensor.

Building and flashing this sample code is very straight-forward:

cd ~/golioth-ncs-workspace/zephyr/samples/bluetooth/peripheral_ht/
west build -b nrf52840dk_nrf52840 .
west flash

Monitor the output

Connect to the nRF9160-DK over serial and you should see the device scan for a compatible peripheral, connect to it, and begin taking temperature readings:

*** Booting nRF Connect SDK v2.5.2 ***
Temperature 23.75C.
Connected: E9:E3:F1:6D:A9:87 (random)
[ATTRIBUTE] handle 25
[ATTRIBUTE] handle 26
[ATTRIBUTE] handle 28
[SUBSCRIBED]
Temperature 23.75C.
Temperature 23.75C.
Temperature 23.75C.
Temperature 23.75C.
Temperature 24C.

Next up, let’s connect Golioth and send these readings to the cloud!

Add Golioth to the project

To Golioth to this project we need three things:

Ensure Golioth is installed as a Zephyr module
Add Kconfig symbols to enable nRF9160 cellular and to add Golioth
Add a few API calls to the central_ht code to connect to Golioth and push the temperature reading to the cloud

In this section we’ll be working with the Zephyr central_ht sample so let’s switch to that directory:

cd ~/golioth-ncs-workspace/zephyr/samples/bluetooth/central_ht/

1. Add Golioth as a Zephyr Module

If you followed our getting started guide for NCS, this is already done. If not, you can follow the Adding the Golioth Firmware SDK to an Existing Zephyr West Project section of our SDK readme.

2. Add Kconfig symbols for cellular and Golioth

For these changes, we’ll crib a lot of code from Golioth’s LightDB Stream sample since we’re using that service to stream the temperature to the cloud.

Add the contents of the nRF9160-DK board file from the Golioth lightdb_stream sample to the board file you previously created in the central_ht Zephyr sample. Here’s what that file should look like now:

# General config
CONFIG_HEAP_MEM_POOL_SIZE=4096
CONFIG_NEWLIB_LIBC=y

# Networking
CONFIG_NET_SOCKETS_OFFLOAD=y
CONFIG_NET_IPV6=y
CONFIG_NET_IPV6_NBR_CACHE=n
CONFIG_NET_IPV6_MLD=n

# Increase native TLS socket implementation, so that it is chosen instead of
# offloaded nRF91 sockets
CONFIG_NET_SOCKETS_TLS_PRIORITY=35

# Modem library
CONFIG_NRF_MODEM_LIB=y
CONFIG_NRF_MODEM_LIB_ON_FAULT_APPLICATION_SPECIFIC=y

# LTE connectivity with network connection manager
CONFIG_LTE_CONNECTIVITY=y
CONFIG_NET_CONNECTION_MANAGER=y
CONFIG_NET_CONNECTION_MANAGER_MONITOR_STACK_SIZE=1024

# Increased sysworkq size, due to LTE connectivity
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048

# Disable options y-selected by NCS for no good reason
CONFIG_MBEDTLS_KEY_EXCHANGE_DHE_PSK_ENABLED=n
CONFIG_MBEDTLS_KEY_EXCHANGE_DHE_RSA_ENABLED=n

# Generate MCUboot compatible images
CONFIG_BOOTLOADER_MCUBOOT=y

# HCI low power UART
CONFIG_NRF_SW_LPUART=y
CONFIG_NRF_SW_LPUART_INT_DRIVEN=y

CONFIG_UART_2_ASYNC=y
CONFIG_UART_2_INTERRUPT_DRIVEN=n
CONFIG_UART_2_NRF_HW_ASYNC=y
CONFIG_UART_2_NRF_HW_ASYNC_TIMER=2

Now in the prj.conf file for the central_ht code sample, add the following Kconfig symbols. These have the effect of enabling the Golioth SDK, enabling the Stream service, and using runtime settings to store your device credentials in the storage partition using the Zephyr shell. The main stack size is also increased to account for some additional memory usage.

CONFIG_BT=y
CONFIG_LOG=y
CONFIG_BT_CENTRAL=y
CONFIG_BT_SMP=y
CONFIG_BT_GATT_CLIENT=y
CONFIG_CBPRINTF_FP_SUPPORT=y
# Golioth Firmware SDK
CONFIG_GOLIOTH_FIRMWARE_SDK=y

# Application
CONFIG_MAIN_STACK_SIZE=2048
CONFIG_GOLIOTH_SAMPLE_COMMON=y
CONFIG_LOG_BACKEND_GOLIOTH=y
CONFIG_GOLIOTH_SETTINGS=y
CONFIG_GOLIOTH_STREAM=y

CONFIG_GOLIOTH_SAMPLE_HARDCODED_CREDENTIALS=n

CONFIG_FLASH=y
CONFIG_FLASH_MAP=y
CONFIG_NVS=y

CONFIG_SHELL=y
CONFIG_SETTINGS=y
CONFIG_SETTINGS_RUNTIME=y
CONFIG_GOLIOTH_SAMPLE_PSK_SETTINGS=y
CONFIG_GOLIOTH_SAMPLE_SETTINGS_AUTOLOAD=y
CONFIG_GOLIOTH_SAMPLE_SETTINGS_SHELL=y

CONFIG_LOG=y
CONFIG_EVENTFD_MAX=14
CONFIG_LOG_PROCESS_THREAD_STACK_SIZE=1536
CONFIG_MBEDTLS_ENABLE_HEAP=y
CONFIG_MBEDTLS_HEAP_SIZE=10240
CONFIG_MBEDTLS_SSL_IN_CONTENT_LEN=2048
CONFIG_MBEDTLS_SSL_OUT_CONTENT_LEN=2048
CONFIG_NETWORKING=y
CONFIG_NET_IPV4=y
CONFIG_POSIX_MAX_FDS=23

3. Add Golioth API calls to main.c

With all the configuration in place, we’re now ready to update main.c to connect to Golioth and push temperature readings to the cloud.

First, add some includes, create a semaphore, and add a callback near the top of main.c:

#include <golioth/client.h>
#include <golioth/stream.h>
#include <samples/common/net_connect.h>
#include <samples/common/sample_credentials.h>

static struct golioth_client *client;
static K_SEM_DEFINE(golioth_connected, 0, 1);

static void on_client_event(struct golioth_client *client, enum golioth_client_event event,
                void *arg)
{
    bool is_connected = (event == GOLIOTH_CLIENT_EVENT_CONNECTED);
    if (is_connected) {
        k_sem_give(&golioth_connected);
    }
    printk("Golioth client %s\n", is_connected ? "connected" : "disconnected");
}

Next, in the notify_func() function, add an API call to send data to Golioth:

char sbuf[32];
snprintk(sbuf, sizeof(sbuf), "{\"temperature\":%g}", temperature);
printf("Sending to Golioth: %s\n", sbuf);


int err = golioth_stream_set_async(client,
                   "sensor",
                   GOLIOTH_CONTENT_TYPE_JSON,
                   sbuf,
                   strlen(sbuf),
                   NULL,
                   NULL);
if (err) {
    printf("Failed to push temperature: %d\n", err);
}

Finally, in main() add code to start the Golioth client connection:

char sbuf[32];
snprintk(sbuf, sizeof(sbuf), "{\"temperature\":%g}", temperature);
printf("Sending to Golioth: %s\n", sbuf);


int err = golioth_stream_set_async(client,
                   "sensor",
                   GOLIOTH_CONTENT_TYPE_JSON,
                   sbuf,
                   strlen(sbuf),
                   NULL,
                   NULL);
if (err) {
    printf("Failed to push temperature: %d\n", err);
}

Running the demo and viewing data on the cloud

The first time you run the nRF9160-DK you need to add device credentials. Open a serial terminal to the device and use the shell to issue the following commands:

uart:~$ settings set golioth/psk-id my-psk-id@my-project
uart:~$ settings set golioth/psk my-psk
uart:~$ kernel reboot warm

With both of our boards programmed and powered on, we can view the terminal output of the nRF9160-DK to see the connection and scanning process:

*** Booting nRF Connect SDK v2.5.2 ***                                                                                                                                                         
[00:00:00.465,972] <inf> fs_nvs: 2 Sectors of 4096 bytes                                                                                                                                       
[00:00:00.466,003] <inf> fs_nvs: alloc wra: 0, fb8                                                                                                                                             
[00:00:00.466,003] <inf> fs_nvs: data wra: 0, 68                                                                                                                                               
[00:00:00.466,278] <inf> golioth_samples: Bringing up network interface                                                                                                                        
[00:00:00.466,308] <inf> golioth_samples: Waiting to obtain IP address                                                                                                                         
[00:00:02.545,684] <inf> lte_monitor: Network: Searching                                                                                                                                       
[00:00:05.688,049] <inf> lte_monitor: Network: Registered (roaming)                                                                                                                            
[00:00:05.689,117] <inf> golioth_mbox: Mbox created, bufsize: 1232, num_items: 10, item_size: 112                                                                                              
[00:00:07.861,846] <inf> golioth_coap_client_zephyr: Golioth CoAP client connected                                                                                                             
Golioth client connected                                                                                                                                                                       
[00:00:07.862,365] <inf> golioth_coap_client_zephyr: Entering CoAP I/O loop                                                                                                                    
[00:00:08.559,051] <wrn> bt_hci_core: opcode 0x0000 pool id 5 pool 0x2000d430 != &hci_cmd_pool 0x2000d488                                                                                      
[00:00:08.600,585] <inf> bt_hci_core: HW Platform: Nordic Semiconductor (0x0002)                                                                                                               
[00:00:08.600,616] <inf> bt_hci_core: HW Variant: nRF52x (0x0002)                                                                                                                              
[00:00:08.600,646] <inf> bt_hci_core: Firmware: Standard Bluetooth controller (0x00) Version 141.732 Build 3324398027                                                                          
[00:00:08.610,504] <inf> bt_hci_core: Identity: E1:99:6F:78:C4:C4 (random)                                                                                                                     
[00:00:08.610,534] <inf> bt_hci_core: HCI: version 5.4 (0x0d) revision 0x1168, manufacturer 0x0059                                                                                             
[00:00:08.610,565] <inf> bt_hci_core: LMP: version 5.4 (0x0d) subver 0x1168                                                                                                                    
Bluetooth initialized                                                                                                                                                                          
Scanning successfully started                                                                                                                                                                  
[DEVICE]: F4:BC:DA:35:5E:68 (public), AD evt type 0, AD data len 27, RSSI -85                                                                                                                  
[AD]: 1 data_len 1                                                                                                                                                                             
[AD]: 9 data_len 12                                                                                                                                                                            
[AD]: 255 data_len 8                                                                                                                                                                           
[DEVICE]: F4:BC:DA:35:5E:68 (public), AD evt type 4, AD data len 31, RSSI -85                                                                                                                  
[DEVICE]: F0:5F:8B:2D:1C:A2 (random), AD evt type 0, AD data len 20, RSSI -82                                                                                                                  
[AD]: 9 data_len 5                                                                                                                                                                             
[AD]: 25 data_len 2                                                                                                                                                                            
[AD]: 1 data_len 1                                                                                                                                                                             
[AD]: 2 data_len 4                                                                                                                                                                             
[DEVICE]: F0:5F:8B:2D:1C:A2 (random), AD evt type 4, AD data len 0, RSSI -81                                                                                                                   
[DEVICE]: E9:E3:F1:6D:A9:87 (random), AD evt type 0, AD data len 11, RSSI -29                                                                                                                  
[AD]: 1 data_len 1                                                                                                                                                                             
[AD]: 3 data_len 6                                                                                                                                                                             
uart:~$ Temperature 21.5C.                                                                                                                                                                     
Sending to Golioth: {"temperature":21.5}                                                                                                                                                       
Connected: E9:E3:F1:6D:A9:87 (random)                                                                                                                                                          
[ATTRIBUTE] handle 25                                                                                                                                                                          
[ATTRIBUTE] handle 26                                                                                                                                                                          
[ATTRIBUTE] handle 28                                                                                                                                                                          
[SUBSCRIBED]                                                                                                                                                                                   
uart:~$ Temperature 21.5C.                                                                                                                                                                     
Sending to Golioth: {"temperature":21.5}                                                                                                                                                       
Temperature 21.25C.                                                                                                                                                                            
Sending to Golioth: {"temperature":21.25}                                                                                                                                                      
Temperature 21.5C.                                                                                                                                                                             
Sending to Golioth: {"temperature":21.5}                                                                                                                                                       
Temperature 21.25C.                                                                                                                                                                            
Sending to Golioth: {"temperature":21.25}

We see the chip boot, connect to the cell network, then connect to Golioth. After that, the Bluetooth scan begins, connecting to devices it finds to query for the desired health temperature service (HTS). Once a service is found, we see temperature readings that are then pushed to the cloud.

Checking on the LightDB Stream tab in the device view of the Golioth web console shows the data arriving on the cloud!

Connecting Bluetooth to the Cloud

This lays the groundwork for connecting your Bluetooth devices to the cloud. One gateway (or a relatively small number of them) can service multiple BLE peripheral devices for both read and write activities. It’s even possible to update firmware over a Bluetooth connection. But that’s a post for a different day.

Give Golioth a try, it’s free for individuals! If you have a need for a more involved Bluetooth to Cellular gateway, we’d love to hear about it. Reach out to the Golioth DevRel team.

If you want to dive into the code from this post, start with the Zephyr central_ht sample and apply the file changes found in this gist.

April 23, 2024/by Mike Szczys

Demo, Event, Zephyr

Demos we’ll have at Embedded World 2024

We are returning to Embedded World in 2024 (EW24) and will be showing off the things we’ve been working on at Golioth since last year. We’re also living a little closer to the edge and bringing some experiments and unreleased items to showcase on the floor. Demo gods, be kind!

We’ll be at the Zephyr booth, helping to showcase one of our favorite Real Time Operating Systems and Ecosystems (though not the only one we support!). Golioth generally only targets one part in our standard hardware setup, but we continuously verify a bunch of hardware on our Firmware SDK. This means that you can be certain that every new feature added to the Golioth Cloud and SDK is fully tested throughout our supported hardware. The great thing about Zephyr is that the underlying hardware works very similarly to all of our CVBs, because of the work from the silicon vendors that participate in the open source project. We’re excited because 2024 saw even more silicon vendors joining the fray!

Golioth Demos

Being part of a large project like Zephyr, we share time at the booth Kiosks. We’ll be at the Zephyr Booth (4-170) at the following times:

Tuesday, April 9th from 1500 to 1800
Wednesday, April 10th from 1200 to 1500
Thursday, April 11th from 0900 to 1130

All times CEST, which is GMT + 1

Hardware we’ll have on hand

Modbus Vibration monitor – This is our newest Reference Design that captures vibration and temperature data from a sensor normally used in industrial environments for large motors. The hardware in the Reference Design talks over RS-485 to the remote sensor and queries all of the available registers and publishes them to the Golioth cloud for processing and viewing.
Air Quality Monitor – It’s always interesting to see the trend lines of air quality metrics throughout the day. There are a lot of people that filter in and through the Nuremberg convention center, so we can track things like CO2 concentration just from the elevated number of bodies. The particulate counter is unlikely to go up a lot, but if there’s something like a cotton candy booth nearby, it could potentially have particulates flying through the air. I’ll be sure to go and investigate and clean up that particular air quality problem if the need arises.
Aludel Elixir – Powering our newer reference designs is the Aludel Elixir. This is an evolution of our previous designs (the Aludel Mini) that stitched together outside development boards that we loved like the Circuit Dojo nRF9160 Feather and the Sparkfun ThingPlus nRF9160. We pulled many of those components onto a bespoke PCB, and also added other components that can enhance any reference design. It all comes together in a custom milled case that enables maximum flexibility.

Aludel Elixir Rev B

Are you interested in your very own Aludel? Shoot me a note to discuss at Embedded World or fill out this form

Partner Demos:

AL2LOG at the AL2TECH booth (3A-335)

A render of the AL2LOG Unit

Our design partner AL2TECH based out of Italy is bringing an exciting new logging project based on the nRF9160 and an STM32 coprocessor. This is meant to target industrial applications that require a flexible way to capture data in a harsh environment.

The AL2LOG is a compact industrial logger with different sensor and actuator interfaces. This is based upon AL2TECH’s broad experience serving their clients in the energy meter/remote sensor monitoring industries.

The AL2LOG PCB (rev 1.0)

Some high level features include Cellular and GPS connectivity (nRF9160), as well as external module comms using USB-C, RS-485, and CAN. There is an Ultra Low Power acquisition and logging sub system (STM32) with a true 14 bit ADC. Industrial inputs including Digital Input, Dry Contact, Pulse Counter, 0-10V Voltage Analog Input, and 4-20mA Current Loop Input. There are outputs like Open Drain Output, and programmable supplies for devices downrange of the logger. The device can also switch external 220V via a relay. For powering the device you can use a 12-24V rail, or depend on the large D-Size High Power LTC Cells.

The device communicates back to Golioth using Zephyr/NCS and can be remotely updated using Golioth’s OTA service. Data routed from the field back to the cloud is accessible via the Golioth REST API.

BT Mesh demo from partner Ambient Sensors at the Zephyr booth (4-170)

BLE Mesh Demo from Ambient Sensors communicating through Golioth, on display at Embedded World

This demo shows Golioth controlling an LTE/BLE bridge (Thingy91) that communicates to a BLE Mesh composed of a series of lights that are running the Light Lightness Controller (the LC Server) model. Eight instances of the BBC micro:bit represent lights in this demo. The boards run BLE Mesh firmware on Nordic Semiconductor’s nRF52833, which could easily be controlling an LED driver instead of the demo LEDs.

The BLE SIG standard LC server code is running on each microbit, and is communicating with the Thingy91’s nRF52840, which communicates on the mesh as an LC client, Generic OnOff Client, and Light Lightness client. Using Golioth, the demo remotely sends a command to turn the LC server on, and lights will then slowly dim over time (as is the functionality of the LC server). Normally lights dim after several hours, but for the purpose of the demo, we set that value to several seconds. Delay values may also be set via Golioth, showing how these settings can be tuned remotely if desired. Additionally, we have created commands to allow the Golioth user to turn on the lights, while turning off the LC Server (to prevent it from dimming the lights), and to set the lightness value to any desired value.

Golioth connectivity demo at the NXP FRDM Lab

Find Golioth at the FRDM Lab at EW24

NXP is another vendor that has great support in Zephyr. We’ll be showcasing Golioth’s solution on NXP boards at their FRDM Lab in the Messepark classroom (the main open area when you walk into the conference). You can also see all of NXPs new solutions at their main booth in the presentation hall, booth 4A-222.

Even more Golioth!

What’s that? You haven’t seen enough of us yet? You’d love to hear more on-site at Embedded World? Well let’s telegraph every place we’ll be! In addition to our time showcasing Golioth at the Zephyr booth, we’ll also be giving a couple of talks at the Zephyr stage (next to the Zephyr booth):

April 9th at 1400: “Building end-to-end Zephyr demos on IoT hardware”
April 10th at 1530: “Multi MCU OTA updates with Golioth and Simple Management Protocol”
April 11th at 1200: “Local toolchains, no more! Using GitHub Codespaces for training people on Zephyr throughout the world”

In the evenings (if we’re still standing), we plan to attend these open events and socialize with others in the industry:

IoT Stars – This is a short speaker event, as well as a networking session. We’ll have some of our portable demos with us.
PCBarts meetup – This is a more social event hosted by a local consulting and manufacturing company. Last year this was a wonderful group of engineers showing off their projects–personal and professional–after a long day at the tradeshow.

We really hope to see you at the conference! If you’d like to talk about Golioth and how we can work together, please fill out this form!

March 21, 2024/by Chris Gammell

Demo, Device, Zephyr

Beyond Arduino: Prototyping with Zephyr and Golioth

Timon Skerutsch is a software and electronics engineer who enjoys the systems design aspect of the work the most, from the cloud all the way down to silicon. He is the founder of the product development consultancy Diodes Delight.

Last year, I made the decision to do a personal prototype project with Zephyr instead of my go-to prototyping choices of Arduino, MicroPython, and CircuitPython. The latter two I still use a lot for rapid prototyping, as scripting languages like Python are hard to beat.

For the forthcoming project, I already knew where I was heading. I needed a sensor for my garden irrigation system that tracks the water level in my water reservoir. I had two requirements:

See the water level live on-site
Have the data transmitted to the internet so that I can see how things are doing while I’m away

The system regularly had leaks and failures in the irrigation pipes, which caused the automated watering pump to not only waste water but also fail to water my plants. This is detrimental during a heat wave! That added an additional requirement of an alarm to alert me of abnormal situations. When I started the project, I reached for Zephyr first. Let’s talk about why.

Why Arduino, anyway?

Change is a constant; no more so, than the early design phases of a product, when requirements rapidly shift. Prototyping is a crucial aspect of any development process and we often employ different tools for this than we do for our production firmware.

In the firmware world, the Arduino framework has been an immense success. Not only for teaching firmware development, but also as a quick and easy way to try out concepts and develop prototype solutions.

Not without controversy though. To this day, many embedded engineers will utter a silent curse on Arduino whenever it is encountered on the way to production. The supposed prototype firmware has morphed into the production firmware and now that needs to be extended with very complex functionality.

When deadlines are near and the stress levels are high, it is very appealing to companies to just keep using the code base that already seems to do 90% of the job. A lot of products end up shipping firmware based on frameworks like Arduino and then try to deal with the “last 10% of the work”. If you have developed software for a while you probably know those famous last 10% can be 90% of the work, completely derailing budgets and timelines.

But what is the issue? At the end of the day, Arduino is just a very light HAL based on C and C++! You can do whatever you could do in a bare metal C project. The emphasis is on “light”, which is very beneficial when you have only very basic requirements. Maybe you can get by just fine with printf() debugging and hitting that compile button in your IDE.

The Arduino HAL’s success was in part due to its simplicity and fully integrated development flow but it offers very few solutions to the modern challenges firmware developers are facing. When you consider the complexities you are faced with in modern devices, Over-the-Air firmware updates, firmware update encryption, Continuous Integration, connection to a cloud service, tracking of device status and metrics, then things start to look different.

Zephyr is changing things up in the industry

The Zephyr Project Real Time Operating System (RTOS) has seen a lot of adoption over the past years, even by Arduino themselves. They became a member of the Zephyr Project in 2023 and now contribute to the code base.

Zephyr is a very “IoT aware” RTOS and offers a lot of robust solutions to many of the very complex topics I mentioned. It is also one of the main targets of the Golioth Firmware SDK for that reason.

When developers first start interacting with Zephyr they often tend to be a bit intimidated. Device Tree and KConfig may be familiar tools for Embedded Linux developers, but not for someone coming from bare metal C or FreeRTOS. (Editor’s note: this is why Golioth offers free training around Zephyr).

Zephyr-specific tooling like the west meta tool means there’s a lot to learn when you start diving into Zephyr. You might start to question if that work is worth it. Especially early on in a development process where you want to move quickly and prove your concepts. You might feel a huge system like Zephyr could slow you down.

Due to the steep learning curve, Zephyr does not really have a reputation for being a tool for prototyping. But I think Zephyr has very much a place in that phase of a project and it comes with a lot of benefits once you move beyond the prototype: You are already in an environment that won’t hold you back when it comes to solving the tough problems of modern production ready firmware.

Now that I am up the (arguably steep) learning curve associated with Zephyr, I think in many cases I can produce a working solution a lot quicker than with Arduino or even MicroPython.

Not just for production grade firmware

Since I was starting a prototype, I chose an ESP32 dev board I had laying around which came with a nice little OLED screen. For the sensor, I opted for an industrial liquid level sensor. They are essentially pressure sensors in a sturdy form factor that measure the pressure differential of the outside air and the pressure seen in the liquid container.

I needed something rugged and precise to track abnormal water usage so that was a perfect solution. I ended up getting a stable 0.2mm resolution for my water column, much more than I needed. The sensor is a simple 4-20mA current loop that you often see in industrial automation and I connected that to an external precision ADC.

My firmware needs included:

WiFi provisioning
Network and application protocol to get the data to a server
OTA to update the device remotely
A GUI to show water levels on the OLED
ADC reads to ingest the sensor data

I opted to use Golioth for networking and OTA. While not a typical service for a hobby project, they have a (recently updated) free tier for individuals and it made the whole thing really easy. It only takes a couple lines of code to integrate into any Zephyr project and makes transmitting data to a database as easy as Serial.print(). Having OTA available is a matter of a KConfig option. Most importantly I don’t need to manage an internet facing server application!

net_connect();

golioth_client_t client;
const golioth_client_config_t* client_config = golioth_sample_credentials_get();
client = golioth_client_create(client_config);
golioth_client_register_event_callback(client, on_client_event, NULL);

err = golioth_lightdb_set_int_sync(client, "water-level", water_level/1000, 2);
if (err) {
    LOG_WRN("Failed to transmit water-level: %d", err);
}

I could have directly implemented CoAP or MQTT and host my own server for the receiving side. Both protocols are natively supported by Zephyr, which means I have flexibility if I change my mind on the server side in the future.

OTA firmware updates is also a concept native to Zephyr and very important: no matter what platform I choose! The Arduino ESP32 core has an option for OTA but if you are looking at any other MCU you would have to implement that from scratch which is a whole project in itself.
In Zephyr this is all enabled by the fantastic MCUBoot bootloader.

Abstractions are your friend in a complex world

The platform agnostic nature of Zephyr is powerful. Say you have already written a lot of code and then notice that your chosen MCU does not actually fulfill your needs. You don’t need to start from scratch because all of these advanced APIs are fully abstracted in Zephyr. You can retarget your code to a different platform with minimal code changes. The primary work will be in recreating your devicetree for the new hardware.

Arduino also is known for abstraction, but when it comes to more complex features the Arduino HAL is not defining an interface. Generally, you tend to need to stick to a particular platform if you want to take advantage of the underlying hardware’s fancy features, that is if they are available at all (in Arduino).

Lock-in with a specific IC is a painful lesson we all learned during the chip shortages of 2021 and 2022. Device Tree overlays are a great tool to stay on-top of changing hardware and describe those changes in a clean way. That flexibility is not only important from a risk perspective. Staying flexible during the prototype stage (where requirements change rapidly) allows you to try out different sensors and peripherals.

Changing a sensor in Zephyr is a matter of changing a dozen lines of devicetree definitions without needing to touch a single line of C code. This is made easier when the sensor is “in tree”, but it not the only way to use a new sensor. Devicetree also becomes a powerful tool in the early days of hardware development where your product might go through many revisions and changes. People on your team might be working with different hardware revisions but require the latest bug fixes.

This can quickly become tough to manage in the firmware if you had pin or even IC changes. No need for a ton of #ifdef‘s; all you need is a set of devicetree overlays that describe your various board revisions, the C code can most often stay the same. This not only makes your life easier but also helps reduce mistakes and stale code.

If you are still trying out options you can also interactively work with sensors through the Zephyr Shell, which makes for a great workflow to quickly try out several sensor candidates without writing firmware.

During my project I was unsure whether to choose a different MCU, because the built-in ADC of the ESP32 is quite noisy. In the end, I kept the ESP32 and chose to use an external ADC. My code did not have to change because the ADC API abstracts that away. It was just a matter of defining what ADC my project should use in the devicetree. My code does not need to care if that is an external I2C device or a peripheral internal to my MCU.

/{
    zephyr,user {
        io-channels =
            <&adc_ext 0>;
    };
};

&i2c0 {
    status = "okay";
    adc_ext: mcp3421@68 {
        compatible = "microchip,mcp3421";
        reg = <0x68>;
        #io-channel-cells = <1>;
    };
};

&adc_ext {
    status = "okay";
    channel@0 {
        reg = <0>;
        zephyr,gain = "ADC_GAIN_1";
        zephyr,reference = "ADC_REF_INTERNAL";
        zephyr,acquisition-time = <ADC_ACQ_TIME_DEFAULT>;
        zephyr,resolution = <18>;
        zephyr,differential;
    };
};

That is the benefit of fully abstracted subsystems, your application’s assumptions can stay the same most of the time. Last minute system changes are less painful during firmware development.

Complex UI’s don’t have to be complex to build

For the GUI, I went with LVGL, a popular UI framework that has been integrated into Zephyr.
That was probably the most eye opening experience to me. Normally you would have to mess with display drivers that all work very differently depending on the plugged in display. Then I would need to write code to manually transfer the rendered framebuffer to that display, which again, tends to work differently with each display.

In Zephyr all I have to do is to modify the devicetree for which display driver my OLED needs, the resolution, and the bus it is connected to.

&spi3 {
    status = "okay";
    st7789v_st7789v_ttgo_128x64: st7789v@0 {
        compatible = "sitronix,st7789v";
        spi-max-frequency = <20000000>;
        reg = <0>;
        cmd-data-gpios = <&gpio0 16 GPIO_ACTIVE_LOW>;
        reset-gpios = <&gpio0 23 GPIO_ACTIVE_LOW>;
        width = <135>;
        height = <240>;
        x-offset = <53>;
        y-offset = <40>;
        vcom = <0x19>;
        rgb-param = [CD 08 14];
    };
};

With that done you can write powerful UIs with just a couple lines of code, all the hard stuff is handled behind the scenes. I figured the UI part would be the majority of work for this project, but ended up being done in under an hour.

Often Arduino prototypes tend to have character displays or use the same old school bitmap font because fonts are hard and font systems even harder. In LVGL, you have an array of modern fonts available and it’s fairly easy to include your own font.

Arranging elements is also trivial in LVGL. No need to manually calculate a bunch of stuff like your text length. It has a lot of functions available for laying out complex arrangements.
You can build some really pretty smartphone level UIs with it. These run on very constrained hardware and it doesn’t cost you your sanity in the process!

const struct device *display_dev;
display_dev = DEVICE_DT_GET(DT_CHOSEN(zephyr_display));

lv_obj_t *level_label;
lv_obj_t *status_label;
static lv_style_t level_style;
lv_style_init(&level_style);
lv_style_set_text_font(&level_style, &lv_font_montserrat_48);

level_label = lv_label_create(lv_scr_act());
lv_obj_align(level_label, LV_ALIGN_CENTER, 0, -20);

status_label = lv_label_create(lv_scr_act());
lv_obj_align(status_label, LV_ALIGN_BOTTOM_MID, 0, -50);
lv_obj_add_style(level_label, &level_style, 0);
// display the current water level
lv_label_set_text_fmt(level_label, "%llu", water_level/1000U);
lv_task_handler();

Quick, once you know how to get around

Within a day I had firmware for my hardware and it even looked…pretty!
Since its creation, my device has dutifully reported the water level and withstood the winter season.

The application is ~180 lines of C code, including a lot of error handling. That is really not a lot of code, for so much complex functionality. This is only possible thanks to all of the available abstractions that make writing the actual application logic a breeze. In the background there are multiple threads running but my code doesn’t even need to be aware of that. It is all handled by the kernel and the well-written libraries.

While simple things like an ADC read can be very verbose in Zephyr and a bit more complicated than in Arduino land, the hard parts–the things we often perceive as the last 10%–are a whole lot easier! You don’t want to start empty handed when you are tasked to implement encrypted firmware updates, OTA, telemetry or integrating the cloud team’s new backend system.

The grass is not always greener, of course. Zephyr is still comparatively young in embedded terms and while there are a lot of devices already supported, it is hard to beat the vast amount of drivers available for Arduino. In some situations you might want to do a first board bring up with an existing Arduino driver before implementing a Zephyr driver. License permitting, the driver code could be the basis for a port to Zephyr.

One reason why I think it is so common for firmware to get “stuck” in Arduino-land is because there has not been a good (free) alternative if you wanted to write C (or C++) and not pigeon hole yourself early on into one specific obscure vendor toolchain. If there is nothing obvious to move to, it can make the decision even harder and it will be put off until it’s usually too late.

Concepts like devicetree and a complex build system like CMake can be daunting at first, but there are ripe benefits at the end of the learning curve. If you want to learn Zephyr, Golioth regularly offers free Zephyr training sessions or you can read some of the great blog posts that cover the more gnarly bits of Zephyr in digestible bite sized portions.

March 12, 2024/by Timon Skerutsch

Demo

We Added Cellular to the Hackaday Superconference Badge

The Hackaday Superconference returned for the second post-lockdown year. This was actually the seventh “Supercon” and Dan and Mike were on hand to represent Golioth. As badge-hacking is a large part of the social scene at the conference, we spent a fair amount of time getting data from the non-connected badge up to the cloud. What started out as a test-equipment related project ended up as a community art project.

Conference Admission with a Decades-Old Bench Tool

If you read the Golioth blog, chances are you already know about Supercon. But if not, it’s a conference aimed at people building and working with electronic hardware. Fittingly, instead of a printed plastic rectangle on a lanyard, the Supercon badge is itself an electronic device. This year it was modelled after a Vectorscope; a type of oscilloscope that plots two voltage signals along X and Y axes, instead of plotting a signal in the time domain.

Hackaday Supercon badge displaying the Golioth logo

We had high ambitions for our badge hack. The original plan was to cache the captured voltages being measured by the badge’s ADCs and send them up to the cloud so they could be replayed later. This was a pretty interesting idea, since every badge has a set of inputs and outputs. We could run custom code on the inputs of our badge and capture data from the unaltered badges of people we ran into. We could potentially catch the vector source by using jumper wires to their output pins.

Animated GIF of the badge showing a vector trace

Vectorscope demonstration. Image credit: Hackaday

Alas, the badge creators did a great job of squeezing impressive performance out of the RP2040-based hardware. A tight chain of DMA and PIO (programmable input-output) kept the pipeline of samples out of the processor. We were able to capture some lossy data, and we think we could have tapped into a data stream on the order of 50 ksps, but then RAM becomes an issue to cache that kind of throughput. This meant our original plan was not to be.

Luckily, Dan noticed that a “sketch” app was included on the badge. A static image is simpler to offload to the cloud and render on a gallery page. We had some of the Golioth Aludel-mini hardware with us that uses an nRF9160 cellular modem. So we set off to make it happen.

Badgecase: the Badge Showcase

The impatient reader can head over to Badgecase.io and see the results of our badge hacking. All of the art on that page was entered on a conference badge and uploaded over cellular.

The Gist of the Hack

The stock badges run Micropython. We compiled a custom version so that we could implement i2c peripheral mode on the badge using the RP2040 hardware peripheral. This makes it look like a sensor; Golioth is great at harvesting sensor data and sending it to the cloud. We grab the image data (ignoring all white background pixels), sliced it up into i2c packets, then combined those into CBOR packets and upload them to LightDB Stream.

This is where the cloud side of things takes over. A frontend written in Rust uses a Websocket listener to react to incoming packets. It queries the Golioth cloud for the unique ID of each image, collects all of the blocks that shared the same UID, and reassembles them into PNG files. To add a maker’s mark, a Golioth remote procedure call (RPC) was used just prior to upload to add a title/name to each piece of artwork. All is hosted on a dynamic page which we will convert to a static entry for posterity. However, we had enough fun with this that we may do more of these shenanigans using different subdomains in the future, so watch this space.

The Firmware

Alterations to the actual Micropython apps running on the badge were minimal, the majority of the work came in the i2c peripheral functions as they chopped up data into packets and handled incoming requests from the i2c controller.

def encode_point_for_upload(x, y, color):
    x_high = (x << 10) & 0b1111110000000000
    y_mid = (y << 4) & 0b0000001111110000
    coord = x_high | y_mid
    if color == gc9a01.RED:
        coord |= 1
    elif color == gc9a01.GREEN:
        coord |= 2
    elif color == gc9a01.BLUE:
        coord |= 3
    elif color == gc9a01.BLACK:
        coord |= 4
    else:
        coord |= 0

    print(coord >> 8, coord & 0x00FF)
    return (coord & 0x00FF, coord >> 8)

def menu(key):       # exit and return to menu
    ostentus_i2c.fifo_init_samples()
    for i,m in enumerate(model):
        for j,n in enumerate(m):
            if n != gc9a01.WHITE:
                print(i,j,n)
                coord, col = encode_point_for_upload(i, j, n)
                ostentus_i2c.fifo_put_point(coord, col)
    ostentus_i2c.fifo_finalize_samples()

    while(ostentus_i2c.outgoing_data_available()):
        time.sleep(1)



    global stopflag
    print("menu")
    if vos_state.active:
        stopflag=True
        joy.detach()
        btn.detach()
        csel.detach()

The highlighted lines above are what was added to the stock Vectorscope sketch app. It works by capturing program flow when the user presses the “menu” button, which signals an exit from the app. The fifo_put_point(coord, col) and related functions are calling C code which was marshalled up to the Micropython layer (more on this in a sec).

The sketch app uses a 40×40 grid with five colors (white, black, red, green, blue). We assume all pixels are white and only upload pixels of a different color. Some run-length encoding could have made things a bit leaner, but a simple approach to bit-packing worked for us.

Most of the firmware work went into the C layer and custom Micropython build. The build itself is… shall we say “bespoke”? (ie. “it’s a hack!”) It was not really in a state to publish to a repo. The custom i2c stuff is an extension of the work we already did on the Golioth Ostentus faceplate. We’ll be publishing that project publicly in a few months, so for now, here’s a Gist of the pertinent code for those who are curious.

For Supercon, the i2c files implement a FIFO, into which data from the Micropython layer may be placed. Two i2c register addresses were added, one indicating data is available, the other will send the data. It forms 36-byte packets, with the first packet containing metadata.

The Software

The Rust backend leveraged Tokio, an asynchronous runtime, and its sibling web server framework, Axum. At startup, a thread was spawned to connect to Golioth over Websockets and listen for new messages. Upon receiving a message, this thread checks whether the data was fragmented by comparing the provided total point count with the number of points in the message. If so, it waits for subsequent messages until all points had been acquired for a given image.

Given the time constraints, we wanted to make the application as lean as possible to simplify connecting to external services when deploying. The information we needed to persist after processing an image included the image itself and the name that had been provided via the previously mentioned RPC. For the image data, a blob storage service was a natural solution.

As a hack around introducing another data store to maintain a list of the images, their links, and their names, we encoded the name and UID into the name of the image file in blob storage bucket. When a request arrives for badgecase.io, the server fetches a list of objects in the buckets, parses the file names to extract the user-chosen image name, and renders an HTML page from a handlebars template using the names and links. Some lightweight server-side caching reduced the overhead of accessing the bucket on every request, and the cache can easily be invalidated upon processing a new image, meaning that new images are displayed in the showcase just moments after submission.

The popular image crate made building the PNG from the encoded 16-bit coordinate points fairly straightforward.

fn build_png(data: Vec<i16>) -> image::ImageBuffer<image::Rgb<u8>, Vec<u8>> {
    let mut buffer = image::DynamicImage::new_rgb8(40, 40).to_rgb8();

    let mut state = [[PixelColor::White; 40]; 40];
    for p in data {
        let (x, y, color) = parse_point(p);
        state[x as usize][y as usize] = color;
    }

    for x in 0..40 {
        for y in 0..40 {
            let pixel = buffer.get_pixel_mut(x, y);
            *pixel = image::Rgb(state[x as usize][y as usize].rgb().into());
        }
    }

    imageops::resize(&buffer, 400, 400, imageops::FilterType::Nearest)
}

Parsing the points was essentially the inverse of the operation in the firmware, but Rust’s enum support made working with our custom color scheme much more enjoyable.

#[derive(Clone, Copy, Debug)]
pub enum PixelColor {
    Black,
    White,
    Red,
    Blue,
    Green,
}

impl PixelColor {
    fn new(i: u8) -> PixelColor {
        match i {
            1 => PixelColor::Red,
            2 => PixelColor::Green,
            3 => PixelColor::Blue,
            4 => PixelColor::Black,
            _ => PixelColor::White,
        }
    }

    fn rgb(&self) -> (u8, u8, u8) {
        match *self {
            PixelColor::Black => (0, 0, 0),
            PixelColor::White => (255, 255, 255),
            PixelColor::Red => (255, 0, 0),
            PixelColor::Blue => (0, 0, 255),
            PixelColor::Green => (0, 255, 0),
        }
    }
}

fn parse_point(point: i16) -> (u8, u8, PixelColor) {
    (
        (point >> 10) as u8 & 0b00111111,
        ((point >> 4) as u8 & 0b00111111),
        PixelColor::new(point as u8 & 0b00001111),
    )
}

Deploying the application involved building an OCI image and pushing it up to Google Artifact Registry, then with a few clicks we had it running on Google Cloud Run. Because we had chosen Google Cloud Storage for storing the images, setting up access from the Cloud Run service was seamless. The last bit was mapping the Cloudflare-managed domain to the service because what is a hack project without a proper catchy domain?

Closing Thoughts

We spent way too much time on this hack… it was so much fun! Although our initial dream of capturing the vector traces didn’t materialize, it was still a blast seeing people make a simple drawing and having it appear almost instantly on the website. It also highlights how much stuff there can be in the process from a cellular device up to Cloud, and how Golioth makes things even easier. We only just got back home and already we can’t wait to see what happens with next year’s badge!

November 28, 2023/by Mike Szczys

What is Batch Data?

Sending Batch Data from an IoT Device

Routing Batch Data using a Golioth Pipeline

Batch Data with Timestamp Extract

Using a Special Path for Batch Data

Sample Firmware

Wrapping Up

Thread Recap

Built it yourself

Hardware

Border Router

Thread Node

Firmware

OTA Updates

Connecting to Golioth Cloud

Learn more

Hardware

Golioth Pipeline

n8n

API service

Logic and alerts

Future demos

The Device Settings

The Remote Procedure Calls

How Does the Code Know about Changes to the Device Settings?

How Does the Code Know About RPCs?

Passing commands between nRF9160 and nRF52840

How does the BLE chip (the nRF52840) send Mesh commands?

And then the micro:bits?

The full signal chain, described

Controlling from afar

How it works

Other applications

The basics of the NMEA standard

NMEA message structure

Zephyr GNSS API

GNSS example in Zephyr

KConfig Changes

Devicetree Changes

Final thoughts

HCI low power UART

1. Build the HCI low power UART

2. Move the Prog/Debug switch and flash

Load the central_ht sample on the nRF9160-DK

1. Add board files to the central_ht sample code

2. Build and flash the central_ht sample code

Load the peripheral_ht sample on the nRF52840-DK

Monitor the output

Add Golioth to the project

1. Add Golioth as a Zephyr Module

2. Add Kconfig symbols for cellular and Golioth

3. Add Golioth API calls to main.c

Running the demo and viewing data on the cloud

Connecting Bluetooth to the Cloud

Golioth Demos

Hardware we’ll have on hand

Partner Demos:

AL2LOG at the AL2TECH booth (3A-335)

BT Mesh demo from partner Ambient Sensors at the Zephyr booth (4-170)

Golioth connectivity demo at the NXP FRDM Lab

Even more Golioth!

Why Arduino, anyway?

Zephyr is changing things up in the industry

Not just for production grade firmware

Abstractions are your friend in a complex world

Complex UI’s don’t have to be complex to build

Quick, once you know how to get around

Conference Admission with a Decades-Old Bench Tool

Badgecase: the Badge Showcase

The Gist of the Hack

The Firmware

The Software

Closing Thoughts

Contact us

Need to send us some mail?