Our lives are built on abstraction. It is what allows us build things much faster than we previously could. A person, or a group of people, decides that everyone else shouldn’t need to think about the gritty details of a problem — and voila, a new layer in the stack!

This is great until someone comes along and builds on that new layer, then discovers it obfuscates a lower level detail that they need. The abstraction that was intended to be, and in many cases is, an enabler has suddenly become a hindrance. There are a few options for how abstraction builders can choose to handle this trade-off.

The simplest solution is to decide that they are targeting a narrow audience, acknowledging that they won’t be a good fit for everyone. In reality, nearly all technology is optimized for some subset of the total possible consumers. Even the most ubiquitous products are disliked by some, and don’t fit the requirements of others. Choosing where you want to exist on the spectrum from a narrow audience to a broad one is critically important when designing a product.

Another option is to break the abstraction layer into a set of smaller, modular abstractions that can be assembled in a variety of configurations. This typically enables faster iteration, and when exposed to end-users, widens the audience for the product. The downside is that you are also pushing more cognitive burden onto the end-user, forcing them to understand multiple components and how they should piece them together.

Like any early-stage company, we at Golioth have wrestled with where we fall on the “audience spectrum”. Historically, we have leaned towards a more opinionated platform, prioritizing simplicity and ease of use over letting customers adjust all the knobs. There is good reason for this approach. Golioth connects two fairly disparate technical domains, firmware and the cloud, and many of our customers have deep expertise in one, but not both. The tricky thing about this strategy is that making the firmware highly customizable and the cloud opinionated, or vice versa, results in half our users having an experience that is not catered to their current capabilities.

We could take the simple solution and decide that we are only really targeting half of the total potential audience. Perhaps we could build an incredible platform for firmware engineers who don’t want to think about the cloud. Unfortunately, our users are building products based on connecting hardware to the internet, which is no simple feat. In the early stages, they typically want to shrink scope as much as possible. If functionality is not absolutely necessary, it can wait. But down the line, their team may grow, their product may expand, and they may decide that a domain they were previously happy to consume abstractly is now an area where they need fine-grained control. Golioth needs to serve users along each step on that journey.

Over the next few weeks, we are rolling out significant updates to Golioth that give you more control over the platform. But don’t worry, the simple, streamlined experience you know and love isn’t going anywhere. Alongside these changes, we’ll also give you more insight into your usage, and if you don’t want to think about managing custom configuration, we’ll provide reasonable defaults that allow you to keep using Golioth as you always have. With these new capabilities, from prototype to production, we can’t wait to see what you will build!

Back in September, we released beta support for Zephyr RTOS in the Golioth Firmware SDK, as an additional option alongside our existing standalone Zephyr SDK. Today, we’re excited to remove the beta tag and announce full support for Zephyr in our cross-platform Firmware SDK. 

 

Over the last several months we’ve been improving the efficiency of the Firmware SDK and it is now on par with the standalone Zephyr SDK. We’ve expanded our automated test coverage, taken a fine-tooth comb through our APIs and documentation, and listened to and incorporated your feedback (thanks!). All of this means that the version of the SDK we’re releasing today is our best yet – and all of our customers will get these improvements at the same time, whether using Zephyr, ESP-IDF, or any other platform.

 

That’s why we chose to incorporate Zephyr support into our Firmware SDK and deprecate our standalone Zephyr SDK. We’re big fans of Zephyr, but our goal is to support the IoT community at large as a universal IoT connector, and that includes folks who choose to use other platforms. In order to provide the highest quality support to all of our users while moving as fast as possible to deliver new features and value, it made the most sense to invest in a single unified Firmware SDK.

 

Of course, our commitment to Zephyr itself hasn’t changed. We continue to be financial sponsors of the project, sit on both the Technical and Marketing Steering Committees, and provide free Zephyr training. And you’ll be able to find us at the Zephyr Developer Summit in Seattle in April.

Ready to get started?

You’ll find the same features and quality you’ve come to expect from Golioth in our Firmware SDK, but we’ve taken this opportunity to rework some of our APIs. Migrating from the Zephyr SDK is easy, and we’ve prepared a migration guide to help you along the way. For those beginning a new project, our getting started documentation will get you up and running in no time.

What does this mean for the Zephyr SDK?

As of today, our standalone Zephyr SDK is deprecated and not recommended for new designs. We may make periodic bug fixes and security updates to the SDK, but we won’t be adding new features or actively developing the standalone Zephyr SDK. This will result in accelerated delivery of features and support in our cross-platform Firmware SDK, as our team is now singularly focused.

 

We will reach formal end-of-life for the standalone Zephyr SDK on July 31, 2024, at which point we will no longer make any updates to the SDK. If you have any questions, please reach out to your Golioth account rep or contact us at [email protected].



Golioth raises $4.6M seed round

Today, we’re announcing open access to a library of new reference designs for embedded engineers to accelerate their time to market, the launch of a Select Partner Program for energy and construction developers, and the completion of a $4.6M round of seed funding led by Blackhorn Ventures and Differential Ventures with participation from existing investors, Zetta Venture Partners, MongoDB Ventures and Lorimer Ventures. We’ve raised a total of $7.1 million to date. 

This funding will help establish our IoT cloud platform more firmly with embedded engineers, and build toward the company’s goal of helping our customers solve the biggest problem in IoT infrastructure – untangling the messy middle between hardware and the cloud.

A fresh look at the biggest problem in IoT

Almost two years ago when Golioth announced our pre-seed funding, We asked ourselves, “How do you bring 30.9B devices online?” 

This question still drives us today. Eventually, almost every device people use will be connected to the internet, no matter where we are or what we’re doing. Our vision is to enable builders to help get us there faster by connecting “sand” to the internet using our platform. 

Today, Golioth empowers engineers to build secure, reliable, and flexible IoT infrastructure so they can connect devices and move from prototype to production in minutes, instead of months. And we’re excited to keep pushing this vision forward.

Solving for the right problems

When we started Golioth, we planned to be an omni-platform that worked with every standard and technology out of the gate. We quickly found out that we didn’t have to be everything for everyone. We could help engineers faster with a narrower focus and extend the platform as we went along. 

More than anything else, engineers struggle with the complexity of connecting their sensors to the web. With limited resources, they must choose between getting a device connected to the internet, or creating a more useful device. And there’s not much time to consider if any of it is secure. 

Our platform is focused on solving that problem. Golioth shortens the time from prototype to production and easily scales that process. This includes tooling and workflows for securing devices, sending software updates over the air, and routing data to the web, all critical problems every IoT developer needs to solve. 

But getting devices online doesn’t start and stop with Golioth. To solve other critical problems developers are facing we’re partnering with leaders across the IoT ecosystem including components manufacturers, developer tools, cloud services, solutions partners, and others. 

In short, engineers do better work, faster with Golioth. 

Helping engineers get ahead with the right references

So what does “building faster” look like? It includes working with partners and companies to build a more resilient platform for industries including climate and energy, shipping and logistics, micromobility, building automation, and others. And with the IoT market expected to hit over $2.7T by 2028, engineers will need a lot of support.

To support embedded engineers in these areas, we created a library of free reference designs using our platform including: 

Golioth’s tight integrations with companies like Nordic Semiconductor, Infineon, NXP Semiconductor, Ubidots and others, lets engineers build with better support, seamless integrations, and faster enablement for all types of projects. In particular we see increasing usage in cellular IoT, 5G, and Thread as the driving force for IoT applications that enable resource efficiency in use cases such as energy and utilities, environmental cases, micromobility and others. 

As part of our seed announcement, we’ve launched a Select Partner Program, which will accept 15 new design partners building solutions for the energy and utility sector, and 15 design partners building solutions for the construction industry by June 1st. To learn more and apply to be a Select Design Partner, go here.

Looking forward to connecting more sand to the cloud, together.

We’re excited to announce a partnership between Golioth and Nordic Semiconductor. As long-time fans of and friends with Nordic Semiconductor, we officially joined as a Solution Partners in the Nordic Partner Program in 2023.

Nordic Semiconductor parts and development kit solutions work well with the Golioth platform, many of which are featured throughout this blog and in our documentation. Briefly:

  • nRF9160-DK is a continuously verified board that we use to test and deploy cellular applications on Golioth.
  • nRF52840-DK is a Continuously Verified Board when used in conjunction with an add-on Wi-Fi modem.
  • nRF52840 chipset is the basis of our OpenThread-based demos, which connect back through Linux-based Border Routers.
  • The recently released nRF7002 already works on Golioth in conjunction with the nRF5340.

Our customers are also well-versed in using Nordic Semiconductor components. One of our largest clients in the micromobility space is using the nRF9160 for connectivity over cellular. Another is using the nRF52 using Thread to network devices in the commercial real estate space. Both of these customers cited ease-of-use and tight coupling of Golioth services with Nordic hardware in their decision to pursue their hardware and IoT path.

What this partnership means for you

We’re taking our collaboration to the next level with faster enablement, seamless integrations, and more comprehensive training.

By partnering with Nordic Semiconductor, we’ll be able to offer a more comprehensive suite of IoT solutions than ever before. As new elements and components are introduced by Nordic Semiconductor, we will enable these technologies on the Golioth platform even quicker.

We are working on integrating cloud components to make it a seamless process to consume Nordic data for your products and act at the device level on that data.

Our training is migrating to use Nordic Semiconductor components so that we can offer direct training in Zephyr using Golioth and the nRF Connect SDK. All of these will enable our customers to accelerate their IoT projects, reduce development costs, and bring their products to market faster.

For customers who are new to Golioth, but longtime users of Nordic Semiconductor components, they will see immediate benefit to trying out our IoT Platform with their hardware. Pulling in Golioth as a feature of a nRF Connect SDK (Zephyr) project gives access to features like:

We believe that this partnership will drive innovation in the IoT space. By working together, we’ll be able to push the boundaries of what’s possible in IoT and create solutions that are more secure, reliable, and scalable than ever before. We’re excited to see what we can accomplish together, and we’re confident that our customers will benefit greatly from this partnership.

Partnership, in action

We showcased our Reference Designs at Embedded World 2023. Each of these solutions integrated the nRF9160 cellular SIP, in addition to customized sensors. The direct integration into the Golioth / Zephyr SDK meant development time was minimal for each additional design. Our team was able to create designs targeting a wide range of customer verticals, all using the same great underlying hardware.

Golioth Reference Designs using the Nordic Semiconductor nRF9160 at Embedded World 2023

At Golioth, we’re committed to providing you with the best possible IoT solutions. Our partnership with Nordic Semiconductor is a major step forward in achieving that goal. We’re excited to continue working closely with the teams at Nordic Semiconductor headquarters and their various FAEs throughout the world to develop and deliver innovative IoT solutions to our customers. Stay tuned for more updates as we continue to push the boundaries of what’s possible in IoT.

Golioth is secure by default, offering a couple of different ways for your devices to establish a secure connection to the Golioth Servers. When trying out features in the lab, pre-shared keys (PSK) are fine. But when moving devices into production, there is no substitute for certificate-based authentication.

Today we are announcing the ability to use Certificates with the Golioth platform.

Certificates deliver numerous security benefits when compared to pre-shared keys. This is especially true when it comes to provisioning your IoT fleet. As devices roll off the assembly line, they can be granted individual device certificates signed using a trusted chain of root certificates and intermediate certificates. At that point, the devices are not yet registered on the Golioth server. When they first connect, the certificates are verified against the chain of trust and a record of the trust-verified device is created. This simplifies the registration of a large influx of new devices, as happens in a production environment.

Let’s walk through the process used to get to that point.

How to generate and use certificates with Golioth

Generating a self-signed root certificate

A root certificate is a cryptographic public/private key pair. The private key is used to sign all device certificates. The public key is uploaded to the Golioth server and used at the project level to verify each device certificate when establishing a secure connection.

SERVER_NAME='golioth'

# Generate an elliptic curve private key 
# Run `openssl ecparam -list_curves` to list all available algorithms
# Keep this key safe! Anyone who has it can sign authentic-looking device certificates
openssl ecparam -name prime256v1 -genkey -noout -out "${SERVER_NAME}.key.pem"

# Create and self-sign a corresponding public key / certificate
openssl req -x509 -new -nodes -key "${SERVER_NAME}.key.pem" -sha256 -subj "/C=BR/CN=Root ${SERVER_NAME}" -days 1024 -out "${SERVER_NAME}.crt.pem"

The code above generates two files. The golioth.key.pem is the private key which you must safeguard. All credentials created from this private root key will be fully trusted. Golioth will not have a copy of this key.

The golioth.crt.pem is the public key that is uploaded to Golioth. This serves as a way to verify device credentials as trusted. It cannot be used to sign new device certificates, only the private key has that power.

Upload the root certificate public key to Golioth

Certificate authentication

As of December 2022, the Golioth console includes a “Certificates” option on the left sidebar menu. The resulting window is used to upload the root certificate public key. This is all that you need to do to prepare the Golioth Cloud for your fleet. As long as each IoT device has its own certificate signed with your root certificate’s private key, it will be added to this project the first time it tries to connect to Golioth.

Generate device certificates

PROJECT_SLUG='project_slug'
PRIMARY_HARDWARE_ID='primary_hardware_id'
SERVER_NAME='golioth'
CLIENT_NAME="${PROJECT_SLUG}-${PRIMARY_HARDWARE_ID}"

# Generate an elliptic curve private key
openssl ecparam -name prime256v1 -genkey -noout -out "${CLIENT_NAME}.key.pem"

# Create a certificate signing request (CSR)
# (this is what you would normally give to your CA / PKI to sign)
openssl req -new -key "${CLIENT_NAME}.key.pem" -subj "/C=BR/O=${PROJECT_SLUG}/CN=${PRIMARY_HARDWARE_ID}" -out "${CLIENT_NAME}.csr.pem"

# Sign the certificate (CSR) using the previously generated self-signed root certificate
openssl x509 -req \
    -in "${CLIENT_NAME}.csr.pem" \
    -CA "${SERVER_NAME}.crt.pem" \
    -CAkey "${SERVER_NAME}.key.pem" \
    -CAcreateserial \
    -out "${CLIENT_NAME}.crt.pem" \
    -days 500 -sha256

# Convert device certificates to DER format
openssl x509 -in ${CLIENT_NAME}.crt.pem -outform DER -out ${CLIENT_NAME}.crt.pem.der
openssl ec -in ${CLIENT_NAME}.key.pem -outform DER -out ${CLIENT_NAME}.key.pem.der

The root certificate is the source of trust, and all device certificates are created using this private key. This is why it is crucial that you safeguard your root certificate(s). Device credentials are created using the private key and added to your fleet during production. If newly created devices are compromised, it will not affect any other device credentials already out in the field because they don’t hold the (root) private key, they were simply generated from it.

The code above uses the Golioth Project ID as the PROJECT_SLUG and the SERVER_NAME to connect to the correct project. The root certificate public key was uploaded to this project in the previous section. When creating device certificates, the PRIMARY_HARDWARE_ID is a unique identifier that you generate (e.g. MAC address, naming scheme, any string will work). You need one of these for each device in your fleet.

As with the root certificate, the device certificate creation process will generate a public/private key pair. Both are loaded onto the device and used when establishing a secure connection to the Golioth Servers.

Golioth uses ECDSA (a note for other encryption geeks like us)

When a device is ready to connect to Golioth, it will request and download a server-side (public) certificate. That too will be signed by a trusted authority. The device will need to verify that certificate to ensure it’s not attempting to connect to a bad actor, such as someone pretending to be Golioth by using a “man-in-the-middle” (MITM) attack.

At Golioth, we are embedded developers as well as Cloud experts, so we chose an encryption that is as friendly as possible to resource-constrained devices. That is why we have all the certificates in the chain use ECDSA keys. They are significantly smaller to transfer and less resource intensive for embedded processing.

Summary

We’ve previously written about the importance of using certificates for Internet of Things (IoT) authentication. Certificate validation scales well, especially when it comes to provisioning devices during manufacture. New credentials can be signed with a root certificate without needing to share them back to the cloud server. And since those devices will be sent out into the field there’s an additional security benefit: compromised devices don’t expose symmetrical keys as is the case with PSK-based encryption.

In most cases, production deployments should consider certificate-based authentication for their IoT fleet. The certificate feature is built into Golioth and ready to use today.

We’d love to hear about how you are securing your company’s IoT fleet. Share your questions and tips on the Golioth Forum and don’t hesitate to reach out to our Developer Relations team if you’re interested in a guided tour of our certificate-based authentication!

We are excited to announce a partnership between Golioth and NXP Semiconductor. The result is a leap forward in your IoT design that will save you time while preparing your fleet to scale.

NXP’s line of Ethernet-enabled parts are feature rich and well supported in the Zephyr ecosystem. When paired with the next-generation device management tools from Golioth, you get to skip the awkward growing pains in IoT. Power up your first proof of concept hardware and immediately do something meaningful with it from the Cloud side. Your i.MX RT based devices will suddenly include Over-the-Air (OTA) updates, command and control, data management, remote logging, and more.

Built on Zephyr

NXP development board

NXP i.MX RT1060 EVKB development board

NXP is a founding and platinum member of the Zephyr project. The engineering team at NXP has spent years actively developing the RTOS. When you choose to build around an NXP part, you can expect excellent stability and up-to-date functionality with Zephyr. This meshes perfectly with Golioth’s embrace of the Zephyr platform (we are silver members of the project).

Mike previously wrote about how much we enjoy working with the MIMXRT1060-EVKB board. As part of this new partnership, we’re excited to add it as a Continuously Verified Board (CVB) on Golioth. This means that our engineering team thoroughly tests this hardware against every release, and we’ve even included it in our Hardware-in-the-Loop (HIL) testing that runs with every commit. You will always enjoy the latest and greatest Golioth support on this hardware.

NXP hardware is industrial at heart

The RT106x line of chips are some of the most powerful microcontrollers on the market. With a 600 MHz clock and huge range of peripherals (CAN, USB, audio, video, display), it is as large a system you might get without a jump to Linux. For the vast majority of IoT applications, operating without Linux is ideal. For instance, as a microcontroller, the RT106x delivers lower-latency response, low-power operation, and it will save on your Bill of Materials (BOM) cost.

If you’re contemplating connectivity in an industrial setting, it’s hard to beat NXP chips with their built-in Ethernet. From a development board standpoint, the EVKB is a great bet. You can use the built-in magnetics and jack for Ethernet, or pop a WiFi module into the m.2 slot. There’s a 20-pin J-Link programming header or you can flash the J-Link firmware to the on-board debugger and it’s ready to program via USB. Since this board has great Zephyr support, the west flash command is all it takes to flash the compiled binary.

Golioth enables microcontroller platforms, large and small

Golioth has already built all of the device management features your IoT fleet needs. Not only for your test fleet in the lab, but for when you scale to millions of devices. It starts with a connection to the cloud that is secure by default. Over this connection we make it easy to perform OTA firmware updates, even on fleets made up of multiple hardware configurations.  Golioth is designed for fleets of constrained devices, so efficient delivery of updates and data are top of mind for all features we provide.

What happens when a device in the field starts acting odd? Remotely adjust the logging levels so you get more information on the problem, or use a remote procedure call (RPC) to reboot.

Now you’ve discovered you need your fleet of 10,000 devices to take sensor readings twice as frequently. If you’re using the Golioth Settings Service, that’s a one-click change that can modify unit. If you’re not using the Settings Service, push an OTA update to enable it… now it’s a one click change.

And what about all that data? Do you have a plan to keep the data from your devices in Decatur, Texas organized and easy to query separately from the devices in Decatur, Georgia? We do, with our flat system of tagging devices in our fleet management console.

These are problems that everyone building IoT must address. Why boostrap your own device management when Golioth makes it secure, dependable, and easy? Take Golioth for a test-drive. With our Dev Tier your first 50 devices are free.

In partnership for you

The combination of great hardware and an excellent device management cloud means a rock-solid base to build from. Choose NXP hardware, take advantage of Golioth, and focus your time on creating a superb product for your company.

Learn more about our partnership on the Golioth site as well as the NXP site.

 

Today, Golioth is releasing an open-source SDK for the Espressif ESP-IDF, making it easier for Espressif-based projects to securely connect to the Golioth Cloud. All Cloud enablements on Golioth (OTA firmware updates, command and control, time-series databases, logging) are immediately available for ESP-IDF projects.

Golioth is about choice

Golioth is a device management Cloud platform designed for hardware and firmware teams to connect a wide range of devices to the internet and make use of device data.

We believe choosing a hardware platform should not lock you out of different ways of connecting to the Cloud. Golioth will support as many hardware platforms as possible to give you the choice you need, and will provide device-side examples and firmware code snippets to quickly prototype with the Golioth Cloud.

Today, the ESP-IDF SDK joins the Zephyr RTOS SDK, and the nRF Connect SDK (based on Zephyr) as supported platforms.

Golioth is a Cloud built to enable hardware

The general requirements to leverage our Cloud services are:

  • A device with a network interface (such as LwIP running on FreeRTOS for ESP-IDF).
  • A CoAP library
  • A way to encrypt packets using DTLS
  • (Optional) CBOR encoding

Any device-side SDK we create will have these elements when talking to our CoAP endpoints. As a result, we now have enabled ESP-IDF based projects to access features such as:

  • Over-the-air updates, including device-specific targeting.
  • Time-series database tracking for easy management of sensor data on the cloud.
  • Command and control capabilities using state-based databases on the cloud.
  • Device logging to pass troubleshooting messages off of individual devices (fewer debug probes and USB cables involved!)
  • Easy export of all cloud side data to 3rd party visualization and cloud platforms.

We are committed to continuous testing for all of our SDKs and you can be confident that the Golioth ESP-IDF SDK will continue to be tested for new hardware from Espressif and any changes that develop on the ESP-IDF ecosystem.

What is the ESP-IDF?

The Espressif IoT Development Framework (ESP-IDF) is a network-enabled Real Time Operating System (RTOS) built for Espressif’s ESP32 family of components. It is a complete software development framework built on the FreeRTOS Kernel. Espressif bundles a network stack using LwIP and adapted the RTOS for their Wi-Fi, Bluetooth, and Thread modems. Additionally, there are external components and libraries available, including a build system and programming tools, so the offering is more like a distribution than a monolithic piece of software.

Why Golioth is interested in the ESP-IDF?

Because our users asked for it! Golioth is committed to providing future device-side SDKs and examples to support platforms that our customers want to use.

The combination of Espressif parts and the ESP-IDF offers a cost-effective solution, one that is increasingly found in consumer-focused Wi-Fi products. There is a large community of people developing in and around the ESP-IDF. Many devs have past experience using FreeRTOS, and wanted a more integrated way to connect to the Golioth Cloud.

We are particularly interested in the growing portfolio of Espressif components. For example, the ESP32-C5 announced last month is based on a RISC-V core and has both 2.4 GHz and 5 GHz radios, a recent development in the low-cost Wi-Fi space. We are also excited by the Thread compliant components being developed, such as the ESP32-H2, which is also RISC-V based and has a 802.15.4 MAC and Phy, allowing these devices to easily join Thread networks, which we have been writing about in the past few weeks. Thread devices targeted for consumer applications are surely also candidates for the forthcoming Matter standard, set to be released later this year.

Golioth’s Future: More Device SDKs

If you have a platform we are not currently developing for, we’d love to hear about it! Email us at [email protected].

For support on the new Golioth ESP-IDF SDK, checkout out the quickstart documentation, come chat about it on our Golioth Discord, post longer form questions on our Forum, and file issues you encounter on the GitHub page. In future posts, we’ll showcase the ESP-IDF SDK with examples in video and blog format. We can’t wait to see what you build!

Hello from ZDS!

This week the Golioth team is at Zephyr Developer Summit. Previously we announced that we’ll be there and shared the talks we are presenting. We will post those shortly after the conference takes place. In the meantime, let’s recap how we got here in the first place and share a little bit more about what we’re showcasing.

Why Zephyr?

In short, because it helps our users. We are members of the Technical Steering Committee (TSC) and have been almost since the inception of the company. We built our first Device SDK on top of Zephyr because of the broad hardware support and high level integration of Golioth features into the Real Time Operating System (RTOS).

The assertion that “Zephyr helps our users” might be extra frustrating to beginners: Zephyr—and RTOSes more broadly—represents a tricky new set of skills that might be foreign to firmware or hardware engineers. For beginners coming from the hobby space, it can be an extra rude introduction into the world of command line compilation and large ecosystem. However, connecting to the internet is a difficult task, especially for custom hardware: we think that Zephyr represents a great first step towards managing those devices over time. We are committed to pushing for more user-friendly code and methods from the Zephyr foundation, and we will continue to publish best practices on our blog and our YouTube channel to help people get connected.

Showcase

One thing we’re excited about is showcasing how Golioth works to members of the community. We have been developing different “color coded” demos to make them a bit more memorable for folks that stop by our booth. Each of these demos feature a hardware (device) component and a dashboard component, in order to visualize the data that is on the Golioth Cloud.

This is the first time we have showcased the “Aludel”, which is our internal platform for prototyping ideas and switching out different development boards and plug-in sensors. We will post more about this in the future, including our talk on the subject.

Red Demo

The Red Demo is our showcase of devices running OpenThread on Zephyr; this is part of our larger interest in Thread, which we see as a very interesting way to connect a large range of sensors to the internet securely. We have been excited to show how we can use low power devices like the Nordic nRF52840 to communicate directly with the Golioth Cloud.

The devices we are using in this case are off-the-shelf multi-sensor nodes from Laird called the BT510. This hardware has additional sensors on the board which we integrated with LightDB Stream to send time-series data back to Golioth. This was fast work, thanks to Laird’s Zephyr support, it was as simple as calling out the board when we compiled the demo firmware.

We then capture the data from these on the Red Demo Dashboard, showing both historical and live data for the sensors.

 

Green Demo

The Green Demo showcases LightDB State, our real-time database that can be used to control a wide range of devices in a deployment. On the device side, it uses the Aludel platform to measure a light sensor, as would happen in a greenhouse. There is also a secondary Zephyr-based device inside a lamp, representing a grow light that might be inside a grow house. The lamp is set up to “listen” to commands from another node, in this case the Aludel.

LightDB State is used to control elements like “update rate” to control regulate flow of information. It also lets us monitor critical device variables on an ongoing basis and set up logic on the web to take actions as a result. Command and control variables can be set from multiple places, including a custom mobile app, the Golioth Console, a visualization platform, a web page, or (as is the case here) even from another device!

Our Green Demo Dashboard (below) again showcases live and historical information, as well as the current status of the connected lamp.

As an added bonus, we control some of the logic on the back end from a Node-RED instance, including control logic. That takes the light intensity sensor output and calculates how bright the lamp should be. Because this is written in Node-RED, we can include an additional input from a mobile app to control the “target intensity”. In this way, people at the booth can adjust the lamp output if the exhibition space is brighter or darker. Plus…it looks cool!

Blue Demo

The Blue Demo helps to showcase how data migrates into and out of Golioth. Using Output Streams, you can export all cloud events to 3rd party providers like AWS, Azure, and Google Cloud. Buttons on the Blue faceplate switch the output being sent back to the cloud. The sensor readings being exported to all 3 clouds can be turned on or off by changing which variables are exported from the device.

On the device side, we capture a sensor using our Aludel platform. The sensor is a BME280 (in-tree sensor in Zephyr), going through a feather form-factor dev board, talking to the network over a WizNet W5500 Ethernet external chip to the network. The Blue Demo Dashboard showcases the live data, and of course the data is being exported simultaneously to the 3 cloud platforms in real-time.

Orange Demo

Golioth is a “middleware” built on top of Zephyr RTOS, which means you can use it to implement new features on top of already-existing hardware. This demo uses the Nordic Semiconductor Thingy91 with custom firmware to send GPS data back over the cellular network to Golioth using LightDB Stream. This demo also has Golioth Logging and Device Firmware Update, which are easy to add to any project as an additional service for troubleshooting or in-field updates.

On the dashboard side, we wanted different ways to showcase this data, including “latest update”. Having access to the raw data is useful for anyone wanting to try asset tracking applications. We’re excited to be able to showcase this data as it dynamically flows into the Golioth Console and back out to the Grafana dashboard.

Future showcases

We’re excited to be showcasing our demos at the Zephyr Developer Summit, but these are moving targets! We will continue to update and pull in new feature for future events. We will be at Embedded World in two weeks (June 20-24th) and will have many of the same demos there.

Golioth has just released a Grafana WebSocket Data Source plugin. This open source plugin allows you to create a graphic dashboard using data from any WebSocket URL that uses JSON formatting. This means you can you directly connect the Golioth Cloud API and have Grafana render incoming data in real time as graphs, charts, maps, etc.

Grafana Takes Care of Data Visualization

Temperature data display as graphs

Realtime data from Golioth devices looks great in Grafana!

Grafana is an open source visualization and analytics software from GrafanaLabs. It allows you to query, visualize, alert on, and understand your metrics no matter where they are stored. We chose Grafana because we heard it mentioned multiples times by our community members and found that it streamlines the process of managing device data and visualizing. You can build from source and run it locally (the power of open source), or use the free tier of the hosted Grafana Cloud platform.

We’re Going to Need WebSockets

At Golioth, we recently added WebSocket support for some of our services like Logs, LightDB State, and LightdDB Stream. Actaully, Chris wrote a blog post all about it. You can now connect to a WebSocket URL and get real-time data updates from your devices without the need to poll for updates like you would with a REST API.

As awesome as it is, Grafana didn’t have a WebSockets plugin, so we created one! It allows you to connect to any WebSocket URL and visualize the data in Grafana. You can get a copy of the WebSockets plugin code in our repo. But we have also submitted it as an official plugin – it should be available in Grafana soon!

Let’s Try the Demo!

Setting Up a Device to Send Data

As a hardware beginner, I followed one of our easiest samples using PlatformIO and our Arduino SDK to get my ESP32 board connected to our platform, see our examples folder on our Arduino SDK repo.

Note: The arduino-sdk repository showcased in this post is deprecated. GoliothLabs has an experimental repository that may work as a replacement:

I’m writing data on both of our services in a 5 seconds loop, updating my device’s state with LightDB State and reporting time series data for our LightDB Stream service.

// Every 5 seconds loop
...
  int core_temp = randomNumber(40, 60);
  int room_temp = randomNumber(27, 33);
  int uptime = millis() / 1000;
  String payload =
    "{
      \"core\":
        {
          \"temp\":" + String(core_temp) + ",
          \"uptime\":" + String(uptime) +
        "},
    \"room\":
      {
        \"temp\":" + String(room_temp) + "
      }
    }";
  
  client->setLightDBStateAtPath("/", payload.c_str());
  client->sendLightDBStream("/", payload.c_str());
...

This small program sends random numbers as core/room temperatures, and the device’s uptime. We have data, now let’s graph it!

Golioth’s WebSockets API

LightDB state data

Devices > Your Device > LightDB State

LightDB Stream Data

Monitor > LightDB Stream

WebSockets “listens” for changes, so every 5 seconds those updates on State and Stream are also available thru Golioth’s WebSocket API. We can follow the WebSockets docs to setup a connection to listen to them.

Below you can see how the WebSocket hosts URLs are formatted. Use the Golioth Console to find the projectId, deviceId, API key, and any paths you need. (This information is also available using the goliothctl command line tool.)

LightDB State

// Websocket URL format: wss://api.golioth.io/v1/ws/projects/{projectId}/devices/{deviceId}/data{/path=**}?{x-api-key|jwt}={API_KEY|JWT}

wss://api.golioth.io/v1/ws/projects/smart-house/devices/61d315e441da400dd6934493/data?x-api-key={projectApiKey}

LightDB Stream

// Websocket URL format: wss://api.golioth.io/v1/ws/projects/{projectId}/stream?{x-api-key|jwt}={API_KEY|JWT}

wss://api.golioth.io/v1/ws/projects/smart-house/stream?x-api-key={projectApiKey}

With those I was able to test the endpoint using websocat to make sure that the data my device was sending created notifications through the WebSocket connection as expected.

websocat command line tool used to test WebSocket feed

Grafana WebSocket Plugin Setup

For this demo I’m running everything locally (both Grafana and the Plugin itself), but we have submitted the signed plugin to Grafana’s Cloud so soon you will be able to find it there by default.

To use the plugin, there are a few configuration steps we need to follow:

  • Once you’ve logged into Grafana, go to the Configuration page (gear icon) and then to Plugins
  • Search for WebSocket API, select and then click on Create a WebSocket API data source

Now we fill in the fields as the API requires and hit save. For Golioth’s WebSocket API you can see I’ve entered the URL, and I’ve specified x-api-key and stored the API key that I generated on the Golioth Console.

Grafana WebSocket data source

Notice that I’m not using the full path, but only through the project level (smart-house). This way we can select the specific path we want to listen to later on the Panel’s Query page. This allows more flexibility.

Building the Dashboard that Shows the Data

With the data source set up, it’s time for us to build the query on an actual panel.

Something to keep in mind is that, right now, it will only listen to future events, so if there’s any update on the Query Field, Path, Panel or Dashboard, the current stored data will be wiped.

Graphing LightDB Stream:

  • Hover on the plus sign at the sidebar and select Dashboard
  • Then click on Add a new panel
  • Select WebSocket API under the Query’s data source
  • Go to Query A Path tab and fill with your desired path. In my case /stream

Grafana WebSockets path setting

  • Under Fields tab, use JSONPath reference to transform the JSON result into something that the dashboard can display. Here I’ve used $.result.data.timestamp, $..core.temp, and $..room.temp
  • I want to display the last 5 minute events of my device’s temperature, both for the core and room. So I changed the panel’s type into Time Series on the top right corner, and filled with the fields required:
  • Hit Save, and its done.

Grafana WebSockets field configuration

As long as data is streaming in from a device, it will be graphed in real time. One tip as you get used to handling data, try starting with Table View on to better understand the JSON by your query.

Displaying LightDB State

To create another panel for the LightDB State we can just duplicate the first one and update his its Path, so:

Grafana state data path configuration

There’s a small differences here, LightDB State can have deeper paths e.g. .../data/core/temp but they will only be notified via WebSockets when there’s an update on them. Because I want to listen to multiple path updates for this Panel, I will leave the path set to the root level, that is .../data.

Grafana state field queries

  • Transform the query fields to use $..core.temp and $..room.temp
  • Hit Save, and it’s another one done.

Final Results

Grafana state field queries

Like that, we have 2 panels ready to listen for updates coming from both LightDB Stream and State in real-time. Feel free to add more data and to listen to different State paths.

Conclusion

Grafana is an awesome tool that allows us to connect and create professional visualizations. Because it is open source, a major part of its improvement over time is a result of its community. We’re building an amazing platform at Golioth and we also want to empower our community with the freedom to build things they love with us. With WebSockets now available on our platform, we built this plugin to share that excitement with our users and with the entire world.

Hopefully you’ll give this a try, and we want your feedback! Get in touch with us on the Golioth Discord server and please join us there for our Office Hours every Wednesday at 10 am Pacific Time.