Today, Golioth is releasing an open-source SDK for the Espressif ESP-IDF, making it easier for Espressif-based projects to securely connect to the Golioth Cloud. All Cloud enablements on Golioth (OTA firmware updates, command and control, time-series databases, logging) are immediately available for ESP-IDF projects.

Golioth is about choice

Golioth is a device management Cloud platform designed for hardware and firmware teams to connect a wide range of devices to the internet and make use of device data.

We believe choosing a hardware platform should not lock you out of different ways of connecting to the Cloud. Golioth will support as many hardware platforms as possible to give you the choice you need, and will provide device-side examples and firmware code snippets to quickly prototype with the Golioth Cloud.

Today, the ESP-IDF SDK joins the Zephyr RTOS SDK, and the nRF Connect SDK (based on Zephyr) as supported platforms.

Golioth is a Cloud built to enable hardware

The general requirements to leverage our Cloud services are:

  • A device with a network interface (such as LwIP running on FreeRTOS for ESP-IDF).
  • A CoAP library
  • A way to encrypt packets using DTLS
  • (Optional) CBOR encoding

Any device-side SDK we create will have these elements when talking to our CoAP endpoints. As a result, we now have enabled ESP-IDF based projects to access features such as:

  • Over-the-air updates, including device-specific targeting.
  • Time-series database tracking for easy management of sensor data on the cloud.
  • Command and control capabilities using state-based databases on the cloud.
  • Device logging to pass troubleshooting messages off of individual devices (fewer debug probes and USB cables involved!)
  • Easy export of all cloud side data to 3rd party visualization and cloud platforms.

We are committed to continuous testing for all of our SDKs and you can be confident that the Golioth ESP-IDF SDK will continue to be tested for new hardware from Espressif and any changes that develop on the ESP-IDF ecosystem.

What is the ESP-IDF?

The Espressif IoT Development Framework (ESP-IDF) is a network-enabled Real Time Operating System (RTOS) built for Espressif’s ESP32 family of components. It is a complete software development framework built on the FreeRTOS Kernel. Espressif bundles a network stack using LwIP and adapted the RTOS for their Wi-Fi, Bluetooth, and Thread modems. Additionally, there are external components and libraries available, including a build system and programming tools, so the offering is more like a distribution than a monolithic piece of software.

Why Golioth is interested in the ESP-IDF?

Because our users asked for it! Golioth is committed to providing future device-side SDKs and examples to support platforms that our customers want to use.

The combination of Espressif parts and the ESP-IDF offers a cost-effective solution, one that is increasingly found in consumer-focused Wi-Fi products. There is a large community of people developing in and around the ESP-IDF. Many devs have past experience using FreeRTOS, and wanted a more integrated way to connect to the Golioth Cloud.

We are particularly interested in the growing portfolio of Espressif components. For example, the ESP32-C5 announced last month is based on a RISC-V core and has both 2.4 GHz and 5 GHz radios, a recent development in the low-cost Wi-Fi space. We are also excited by the Thread compliant components being developed, such as the ESP32-H2, which is also RISC-V based and has a 802.15.4 MAC and Phy, allowing these devices to easily join Thread networks, which we have been writing about in the past few weeks. Thread devices targeted for consumer applications are surely also candidates for the forthcoming Matter standard, set to be released later this year.

Golioth’s Future: More Device SDKs

If you have a platform we are not currently developing for, we’d love to hear about it! Email us at [email protected].

For support on the new Golioth ESP-IDF SDK, checkout out the quickstart documentation, come chat about it on our Golioth Discord, post longer form questions on our Forum, and file issues you encounter on the GitHub page. In future posts, we’ll showcase the ESP-IDF SDK with examples in video and blog format. We can’t wait to see what you build!

Hello from ZDS!

This week the Golioth team is at Zephyr Developer Summit. Previously we announced that we’ll be there and shared the talks we are presenting. We will post those shortly after the conference takes place. In the meantime, let’s recap how we got here in the first place and share a little bit more about what we’re showcasing.

Why Zephyr?

In short, because it helps our users. We are members of the Technical Steering Committee (TSC) and have been almost since the inception of the company. We built our first Device SDK on top of Zephyr because of the broad hardware support and high level integration of Golioth features into the Real Time Operating System (RTOS).

The assertion that “Zephyr helps our users” might be extra frustrating to beginners: Zephyr—and RTOSes more broadly—represents a tricky new set of skills that might be foreign to firmware or hardware engineers. For beginners coming from the hobby space, it can be an extra rude introduction into the world of command line compilation and large ecosystem. However, connecting to the internet is a difficult task, especially for custom hardware: we think that Zephyr represents a great first step towards managing those devices over time. We are committed to pushing for more user-friendly code and methods from the Zephyr foundation, and we will continue to publish best practices on our blog and our YouTube channel to help people get connected.

Showcase

One thing we’re excited about is showcasing how Golioth works to members of the community. We have been developing different “color coded” demos to make them a bit more memorable for folks that stop by our booth. Each of these demos feature a hardware (device) component and a dashboard component, in order to visualize the data that is on the Golioth Cloud.

This is the first time we have showcased the “Aludel”, which is our internal platform for prototyping ideas and switching out different development boards and plug-in sensors. We will post more about this in the future, including our talk on the subject.

Red Demo

The Red Demo is our showcase of devices running OpenThread on Zephyr; this is part of our larger interest in Thread, which we see as a very interesting way to connect a large range of sensors to the internet securely. We have been excited to show how we can use low power devices like the Nordic nRF52840 to communicate directly with the Golioth Cloud.

The devices we are using in this case are off-the-shelf multi-sensor nodes from Laird called the BT510. This hardware has additional sensors on the board which we integrated with LightDB Stream to send time-series data back to Golioth. This was fast work, thanks to Laird’s Zephyr support, it was as simple as calling out the board when we compiled the demo firmware.

We then capture the data from these on the Red Demo Dashboard, showing both historical and live data for the sensors.

 

Green Demo

The Green Demo showcases LightDB State, our real-time database that can be used to control a wide range of devices in a deployment. On the device side, it uses the Aludel platform to measure a light sensor, as would happen in a greenhouse. There is also a secondary Zephyr-based device inside a lamp, representing a grow light that might be inside a grow house. The lamp is set up to “listen” to commands from another node, in this case the Aludel.

LightDB State is used to control elements like “update rate” to control regulate flow of information. It also lets us monitor critical device variables on an ongoing basis and set up logic on the web to take actions as a result. Command and control variables can be set from multiple places, including a custom mobile app, the Golioth Console, a visualization platform, a web page, or (as is the case here) even from another device!

Our Green Demo Dashboard (below) again showcases live and historical information, as well as the current status of the connected lamp.

As an added bonus, we control some of the logic on the back end from a Node-RED instance, including control logic. That takes the light intensity sensor output and calculates how bright the lamp should be. Because this is written in Node-RED, we can include an additional input from a mobile app to control the “target intensity”. In this way, people at the booth can adjust the lamp output if the exhibition space is brighter or darker. Plus…it looks cool!

Blue Demo

The Blue Demo helps to showcase how data migrates into and out of Golioth. Using Output Streams, you can export all cloud events to 3rd party providers like AWS, Azure, and Google Cloud. Buttons on the Blue faceplate switch the output being sent back to the cloud. The sensor readings being exported to all 3 clouds can be turned on or off by changing which variables are exported from the device.

On the device side, we capture a sensor using our Aludel platform. The sensor is a BME280 (in-tree sensor in Zephyr), going through a feather form-factor dev board, talking to the network over a WizNet W5500 Ethernet external chip to the network. The Blue Demo Dashboard showcases the live data, and of course the data is being exported simultaneously to the 3 cloud platforms in real-time.

Orange Demo

Golioth is a “middleware” built on top of Zephyr RTOS, which means you can use it to implement new features on top of already-existing hardware. This demo uses the Nordic Semiconductor Thingy91 with custom firmware to send GPS data back over the cellular network to Golioth using LightDB Stream. This demo also has Golioth Logging and Device Firmware Update, which are easy to add to any project as an additional service for troubleshooting or in-field updates.

On the dashboard side, we wanted different ways to showcase this data, including “latest update”. Having access to the raw data is useful for anyone wanting to try asset tracking applications. We’re excited to be able to showcase this data as it dynamically flows into the Golioth Console and back out to the Grafana dashboard.

Future showcases

We’re excited to be showcasing our demos at the Zephyr Developer Summit, but these are moving targets! We will continue to update and pull in new feature for future events. We will be at Embedded World in two weeks (June 20-24th) and will have many of the same demos there.

Golioth has just released a Grafana WebSocket Data Source plugin. This open source plugin allows you to create a graphic dashboard using data from any WebSocket URL that uses JSON formatting. This means you can you directly connect the Golioth Cloud API and have Grafana render incoming data in real time as graphs, charts, maps, etc.

Grafana Takes Care of Data Visualization

Temperature data display as graphs

Realtime data from Golioth devices looks great in Grafana!

Grafana is an open source visualization and analytics software from GrafanaLabs. It allows you to query, visualize, alert on, and understand your metrics no matter where they are stored. We chose Grafana because we heard it mentioned multiples times by our community members and found that it streamlines the process of managing device data and visualizing. You can build from source and run it locally (the power of open source), or use the free tier of the hosted Grafana Cloud platform.

We’re Going to Need WebSockets

At Golioth, we recently added WebSocket support for some of our services like Logs, LightDB State, and LightdDB Stream. Actaully, Chris wrote a blog post all about it. You can now connect to a WebSocket URL and get real-time data updates from your devices without the need to poll for updates like you would with a REST API.

As awesome as it is, Grafana didn’t have a WebSockets plugin, so we created one! It allows you to connect to any WebSocket URL and visualize the data in Grafana. You can get a copy of the WebSockets plugin code in our repo. But we have also submitted it as an official plugin – it should be available in Grafana soon!

Let’s Try the Demo!

Setting Up a Device to Send Data

As a hardware beginner, I followed one of our easiest samples using PlatformIO and our Arduino SDK to get my ESP32 board connected to our platform, see our examples folder on our Arduino SDK repo. I’m writing data on both of our services in a 5 seconds loop, updating my device’s state with LightDB State and reporting time series data for our LightDB Stream service.

// Every 5 seconds loop
...
  int core_temp = randomNumber(40, 60);
  int room_temp = randomNumber(27, 33);
  int uptime = millis() / 1000;
  String payload =
    "{
      \"core\":
        {
          \"temp\":" + String(core_temp) + ",
          \"uptime\":" + String(uptime) +
        "},
    \"room\":
      {
        \"temp\":" + String(room_temp) + "
      }
    }";
  
  client->setLightDBStateAtPath("/", payload.c_str());
  client->sendLightDBStream("/", payload.c_str());
...

This small program sends random numbers as core/room temperatures, and the device’s uptime. We have data, now let’s graph it!

Golioth’s WebSockets API

LightDB state data

Devices > Your Device > LightDB State

LightDB Stream Data

Monitor > LightDB Stream

WebSockets “listens” for changes, so every 5 seconds those updates on State and Stream are also available thru Golioth’s WebSocket API. We can follow the WebSockets docs to setup a connection to listen to them.

Below you can see how the WebSocket hosts URLs are formatted. Use the Golioth Console to find the projectId, deviceId, API key, and any paths you need. (This information is also available using the goliothctl command line tool.)

LightDB State

// Websocket URL format: wss://api.golioth.io/v1/ws/projects/{projectId}/devices/{deviceId}/data{/path=**}?{x-api-key|jwt}={API_KEY|JWT}

wss://api.golioth.io/v1/ws/projects/smart-house/devices/61d315e441da400dd6934493/data?x-api-key={projectApiKey}

LightDB Stream

// Websocket URL format: wss://api.golioth.io/v1/ws/projects/{projectId}/stream?{x-api-key|jwt}={API_KEY|JWT}

wss://api.golioth.io/v1/ws/projects/smart-house/stream?x-api-key={projectApiKey}

With those I was able to test the endpoint using websocat to make sure that the data my device was sending created notifications through the WebSocket connection as expected.

websocat command line tool used to test WebSocket feed

Grafana WebSocket Plugin Setup

For this demo I’m running everything locally (both Grafana and the Plugin itself), but we have submitted the signed plugin to Grafana’s Cloud so soon you will be able to find it there by default.

To use the plugin, there are a few configuration steps we need to follow:

  • Once you’ve logged into Grafana, go to the Configuration page (gear icon) and then to Plugins
  • Search for WebSocket API, select and then click on Create a WebSocket API data source

Now we fill in the fields as the API requires and hit save. For Golioth’s WebSocket API you can see I’ve entered the URL, and I’ve specified x-api-key and stored the API key that I generated on the Golioth Console.

Grafana WebSocket data source

Notice that I’m not using the full path, but only through the project level (smart-house). This way we can select the specific path we want to listen to later on the Panel’s Query page. This allows more flexibility.

Building the Dashboard that Shows the Data

With the data source set up, it’s time for us to build the query on an actual panel.

Something to keep in mind is that, right now, it will only listen to future events, so if there’s any update on the Query Field, Path, Panel or Dashboard, the current stored data will be wiped.

Graphing LightDB Stream:

  • Hover on the plus sign at the sidebar and select Dashboard
  • Then click on Add a new panel
  • Select WebSocket API under the Query’s data source
  • Go to Query A Path tab and fill with your desired path. In my case /stream

Grafana WebSockets path setting

  • Under Fields tab, use JSONPath reference to transform the JSON result into something that the dashboard can display. Here I’ve used $.result.data.timestamp, $..core.temp, and $..room.temp
  • I want to display the last 5 minute events of my device’s temperature, both for the core and room. So I changed the panel’s type into Time Series on the top right corner, and filled with the fields required:
  • Hit Save, and its done.

Grafana WebSockets field configuration

As long as data is streaming in from a device, it will be graphed in real time. One tip as you get used to handling data, try starting with Table View on to better understand the JSON by your query.

Displaying LightDB State

To create another panel for the LightDB State we can just duplicate the first one and update his its Path, so:

Grafana state data path configuration

There’s a small differences here, LightDB State can have deeper paths e.g. .../data/core/temp but they will only be notified via WebSockets when there’s an update on them. Because I want to listen to multiple path updates for this Panel, I will leave the path set to the root level, that is .../data.

Grafana state field queries

  • Transform the query fields to use $..core.temp and $..room.temp
  • Hit Save, and it’s another one done.

Final Results

Grafana state field queries

Like that, we have 2 panels ready to listen for updates coming from both LightDB Stream and State in real-time. Feel free to add more data and to listen to different State paths.

Conclusion

Grafana is an awesome tool that allows us to connect and create professional visualizations. Because it is open source, a major part of its improvement over time is a result of its community. We’re building an amazing platform at Golioth and we also want to empower our community with the freedom to build things they love with us. With WebSockets now available on our platform, we built this plugin to share that excitement with our users and with the entire world.

Hopefully you’ll give this a try, and we want your feedback! Get in touch with us on the Golioth Discord server and please join us there for our Office Hours every Wednesday at 10 am Pacific Time.

Welcome to the Dev Tier

Golioth, the straightforward commercial IoT development platform built for scale, is now generally available. This means that IoT deployments from one to one million devices can now join the platform, prototype new business concepts, and immediately scale.

We recently completed our “beta” program which was an opportunity to work with engaged hardware, firmware, and software developers. People who are building interconnected systems of hardware and cloud connectivity (AKA “IoT deployments”). We learned what this group needs on a daily and longer-term deployment basis and have refined the Golioth offering to better serve developers throughout the product development lifecycle. We now offer these services to the wider hardware development community with the click of a button.

For those just getting started, our “Dev” tier provides everything you need to evaluate and develop your proof of concept, all for free. What does that include?

  • Our open source Zephyr® SDK integrated with the Golioth Cloud
  • Support for 100+ hardware components, including the latest cellular nRF9160 from Nordic Semiconductor & the ESP32-C3 Wi-Fi SoC based on RISC-V from Espressif Systems
  • “Secure by Default” communication over an ultra-efficient protocols like CoAP
  • Access to powerful Device Services like:
    • Software updates that include secure boot and firmware management
    • A realtime NoSQL database, useful for creating Digital Twins and synchronization
    • A time-series database for storing and querying sensor data
    • A flexible logging system to collect OS & application status and troubleshoot from the field
  • A browser-based app to manage your fleet, as well as command line tools and RESTful APIs for automation
  • Community support via our Discord server and GitHub
  • Enough space to get your early deployment off the ground (see rates and limits)

For those ready to scale their product, we offer additional features including advanced reporting, white-glove service, and on-demand support.

Contact us at [email protected] about Pilots and beyond.

A brief recap of Golioth

New to Golioth? Here’s a quick overview of who we are, what we do, and the problems we’re solving.

What is Golioth?

First and foremost, Golioth is a cloud platform. We offer APIs in the form of “Device Services” and cloud processing for your IoT fleet. For developers, this means you can point their embedded devices at these endpoints on the web and have access to some really advanced capabilities.

Why build Golioth at all? We see product teams creating these same functions over and over again with their cloud teams, with no standardization and without flexibility across cloud providers. What’s worse, these features are tough to build until development teams grow to a certain size. With Golioth, we have standardized an offering of key services and make them available from day one of hardware development.

An IoT deployment is nothing without hardware, and we want to support as much choice as possible. That’s why we support our community by offering the Golioth SDK built on top of Zephyr®. This is a Real Time Operating System that makes it easy to implement the networking capabilities and get started quickly. There is support across hundreds of boards and platforms, so you can customize your hardware to fit your end customers’ needs. Zephyr® is flexible enough to allow switching between platforms, which is extra critical during the chip shortages in 2021 and beyond.

What are the problems we’re solving?

Complexity

The Golioth SDK is incredibly simple to set up with only 4 lines in a configuration file. Once you pull in the SDK, you not only get access to sample code to test our services, you get access to services built for hardware developers: The ability to troubleshoot, the ability to fix problems remotely, the ability to get readings from your devices, and more.

Lock-In

2021 has been a challenging year for sourcing hardware. Building on top of an open source Real Time Operating System (RTOS) like Zephyr® allows much needed flexibility. By abstracting away some of the hardware specificity, you gain access to a wide range of devices and types of connectivity. The Golioth Device Catalog shows a range of hardware already tested with Golioth services and many more devices that will be supported soon.

Security

Your IoT deployment is secure from the first packet you upload to the Golioth cloud. All devices use a flexible, yet secure DTLS protocol with Pre-Shared Key authentication. The data you push from your device to the cloud will be encrypted and easily authenticated and decoded in the Golioth cloud. Our REST API uses OAuth 2.0 for authorization and web side access to the Golioth cloud conforms to industry best practices. Hardware and firmware developers can prototype within minutes, and the rest of their team will trust the incoming data from a trusted data pipeline such as Golioth. Pushing firmware updates back down to the device are also secure by default, passing signed images over an encrypted connection down to each device.

Continuous Improvement

At Golioth, we strive to continuously improve our offering for developers and their IoT deployments. What’s more, we actively engage with our community to find out what will make it more straightforward to deploy devices out into the world. Top requested features under development include security certificates, MQTT, WebSockets, with even more on the long-term roadmap.

Long Term Support

After our first year of operation, we have seen the pain that IoT companies of all sizes experience. But in this final point about who we are and what we do, we want to emphasize our support for you, the developer, and the company deploying IoT devices. We offer community-based support as part of all tiers, including the Dev Tier. For those who need or desire more hands-on support, we have white-glove service for our Pilot customers (reach out to us if this is you!). With Golioth, your IoT deployment will not only be easier to get off the ground, it will also have the ability to scale across devices, across divisions in your company, and across the globe.

Are you ready to build?

Golioth is ready for your next IoT deployment. We can help you to connect an older industrial product, your brand new asset tracker, or help you to deploy a never-before-seen device. See how Golioth speeds up development and increases the chances that your next pilot hardware will be put into production. Get started today for free with Golioth’s brand new Dev Tier.

Get started today

Sign up:

Follow us:

Today, we’re excited to announce our $2.5 million Seed funding led by Zetta Venture Partners, with participation from additional investors including Chris Aniszczyk (CTO of CNCF), Sam Ramji (CSO of Datastax), Shiva Rajaraman (VP Commerce at Facebook and former CTO WeWork), and Stephen Blum (CTO of PubNub). We’re also opening up Golioth to beta testers and are proud to share that we’ve joined the Zephyr™ Project as a Silver Member.