How to Connect Live Golioth IoT Data to Grafana Cloud using WebSockets

Live data visualization shows you what is going on with your IoT fleet right now, and it makes for impressive customer demos. We previously wrote about how to visualize live data on the self-hosted version of Grafana using the WebSocket plugin we wrote. Today let’s walk through how to connect live Golioth IoT data to Grafana Cloud.

Overview

Golioth is the translation layer between your fleet of IoT devices and the cloud. Among our many ways of accessing the collected data, WebSockets are one way to get live updates as soon as the data arrives.

We wrote a WebSockets plugin for Grafana, the monitoring and observability software. The plugin is open source (here’s the code repository) and as of just a few days ago it’s available in Grafana Cloud, the cloud-hosted version of the software. Today I’ll guide you through:

1. Sending some temperature sensor data to Golioth
2. Configuring the Golioth WebSocket API on the Grafana Cloud
3. Graphing the temperature data in realtime

This example has everything you need to adapt it for your own needs.

1. Send Sensor Data to Golioth

For this guide I’m using the LightDB Stream example that is part of the Golioth SDK, and running it on an ESP32. This example is a good one, since it will either use a temperature sensor you have configured in the devicetree, or simulate a temperature sensor if there is not one available.

First, find your device credentials in the Golioth Console by selecting Devices from the left sidebar menu and clicking the name of your device on the resulting list.

CONFIG_GOLIOTH_SYSTEM_CLIENT_PSK_ID="esp32-temperature-pskid@ttgo-demo"
CONFIG_GOLIOTH_SYSTEM_CLIENT_PSK="a_strong_password"

CONFIG_ESP32_WIFI_SSID="your_wifi_ssid"
CONFIG_ESP32_WIFI_PASSWORD="your_wifi_password"

Now add the device credentials–Identity (PDK-ID) and Pre-Shared Key (PSK)–and your WiFi access point credentials to the project’s prj.conf file. This file should be located at ~/zephyrproject/modules/lib/golioth/samples/lightdb_stream/prj.conf. Add the lines in the format shown above.

$ west build -b esp32 samples/lightdb_stream -p
$ west flash

Finally, use the commands above to build and flash the sample to your ESP32.

If you go to Monitor→LightDB Stream you will see new temperature data coming in every five seconds. In this window, choose the Last 4h setting from the time selector, your device from the device selector, and enable 5s auto-refresh in the upper right.

Each time one of these packets comes in, the data will be automatically updated in Grafana. We just need to set that up, which is next!

2. Configure the Golioth WebSocket API in Grafana Cloud

The prerequisite for this step is to get an API key from Golioth.

1. Use the API Keys option on the left sidebar of the Golioth Console
2. Click the Create button in the upper right of the main window
3. In the resulting dialog (show above), press the Create button

The final dialog displays the key and some useful path information. We will use this path along with the wss://api.golioth.io/v1/ws/ WebSocket endpoint which I looked up in the Golioth WebSocket docs.

Log into Grafana Cloud. There are a number of services offered, the one we want is simply called Grafana.

On the left sidebar, hover over the gear icon and select Plugins. Search for WebSocket on the next screen and click the “WebSocket API” tile. Follow this 3-step process to add the WebSocket plugin (shown in the screenshots above):

1. Click the Install via grafana.com button
2. On the next screen click the Install plugin button
3. Go back to the previous screen and refresh that page. It may take a minute or two, but when the new button appears click Create a WebSocket API data source

This brings us to setting up our data source.

1. Name can be anything you want
2. Host is the wss://api.golioth.io/v1/ws/ from the Golioth Docs
3. Click the Add header button
   1. Enter x-api-key for the header value
   2. Paste your API key from the Golioth Console in the Value box
4. Click the Save & test button

We have completed the WebSocket connect between Grafana and Golioth. Now it’s time to visualize the incoming data!

3. Graphing Temperature Data in Grafana

In the left sidebar of Grafana, click the big plus icon and choose Dashboard, then click Add Panel.

1. In the Data source box on the left, select the name of the data source you just set up.
2. For now, enter just a $into the Field box
3. Click the Path tab
   1. Follow this format which was shown on the confirmation dialog when you created your API key:/projects/<name of Golioth project>/stream
   2. Notice I have switched from /devices to /stream.
4. At the top of the window in the center, turn the Table view on

You should begin to see JSON data come in over the table view. Remember that WebSockets visualize realtime data, so JSON will only appear when new packets are sent.

Now that we’ve seen the JSON data coming in, we can use it to identify two types of data to use for the graph: temperature and timestamp.

Going back to the Fields tab, we can change the $ character (which means to show all the JSON coming in) to a specific key/value pair.

• Enter $.result.data.timestamp
  • You will see the data in the table-view change as you type, this is helpful in navigating the hierarchy
• Click the + button all the way to the right of the Field box to add another field
• Enter $.result.data.data.temp

When you click the Table view button at the top, the view will be replaced by a visualization of your data. Here you can see it is obvious that I’m using simulated temperature data. You can also change the way it is visualized by clicking on the Time series menu in the upper right.

Conclusion

This is one simple example that is graphing a single stream from one device. But the principle scales with your IoT fleet. Graphing large numbers of devices across multiple projects takes little more than changing how you use your Path and Field settings on the panel. With Grafana you can easily group together multiple panels into a dashboard, with visualization options tailored to almost any data type.