,

Low Volume, High Mix Mechanical Enclosures for IoT Projects

Most of the time we’re writing about how to build out the firmware that will go onto your IoT Devices. Our device Software Development Kits (SDKs) are a key offering that Golioth provides to hardware and firmware engineers; we want it to be easier for you to build the code that will go onto the device out in the field. Once the code is ready to go, you need to think about how you are going to enclose the thing you just made, whether it’s a product or a prototype.

We have been building prototypes in the form of Golioth Reference Designs. These are end-to-end demos that include hardware, firmware, software, and visualization, and we’ve been building a bunch of them. That has meant we are developing a “high mix, low volume” set of products (really prototypes) and have needed to figure out how to utilize enclosures for the different things we’re trying to do.

What’s in the box

Most of our current Reference Designs contain a few key things:

  • A development board, such as the Sparkfun Thing Plus nRF9160 (our current most-used board)
  • Sensor boards or interfaces to external items
  • A battery
  • A centralized board that makes it easy to mount each of the above items

Off-the-shelf hardware like this is less a “product” than it is a first round prototype or even Proof-of-Concept. It can be used to test a business idea or to build requirements for a future high volume product.

Requirements influence design

We have tried a few different ways of enclosing our projects so far. As you’ll read below, each time we built something, we took the lessons we learned and modified our next iteration.

Color demos (Aludel)

Our first round of assumptions was that we needed to make everything waterproof and have maximum flexibility. As such, we decided on the Bud model PN-1324-CMB. It’s a normally gray opaque base with a clear polycarbonate lid where we could have a display element on a top PCB. The main function was that we could implement lots of cabling internally but still have access to Click boards (based on the Mikroelectronica MikroBus standard). The cabling would all go through cable glands to keep out moisture. I also liked that the box had flanges so we could screw this unit to a wall or a display stand.

This method worked out OK but had the downside of requiring each sensor to be either outside the box and cabled in or we needed to “break” the waterproof nature to monitor things like air humidity. Most things we were planning to do involved direct monitoring of the environment. This, in combination with the large overall size of the demos, meant that we thought we could improve on future revisions.

You can see these cases in action (including cover plates that hid the base PCB) in the green and blue demos that we took to Embedded World in 2022.

Trashcan Reference Design – First iteration

Since we were monitoring using a time-of-flight sensor on the Trashcan Reference Design, I knew we would need the sensor to be external to the case. Or at least it would need to be pointing outside the envelope of the case, as shown below. I used a Sparkfun breakout board for the VL53L0X sensor from ST Micro and drilled out a hole to allow the signal to point down into a trashcan and reflect off the top of the trash piled up inside that can. Then I hot glued it in place.

Open case

This is unrealistic in a deployment as a trashcan is a harsh environment, especially deployed somewhere like a national park. The moisture and heat inside a trashcan would require the device be more watertight. After building this iteration and realizing we were not designing something deployable, my mindset around how we needed to make our cases began to change. It would be imperative on anyone who utilizes our reference designs to make revisions, optimizing it for manufacture and hardening it for their specific environmental needs. What is the impact if our Reference Designs are no longer planned to be field ready or watertight?

Trashcan Reference Design – Second Iteration

In the second iteration of the Trashcan Reference Design, I started to optimize for some different elements, most noticeably the repeatability of the design. There is nothing special about the PCB shown below, only that it can be manufactured more easily than the hotglue-laden prototype in the first iteration. Again, we see that the design is not water tight. I did focus on maintaining a smaller form factor than the Aludel, with a low cost case.

Side view of case open

The IoT Trashcan Reference Design with the top of the case open

This one is the Bud CU-1937-MB, part of their “Utilibox” line. Not designed to show much of anything, it’s meant to really be screwed to the wall and forgotten (as many IoT things are). I once again chose a design that had flanges as part of the case for easy mounting on a wall; in this case, it’d be mounted on the underside of a trashcan lid.

The difficulty was in a non-standard cutout. Also the fact that I was using a Dremel hand tool instead of something more precise like a milling machine. This goes back to the “high mix” element, it was tough to justify buying a milling machine and then programming it or getting better at manual machining. I took my best shot with a Dremel (spending quite a bit of time in the process) and then 3D printed a thin case over top of the quite ragged holes that I cut.

Side view of case closed

A side view of the IoT Trashcan Reference Design including 3D printed cover

Most people wouldn’t/won’t see this element of the design, but it’s still important not to have holes cut out that are all jagged and rough.

Aludel Mini

In our most current iteration on display at Embedded World coming up in 2 weeks, I decided to take a different tack, based on some brainstorming with my coworker Mike. We had already re-designed the landing board to once again utilize Click boards from Mikroelectronica, and to fit in the same case (CU-1937-MB); this was called the “Aludel mini”, playing off the name of the original Aludel case design.

It was unclear at first if we could fit the size of the headers (in width and height) into that case, but it turns out we can. We really liked that there is a wide variety of sensors available in that form factor and we can also buy things like the Arduino to Click converter boards to make the same sensors available on platforms like the nRF9160-DK.

So now that we know we are using that same case and that we are somewhat OK with a 3D printed element on that case, I decided to take a leap and have Bud modify the boxes to plan for this flexibility. Instead of having them mill out just the button and USB C port cutouts, I decided to have them basically wipe out two of the walls of the case. That might be a bit extreme, but it allows for ultimate flexibility.

This was a custom order (with an upcharge for doing so, understandably), but it was a pretty seamless process to go back and forth with the manufacturer rep (Pinnacle Marketing, based on my location in the US Southeast) and the manufacturer (Bud Industries) and get these made. They did warn that the structural integrity of the case would be impacted, but I moved forward knowing we would be able to reinforce with the inserts we design.

Now the trick was to design something that could fit this shape. They have a high draft angle, ostensibly to make it an easier ejection from injection molding equipment, but that makes for some odd trapezoidal walls. I brought the supplied 3D model into FreeCAD and set to work printing and iterating on the design. Here’s the shape I ended up with:

I utilized the mounting posts for the top case as a reinforcing element. For most of the designs I put this into, I’ll glue the plate in place, but it is also designed to slide in and out for prototypes.

On the other side of the board, we expect there to be much more variation. Often this is where wires or extension cables are interfacing with the case. I took the base plate and was able to draw simple shapes onto the face plate to create a variation for each reference design that we make. This will depend on the type of Click board we have plugged into the Aludel Mini. For the Soil Moisture Reference Design, you can see that I made a cutout that allows the Click Shuttle Expander to go out to individual boards:

Soil Moisture Monitor closed with sensors closed

The obvious question

With all of these modifications, the obvious question is: Why not 3D print the entire thing?

A fair question, and definitely something that I might consider in a future iteration. I would still want to maintain many of the same features that we have put into the existing designs (relatively small envelope, mounting flanges, plain exterior). The cost would be roughly the same if sent to China for 3D printing (depending on material used), when compared to a low volume run of modified cases + printed inserts. I also like the implied constraints of choosing an enclosure and then needing to work around those constraints, although the exercise changes when designing a product. It prevents adding “just one more thing”, when in our case we’re primarily trying to showcase the platform and not the hardware itself.

High mix is not the norm

Most product companies work differently. If you are spending enough money to hire engineers to design a product, you are likely making a higher volume of devices to support the salaries of engineers. In that case, your constraints change quite a bit. When moving to a higher volume manufacturing run, it makes sense to take everything you learn from a Golioth Reference Design and spin it into a higher volume product (custom PCBs, integrated sensors, more complex battery management, bespoke watertight enclosures, etc). If you want to take Reference Designs for a spin and use them as the basis of your next product, we’d love to talk. If you’re more interested in chatting about how to optimize cases for a variety of different designs, post about it on our forum!

/by

Nordic’s brand new nRF7002 Wi-Fi chip already works with Golioth

Eli Hughes is the principal of Wavenumber LLC which delivers positive outcomes in the areas of embedded systems, software, IOT, audio, acoustics, industrial design, and content creation.

My experience with Nordic microcontrollers and modern cloud connectivity began in December of 2017. However, it is only this month that Nordic Semiconductor finally released a Wi-Fi capable component. The nRF7002 is a low power Wi-Fi 6 helper chip (no microcontroller internally) that is deeply integrated with the Nordic Connect SDK (NCS). This article details how I got it booted and why I decided to make my first foray into using it with the Golioth Cloud.

Using Golioth for my 1st nRF7002 project

The nRF7002-DK arrives on my bench

Most of my experiences straddle the worlds of hardware and software. Network connectivity, security, and cloud “backend” can get complex quickly. For hardware-oriented minds, navigating the complexities of modern data transport are daunting. Gone are the days when adding embedded “internet connectivity” meant all one had to do was send a raw UDP packet!

Likewise, folks experienced in modern web backend technologies infrequently approach embedded development. Golioth appeared to be a bridge between these two worlds. Providing engineers on the embedded side tools for easily and securely connecting to the cloud is a big deal. Likewise, an experienced backend engineer will love hearing about access via REST APIs.
The arrival of the nRF7002-DK was a great opportunity for me to “kick the tires”. I also had personal interest in the secure approach to CoAP with DTLS.

Setup to “Live on the Edge” with nRF7002-DK

At the time of this writing, Zephyr support for the nRF7002 is still in active development. One of my soapbox pitches for Zephyr is that it is much more than an RTOS. It comes with a testing framework, open community development model, and packaging system. I could see the current nRF7002 activity on the main branch of the sdk-nrf repository. The last official release was v2.2. To experiment with the nRF7002 I needed to work from main or with one of the v2.2.99 development tags.

Observing nRF7002 development activity

Zephyr’s west tool is used to manage packages and dependencies through a west manifest. Golioth takes full advantage of this system by building their Golioth Zephyr SDK with the NCS in mind. I knew in advance that I would be hacking on samples in the Golioth SDK tree and that I would also need to use a bleeding edge version of the nRF SDK. In this case, I chose to use the fork approach for development. Namely, I forked the Golioth SDK to give myself a personal sandbox.

Forking the Golioth Zephyr SDK

This approach has the advantage that I can experiment with the Golioth samples and submit a pull request to the Golioth team in case I come up with something useful. With my fork in place, the west manifest can be modified so I can be working with the latest nRF7002 examples.
Inside the Golioth SDK is west-ncs.yml, the manifest file for Nordic specific development.

A “one liner” to be on the bleeding edge

It literally is a “one-liner” to be on the bleeding edge, From here, I pull everything down to my local machine to get started.

Note: The assumption here is that your machine has already been setup using the Zephyr getting started guide

Initializing my “sandbox” fork of the Golioth SDK

The west update operation can take a while the first time. It is pulling down all the repositories specified in the manifest.

Basic Verification of the nRF7002

Before I dove into the Golioth samples, I loaded a nRF7002 sample to verify Wi-Fi connectivity. Inside of nrf/samples/wifi/shell is a shell sample which exposes Wi-Fi and network behaviors.

Kicking off a “pristine” build using the nRF7002dk_nrf5340_cpuapp board

The nRF7002-DK has a J-Link debugger onboard and I prefer the Ozone programming/debugging experience. Note that the nRF7002 is a Wi-Fi companion the nRF7002-DK uses a nRF5340 as the host processor. The Zephyr Wi-Fi shell is a helpful tool. I could quickly connect to my guest network and perform network operations such as a ping and DNS query.

Using the Zephyr Wi-Fi Shell for connectivity verification

Getting to Golioth

I choose to start with the Golioth logging sample located in modules/lib/golioth/samples/logging inside the forked repository. Logging is a fundamental Zephyr feature and Golioth implements a logging backend to get data up to the cloud with minimal hassle. Since the nRF7002 board is on the bleeding edge, an nRF7002dk_nrf5340_cpuapp.conf board configuration had to be added to the logging sample. The Golioth examples have board Kconfig overlays located in boards directory inside of the example application folder. These overlays are used to select settings tailored for specific board or hardware platforms.

Adding a nRF7002dk_nrf5340_cpuapp board Kconfig overlay for the Golioth Logging Sample

I started by using the esp32.conf as a template, renamed it to nRF7002dk_nrf5340_cpuapp.conf and then started hacking. Since I had just run the Wi-Fi shell example, I copied all the settings in the prj.conf from the shell application example. This approach is a bit overkill, but it is easy to remove unneeded settings once everything is functional.

I also had a hunch that TLS configuration might be an issue. Nordic has their own fork of mbedTLS, so I copied in the TLS settings from nrf9160dk_nrf9160ns.conf in the logging/boards folder

Working in a Nordic Specific TLS Configuration

Next I configured the application prj.conf with a device PSK_ID and PSK that I created via the Golioth control panel. FluffyBunny1 is now provisioned!

Adding in a Test PSK and ID

To get the Golioth Wi-Fi sample to auto-magically connect to my network at boot, the Wi-Fi SSID and password were added to the prj.conf as well. This is required because we haven’t pulled in the settings subsystem, which would allow me to configure things over UART.

Configuring the sample to auto connect to my Wi-Fi Network

One of the most frustrating “living on the edge” was trying to remember the password for my Wi-Fi network. Is it possible this task consumed the majority of my time on getting things booted? My frantic, scattered brain almost resorted to the “emergency” button on the route. Lucky for me, my spouse came to the rescue. Starting with the stock logging example, I personalized the main loop to be FluffyBunny1 friendly.

The FluffyBunny1 Logging Experiment

FluffyBunny1 is reporting!

Great success! Secure communication to the cloud in minutes. Mind you, I had to do some hacking to get support for bleeding edge hardware, but getting this result was impressive.
Me being me, I couldn’t stop after seeing log messages flow to the Golioth backend.

Enabling Golioth RPC functionality

I had read about the ability to trigger a function on the device by using Remote Procedure Calls (RPCs) on Golioth and wanted to try extending this demo. I sprinkled in a bit of code to define an RPC callback and register it. It was surprisingly simple.

Code modifications to add a bit of RPC functionality

Adding in this function allowed us to “pet the fluffy bunny” directly from the control panel. Let’s look at how it happens on the console + the uart output.

Fluffy Bunny demo

Wrapping Up

My experience with Golioth and the nRF7002 was very smooth! I was quite impressed how quickly I achieve communication with the cloud backend. Even with the “bleeding edge” nature of the nRF7002, the Zephyr ecosystem simplified bring-up. Having Golioth essentially work “out of the box” on new hardware is a big plus.

I already have an idea for a new project using Golioth with custom sensors and the nRF7002. In a previous life I was involved with “raw” postgres and TimeScaleDB. After seeing Golioth LightDB Streams, I believe I have an approach to my newfound hobby: Bonsai monitoring.

I hope you can check out Golioth for your next connected project. The overall experience was Zen-like.

/by
, ,

Using VScode for ESP-IDF and Golioth

Visual Studio Code, colloquially known as VScode, is among the most popular integrated development environments (IDEs). Today we’re going to walk through the process of setting up ESP-IDF in VScode and using it to run Golioth device management example code on an ESP32.

Not everyone likes to live their lives hammering away at a command prompt. What we’ll cover today is another option which uses Espressif’s VScode extension (plugin) to largely automate how you work with the Espressif IoT Development Framework (ESP-IDF). That means nice buttons and interfaces to build, flash, and monitor applications for the ESP32 family of chips.

Installing VScode and the ESP-IDF extension

As a prerequisite you will need to have VScode installed. If you don’t, head over to the download page and do so now.

ESP-IDF VScode extension

Open VScode and click on the extensions icon (looks like four boxes) on the left sidebar. Type esp-idf into the search bar that appears and the top result will be “Espressif IDF”. Click the install button and you’re off to the races.

Configure ESP-IDF VScode Extension

You will be greeted with options for installing the various ESP-IDF tools. If you don’t have an opinion on how things are installed you can choose the automatic route. I wanted to specify what directories were used during the install so I chose the manual route and used the settings above.

It will take a few minutes for everything to download. You will want to click on the “Download ESP-IDF Tools” to ensure that the compilers and other tools are downloaded (in addition to the Espressif SDK). If you need more help, check out Espressif’s installation guide.

Installing the Golioth Firmware SDK

Now that VScode and the ESP-IDF are installed, let’s take a moment to install the Golioth Firmware SDK. This provides the tools and sample code for connecting your ESP32 to Golioth.

Cloning the Golioth Firmware SDK

VScode has a handy tool for cloning git repositories. Bring up the command palette (ctrl-shift-p), type in gitcl, and press enter. A prompt will open in the same window for you to enter the following URL:

https://github.com/golioth/golioth-firmware-sdk.git

After pressing enter, a window will open where you can select a folder to store the repository. A folder called golioth-firmware-sdk will be placed in that location.

VScode will ask if you want to open the cloned repository. Please click Cancel on this window. The Golioth Firmware SDK supports multiple platforms and we will open the ESP-IDF specific directory in the next step.

Open the project and update the git submodules

Now that the repository has been cloned, let’s open the sample code in VScode. Click File→Open Folder and navigate to the golioth-firmware-sdk/examples/esp_idf/golioth_basicsfolder, then click Open.

The Golioth SDK includes a few packages as submodules and these must be updated before continuing. We’re going to use the terminal for this step. In VScode click Terminal→New Terminal. A terminal window will open in the golioth-basics folder. Type this command to update the submodules:

git submodule update --init --recursive

Type exit to close the terminal window.

Build, flash, and monitor the golioth-basics application

Now that everything is installed we get to see the ease of using an IDE.

ESP-IDF Build, Flash and Monitor

The bar along the bottom of the VScode window includes icons for working with the ESP-IDF tools. Make sure your ESP32 is plugged into USB. Click the flame-shaped icon which will build the project, flash it to the ESP32, and open a serial connection to the chip.

The build will take place and then VScode will open a window in the top center prompting you to select JTAG/UART/DFU. We will be using UART. Also note that there is a selection in the bottom menu bar where you can set the port that will be used when flashing (the image above shows /dev/ttyUSB1 in my case).

Assign device credentials in the monitor window

ESP32 running the golioth-basics app

The golioth-basics app will begin running immediately and you should see an output from the chip in a window inside VScode. We need to give the chip WiFi and Golioth credentials so that it can connect to the cloud.

If you have not yet signed into Golioth, our Dev Tier is free for your first 50 devices. (Tip: there is a Console Overview on our docs that will walk you through creating a set of device credentials.) Get your Golioth credentials, and the login info for your WiFi access point, and pass them to the chip using this command format:

settings set wifi/ssid YourWiFiAccessPointName
settings set wifi/psk YourWiFiPassword
settings set golioth/psk-id YourGoliothDevicePSK-ID
settings set golioth/psk YourGoliothDevicePSK

assign credentials to the device

Once you’ve set the credentials, type reset and the ESP32 will reboot, connect to WiFi and then to Golioth.

Successful connection to Golioth

Wrapping up

You’ve successfully compiled, flashed, and run a demo Golioth application for ESP-IDF using VScode. The same principles can be applied to your own projects.

If you’d like to dig deeper into how the golioth-basics code works, I encourage you to study the golioth_basics.c file in the golioth-firmware-sdk/examples/common folder. It demonstrates all of the Golioth device management features like OTA firmware updates, remote procedure calls (RPC), IoT fleet settings service, LightDB State and LightDB Stream data services, and remote logging.

We’d love to hear about the projects your working on. Share your successes and post your question on the Golioth Forums. If you’re interested in learning how to add Golioth to your IoT fleet, get in touch with our Developer Relations crew.

References:

 

/by

Trialing PlatformIO and Arduino with Golioth (Labs)

Over the holidays I tried out a fun experiment that combines Arduino and PlatformIO with Golioth. The work was prompted by a customer with an existing Arduino code base. Their long-term plan is to port the code over to ESP-IDF, but this serves as an intermediary step while they work out their own library transition plans. It works for them and today I’ll cover how you can give it a try for yourself. But first, some disclaimers.

Golioth Labs means Experimental

We use the Golioth Labs organization on GitHub for our experimental projects. That means you should not build for production devices using any of those repositories. This experiment targets the ESP32 family of chips. We have full support for ESP-IDF in the Golioth Firmware SDK, which is what we recommend using.

That being said, what if you have an existing Arduino project and want to test out some Golioth features? This is a “what am I getting myself into” step, and depends on your code being able to run on an ESP32 since the experiment builds Arduino as a component of ESP-IDF. The “cherry on top” wraps the project up into PlatformIO, an IDE built around VScode that has become a popular alternative to the native Arduino IDE.

Test Driving Golioth + PlatformIO + Arduino

As a prerequisite you need to install VScode and the PlatformIO extension for it.

With those tools in place, let’s walk through how to clone the Golioth repository and set up submodules, then finish up by building and running the demo program.

Clone the Repository and Submodules

By default, PlatformIO stores new projects in the ~/Documents/PlatformIO/Projects folder. You may clone this project anywhere you like, but that’s as good of a place as any.

Navigate to your preferred folder and clone the GoliothLabs repo and submodules:

# Clone the repository
git clone https://github.com/goliothlabs/golioth_platformio_arduino.git
# Enter the newly created folder
cd golioth_platformio_arduino
# Initialize the submodules (this will take more than 10 minutes)
git submodule update --init --recursive
# Clone a nested repository (yes, this is odd)
git clone https://github.com/hathach/tinyusb.git third_party/esp32-arduino-lib-builder/components/arduino_tinyusb/tinyusb

That’s it for setup. The last step in the commands listed above is an odd one. According to Espressif’s Arduino as an ESP-IDF component documentation, the tinyusb library needs to be nested inside of the esp32-arduino-lib-builder library’s components folder.

Build and Run the Golioth Sample

  1. Start by importing the project into PlatformIO.In VScode, click the alien icon on the left sidebar, choose PIO Home→Open from the sidebar that appears, and select Open Project from the PlatformIO home screen.PlatformIO open project
  2. Next, set up your credentials.Make a copy of the credentials.h_example file and rename it credentials.h. In this file, place the credentials for your WiFi access point, and the PSK-ID/PSK for the device you created using the Golioth Console.Note: With our Dev tier, your first 50 devices are free. Use the Credentials tab when viewing a device to access the PSK-ID/PSK.credentials.h file
  3. Now build the project.Click on the alien icon on the left sidebar and choose Build from the Project Tasks menu. A terminal window will open and the build process will begin.PlatformIO build process
  4. Finally, flash the device and observe the output.From the Project Tasks menu, click on Upload and Monitor. The project will build, upload to the ESP32, and then open a terminal window to show the serial output from the ESP32.

You will see an output similar to the following:

I (942) wifi:new:<6,0>, old:<1,0>, ap:<255,255>, sta:<6,0>, prof:1
I (944) wifi:state: init -> auth (b0)
I (952) wifi:state: auth -> assoc (0)
I (957) wifi:state: assoc -> run (10)
I (970) wifi:connected with golioth-staff, aid = 1, channel 6, BW20, bssid = c2:ff:d4:a8:fa:10
I (970) wifi:security: WPA2-PSK, phy: bgn, rssi: -39
I (977) wifi:pm start, type: 1

I (1041) wifi:AP's beacon interval = 102400 us, DTIM period = 2
.W (1556) wifi:<ba-add>idx:0 (ifx:0, c2:ff:d4:a8:fa:10), tid:0, ssn:1, winSize:64
␛[0;32mI (1727) esp_netif_handlers: sta ip: 192.168.1.157, mask: 255.255.255.0, gw: 192.168.1.1␛[0m
.IP address: 
192.168.1.157
␛[0;32mI (1936) golioth_mbox: Mbox created, bufsize: 2184, num_items: 20, item_size: 104␛[0m
␛[0;32mI (1937) golioth_fw_update: Current firmware version: 1.0.0␛[0m
Hello, Golioth! #0
␛[0;32mI (1953) golioth_example: Updating counter to: 0␛[0m
␛[0;32mI (1987) golioth_coap_client: Start CoAP session with host: coaps://coap.golioth.io␛[0m
␛[0;32mI (1988) golioth_coap_client: Session PSK-ID: platformio-test@golioth-arduino-demo␛[0m
␛[0;32mI (1998) libcoap: Setting PSK key
␛[0m
␛[0;32mI (2005) golioth_coap_client: Entering CoAP I/O loop␛[0m
Hello, Golioth! #1
␛[0;32mI (6955) golioth_example: Updating counter to: 1␛[0m
␛[0;33mW (9968) golioth_coap_client: CoAP message retransmitted␛[0m
␛[0;32mI (11492) golioth_coap_client: Golioth CoAP client connected␛[0m
Hello, Golioth! #2
␛[0;32mI (11957) golioth_example: Updating counter to: 2␛[0m
␛[0;32mI (13544) golioth_fw_update: Waiting to receive OTA manifest␛[0m
Hello, Golioth! #3
␛[0;32mI (16959) golioth_example: Updating counter to: 3␛[0m
Hello, Golioth! #4
␛[0;32mI (21961) golioth_example: Updating counter to: 4␛[0m
Hello, Golioth! #5
␛[0;32mI (26963) golioth_example: Updating counter to: 5␛[0m
^CHello, Golioth! #6
␛[0;32mI (31965) golioth_example: Updating counter to: 6␛[0m
Hello, Golioth! #7
␛[0;32mI (36967) golioth_example: Updating counter to: 7␛[0m

In this example, we are sending Arduino Serial.print() commands and Golioth logs that are displayed on the terminal and sent to the server. They show an upcounting timer that is stored on the Golioth LightDB state service. Over-the-Air (OTA) firmware update is also available.

While this shows a very small subset of Golioth features, everything shown in our golioth_basics example code can be used with this project.

What to do Next

As I mentioned earlier, you should not build production hardware around this repo. This is merely a way to test drive Golioth with your existing Arduino code base. The longer term goal should be to transition your ESP32-based hardware over to the ESP-IDF flavor of the Golioth Firmware SDK. If you’re using non-ESP32 hardware, you should target the Golioth Zephyr SDK.

While Arduino and PlatformIO are great platforms, directly targeting the ESP-IDF framework results in a more stable and predictable build. The Golioth SDK is built as a component of ESP-IDF for rock-solid performance. Building for ESP-IDF is definitely the end goal if you are migrating an existing fleet, and the best place to start if you are bootstrapping new designs.

Do you have questions about how to best integrate Golioth for managing your IoT devices? We’d love to hear from you! Post a message on the Golioth Forum, or reach out to our Developer Relations folks to set up a meeting.

/by
,

Golioth Ran My Furnace This Christmas

The human body is surprisingly adept at sensing temperature. And so when I sat up in bed my body immediately informed me something was not right with my furnace. Yes, it’s winter in Wisconsin, but it shouldn’t feel that cold inside the house. The furnace was fine, but the thermostat was not, and so begins the story of how Golioth ran my furnace over Christmas.

Twas the Week Before Christmas and Cold in the House

HVAC control panel

EIM that connects to the furnace/AC/HRV. The red/green wires exiting the bottom of the photo go to the Golioth Greenhouse Controller

I arose from my slumber and went to check the thermostat to find it displaying a message that it could not connect to the Equipment Interface Module (EIM). We have a furnace, air conditioner, and a heat recovery ventilator (HRV is a fancy name for a pair of fans that exchange stale inside air with fresh outside air). Being a geek, I wanted one way to control them all. That meant installing the Honeywell EIM to switch those subsystems. The thermostat then connects wirelessly to the EIM to provide automated control. But the chilly morning we’re discussing here, the thermostat wasn’t connecting at all!

After basic troubleshooting I used a jumper wire to short the white “run the furnace now” wire to the 24-volt red wire. The furnace kicked on and started warming up the house. There was nothing wrong with the furnace, but I no longer had an automatic way to control it.

But wait…industrial control is one place where Golioth shines! Could I use Golioth as my furnace controller until the built-in system was repaired?

Connecting the Golioth Greenhouse Controller to My House

Golioth Greenhouse Controller connected to the EIM

The Golioth Greenhouse Controller was placed outside of the utility closet to sense temperature readings from the room.

Just a week before this happened I wrote a blog post about the Golioth IoT Greenhouse Controller. It includes relays that can be switched automatically based on a temperature sensor. This is the definition of a thermostat. And these Golioth reference designs are built to be easily adaptable to similar applications. The hardware connection was dead simple: just run the red and white wires to the temperature-controlled relay on the greenhouse controller.

The logic in the firmware needed a quick tweak as greenhouses regulate temperature by taking action when it gets too hot (the opposite of heating a house). This required the “expert” firmware engineering step of changing a greater-than sign to a less-than sign. Luckily all of our reference designs have Over-the-Air (OTA) firmware update built-in so I was able to make this change while the device was in my basement, still connected to the furnace.

The temperature threshold is set via Golioth’s web console, so at that point I had actually upgraded my home system to include control from anywhere in the world! I also got a dashboard to keep my eye on temperature, pressure, and humidity.

Eating Your Own Dog Food (and Enjoying the Flavor)

So, why didn’t I just go out and buy another thermostat? The first reason is the ongoing chip shortage (now into it’s third year). The EIM part that was broken is about $80 normally, but there’s a 31 week lead time on it right now. The second reason is that the only wires running from the furnace to the thermostat are for supplying power (+24 V and Ground). Not only would a replacement thermostat be temporary, I would need to run new wires to give it furnace control or leave it in the basement.

For me this was a great philosophy experiment. We spend a lot of time planning and developing reference designs. Now I know a bit more about the user experience, and I like it! The hardware form-factor was easy to work with, and the approaches we take to control and feedback worked well for my application.

Temperature graph with many on/off cycles

Temperature control without hysteresis

That doesn’t mean it was a perfect fit. Our basic design doesn’t have hysteresis built in, which means that the controller was trying to cycle more often than normal as there was no deadband to allow the house to warm (or cool) a bit past the target temperature. I implemented a simple cycle time threshold that prevented switching between on and off more than once in a 15 minutes period. It’s an application-specific topic that could be added to an implementation guide for this reference design.

Temperature and humidity graphs with smooth rise and smooth response due to cycle-time control.

Temperature response over the same time period was much smoother with simple cycle-time control

I called several HVAC shops in town and the earliest estimates for a new part was the end of March. Online I was able to find a Honeywell “starter” kit included the replacement EIM with just a week before delivery. During this time the Golioth-controlled furnace kept the house warm, but also getting very very humid without the HRV to help regulate. Good news everyone, the Greenhouse demo has a humidity sensor and a second relay. You guessed, it. I added control for the ventilation as well!

I Built An IoT Thermostat in 20 Minutes

Now, I was already familiar with this demo unit, so take this with a grain of salt. But the purpose of the Golioth reference designs is to give businesses a running start when developing a proof of concept. I was able to substitute this one in place of my thermostat with just about 20 minutes of effort. It kept my house comfortable in the harshest conditions of December’s arctic blast, and it let me check temperature and adjust settings while I was working outside of the home during that time. This is the Internet of Things, and it’s never been easier.

/by
,

How to add Golioth to any ESP-IDF project

This post has been superseded by a new post on how to add the Golioth FW SDK to an ESP-IDF project, as of December 2024. The post below will be kept for historical reasons, but please utilize the steps in the new post going forward.

Adding Golioth’s device management features to an existing ESP-IDF project is a snap. The Golioth Firmware SDK has excellent support for Espressif parts. Simply add Golioth as a component in your build, enable it in the CMake and Kconfig files, and start using the API functions.

Today I’m going to walk through how to do this using an ESP32 development board, however, any of the ESP32 family of chips (ESP32s2, ESP32c3, etc.) will work. Advanced users will also want to review the Golioth Firmware SDK Integration Guide which is the bases for this post.

Walkthrough

Since this article is about adding Golioth to an existing ESP-IDF application, I assume you already have the ESP-IDF installed. If you do not, you may consider following our ESP-IDF quickstart guide before continuing.

For illustration purposes, I’ve copied Espressif’s /esp-idf/examples/wifi/getting_started/station example code to a folder called esp_hello_golioth and will add the Golioth device management features to the program.

cp -r ~/esp-idf/examples/wifi/getting_started/station esp_hello_golioth

If you are following along, you will want to set your WiFi credentials either by using menuconfig or by changing the default values in main/Kconfig.projbuild.

1. Install Golioth as a component

Add the Golioth Firmware SDK as a submodule of your project. (Note that this means you need to have your project under version control. Hint: git init)

cd ~/esp_hello_golioth
git submodule add https://github.com/golioth/golioth-firmware-sdk.git third_party/golioth-firmware-sdk
git submodule update --init --recursive

2. Add the Golioth SDK to your CMake configuration

We need to tell CMake to include the Golioth SDK in the build. This is done by editing the project’s top-level CMakeLists.txt file and adding the following line before the project() directive in that file:

list(APPEND EXTRA_COMPONENT_DIRS third_party/golioth-firmware-sdk/port/esp_idf/components)

We also need to add the Golioth SDK as a dependency. This is done by editing the deps list in the CMake file in the main directory of the project. For this ESP-IDF example code, there are no existing deps so I added the following to the top of main/CMakeLists.txt:

set (deps
     "golioth_sdk"
)

3. Add Kconfig settings

Golioth needs MbedTLS so it must be enabled via the Kconfig system. I created an sdkconfig.defaults file in the top-level folder and added the following:

CONFIG_MBEDTLS_SSL_PROTO_DTLS=y
CONFIG_MBEDTLS_PSK_MODES=y
CONFIG_MBEDTLS_KEY_EXCHANGE_PSK=y
CONFIG_GOLIOTH_AUTO_LOG_TO_CLOUD=1
CONFIG_GOLIOTH_COAP_REQUEST_QUEUE_MAX_ITEMS=20

The final two symbols were added because I plan to send logging messages to the Golioth servers.

4. Use Golioth API calls in your C code

Now that we’ve unlocked Golioth we can start making API calls. Gain access to the functions by including the Golioth header file:

#include "golioth.h"

For this demo I created a counter to keep track of a forever loop. After a five-second pause, the counter value is sent to Golioth as a Log message, then sent as a LightDB State value. The highlighted code beginning at line 14 is what I added.

void app_main(void)
{
    //Initialize NVS
    esp_err_t ret = nvs_flash_init();
    if (ret == ESP_ERR_NVS_NO_FREE_PAGES || ret == ESP_ERR_NVS_NEW_VERSION_FOUND) {
      ESP_ERROR_CHECK(nvs_flash_erase());
      ret = nvs_flash_init();
    }
    ESP_ERROR_CHECK(ret);

    ESP_LOGI(TAG, "ESP_WIFI_MODE_STA");
    wifi_init_sta();

    //Everything above this line is unchanged from the ESP-IDF example code

    const char* psk_id = "your-golioth@psk-id";
    const char* psk = "your-golioth-psk";

    golioth_client_config_t config = {
            .credentials = {
                    .auth_type = GOLIOTH_TLS_AUTH_TYPE_PSK,
                    .psk = {
                            .psk_id = psk_id,
                            .psk_id_len = strlen(psk_id),
                            .psk = psk,
                            .psk_len = strlen(psk),
                    }}};
    golioth_client_t client = golioth_client_create(&config);

    uint16_t counter = 0;
    while(1) {
        GLTH_LOGI(TAG, "Hello, Golioth! #%d", counter);
        golioth_lightdb_set_int_async(client, "counter", counter, NULL, NULL);
        ++counter;
        vTaskDelay(5000 / portTICK_PERIOD_MS);
    }
}

For simplicity I hardcoded the Golioth credentials (PSK-ID/PSK) which is not a best practice as hardcoded credentials will not survive a firmware upgrade. Both shell and Bluetooth provisioning are demonstrated in the golioth_basics example code which stores these credentials in non-volatile flash.

See the results on the Golioth Console

The serial output of this app shows the Golioth client connecting and Log messages being sent.

I (937) wifi:AP's beacon interval = 102400 us, DTIM period = 2
I (1647) esp_netif_handlers: sta ip: 192.168.1.159, mask: 255.255.255.0, gw: 192.168.1.1
I (1647) wifi station: got ip:192.168.1.159
I (1647) wifi station: connected to ap SSID:TheNewPeachRepublic password:123456789101112internetplease
I (1657) golioth_mbox: Mbox created, bufsize: 2184, num_items: 20, item_size: 104
I (1667) wifi station: Hello, Golioth! #0
W (1677) wifi:<ba-add>idx:0 (ifx:0, c6:ff:d4:a8:fa:10), tid:0, ssn:1, winSize:64
I (1677) golioth_coap_client: Start CoAP session with host: coaps://coap.golioth.io
I (1697) libcoap: Setting PSK key

I (1697) golioth_coap_client: Entering CoAP I/O loop
I (1917) golioth_coap_client: Golioth CoAP client connected
I (6707) wifi station: Hello, Golioth! #1
I (11707) wifi station: Hello, Golioth! #2
I (16707) wifi station: Hello, Golioth! #3
I (21707) wifi station: Hello, Golioth! #4
I (26707) wifi station: Hello, Golioth! #5
I (31707) wifi station: Hello, Golioth! #6
I (36707) wifi station: Hello, Golioth! #7
I (41707) wifi station: Hello, Golioth! #8
I (46707) wifi station: Hello, Golioth! #9

Both Log messages and State data can be accessed from the tabs along the top of the device view in the Golioth Console. Navigate there by selecting Devices from the left sidebar menu and then choosing your device from the resulting list.

The log view shows each message as it comes in, and from the timestamps we can see that the five-second timer is working. For long-running operations, there are time-window selection tools as well as filters for log levels, devices, and modules.

The LightDB State view shows persistent data so you will always see the most recently reported value at the “counter” endpoint. Make sure the refresh dialog in the upper right is set to Auto-Refresh: Real-time to ensue you see the updates as they arrive from the device.

Give Golioth a try!

We built Golioth to make IoT design easier for firmware developers. We’d love it if you took our platform for a test-drive! With our Dev Tier, your first 50 devices are always free. Grab an ESP32 and you’ll have data management, command and control, and OTA firmware update in your toolbelt in no time.

/by
, ,

ZDS: What Chip Shortage? How We Use Zephyr for Truly Modular Hardware

Even people who have been living under a rock for the past couple of years know that there’s a global chip shortage. The correct response from the engineering community should be changes to our design philosophy and that’s the topic of the talk that Chris Gammell and I gave at the 2022 Zephyr Developer Summit back in June. With the right hardware and firmware approach, it’s possible to design truly modular hardware to respond to supply chain woes.

Standardization enabled the industrial revolution, and the electronics field is a direct descendant of those principles. You can rest easy in knowing that myriad parts exist to match your chosen operating voltage and communication scheme. But generally speaking it’s still quite painful to change microcontrollers and other key-components mid-way through product design.

Today’s hardware landscape has uprooted the old ways of doing things. You can’t wait out a 52-week lead time, so plan for the worst and hope for the best. Our talk covers the design philosophies we’ve adopted to make hardware “swapability” possible, while using Zephyr RTOS to manage the firmware side of things.

Meet the Golioth Aludel

Golioth exists to make IoT development straightforward and scalable–that’s not possible if you’re locked into specific hardware, especially these days. So we designed and built a modular platform to showcase this flexibility to our customers.

Golioth Aludel prototyping platform internal view

Called Aludel, Chris Gammell centered the design around a readily available industrial enclosure. The core concept is a PCB base that accepts any microcontroller board that uses the Feather standard. This way, every pin-location and function is standardized. Alongside the Feather you’ll find two headers that use the Click boards standard (PDF), a footprint for expansions boards facilitating i2c, SPI, UART, and more. The base includes Qwiic and Stemma headers for additional sensor connectivity, as well as terminal blocks, room for a battery, and provisions for external power.

The key customization element for Aludel is the faceplate. It’s a circuit board that can deliver a beautiful custom design to match any customer request. But on the inside, each faceplate has the same interface using a flat cable connection to the base board. Current versions of the faceplate feature a a screen, an EEPROM to identify the board, and a port expander that drives buttons, LEDs, and whatever else you need for the hardware UI.

The beauty of this is that electrically, it all just works. Need an nRF52, nRF9160, ESP32, STM32, or RP2040? They all already exist in Feather form factor. Need to change the type of temperature sensor you’re using? Want to add RS485, CANbus, MODBUS, 4-20 ma communication, or some other connectivity protocol? The hardware is ready for it–at most you might need to spin your own adapter board.

Now skeptics may look at the size of this with one abnormally high-arched eyebrow. But remember, this is a proof-of-concept platform. You can set it up for your prototyping, then take the block elements and boil them down into your final design. Prematurely optimizing for space means you will have many many more board runs as the parts change.

When you do a production run and your chip is no longer available, the same concepts will quickly allow you to test replacements and respin your production PCB to accommodate the changes.

Zephyr Alleviates your Firmware Headaches

“But wait!” you cry, “won’t someone think of the firmware?”. Indeed, someone has thought of the firmware. The Linux Foundation shepherds an amazing Real-Time Operating System called Zephyr RTOS that focuses on interoperability across a vast array of processor architectures and families. Even better, it handles the networking stack and has a standardized device model that makes it possible to switch peripherals (ie: sensors) without your C code even noticing.

We use Zephyr to maintain one firmware repository for the Aludel hardware that can be compiled for STM32F40, nRF52840, nRF9160, and ESP32 using your choice of Ethernet, WiFi, or Cellular connections. How is that even possible?

&i2c0 {
	bme280@76 {
		compatible = "bosch,bme280";
		reg = <0x76>;
		label = "BME280_I2C";
	};
};

&spi1 {
	compatible = "nordic,nrf-spi";
	status = "okay";
	cs-gpios = <&gpio0 3 GPIO_ACTIVE_LOW>,
	           <&gpio0 27 GPIO_ACTIVE_LOW>,
	           <&gpio1 8 GPIO_ACTIVE_LOW>;
	test_spi_w5500: w5500@0 {
		compatible = "wiznet,w5500";
		label = "w5500";
		reg = <0x0>;
		spi-max-frequency = <10000000>;
		int-gpios = <&gpio0 2 GPIO_ACTIVE_LOW>;
		reset-gpios = <&gpio0 30 GPIO_ACTIVE_LOW>;
	};
};

/ {
    aliases {
        aludeli2c = &i2c0;
        pca9539int = &interrupt_pin0;
    };
    gpio_keys {
        compatible = "gpio-keys";
        interrupt_pin0: int_pin_0 {
			gpios = < &gpio0 26 GPIO_ACTIVE_LOW >;
			label = "Port Expander Interrupt Pin";
		};
	};
};

Zephyr uses the DeviceTree standard to map how all of the hardware is connected. A vast amount of work has already been done for you by the manufacturers who maintain drivers for their chips and supply .dts (DeviceTree Syntax) files that specify pin functions and mappings. When writing firmware for your projects you supply DeviceTree overlay files that indicate sensors, buttons, LEDs, and anything else connected to your design. The C code looks up each device to receive all relevant information like device address, GPIO port/pin assignment and function.

const struct device *weather_dev = DEVICE_DT_GET_ANY(bosch_bme280);
sensor_sample_fetch(weather_dev);
sensor_channel_get(weather_dev, SENSOR_CHAN_AMBIENT_TEMP, &temp);
sensor_channel_get(weather_dev, SENSOR_CHAN_PRESS, &press);
sensor_channel_get(weather_dev, SENSOR_CHAN_HUMIDITY, &humidity);

The hardware abstraction doesn’t stop there. Zephyr also specifies abstraction for sensors. For instance, an IMU will have an X, Y, and Z output — Zephyr makes accessing these the same even if you have to move to an IMU from a different manufacturer due to parts shortages. You update the overlay files for the new build, and use a configuration file to turn on any specific libraries for the change, but the code you’re maintaining can continue on as if nothing happened.

Of course there are caveats which we cover in the talk. There is no one-size-fits-all in embedded engineering so you will eventually need to define your own .dts files for custom boards, and add peripherals that are unsupported in Zephyr. This is not a deal breaker, there is a template example for adding boards (and I highly recommend watching Jared Wolff’s video on the subject). And since Zephyr is open source, your customizations can be contributed upstream so others may also benefit.

Hardware will never again be the same

Electronics manufacturing will eventually emerge from the current shortages. But there are no signs of this happening soon, and it’s been ongoing for two years. We shouldn’t be trying to return to “normal”, but planning a better future. That’s what Golioth is doing for the Internet of Things. Choosing an IoT management platform shouldn’t require you to commit to one type of hardware. Your fleets should evolve, and our talk outlines how they can evolve. Golioth helps to manage your diverse and growing hardware fleet. Golioth is designed to make it easy to keep track of and control all of that hardware, whether that’s 100 devices or 100,000. Take Golioth for a test drive today.

/by
,

New Feature: Visualizing IoT Data Using Ubidots

How to use Ubidots data visualization

Golioth and Ubidots have teamed up to make it easy to visualize your Internet of Things device data. Ubidots makes it easy to white-label visualizations to produce a branded version of your service for end customers; you can also create end-users, in order to allow access to custom dashboards.

Our partnership has produced an integration that makes it a snap to connect your device data to an Ubidots dashboard so that it looks great for you and for your clients. Today I’ll walk you through the process of setting up a connection.

Ubidots data visualization dashboard

What you need to get started

To get started you just need a few basic things:

  • An account on Golioth (sign up)
  • An account on Ubidots (sign up)
  • A device that is sending data to Golioth

Creating your accounts is pretty easy; Golioth has a free dev tier, and Ubidots has a free Educational/Personal use option or a free trial period for their Business users. Everyone who has followed our Getting Started guide will already has a Golioth account.

If you don’t already have a device set up to send data, you can use the Golioth LightDB Stream example. It will simulate temperature data and send it back to the Golioth Cloud, a perfect dataset for your first Ubidots dashboard!

Setting up a connection between Golioth and Ubidots

To connect Golioth and Ubidots we need to take the follow steps:

  1. Configure an Ubidots plugin
  2. Configure a Golioth output stream
  3. Set up an Ubidots dashboard

1. Configure the plugin in Ubidots

Ubidots has a plugin specifically for Golioth. But before configuring the plugin we want to make sure we have a token to use for this project. From the Ubidots dashboard, click your profile picture in the upper right, choose API Credentials and click More under the tokens menu that appears. This will open the API Credentials page.

How to create an ubidots token

Use the plus sign to add a new token, then use the pencil icon to edit the name of the token.

Next, choose Devices→Plugins from the top menu. Click the plus to add a plugin, and choose Golioth. Click the arrow icon to get to the plugin configuration menu.

Choose the token you just created from the dropdown menu. In the LightDB Stream field, use the name of the endpoint where your device is sending data.

If you have a nested JSON, like {"env":{"temp":20.5}}, then you would set this value to “env” and the Ubidots plugin would pick up all of the key/value pairs inside of that JSON object.

If you are following along with the Golioth Stream sample, leave the LightDB Stream field blank, since we publish to the root-level directly. You can see in the code example below, we are setting the temperature value to the root-level “temp” key.

err = golioth_lightdb_set(client,
			GOLIOTH_LIGHTDB_STREAM_PATH(&amp;amp;quot;temp&amp;amp;quot;),
			COAP_CONTENT_FORMAT_TEXT_PLAIN,
			str_temperature,
			strlen(str_temperature));

Click the right arrow, give your plugin a name and description, and click the check mark icon to save. You will be redirected to a list of your plugins.

[screenshot]

Click on the one you just created and then select Decoder from the left sidebar.

Generally speaking, this page is already configured to work. We need to copy the Ubidots HTTPs Endpoint URL and use it to create a Golioth Output Stream in the next step.

If you want to learn more about the data format so that you can customize the decoder on this page, look at the DeviceStreamMessage in our Event Types documentation. For today’s example, the defaults will work.

2. Configure a Golioth output stream

Choose Ubidots as Output Stream

In the Golioth Console, select Output Streams from the left sidebar, click on Create an Output Stream, and choose Ubidots from the list.

Give your new Output Stream a name, and choose DEVICE_STREAM_TYPE as the event filter. The HTTPS Endpoint URL comes from the Ubidots dashboard. So does the auth token, which you can copy from the Ubidots dashboard by clicking your avatar in the upper right and choosing API Credentials(this reveals a panel that lets you copy tokens to the clipboard). Click save to finish setting up your Output Stream.

3. Set up an Ubidots dashboard

We’re ready to start visualizing! At the top of the Ubidots dashboard, select Data→Dashboards. You can create a new dashboard using the ☰ icon (sometimes called a “hamburger menu”) in the upper left.

On your new dashboard, click Add new Widget and choose a Gauge from the list.

Ubidots dashboard gauge settings

In the data section, click Add Variable, choose your device, and select the data you want to show. If your device is not listed, the endpoint may be configured wrong, or the device has not yet sent any data. There is a Logs section in the Ubidots plugins settings page that will be helpful in troubleshooting the issue.

The rest of the settings in this dialog decide how your data will be displayed. Set the name of the widget, and choose the range that suits you best before clicking the check mark icon to save the changes.

Temperature gauge

Now you have a gauge that shows temperature readings from your device. Of course there are many other visualization methods available to match your needs, so spend some time playing around with your new Ubidots dashboard!

/by
,

How to use Node-RED to control IoT Devices on Golioth

Node-RED is a graphical programming tool that makes it easy to interact with web-based events. It’s called a “Low-code” solution because you can do a lot without writing much code. I love it because it’s the right balance of handling the hard stuff for me. Initial ease of use, while still making it possible to drop in custom code when I need it. I’ve been using that to control my IoT devices on the Golioth Cloud, and today I’m sharing how I do it.

Note: the concept of a node will be highlighted with bold text throughout this article to help clarify we’re discussing one of the blobs shown on screen in Node-RED

Why use Node-RED with Golioth

Earlier this year Ben Mawbey showed us how he uses Node-RED to manipulate data as it arrives on the Golioth cloud. This approach listens for realtime data using WebSockets. But this has a couple of limitations:

  • We will only see the data when it changes
  • We aren’t able to send or update data using WebSockets.

On that second point, the Golioth REST API lets us send/update data and query stored data.

Node-RED is a good choice here, since it’s relatively easy to set up a connection to WebSockets and the REST API. It’s powerful enough to do any data manipulation we want. It does need to run on a server, but on the upside that means it is always running and can be accessible to multiple end users via any web browser, no need to install an app. Let’s dive in!

Prerequisites

This example assumes you have already taken care of the following:

  • a Node-RED server
    • The Get Started page offers several solutions like running it locally on a Raspberry Pi, or on a cloud server
  • Golioth account, and a device added on the Golioth Console
    • As always, our getting started guide will walk you through all of these details
    • You can follow the demo below without having a hardware device, we just need to create a device on the Golioth Console to send data back and forth to the cloud

Connect Node-RED to WebSockets

The secret sauce in connecting a WebSocket is the URL which includes the Golioth API key. Go to the Golioth Console and choose API Keys from the left sidebar to create a new key:

The exact formatting of the WebSockets URL is detailed on our reference page, but I’ll show you how to do it here. Notice the three pieces of important data in the diagram below: project id, device id, and finally the API key.

I have chosen to connect to the dataendpoint to monitor LightDB state data. (If you want to monitor LightDB stream you can change data to stream.)

wss://api.golioth.io/v1/ws/projects/node-red-demo/devices/62695497404e12cd12628117/data?x-api-key=ynhJQQjLautKYLxQESuqB8VJLWMEVpH7

Setting up the Node-RED flow begins by adding a WebSocket-in node and connecting it to a debug node. I have also connected a JSON node for convenient data access later.

The WebSocket-in node is configured as follows:

The WebSocket type is “Connect to” which allows us to add a new URL. Click the pencil icon or the right side of that field and in the new window paste the URL. Because we are passing the API key as part of the URL, we do not need to add a TLS configuration.

Data will only appear on this WebSocket-in node when it first arrives on the Golioth Cloud. So generate an update by going to the Golioth Console and adding the key/value pairs shown above. In the Node-RED shown in the flow setup step, notice the debug output includes the raw payload as well as the JSON object.

Connect Node-RED to REST API

Setting up the REST API connection requires a bit of code, but uses the same project, device, and API key info from the last step. Two inject nodes (that send a 1 or a 0), a function node, an https request node, and a debug node complete this flow:

The inject node and http request node need a quick settings adjustment:

Set the payload of the inject node to pass 0 or 1 as a number (not a string). In the http request node, change the method to “set by msg.method”. The magic will all happen in the function node:

Code for this function is listed below. The variables at the top are used to set up the API URL, so you will need to enter your project id, device id, API key, and the LightDB state key that makes up the endpoint (led0 in my example).

var proj_id = "node-red-demo"
var dev_id = "62695497404e12cd12628117";
var api_key = "ynhJQQjLautKYLxQESuqB8VJLWMEVpH7";
var endpoint_name = "led0";

var dev_url = "https://api.golioth.io/v1/projects/" + proj_id + "/devices/";
var endpoint = "/data/" + endpoint_name;

var data = msg.payload;

var msg = {
	"method" : "PUT",
	"url" : dev_url + dev_id + endpoint,
	"headers" : {
		"Content-Type": "application/json",
		"x-api-key": api_key
	},
	"payload" : JSON.stringify(data)
};

return msg;

This code intercepts the incoming data (which will be a 0 or 1 from the inject node), formats it as a PUT command, and sends it to the http request node which will submit it to Golioth. In this demo, clicking one of the inject nodes updates the led0 value in LightDB state.

Head over to the Golioth Console and look at the LightDB state data for your device to see the changes. In practice, you can implement the desired state versus actual state principles discussed in my recent article and use this flow to update an LED on an actual piece of hardware.

Further Exercises: Dashboard/Web App

I’ve covered a lot of ground in this article and unfortunately have run out of column inches. But before signing off, I want to mention the potential for turning Node-RED flows into web apps.

The Node-RED dashboard node adds a UI which can be loaded on any browser. It looks spectacular with almost no work from us. Here you can see I’ve set up a display that shows the state of the LED, displays the latest value of the counter, and adds two buttons to turn the LED on or off.

If you want to test this out, here is an export of this flow that you can import into your Node-RED.

A word of caution: your flow contains an API key for accessing device information on Golioth. This must be kept secure. Please make time to review how to secure Node-RED and ensure that you authenticate users if you decide to build and share a web interface.

/by

How to Manually Connect to Golioth with the ESP-IDF

How to connect to Golioth with the ESP-IDF
PLEASE NOTE: This post was published in April of 2022. In July 2022, we released the Golioth ESP-IDF SDK, which is our recommended path for developing with the ESP-IDF. The post below is a great look “under the hood” to see what it took to connect to Golioth CoAP endpoints before our ESP-IDF SDK existed.

Golioth has great support for ESP32 devices. While our official SDK is built on the Zephyr RTOS, and we understand that there are some use cases where you might need to use the ESP-IDF instead.

The good news is that Golioth uses the CoAP protocol. So anything that can authenticate with Golioth and communicate over CoAP can be use with our platform. The ESP-IDF includes some CoAP examples, and allows pre-shared keys to be used for authentication, which means it will work with Golioth.

In this article, I’ll show you how to use VS Code to develop an embedded application that will connect to the Golioth Platform using the ESP-IDF instead of Zephyr. Let’s dive in.

Requirements

To follow this tutorial, you first need to have

  • A Golioth account
  • A Golioth project with at least one device
  • PSK-ID and PSK of your device
  • Visual Studio Code (VS Code) pre-installed

If you followed the Golioth Getting Started guide, you have already satisfied the first three requirements and will just need to install VS Code.

Install the ESP-IDF in VS Code

The ESP-IDF can be installed from VS Code. I suggest following this quick tutorial to install the ESP-IDF extension for VS Code, even if you already have manually installed it. We’ll only use the Graphical Interface provided by the extension in this tutorial.

Create a coap-client project from the ESP-IDF examples

Open the VS Code command pallet (ctrl+shift+p) and search for “ESP-IDF: New Project“

In the New Project window, enter:

  • project name: coap-golioth-hello
  • project location: <path-to-your-project>/
  • ESP-IDF board: custom board
  • ESP-IDF Target: Esp32 module
  • Serial Port: choose the serial port that your device is connected (in my case, /dev/ttyUSB0)

Click on “Choose Template” button, then click in the selection box and select “ESP-IDF” option

Next, a new windows will be open with ESP-IDF samples.

Scroll down and go to Protocol section and select coap_client sample option. Then click on “Create project using template coap_client”.

A status will appear at the bottom right corner showing the progress.

When finished, a new window will appear on the bottom right asking if you want to open the recently created project in a new window.  Click Yes.

After you create the Project and open it in VS Code, you’ll see the project structured as shown above.

Set up the Golioth platform as the CoAP destination

You will not need to change the the sample code contained in the main folder to make this tutorial work. The only change needed is to create a configuration file and set some variables.

To create the complete config file, we’ll use the menuconfig tool that the ESP-IDF extension makes available for us. Click the gear button in the VS Code bottom tool bar:

A new SDK Configuration Editor window will open:

We will set some configs in this windows.

First, at the left menu bar, click the Example CoAP Client Configuration:

Next, set the following values:

  • Target Uri to coaps://coap.golioth.net/hello
  • Preshared Key (PSK) to the PSK of your device which you’ll find in the Golioth Console
  • PSK Client Identity to the PSK-ID of your device which you’ll find in the Golioth Console
  • WiFi SSID and WiFi Password to the name and password of your WiFi network

There’s one last setting needed before saving your configuration. In the left menu bar, go to CoAP configuration and set the CoAP Encryption method to Pre-Shared Keys.

Finally, click save.

When your configuration is saved, the ESP-IDF extension will make some changes in your structure. It creates an sdkconfig.old file that contains the previous config before saving the new one, and also a new sdkconfig file that contains all the new configuration variables.

At this point, our tutorial is ready to be built and flashed to the device. If you check the Golioth Console before you first run the example, you’ll notice the device is reported as offline. If we did everything right, that’s about to change!

Run the CoAP example on ESP32

To build the firmware, click the cylinder button at the bottom tool bar and wait for the build process finish:

After the build finishes, ensure your device is connected to the USB port, click the flash button and, again, wait for the process to finish:

At this point, you have the firmware running on your device. But how can we see if it’s running properly?  Just click the monitor button, and you’ll start to see all the logs coming from your device through the serial port:

Above, you see some logs generated when the device is booting which includes some interesting information about the device.

Below, you can see logs related to the Golioth communication itself. As we set our Target Uri to coap.golioth.io/hello, this endpoint sends back a hello message every time the device connects to it:

So, from the device perspective, you can see that it managed to connect to the Golioth, right?

But let’s see from the Golioth Console perspective. Go to the Device section in the console and you will see the device is now reported as online:

Conclusion

At this point we can see that connecting a device to the Golioth using the Espressif ESP-IDF and the CoAP protocol is not hard. This post serves as a “Hello World!” to get the connection up and running. Now that you’ve done that, you can easily transfer data between your ESP32 and Golioth.

Espressif has included a lot of examples with their VS Code extension that will help you better understand how to use it and are worth further study. In my next tutorial I’ll continue using the CoAP protocol but this time we’ll see how to send a LightDB Stream package, which includes time-series data perfect for visualizing the information being collected by your devices.

/by