,

How to Control LTE with the Nordic nRF9160 and Golioth

The Nordic nRF9160 is a fantastic solution for cellular connectivity. All of the Golioth sample code runs on the nRD9160-DK without any changes to configuration so you can test out all of our features. But eventually you’re going to start scratching your head about how the LTE connection works. This is especially true because the device will do nothing while first establishing the connection, which can take over one minute depending on your network. The answer is to use the LTE link controller library.

Using Automatic LTE Control

The Nordic nRF Connect SDK (based on Zephyr) makes it really easy to automatically connect to LTE. Simply put, there’s a Kconfig symbol that causes the modem to connect to the cellular network when the device powers up:

CONFIG_LTE_AUTO_INIT_AND_CONNECT=y

While this takes care of the connection, it does so before main() begins running, and it blocks program execution until the network is connected. That can take more than a minute and depends on things like your distance to your closest tower and the strength of the signal in your office. Since you can’t write log messages, toggle LEDs, or write to a screen, it can look to a user like the device is stuck.

For battery-controlled devices you want to carefully control when the radio is on and when it is off. In this case, automatic control is usually not an option.

Using Manual LTE Control

Nordic’s docs for the LTE link controller are fantastic, you should spend the time to read through them. We’ll discuss the most basic form of link control: manually establishing a connection.

To use the link controller, first select the library using Kconfig:

CONFIG_LTE_LINK_CONTROL=y

Then include the library in your c file:

#include <modem/lte_lc.h>

We can now start using the link controller functions. For me, the most interesting ones are the async functions. For instance:

int err;

//Initalize the modem and connect to the network (register a callback)
err = lte_lc_init_and_connect_async(lte_handler);

/* Do some things on the network */

//Place the modem in offline mode
err = lte_lc_offline();

/* Do some offline things */

//Modem already initialized, just reconnect again
err = lte_lc_connect_async(lte_handler);

Notice that lines 4 and 14 register a callback. Nordic has a nice example of what that callback should look like:

/* Semaphore used to block the main thread until the link controller has
 * established an LTE connection.
 */
K_SEM_DEFINE(lte_connected, 0, 1);

static void lte_handler(const struct lte_lc_evt *const evt)
{
     switch (evt->type) {
     case LTE_LC_EVT_NW_REG_STATUS:
             if ((evt->nw_reg_status != LTE_LC_NW_REG_REGISTERED_HOME) &&
             (evt->nw_reg_status != LTE_LC_NW_REG_REGISTERED_ROAMING)) {
                     break;
             }

             printk("Connected to: %s network\n",
             evt->nw_reg_status == LTE_LC_NW_REG_REGISTERED_HOME ? "home" : "roaming");

             k_sem_give(&lte_connected);
             break;
     case LTE_LC_EVT_PSM_UPDATE:
     case LTE_LC_EVT_EDRX_UPDATE:
     case LTE_LC_EVT_RRC_UPDATE:
     case LTE_LC_EVT_CELL_UPDATE:
     case LTE_LC_EVT_LTE_MODE_UPDATE:
     case LTE_LC_EVT_TAU_PRE_WARNING:
     case LTE_LC_EVT_NEIGHBOR_CELL_MEAS:
     case LTE_LC_EVT_MODEM_SLEEP_EXIT_PRE_WARNING:
     case LTE_LC_EVT_MODEM_SLEEP_EXIT:
     case LTE_LC_EVT_MODEM_SLEEP_ENTER:
             /* Callback events carrying LTE link data */
             break;
     default:
             break;
     }
}

Also notice that they recommend using a semaphore. Because this runs asynchronously, checking this semaphore is a good way for the rest of your code to know if an LTE connection has been established.

Considerations when Using Golioth with Manual Control

Golioth depends on a network connection. When you manually control the network, you should also take Golioth into consideration. You need to wait until LTE has been connected to start the Golioth System Client. A good place to do this is in the callback:

case LTE_LC_EVT_NW_REG_STATUS:
    if ((evt->nw_reg_status != LTE_LC_NW_REG_REGISTERED_HOME) &&
     (evt->nw_reg_status != LTE_LC_NW_REG_REGISTERED_ROAMING)) {
        break;
    }

    LOG_INF("Connected to LTE network. Starting Golioth System Client...");

    golioth_system_client_start();

    break;

Before going offline or putting the modem into a sleep mode it is recommended that you stop the Golioth client:

golioth_system_client_stop();

Calls to Golioth services (LightDB State, LightDB Stream, etc.) will cause errors if the client is not connected. You may choose simply to ignore the errors in the serial terminal, however, gating those function calls with a semaphore is another option.

For applications that utilize an intermittent network connection, we like using message queues to cache data. Timestamps may be added to each reading so that the data is properly recorded the next time a connection is available. We have previously discussed using Zephyr message queues for this purpose.

Conclusion

Automatic LTE control is great for trying out demo code. However, we think in most applications you’ll want to decide when and how to use the modem. Luckily, for this particular SIP, Nordic has made the control library really easy to use.

Do you have questions about cellular modem control with your IoT fleets? We’d love to hear from you! Open a new thread on the Golioth Forum, or set up a video call with our Developer Relations crew to discuss your use case.

/by
, ,

Using VScode for ESP-IDF and Golioth

Visual Studio Code, colloquially known as VScode, is among the most popular integrated development environments (IDEs). Today we’re going to walk through the process of setting up ESP-IDF in VScode and using it to run Golioth device management example code on an ESP32.

Not everyone likes to live their lives hammering away at a command prompt. What we’ll cover today is another option which uses Espressif’s VScode extension (plugin) to largely automate how you work with the Espressif IoT Development Framework (ESP-IDF). That means nice buttons and interfaces to build, flash, and monitor applications for the ESP32 family of chips.

Installing VScode and the ESP-IDF extension

As a prerequisite you will need to have VScode installed. If you don’t, head over to the download page and do so now.

ESP-IDF VScode extension

Open VScode and click on the extensions icon (looks like four boxes) on the left sidebar. Type esp-idf into the search bar that appears and the top result will be “Espressif IDF”. Click the install button and you’re off to the races.

Configure ESP-IDF VScode Extension

You will be greeted with options for installing the various ESP-IDF tools. If you don’t have an opinion on how things are installed you can choose the automatic route. I wanted to specify what directories were used during the install so I chose the manual route and used the settings above.

It will take a few minutes for everything to download. You will want to click on the “Download ESP-IDF Tools” to ensure that the compilers and other tools are downloaded (in addition to the Espressif SDK). If you need more help, check out Espressif’s installation guide.

Installing the Golioth Firmware SDK

Now that VScode and the ESP-IDF are installed, let’s take a moment to install the Golioth Firmware SDK. This provides the tools and sample code for connecting your ESP32 to Golioth.

Cloning the Golioth Firmware SDK

VScode has a handy tool for cloning git repositories. Bring up the command palette (ctrl-shift-p), type in gitcl, and press enter. A prompt will open in the same window for you to enter the following URL:

https://github.com/golioth/golioth-firmware-sdk.git

After pressing enter, a window will open where you can select a folder to store the repository. A folder called golioth-firmware-sdk will be placed in that location.

VScode will ask if you want to open the cloned repository. Please click Cancel on this window. The Golioth Firmware SDK supports multiple platforms and we will open the ESP-IDF specific directory in the next step.

Open the project and update the git submodules

Now that the repository has been cloned, let’s open the sample code in VScode. Click File→Open Folder and navigate to the golioth-firmware-sdk/examples/esp_idf/golioth_basicsfolder, then click Open.

The Golioth SDK includes a few packages as submodules and these must be updated before continuing. We’re going to use the terminal for this step. In VScode click Terminal→New Terminal. A terminal window will open in the golioth-basics folder. Type this command to update the submodules:

git submodule update --init --recursive

Type exit to close the terminal window.

Build, flash, and monitor the golioth-basics application

Now that everything is installed we get to see the ease of using an IDE.

ESP-IDF Build, Flash and Monitor

The bar along the bottom of the VScode window includes icons for working with the ESP-IDF tools. Make sure your ESP32 is plugged into USB. Click the flame-shaped icon which will build the project, flash it to the ESP32, and open a serial connection to the chip.

The build will take place and then VScode will open a window in the top center prompting you to select JTAG/UART/DFU. We will be using UART. Also note that there is a selection in the bottom menu bar where you can set the port that will be used when flashing (the image above shows /dev/ttyUSB1 in my case).

Assign device credentials in the monitor window

ESP32 running the golioth-basics app

The golioth-basics app will begin running immediately and you should see an output from the chip in a window inside VScode. We need to give the chip WiFi and Golioth credentials so that it can connect to the cloud.

If you have not yet signed into Golioth, our Dev Tier is free for your first 50 devices. (Tip: there is a Console Overview on our docs that will walk you through creating a set of device credentials.) Get your Golioth credentials, and the login info for your WiFi access point, and pass them to the chip using this command format:

settings set wifi/ssid YourWiFiAccessPointName
settings set wifi/psk YourWiFiPassword
settings set golioth/psk-id YourGoliothDevicePSK-ID
settings set golioth/psk YourGoliothDevicePSK

assign credentials to the device

Once you’ve set the credentials, type reset and the ESP32 will reboot, connect to WiFi and then to Golioth.

Successful connection to Golioth

Wrapping up

You’ve successfully compiled, flashed, and run a demo Golioth application for ESP-IDF using VScode. The same principles can be applied to your own projects.

If you’d like to dig deeper into how the golioth-basics code works, I encourage you to study the golioth_basics.c file in the golioth-firmware-sdk/examples/common folder. It demonstrates all of the Golioth device management features like OTA firmware updates, remote procedure calls (RPC), IoT fleet settings service, LightDB State and LightDB Stream data services, and remote logging.

We’d love to hear about the projects your working on. Share your successes and post your question on the Golioth Forums. If you’re interested in learning how to add Golioth to your IoT fleet, get in touch with our Developer Relations crew.

References:

 

/by
, ,

How to optimize power in Celluar IoT deployments (with Jared Wolff)

How do you ensure you achieve the lowest possible power with your Cellular IoT Devices? We held a webinar on January 18th, 2023 where we tackled this question, alongside our friend Jared Wolff of CircuitDojo. In this video, we review how to measure and optimize your hardware, firmware, and software. The main focus of this video is the nRF9160 from Nordic Semiconductor, which runs NCS / Zephyr. Jared shared many of the tips and tricks he has learned while building the CircuitDojo Feather nRF9160.

A three pronged approach

There are three key areas to focus on when trying to optimize your design for low power.

  • Hardware – Jared talked about the importance of part choice, especially on the switching regulator. The nRF9160 has a low quiescient current if everything is configured properly.
  • Firmware – In order to take advantage of the low current draw of the part in question, we need to understand and enable features within the nRF9160. This would be true of any part we chose to use, the configuration is crucial. Understanding how the cellular modem is communicating with the tower is another important step, since each time you ping the tower it takes energy to check in.
  • Software – In this case, we mean Cloud software, such as the capabilities that Golioth provides to its users. The Golioth platform enables efficiencies like CoAP communication and CBOR encoding, lowering the size (and power draw) of each packet you decide to send to the Cloud. Things like the Golioth Setting service allows you to enable and disable different modes on your device to achieve fine-grained control of devices in your fleet.

Take your design (and battery life) further

Golioth is focused on building better cellular IoT deployments. We enable engineers to build customizable and reliable device fleets that monitor and impact the world around them. If you need any help optimizing your fleet, let us know on our forum or drop us a note.

/by
,

Move over Internet of Things, I built the Internet of Memes

Mepaper hackathon

I built the Internet of Memes, making it possible for anyone at my company to push funny pictures to the ePaper display on the desk of all our coworkers.

I work at Golioth, but I don’t really work at Golioth. We’re headquartered in San Francisco but we’re also a completely remote company spread across the US, Brazil, and Poland. I love working with insanely talented and interesting people, but we don’t get to see each other face to face in the office each day. So at risk of “all work and no play”, I took on an after-hours project to make sharing memes with each other a bit more IRL.

ePaper is Standard-Issue at this Company

MePaper powered by Golioth

As part of onboarding, each new employee is shipped is a small development board that includes an ePaper display, RGB LEDs, buttons, a speaker, accelerometer, and WiFi. It’s Adafruit’s MagTag and we use it for developer training so it makes sense to hand one out to everyone when they join up. But the boards sit unused the vast majority of the time. Why not use that messaging real estate for something fun?

U Got IoT! meme

The project comes in two parts: firmware for the ESP32-S2 that drives the MagTag and a Slack Bot that handles the image publishing. Let’s jump into the device-side first.

Espressif’s ESP-IDF, Golioth’s Device Management, and Google Drive

The device firmware got a bit of a head start because we have a MagTag-based demo that’s part of the Golioth Firmware SDK. The demo can already drive the hardware and use all Golioth features.

RAM considerations

The images for the display are 296×128 black and white so you need 4.7 kB of RAM to download an image. This was a challenge for the on-chip SRAM, but the ESP32-S2 WROVER module on these boards include a 2MB SPI PSRAM chip built into the module. I enabled this in Kconfig and used malloc() to place a framebuffer for the display in external ram.

#define FRAME_BUFFER_SIZE   5000

/* Get memory space for frame buffer */
fb = (uint8_t *) malloc(FRAME_BUFFER_SIZE);

The buffer is slightly oversized because it’s used to store an HTTP response that includes a bit of header data.

Leverage the ESP-IDF http_client

The next challenge was how to get the images onto the device. It is possible to repurpose Golioth’s OTA system to supply these images, but for reasons I’ll get to in the next section that’s is an inefficient approach. What I really want is a way to upload images to one single location and have many devices access them. That’s how webpages work, right?

I adapted the https_with_url() example from the ESP-IDF esp_http_client_example code. Just supply a URL to the image and the http(s) client will grab the page and dump it into the frame buffer.

void fetch_page(void) {
    ESP_LOGI(TAG, "MEME FILE");
    const char* url_string_p = mepaper_provisional_url;
    esp_http_client_config_t config = {
        .url = url_string_p,
        .event_handler = _http_event_handler,
        .crt_bundle_attach = esp_crt_bundle_attach,
        .user_data = (char*)fb,
        .timeout_ms = 5000,
    };
    esp_http_client_handle_t client = esp_http_client_init(&config);
    esp_err_t err = esp_http_client_perform(client);

    if (err == ESP_OK) {
        ESP_LOGI(TAG, "HTTPS Status = %d, content_length = %d",
                esp_http_client_get_status_code(client),
                esp_http_client_get_content_length(client));
                validate_and_display();
    } else {
        ESP_LOGE(TAG, "Error perform http request %s", esp_err_to_name(err));
        xEventGroupSetBits(_event_group, EVENT_DOWNLOAD_COMPLETE);
    }
    esp_http_client_cleanup(client);
}

A question of image formats

What kind of image should this be, anyway? My friend Larry Bank has written a bunch of image decoders for embedded systems which I considered using. But I ended up going with an easier solution. A format I had never heard of before but that’s perfectly suited for a 1-bit-per-pixel display like this one is called PBM or “Portable Bit Map”.

The PBM specification is for a 1-bit color depth image. The byte-ordering and endianness is exactly what is needed for this ePaper display, and the header is super simple to parse. When the file is downloaded to the PSRAM, the firmware searches for a header. Once found, the data after that header is piped directly to the display. Voila!

static const uint8_t HEADER_PATTERN[] = { 0x31, 0x32, 0x38, 0x20, 0x32, 0x39, 0x36, 0x0A };
static const uint8_t WHITESPACE[] = { 0x0A, 0x0B, 0x0D, 0x20 };

static bool is_whitespace(uint8_t test_char) {
    for (uint8_t i=0; i<WHITESPACE_SIZE; i++) {
        if (test_char == WHITESPACE[i]) {
            return true;
        }
    }
    return false;
}

static bool headers_match(uint8_t *slice) {
    // Spec is complicated by "whitespace" instead of a specific character
    for (uint8_t i=0; i<HEADER_LEN; i++) {
        if (i == WHITESPACE_IDX) {
            if (is_whitespace(slice[i]) == false) {
                return false; //this char must be whitespace
            }
        }
        else if (HEADER_PATTERN[i] != slice[i]) {
            return false; //members don't match
        }
    }
    return true;
}

Images are created in your favorite editing software (I use GIMP). Set the mode to 1-bit depth size it to 296×128 (then rotate counterclockwise by 90 degrees) and export it to as PBM.

A Slack Bot Made of Python

With the device side proved out, I turned my mind to the “distribution” problem. How do I let different users control the fleet? Here is a list of consideration on my mind:

  • How can users register their own devices for this service?
  • How can users push their own images to the entire fleet?
  • How will firmware upgrades be managed?

As a fully-remote company, we’re on Slack all day long. A Slackbot (a daemon that sits in the chat listening for commands) makes the most sense for granting some fleet control to everyone in the company.

How should the devices be provisioned so that they are all accessible by the bot? This topic could itself be a whole post. I decided the best way was to let each person add their MagTag to a Golioth project they control, assigning it the mepaper tag (“meme” + “epaper” = mepaper). They then share an API key to that project to receive notifications when there is a new image available for the screen. OTA firmware updates can also be staged for registered devices.

Starting from a stock MagTag, you download the latest firmware release and flash it to the board using esptool.py (the programming tool from Espressif). When the board boots, a serial shell is available via USB where WiFi credentials and Golioth device credentials can be assigned by the user. That’s it for setup, the bot takes care of the rest.

Mepaper Slack bot publish command

Each device is observing a mepaper endpoint on the Golioth LightDB State. It’s a URL to the image to display, and whenever it changes the boards get notified, then uses the https client to download them. The bot listens for a publish <url_of_image> message on the Slack channel, validates the file as existing and being in the proper format. Then the bot iterates through each API key it has stored, updating the LightDB State endpoint for each board tagged as mepaper.

This was chosen instead of pushing OTA artifacts for each image. That’s because the image would need to be uploaded multiple times (one for each project API-key registered) and a copy of all past memes would remain as artifacts. By uploading the images to our Google Drive, they are only stored one time, and all devices can access them as long as they have a copy of the share link.

Mepaper Slack bot commands

Of course, as a quick and dirty hack, the original version of the firmware wasn’t the greatest. I made sure to use the OTA firmware upgrade built into our sample code. As with the image updates, users who the bot recognizes as admins can issue /register ota and /register rollout commands. These accept parameters like version, package name, and the filename of the new firmware. For security, the bot will only upload firmware that is located in the same directory as the bot itself.

Oh the Fun of a Side-Channel for Memes!

Having a dedicated, non-intrusive screen for Memes turns out to be an excellent boost to the remote workplace vibe. Your computer doesn’t need to be unlocked, and you don’t need to switch context to see the newest message. The reason for pushing a meme doesn’t have to be meaningful, it is purely a mechanism for spreading joy. And it worked!

We’ve previously written about the Demo Culture at Golioth. The “Demo early, demo often” mantra is an effort to break out of a rut where you only communicate with your coworkers for Official Business™. Sending memes is one more way to be silly, and enjoy each others’ company, even if we can’t be in the same room. It’s also a neat way to eat our own dogfood.

Future Improvements

There’s lots of room to improve this project, but the first area I’ve been actively working on is the fleet organization itself. When starting out I wanted each user to be able to provision their own device so I chose to have users add the device to their project and share an API key with the bot using a private command on Slack. This an interesting application for controlling devices across multiple projects, but it’s overly complex for this particular application.

Bot-created, credentials, all on the same project

Currently, users need to generate credentials, assign them to the device via serial, and share an API key using the Slack bot. A better arrangement would have been to use the Slackbot to acquire the credentials in the first place. A future improvement will change the /register command to generate new PSK-ID/PSK credentials and send them to the user privately in Slack. This way all devices can be registered on one project, even though the user doesn’t have access to that project’s Golioth Console.

OTA is crucial to fleet management

With the fleet already deployed, how do you make these changes? Via OTA firmware update, of course. I have already implemented a key rotation feature using Golioth’s Remote Procedure Call. The bot can send each device a new PSK-ID/PSK, which will be tested and store, move each member of the fleet over to one singular project.

/by
,

Golioth Ran My Furnace This Christmas

Golioth Heated My Home

The human body is surprisingly adept at sensing temperature. And so when I sat up in bed my body immediately informed me something was not right with my furnace. Yes, it’s winter in Wisconsin, but it shouldn’t feel that cold inside the house. The furnace was fine, but the thermostat was not, and so begins the story of how Golioth ran my furnace over Christmas.

Twas the Week Before Christmas and Cold in the House

HVAC control panel

EIM that connects to the furnace/AC/HRV. The red/green wires exiting the bottom of the photo go to the Golioth Greenhouse Controller

I arose from my slumber and went to check the thermostat to find it displaying a message that it could not connect to the Equipment Interface Module (EIM). We have a furnace, air conditioner, and a heat recovery ventilator (HRV is a fancy name for a pair of fans that exchange stale inside air with fresh outside air). Being a geek, I wanted one way to control them all. That meant installing the Honeywell EIM to switch those subsystems. The thermostat then connects wirelessly to the EIM to provide automated control. But the chilly morning we’re discussing here, the thermostat wasn’t connecting at all!

After basic troubleshooting I used a jumper wire to short the white “run the furnace now” wire to the 24-volt red wire. The furnace kicked on and started warming up the house. There was nothing wrong with the furnace, but I no longer had an automatic way to control it.

But wait…industrial control is one place where Golioth shines! Could I use Golioth as my furnace controller until the built-in system was repaired?

Connecting the Golioth Greenhouse Controller to My House

Golioth Greenhouse Controller connected to the EIM

The Golioth Greenhouse Controller was placed outside of the utility closet to sense temperature readings from the room.

Just a week before this happened I wrote a blog post about the Golioth IoT Greenhouse Controller. It includes relays that can be switched automatically based on a temperature sensor. This is the definition of a thermostat. And these Golioth reference designs are built to be easily adaptable to similar applications. The hardware connection was dead simple: just run the red and white wires to the temperature-controlled relay on the greenhouse controller.

The logic in the firmware needed a quick tweak as greenhouses regulate temperature by taking action when it gets too hot (the opposite of heating a house). This required the “expert” firmware engineering step of changing a greater-than sign to a less-than sign. Luckily all of our reference designs have Over-the-Air (OTA) firmware update built-in so I was able to make this change while the device was in my basement, still connected to the furnace.

The temperature threshold is set via Golioth’s web console, so at that point I had actually upgraded my home system to include control from anywhere in the world! I also got a dashboard to keep my eye on temperature, pressure, and humidity.

Eating Your Own Dog Food (and Enjoying the Flavor)

So, why didn’t I just go out and buy another thermostat? The first reason is the ongoing chip shortage (now into it’s third year). The EIM part that was broken is about $80 normally, but there’s a 31 week lead time on it right now. The second reason is that the only wires running from the furnace to the thermostat are for supplying power (+24 V and Ground). Not only would a replacement thermostat be temporary, I would need to run new wires to give it furnace control or leave it in the basement.

For me this was a great philosophy experiment. We spend a lot of time planning and developing reference designs. Now I know a bit more about the user experience, and I like it! The hardware form-factor was easy to work with, and the approaches we take to control and feedback worked well for my application.

Temperature graph with many on/off cycles

Temperature control without hysteresis

That doesn’t mean it was a perfect fit. Our basic design doesn’t have hysteresis built in, which means that the controller was trying to cycle more often than normal as there was no deadband to allow the house to warm (or cool) a bit past the target temperature. I implemented a simple cycle time threshold that prevented switching between on and off more than once in a 15 minutes period. It’s an application-specific topic that could be added to an implementation guide for this reference design.

Temperature and humidity graphs with smooth rise and smooth response due to cycle-time control.

Temperature response over the same time period was much smoother with simple cycle-time control

I called several HVAC shops in town and the earliest estimates for a new part was the end of March. Online I was able to find a Honeywell “starter” kit included the replacement EIM with just a week before delivery. During this time the Golioth-controlled furnace kept the house warm, but also getting very very humid without the HRV to help regulate. Good news everyone, the Greenhouse demo has a humidity sensor and a second relay. You guessed, it. I added control for the ventilation as well!

I Built An IoT Thermostat in 20 Minutes

Now, I was already familiar with this demo unit, so take this with a grain of salt. But the purpose of the Golioth reference designs is to give businesses a running start when developing a proof of concept. I was able to substitute this one in place of my thermostat with just about 20 minutes of effort. It kept my house comfortable in the harshest conditions of December’s arctic blast, and it let me check temperature and adjust settings while I was working outside of the home during that time. This is the Internet of Things, and it’s never been easier.

/by
,

How to Debounce Button Inputs in a Real Time Operating System

Luis Ubieda is the Lead Firmware Engineer at Croxel. He has a background in Electrical Engineering and is passionate about Electronics, Embedded Systems, and IoT Technology.

Embedded systems are riddled with complexity, mainly because they are at the intersection of expertise. Problem are often a mixture of software, electrical and mechanical issues. This is the case even for seemingly simple tasks, such as reliably detecting a button-press.

“Wait — a button-press??”

Let’s think for a second, how a button press is detected:

  1. Initial State: a button-press typically consists of a “normally-open switch”, which through a pull-up/down resistor, is normally “high” or normally ‘low’: this is the initial state.
  2. Press Event: then, when the button is pressed, the switch is closed and the signal transitions towards the opposite state (e.g, low for normally “high” state), and it’s sustained for as long as the button is held by the user.
  3. Release Event: Finally, when the button is released, it goes back to its initial state.
diagram showing the busy electrical signal produced at the beginning of a button press

Image: Signal during a button-press/release sequence where the transitions are outlined; and the scoped signal has noise. Source: GeeksforGeeks.

In an ideal world, we could just look at the signal edges to keep track of the transitions and assume a falling edge is a “press-event” and rising edge the “release event”. In our world, these transitions are affected by electrical transients caused by the mechanical properties of the button actuator. The suppression of this noise it’s commonly called “debouncing”.

“Ok, I get it. How can we `debounce` button-presses?”

There are two main ways we can approach this: the hardware-way and the software-way. Today I’ll touch on both, and detail the technique I prefer to use when debouncing button input with the Zephyr RTOS.

The Hardware Way

The hardware-way focuses on the root cause: the electrical noise. It guarantees the digital signal won’t have such noise during transitions; and it does it through the use of low-pass filters (most probably RC-filters). There are some pretty cool articles that detail this approach (see references at the end of the article).

The Software Way

On the other hand, the software-way is about “ignoring” these false positives on the signal transitions to determine which ones are the real events we’re looking for, and which ones aren’t. Even though there are many ways implement software debouncing, there are two main approaches, depending on whether the variable of interest is the signal state or the transitions: periodic sampling and tracking edge interrupts.

A. Periodic Button State Sampling

Button sampling works by periodically acquiring the signal state, which is buffered on a continuously rolling sample-set. Through detection of consecutive states, the signal change event is detected (either pressed or released). The rule is simple: if there are X-number of consecutive samples with a changed state, we assume the transition really happened. This periodic sampling is often in the order of 10 to 25-ms and is commonly paced by a hardware timer to guarantee fixed intervals and to free some CPU usage.

diagram showing a series of 1 and 0 signals sampled to detect a button press

B. Tracking Edge-Interrupts with Minimum Cooldown

This works by coordinating the detection of edge changes with the spacing between these: there must be a minimum duration before legitimate signal transition. This cooldown phase is commonly implemented through a timer, which kicks-off on the edge-interrupts: the timer gets restarted on each transition and only when it expires (after 10-ms to 25-ms of no edge-changes), the firmware handles the transition as an event.

diagram showing the ignored edges of a button press signal

Software Approach B: Tracking Edge-Interrupts with Minimum Cooldown

In multi-threaded systems, we can leverage the use of RTOS primitives to make the code more modular while simplifying the logic to achieve the same purpose: featuring a thread and semaphores to control the state transitions and decide when to notify the user of the module when an event occurred.

Example Code – Zephyr RTOS

The following code presents an example of achieving the software approach B on Zephyr, with the following observations:

  • We’re using Zephyr GPIO interrupt APIs to keep track of the edge-changes.
  • We’re using the system workqueue as the cooldown mechanism for false positives.
  • Our button-detection module, abstracts both of these details, and only notifies the user of the relevant events: pressed or released.
  • Note: the user callback context is the workqueue handler, therefore: actions on this context shall be kept brief to allow proper functioning of other parts of the system.
#ifndef _BUTTON_H_
#define _BUTTON_H_
 
enum button_evt {
   BUTTON_EVT_PRESSED,
   BUTTON_EVT_RELEASED
};
 
typedef void (*button_event_handler_t)(enum button_evt evt);
 
int button_init(button_event_handler_t handler);
 
#endif /* _BUTTON_H_ */

#include <zephyr/kernel.h>
#include <zephyr/drivers/gpio.h>
#include "button.h"
 
#define SW0_NODE    DT_ALIAS(sw0)
 
static const struct gpio_dt_spec button = GPIO_DT_SPEC_GET(SW0_NODE, gpios);
static struct gpio_callback button_cb_data;
 
static button_event_handler_t user_cb;
 
static void cooldown_expired(struct k_work *work)
{
   ARG_UNUSED(work);
 
   int val = gpio_pin_get_dt(&button);
   enum button_evt evt = val ? BUTTON_EVT_PRESSED : BUTTON_EVT_RELEASED;
   if (user_cb) {
       user_cb(evt);
   }
}
 
static K_WORK_DELAYABLE_DEFINE(cooldown_work, cooldown_expired);
 
void button_pressed(const struct device *dev, struct gpio_callback *cb,
           uint32_t pins)
{
   k_work_reschedule(&cooldown_work, K_MSEC(15));
}
 
int button_init(button_event_handler_t handler)
{
   int err = -1;
 
   if (!handler) {
       return -EINVAL;
   }
 
   user_cb = handler;
 
   if (!device_is_ready(button.port)) {
       return -EIO;
   }
 
   err = gpio_pin_configure_dt(&button, GPIO_INPUT);
   if (err) {
       return err;
   }
 
   err = gpio_pin_interrupt_configure_dt(&button, GPIO_INT_EDGE_BOTH);
   if (err) {
       return err;
   }
 
   gpio_init_callback(&button_cb_data, button_pressed, BIT(button.pin));
   err = gpio_add_callback(button.port, &button_cb_data);
   if (err) {
       return err;
   }
 
   return 0;
}

Check out the working sample code on https://github.com/ubieda/zephyr_button_debouncing

Conclusion

The most important part of debouncing inputs is to understand how far you should go and which approach best suits you. Cost sensitivity tends to favor software debouncing, whereas less CPU usage favors offloading it to the hardware approach. Like any engineering problem, there are 1000 ways to solve it: always favor the simplest (yet effective) solution that works for you.

References

/by
, ,

IoT Greenhouse Controller: A Golioth Reference Design

The Internet of Things (IoT) can make existing infrastructure more useful and easier to operate, with the added benefit that you don’t need to be on-site to make changes. This is the case with Golioth’s latest reference design: a greenhouse controller that adjusts ventilation and grow lighting based on sensor readings. It also provides manual control from the cloud.

Whether it’s too hot or too cold, tightly monitoring and regulating greenhouse temperature has a huge effect on crop yield and growing time. The same can be said for lighting conditions. At this time of year (winter), consider the poinsettia: it requires intense light during the day, and at least 12 hours of total darkness over night in order to turn a vibrant shade of red. Sounds like a great job for an automated controller.

But think beyond one type of plant and one time of year. The agriculture industry uses automated control to implement different growing conditions based on the cultivar. A cloud-connected controller makes it much easier to update (and keep track of) the growth profiles.

The IoT Greenhouse Controller

An IoT Greenhouse Controller continues to show that simply connecting sensors to the internet is impactful. From one online dashboard you can see how the light, temperature, pressure, and humidity is trending across all of your planthouses. For this reference design we added two mains-rated relays to add control to the equation.

The cellular modem sends sensor data back to Golioth, and monitors the cloud for updates in target temperature and light intensity. A threshold setting for light level automatically controls when the grow lights are turned on or off. The same is true for a temperature threshold that is monitored for control of the ventilation system. Of course both of these relays can be controlled manually.

Let’s take a look at the hardware involved in this Reference Design

Hardware

We’re favoring off-the-shelf hardware for ease of implementation. Most of the modules that we use are simple breakout boards that aren’t much more than a sensor or two, some power handling, and interconnects like cabling. The idea is that someone could take this setup and choose which sensors they want to put onto their custom hardware design that will go out in the field.

IoT Greenhouse controller internals

Golioth Greenhouse Controller reference design internals.
Left to right: light sensor, weather sensor, relays

The full parts list is on our Golioth Projects page, but the key components involved are the Nordic Semiconductor nRF9160 cellular system-in-package (SIP), a BME280 weather sensor, an APDS9960 light intensity sensor, and a set of relays.

The nRF9160 was chosen because it is one of our best supported parts on Golioth. A cellular modem may seem like an odd choice for infrastructure-based controllers, but in combination with the lithium battery you will still be able to monitor greenhouse conditions during a power outage. There is no better peace of mind than being able to answer the question: when the power was out, how cold did my plants get and for how long?

As with many Golioth designs, our wide ranging SDK support means you can retarget the same control code to different hardware in the future. If you want to take the reference design and target a Wi-Fi, Ethernet, or Thread solution, it’s a couple of files configured differently and you have similar functionality with a whole new connectivity medium.

Greenhouse Controller block diagram

Firmware

The firmware for the Greenhouse Controller reference design uses the Golioth Settings Service. This is ideal as it facilitates control of a large fleet of these devices, allowing settings to be adjust for all at once, in groups, and of course down to individual units.

Golioth Settings Service for the Greenhouse Controller reference design

Golioth Settings Service for the Greenhouse Controller reference design

Here you can see the loop delay which indicates how long the device should sleep between sensor readings (in seconds). The light and temperature thresholds control the on/off point of the relays, and finally the auto settings indicate if the relays should be switched automatically based on those thresholds. The controller monitors Golioth’s LightDB state system for manual control commands, which do not interfere when the automated option is enabled.

All of the Golioth reference designs include Over-the-Air (OTA) firmware updates so changes to how the firmware works don’t require an on-site visit. While the current firmware doesn’t implement a schedule-based system, the concept is easy to add and install on the device using OTA.

Cloud Software / Dashboard

The Golioth Zephyr SDK takes care of the cloud connection for all of your devices. When writing firmware, just use the API to set/get/observe your data and Golioth handles the rest:

  • Sensor readings are stored as time-series data on LightDB Stream
  • Device settings are monitored in real-time, with the device reacting to your changes as soon as you make them.
  • The Golioth Console tracks the latest device state, including device health
  • Current firmware version is monitored, with the ability to rollout new OTA updates, and one-click roll-back if you need it
  • Golioth’s convenient REST API delivers easy access to the data for visualization or export to any of your favorite cloud server platforms.

We love using Grafana dashboards to visualize IoT data. The dashboard talks to the Golioth REST API to monitor the IoT sensors and the state of the lighting/ventilation. Of course you could use WebSockets to get live updates as the data arrives at the Golioth servers. For this application, it’s likely that sensor readings are being recorded every few minutes so a dashboard that reloads on its own works well.

Golioth Greenhouse Controller Grafana dashboard

More Golioth Reference Designs

Our reference designs are meant to get you through the initial steps of proving out your IoT-based business. You can buy the readily-available parts used for this Greenhouse controller and with our reference design resources you’ll be on your way to a proof of concept in days instead of weeks. This means you’re fleshing out features and heading toward a hardware prototype with actual performance data. Golioth is designed to scale, so the same connections and features that you use for your first prototype remain in place, with a platform that can handle a number of devices beyond your wildest imagination.

We are busy building out more reference material for you to take and customize for your business needs. We recently launched an Industries section of the Golioth website, which lays out some of the other areas we are targeting and Reference Designs we are building. If any of them interest you, click the “Schedule Demo” button for the one that best matches your needs. You can also drop a note on our Forum or on our Discord if you have ideas of other IoT prototypes you need help with or would like to see us build.

 

/by
,

Zephyr’s New ZBus Feature with Rodrigo Peixoto

As members and contributors to the Zephyr Project, we keep an eye on new developments. A recently published feature of particular interest because it represents a new way to structure programs and messaging between different parts of your program.

ZBus (Zephyr Bus) is a recently merged feature on the Zephyr Project, which brings a standardized version of event driven architecture in the form of a publish and subscribe model inside of your program. The lead author Rodrigo Peixoto spoke with Golioth about the details of this new feature and how it might help Golioth users to make more responsive, flexible programs. In the associated video, Rodrigo walks through the history and capabilities of ZBus.

Why should you consider an event-driven bus architecture?

The decision to take on a new system architecture is not something to do willy nilly. It’s important to understand where an event driven architecture is a good fit.

The first that I think about is scalability. When you have a bus architecture, adding an additional “listener” can be done with much less work.

Consider the alternative to event-driven architecture. When you want to add a new action (say some code that initiates a sensor reading), you need to call that new code from your trigger event (say a timer being finished). That means you need to know where the trigger is located in the code and make changes there to add the call to your new task. Once you add in the required testing, each additional feature can become burdensom. This scales poorly.

With an event-driven system, the trigger is already set up to publish an event. New tasks can be added that look for the event. You don’t need to change any trigger code, you don’t even need to know where that code lives. This performs well as the amount of data increases, which will ultimately depend on how large you think your system will be.

Flexibility is another consideration. An event-driven bus allows the system to be easily adapted to handle new events or changes in the environment. This means that the system can be easily updated and modified without having to completely rewrite large swaths of code. Another type of “flexibility” is how and where you can re-use your code. This makes it easier to develop and test your system, as well as to troubleshoot and fix any problems that may arise.

Finally, if your device needs to meet critical timing, an event-driven system will not only deal with higher levels of complexity, but also respond quickly to new inputs to the system, such as external events. For example, an embedded system might be designed to control a robot, and it would use event-driven architecture to respond to sensor data from the robot’s environment and control its movements accordingly.

How ZBus implements an event-driven architecture

In ZBus, there are “producers” that generate messages and “consumers” that act upon them. There are also “Filters” help to process raw data (such as a sensor output). Each of these are organized into different “channels” to allow listening on a particular lane of data being produced.

Source: Rodrigo’s ZBus presentation (click for link)

Source: Rodrigo’s ZBus presentation (click for link)

Source: Rodrigo’s ZBus presentation (click for link)

Each of these are built into normal scenarios such as listening for timers and taking a reading from a sensor and then alerting other parts of the program that the data is now available. A common scenario is show below:

How will you use ZBus?

Microcontrollers are used in increasingly complex scenarios and are being asked to do more and more. Connecting a low power device to the internet often requires higher levels of complexity that Zephyr helps with. We expect to see more devices using Ecosystems and RTOSes like Zephyr in the future, and implementing ZBus on high complexity devices.

Are you looking at using an event-driven architecture in your system? Let us know on our forum and tell us how we can help!

/by

How to do ESP32 OTA Updates using Golioth and ESP-IDF

The most sought-after Golioth feature is OTA, also known as Over-the-Air firmware updates. When you put an IoT device into the field it’s crucial that you be able to push firmware updates to it without human intervention. Golioth makes simplifies the process for your ESP-IDF projects.

Today we’re walking through the OTA process:

  • Build and flash the initial firmware to the device
  • Provision the device with credentials that will be persistent across firmware updates
  • Build a new revision of the firmware
  • Upload the firmware to Golioth and roll it out as a release
  • Observe the device detecting, downloading, and running the new firmware

Prerequisites:

Please ensure that you have installed a copy of the the ESP-IDF v4.4.2 to your computer. Today’s article will use an ESP32 but this will work with other variants like the ESP32s2, ESP32c3, etc.

Clone a copy of the Golioth Firmware SDK (which includes ESP-IDF support). To do, please follow the “Cloning this repo section” in the README.

Commands in this guide are based on a Linux operating system with the ESP-IDF and Golioth Firmware SDK installed in the home directory (~/). However, these are cross-platform tools and are easy to adapt to your system and your preferred install directories.

Build and flash the initial firmware

In a classic Chicken-or-Egg scenario, to perform a Golioth OTA update your device needs to be running firmware built for Golioth OTA. We can use the golioth_basics example which is ready to run without changes.

First, let’s make sure our ESP-IDF is set to the correct version and enabled for this session:

cd ~/esp-idf
git fetch
git checkout v4.4.2
git submodule update --init --recursive ./install.sh all
source export.sh

Now move to the ESP-IDF section of the Golioth Firmware SDK, specifically the golioth_basics example. We’ll build, flash, and run this code on the ESP32:

cd ~/golioth-firmware-sdk/examples/esp_idf/golioth_basics
idf.py build
idf.py flash
idf.py monitor

On some systems you will need to hold the boot button on the ESP32 in order to flash the code. I find to get the monitor command to work I need to first hold the boot button, then press reset when the screen says “waiting for download”. One last tip: CTRL-] is used to exit from the idf.py monitor screen.

Assign device credentials and connect

There are a number of ways to assign credentials to your device (including Bluetooth via your browser!) but perhaps the easiest is to type them into the shell. Head over to the Golioth Console and select your device’s credentials tab. (If you don’t have an account, sign up for the Dev Tier now, your first 50 devices are free.)

Use your the shell window to set the credentials. Here you can see the process, with the four commands for Golioth and WiFi credentials highlighted:

Type 'help' to get the list of commands.
Use UP/DOWN arrows to navigate through command history.
Press TAB when typing command name to auto-complete.
esp32> W (2212) golioth_example: WiFi and golioth credentials are not set
W (2212) golioth_example: Use the shell settings commands to set them, then restart
esp32> 
esp32> settings set golioth/psk-id [email protected]
Setting golioth/psk-id saved
esp32> settings set golioth/psk 5bfb64ad29dce4e3dd30ab10c5b95a6a
Setting golioth/psk saved
esp32> settings set wifi/ssid MyWifiAp
Setting wifi/ssid saved
esp32> settings set wifi/psk MyWifiPassword
Setting wifi/psk saved
esp32> reset

The final command resets the device so that it will use the new credentials. You should see this output:

I (4235) esp_netif_handlers: sta ip: 192.168.1.159, mask: 255.255.255.0, gw: 192.168.1.1
I (4235) example_wifi: WiFi Connected. Got IP:192.168.1.159
I (4235) example_wifi: Connected to AP SSID: MyWifiAp
I (4255) golioth_mbox: Mbox created, bufsize: 2184, num_items: 20, item_size: 104
I (4255) golioth_basics: Waiting for connection to Golioth...
W (4295) wifi:<ba-add>idx:0 (ifx:0, c6:ff:d4:a8:fa:10), tid:0, ssn:1, winSize:64
I (4395) golioth_coap_client: Start CoAP session with host: coaps://coap.golioth.io
I (4405) libcoap: Setting PSK key

I (4415) golioth_coap_client: Entering CoAP I/O loop
I (4805) golioth_basics: Golioth client connected
I (4805) golioth_basics: Hello, Golioth!
I (4815) golioth_coap_client: Golioth CoAP client connected
I (4815) golioth_fw_update: Current firmware version: 1.2.5
I (5735) golioth_fw_update: Waiting to receive OTA manifest
I (5835) golioth_basics: Synchronously got my_int = 42
I (5845) golioth_basics: Entering endless loop
I (5845) golioth_basics: Sending hello! 0
I (5935) golioth_fw_update: Received OTA manifest
I (5935) golioth_fw_update: Manifest does not contain different firmware version. Nothing to do.
I (5945) golioth_fw_update: Waiting to receive OTA manifest
I (6545) golioth_basics: Callback got my_int = 42
W (9805) golioth_coap_client: CoAP message retransmitted
W (10335) golioth_coap_client: 4.00 (req type: 3, path: .c/status), len 59
I (15845) golioth_basics: Sending hello! 1
I (21965) wifi:bcn_timout,ap_probe_send_start
I (25855) golioth_basics: Sending hello! 2
I (35005) wifi:bcn_timout,ap_probe_send_start
I (35855) golioth_basics: Sending hello! 3

First the ESP32 connects to WiFi, then Golioth. After checking (and not finding) a firmware update available, this example begins sending hello messages every few seconds.

Now let’s do an OTA firmware update

We connected to Golioth with the device, now let’s build and upload a new firmware package to test the OTA capabilities. We’ll use the same code, updating the Current Version number which the device uses to identify when an update is needed. We’ll also change the string used in the log messages so it’s easy to recognize that our new firmware is running.

Change the source code version and rebuild

The file we need to update is a common file used by multiple Golioth SDKs. Edit the ~/golioth-firmware-sdk/examples/common/golioth_basics.c file:

#define TAG "golioth_basics_new"

// Current firmware version
static const char* _current_version = "1.2.6";

You can see I’ve appended “_new” to the tag name and incremented the version number to 1.2.6. Now we’re ready to rebuild… but remember, don’t flash this to your ESP32. We’re going to upload it to Golioth and perform a remote firmware update!

cd ~/golioth-firmware-sdk/examples/esp_idf/golioth_basics
idf.py build

The newly built binary is located in the build folder.

Upload firmware to Golioth and roll out a release

After much preamble we’ve arrived at the important moment.

To set the scene, the ESP32 that’s running on your desk is a remote IoT device taking sensor readings in a brick-and-mortar retail establishment in Waldorf, Maryland. We’ll push an update to it using a three-step process:

  1. Upload the binary as an “artifact”
  2. Create a “release” using the artifact
  3. Click the “rollout” button to make the release live

Go to the Golioth Console and select Firmware Update→Artifacts from the left sidebar. Click the “Create” button.

Golioth OTA create artifact

The only thing we’re going to change on this window is the “Artifact Version”. Type in the same version number you entered in the sourcecode for this firmware (probably 1.2.6 if you’re following along). Click the upload icon in the middle of the window and choose the “golioth_basics.bin” file from the build directory where you ran the idf.py build command. Finally, click the Upload Artifact button.

You have the option here to use a Blueprint. I’m not detailing that today for brevity, but it’s a good practice to use Blueprints to organize your production devices.

Now let’s create a release based on the artifact. Click Firmware Update→Release from the left sidebar and click the Create button.

Golioth OTA Release

All we’re going to do here is to chose the artifact we previously created in the Artifacts box and press Create Release.

You have some options here, most notably you can choose to start the rollout as soon as the release is created. I prefer to wait and roll it out as a separate confirmation step in case I made some mistake along the way.

Note that you have the option here of selecting device Blueprints and Tags to make this a more targeted release. This window is telling me the release will apply to 51 devices (!). That’s okay here because these are all test devices on a test project that we use for training.

Finally, let’s roll out the release to our devices:

Golioth OTA rollout

The Rollout button is all that stand between you and automatic updates. Click it and you will (almost) immediately see your device begin to download the new binary.

Here’s an awesome feature to keep in mind. When you have more than one release, you can use this button to rollback to previous versions. This means if you realize you released an update that has a bug, you can just toggle this button and all of your devices will automatically download the next-newest release that has rollout selected.

Watch your ESP32 update

If you read the source code for the golioth_basics example you will notice that it calls golioth_fw_update_init(client, _current_version);. That means the device has registered with the Golioth servers to receive updates when new firmware is available. Look in the terminal output and you will see the result:

I (175827) golioth_basics: Sending hello! 17
I (185007) golioth_fw_update: Received OTA manifest
I (185007) golioth_fw_update: Current version = 1.2.5, Target version = 1.2.6
I (185017) golioth_fw_update: State = Downloading
I (185317) golioth_fw_update: Image size = 1211744
I (185327) golioth_fw_update: Getting block index 0 (1/1184)
I (185827) golioth_basics: Sending hello! 18
W (187867) golioth_coap_client: CoAP message retransmitted
I (187947) fw_update_esp_idf: Writing to partition subtype 17 at offset 0x1a0000
I (187947) fw_update_esp_idf: Erasing flash
I (191627) golioth_fw_update: Getting block index 1 (2/1184)
I (191837) golioth_fw_update: Getting block index 2 (3/1184)
I (192037) golioth_fw_update: Getting block index 3 (4/1184)
I (192187) golioth_fw_update: Getting block index 4 (5/1184)
I (192447) golioth_fw_update: Getting block index 5 (6/1184)
I (192597) golioth_fw_update: Getting block index 6 (7/1184)

... snip ...

I (279837) golioth_fw_update: Getting block index 1181 (1182/1184) 
I (280107) golioth_fw_update: Getting block index 1182 (1183/1184) 
I (280317) golioth_fw_update: Getting block index 1183 (1184/1184) 
I (280457) golioth_fw_update: Total bytes written: 1211760 
I (280467) esp_image: segment 0: paddr=001a0020 vaddr=3f400020 size=29df0h (171504) map 
I (280527) esp_image: segment 1: paddr=001c9e18 vaddr=3ffbdb60 size=05868h ( 22632) 
I (280537) esp_image: segment 2: paddr=001cf688 vaddr=40080000 size=00990h ( 2448) 
I (280547) esp_image: segment 3: paddr=001d0020 vaddr=400d0020 size=d9a64h (891492) map 
I (280847) esp_image: segment 4: paddr=002a9a8c vaddr=40080990 size=1e2a0h (123552) 
I (280887) esp_image: segment 5: paddr=002c7d34 vaddr=50000000 size=00010h ( 16) 
I (280887) golioth_fw_update: State = Downloaded 
I (281127) golioth_fw_update: State = Updating 
I (281327) fw_update_esp_idf: Setting boot partition 
I (281337) esp_image: segment 0: paddr=001a0020 vaddr=3f400020 size=29df0h (171504) map 
I (281397) esp_image: segment 1: paddr=001c9e18 vaddr=3ffbdb60 size=05868h ( 22632) 
I (281417) esp_image: segment 2: paddr=001cf688 vaddr=40080000 size=00990h ( 2448) 
I (281417) esp_image: segment 3: paddr=001d0020 vaddr=400d0020 size=d9a64h (891492) map 
I (281717) esp_image: segment 4: paddr=002a9a8c vaddr=40080990 size=1e2a0h (123552) 
I (281757) esp_image: segment 5: paddr=002c7d34 vaddr=50000000 size=00010h ( 16) 
I (281827) golioth_fw_update: Rebooting into new image in 5 seconds 
I (282827) golioth_fw_update: Rebooting into new image in 4 seconds 
I (283827) golioth_fw_update: Rebooting into new image in 3 seconds 
I (284827) golioth_fw_update: Rebooting into new image in 2 seconds 
I (285827) golioth_fw_update: Rebooting into new image in 1 seconds

... snip ...

I (4267) esp_netif_handlers: sta ip: 192.168.1.159, mask: 255.255.255.0, gw: 192.168.1.1
I (4267) example_wifi: WiFi Connected. Got IP:192.168.1.159
I (4277) example_wifi: Connected to AP SSID: TheNewPeachRepublic
I (4287) golioth_mbox: Mbox created, bufsize: 2184, num_items: 20, item_size: 104
I (4287) golioth_basics_new: Waiting for connection to Golioth...
W (4297) wifi:<ba-add>idx:0 (ifx:0, c6:ff:d4:a8:fa:10), tid:0, ssn:1, winSize:64
I (4307) golioth_coap_client: Start CoAP session with host: coaps://coap.golioth.io
I (4307) libcoap: Setting PSK key

I (4317) golioth_coap_client: Entering CoAP I/O loop
I (4637) golioth_basics_new: Golioth client connected
I (4647) golioth_coap_client: Golioth CoAP client connected
I (4657) golioth_basics_new: Hello, Golioth!
I (4657) golioth_fw_update: Current firmware version: 1.2.6
I (4657) golioth_fw_update: Waiting for golioth client to connect before cancelling rollback
I (4677) golioth_fw_update: Firmware updated successfully!
I (4727) golioth_fw_update: State = Idle
I (5937) golioth_basics_new: Synchronously got my_int = 42
I (5937) golioth_basics_new: Entering endless loop
I (5937) golioth_basics_new: Sending hello! 0

First, the device compares version numbers and then begins to download blocks of the new firmware. Once downloaded it will reboot, connect to Golioth, and verify that it is running the newest version. The log labels near the end of the output now show golioth_basics_new, confirming one of the changes we made to our source code.

Golioth OTA report firmware version

On the Golioth Console, the summary view for this device confirms the currently running firmware version is 1.2.6!

With Golioth, OTA is built into the SDK

Golioth has done the heavy lifting so that you don’t need to. Our SDK uses just the single API call to register your devices for firmware updates. Use the fleet management tools on the Golioth Cloud to provision your devices in groups and by hardware variants. These make it possible to target your test devices for the first round of updates, or push new features just to the devices on the fourth floor of your Des Moines plant.

These robust tools are crucial for successful, long-lasting IoT deployments, and and they’re ready for you to start using right now. If you have any questions, we’d love to talk! Reach out to us on the Golioth Forum or get in touch with the DevRel team for demo.

/by
, ,

The Road to Firmware as a Hardware Engineer

How the Golioth Developer Training paved the way

This is a guest post from Shrouk El-Attar, discussing her journey from the hardware space into the firmware space and how Golioth training has helped her understand building out IoT systems using Zephyr.

The Journey Started with Hardware

To me, hardware just makes sense! You have specific requirements, find a part that can fulfill them, read its datasheet, and then execute. Voila! Successful design achieved.

The worst thing about the process? I don’t know…maybe the Googling? Getting a calculation out by a factor of ten? The tedious BOM stock checking process? These things can be long and tiring, but still, the concepts make sense. Designing a PCB is easy to understand; you follow simple rules and let the copper tracks do the rest. Even things in the “RF Voodoo” territory make sense with well-developed RF design tools these days.

Not too long ago, I finally took the leap into the world of consulting as a self-proclaimed “Hardware Queen”. With that, I saw the terrifying rise in demand for the 2-in-1 engineering consultants: a hardware engineer who is also a firmware engineer.

The Firmware Roadblock

Firmware feels to me like hardware’s anarchist sister. I know her a little; we have some history. It might surprise you to know that she hasn’t always been this way with me.

Once upon a time, there was Arduino (I know, I know). I probably picked up my first Arduino in 2012. I’d heard about them long before then, but as an asylum seeker, I couldn’t afford to own one. That is until the UK officially recognised me as a refugee, and I could finally access higher education and buy my own shiny Arduino.

The first time I got to Blinky on my Arduino, it was like something clicked. I understood it. I understood every single bit of code I was writing on it. “I have a gift,” I thought. Perhaps learning Visual Basic (yup) back in 2007 laid down those basics for me to get to Blinky on Arduino and be a total boss at it.

From then on, I felt invincible. Whatever I could think of, I was able to make it. Self-watering plant? Check. NFC easter egg hunt? Check. Twitter-powered bubble machine? Check. Arduino felt like hardware and software meeting in perfect harmony in a world where anything I could think of was possible.

Shrouk’s Arduino Connected Plant: Light data is sent to cloud platform (left), phone notifications are sent when light data is critical (middle), plant “tweets” when light data reaches below a certain level (right).

My initial success was a sign I was going to be a firmware engineer. But soon after, my firmware dreams were shattered.

Stuck in the Firmware Valley

My first full-time engineering role was at Intel back in 2015, where I specialized in the then-brand-new field called IoT. “Can you imagine that by 2020 there will be 50 billion connected devices?” I thought to myself. I was so excited that I would be one of the people developing those devices. It was the future.

One of the first things I couldn’t wait to get my hands on was the all-shiny Intel Realsense 3D Camera. But when I downloaded my SDK, I didn’t know where to start. After days of struggling on my own, I could get some things working with the help of my senior colleagues. But none of it clicked. I had no idea what I was doing. I was not gifted. Frankly, I sucked at firmware.

I started specializing in hardware. As the years passed, some of my all-time favorite chips became the nRF5 MCUs by Nordic Semiconductor. What a hardware engineer’s dream. Can’t get to a specific pin during routing? Don’t worry; choose literally any other pin you can reach more easily, and re-route there! The nRF5 had an excellent reputation amongst my firmware colleagues too.

In 2019, I decided to give firmware another go on my favorite chip. I wasn’t too far down the toolchain setup, and I’d already started regretting it. What the heck were Make files?! It was an excruciating process. But–no exaggeration–one week later, I’d managed to set up the toolchain. Great, let’s get to Blinky! Save, build, flash. Error.

I spent hours, then finally figure out the error. Save, build, flash—another error. Repeat a zillion times. It turns out I had never set up the toolchain correctly in the first place. Months later, I finally found a training that would work, or so I thought. This turned out to be my biggest disappointment. Nothing in the training was up to date. Nothing worked the way it was supposed to. This was indeed the final time I’d try to touch firmware again. I was stuck. I was done.

The Golioth Way

Back in present day, as I was realizing the demand for the 2-in-1 hardware and firmware engineering consultant, a well timed post from Chris Gammell appeared on my feed. The post was about the Golioth Developer Training, targeted explicitly at hardware engineers. The 2-in-1 engineer could be me. Was I going to try firmware again? Absolutely.

The training took place entirely remotely with hardware engineers from all over the world. This is going to be it, for sure! But, from the very beginning, I was already struggling. “Here we go again”, I thought. I shouldn’t have thought I could do it. I struggled with minor details at first, like appending -d instead of -D to a build command, which apparently gives wildly different results. I copied and pasted the correct information but in the wrong fields. And each exercise took me way longer than the “estimated time” suggested for each section.

The training happened in a large group, but we did the exercises independently. Chris and Mike were checking in on each of us throughout, so they were able to help me fix anything I missed very quickly on a one-to-one basis. As the training went on, I started to notice that I needed their help less and less. What is this I feel? Some cautious optimism, perhaps?

The training method was active. I wasn’t sitting at my desk blindly following instructions. No, I was pushed to figure out the correct way to solve a problem at every stage. It was presented in bite-size information with many collapsed sections. The training also included links referencing concepts, e.g. Zephyr Pin Control, throughout, should I want to learn more. For the most part, I didn’t click those because I wanted to stay on track. My head was already full of too much firmware to learn. But I enjoyed that they were there so I could refer back to them in the future.

Then the happiest of accidents happened. I picked up a later stage of the training on my own, then realized that I had to redo the earlier parts to get to where I wanted to pick up from. Not going to lie; I was slightly annoyed. But while re-doing them, I saw that it wasn’t taking me very long at all. This time, I was well within the time estimates suggested for each exercise, if not much quicker. Was this working for me? Did I finally find a method to learn commercially viable firmware that works?

A light bulb went off when I realized that it took me over a week to get to Blinky on my previous firmware training, but within a couple of hours, I was already speaking to my Wi-Fi device through the Golioth Console. Not only that, but it was all through the Zephyr RTOS with Golioth. The same Zephyr RTOS I struggled endlessly with on the nRF5 hardware that I love. It was a miracle to think about being enabled for firmware on my favorite hardware platform. Coupled with the fact that Golioth is entirely scalable, i.e. the idea that I can start from an idea to production on the same platform, made Golioth a dream.

Golioth also helped me develop a community, some of whom I became close Discord friends with. We all need a Discord buddy to complain to about daily life as an electronics engineering consultant (Hi Seth!). And a decade after I picked up my first Arduino, Golioth Developer Training helped me finally break through that commercially viable firmware ceiling.

Next, I’ll work on reference designs and put the skills I learnt from the training to the test. Watch this space.

Editor note: If you’re interested in taking part as an individual or as a company, sign up for future training here.

/by