Download Images to IoT Devices Using OTA

Golioth Over-the-Air (OTA) Updates in most common cases are used for single-image firmware upgrade purposes. In that scenario, a device is notified about a new release. Such notification includes a release manifest, which contains information about new firmware. The most important metadata that a device gets is firmware version, hash, and URL (used to download the firmware).

Firmware is the only artifact that is tied to OTA release in that scenario. But the Golioth OTA service may also include multiple artifacts. This allows you to implement multi-image upgrades, e.g. when there are many MCUs on a single device. Golioth OTA even supports artifacts that are not firmware, but large blobs of data of any kind. Examples include AI models, images and arbitrary binary blobs.

Device with a display

This article shows an example application running on a device with a display. Implemented functionality is simple: just displaying an arbitrary image. In the future we would like to add more capabilities, so firmware upgrade will be implemented. Additionally we would like to change the displayed image without upgrading the whole firmware.

Multi-component OTA

The Golioth SDK exposes high-level APIs to easily setup single-image firmware upgrade (golioth_fw_update_init()). This automatically creates a thread that observes newest firmware release, upgrades it when notified and reboots to run the new version.

In the case of multi-component releases we will handle the manifest in application code. Let’s first implement a callback that gets executed when a new release is available:

struct ota_observe_data
{
    struct golioth_ota_manifest manifest;
    struct k_sem manifest_received;
};

static void on_ota_manifest(struct golioth_client *client,
                            const struct golioth_response *response,
                            const char *path,
                            const uint8_t *payload,
                            size_t payload_size,
                            void *arg)
{
    struct ota_observe_data *data = arg;

    LOG_INF("Manifest received");

    if (response->status != GOLIOTH_OK)
    {
        return;
    }

    LOG_HEXDUMP_INF(payload, payload_size, "Received OTA manifest");

    enum golioth_ota_state state = golioth_ota_get_state();
    if (state == GOLIOTH_OTA_STATE_DOWNLOADING)
    {
        GLTH_LOGW(TAG, "Ignoring manifest while download in progress");
        return;
    }

    enum golioth_status status =
        golioth_ota_payload_as_manifest(payload, payload_size, &data->manifest);
    if (status != GOLIOTH_OK)
    {
        GLTH_LOGE(TAG, "Failed to parse manifest: %s", golioth_status_to_str(status));
        return;
    }

    if (data->manifest.num_components > 0) {
        k_sem_give(&data->manifest_received);
    }
}

The above code checks whether the release manifest was received correctly and OTA is not already in progress. Then the CBOR encoded manifest is decoded with golioth_ota_payload_as_manifest(). If the manifest is valid and it contains at least one component, the main application thread is notified by releasing a semaphore with k_sem_give(&data->manifest_received).

Now it is time to start manifest observation in main() and wait until a release manifest is received:

int main(void)
{
    struct ota_observe_data ota_observe_data = {};

    /* ... */

    golioth_ota_observe_manifest_async(client, on_ota_manifest, &ota_observe_data);

    k_sem_take(&ota_observe_data.manifest_received, K_FOREVER));

    /* ... */
}

At this point the application continues execution after the manifest is successfully received and parsed. The next step is handling of received components:

int main(void)
{
    /* ... */

    for (size_t i = 0; i < ota_observe_data.manifest.num_components; i++) {
        struct golioth_ota_component *component = &ota_observe_data.manifest.components[i];
        uint8_t hash_bin[32];

        hex2bin(component->hash, strlen(component->hash), hash_bin, sizeof(hash_bin));

        struct component_desc *desc = component_by_name(component->package);
        if (!desc) {
            LOG_WRN("Unknown '%s' artifact package", component->package);
            continue;
        }

        if (desc->version ?
            (component_version_cmp(desc, component->version) == 0) :
            (component_hash_cmp(desc, hash_bin) == 0)) {
            continue;
        }

    LOG_INF("Updating %s package", component->package);

        status = golioth_ota_download_component(client, component, desc->write_block, NULL);
        if (status == GOLIOTH_OK) {
            reboot = true;
        }
    }

    /* ... */
}

Information about each component is stored in the ota_observe_data.manifest.components[] array. Either version or hash is compared with the received component. When it differs, the new component is downloaded with golioth_ota_download_component() API.

Firmware and background components require different handling. This is achieved with component_descs[] array and helper functions:

struct component_desc
{
    const char *name;
    const char *version;
    uint8_t hash[32];
    ota_component_block_write_cb write_block;
};

static struct component_desc component_descs[] = {
    { .name = "background", .write_block = write_to_storage },
    { .name = "main", .write_block = write_fw, .version = _current_version },
};

static int component_hash_update(struct component_desc *desc, uint8_t hash[32])
{
    memcpy(desc->hash, hash, 32);

    return 0;
}

static int component_hash_cmp(struct component_desc *desc, const uint8_t hash[32])
{
    return memcmp(desc->hash, hash, 32);
}

static int component_version_cmp(struct component_desc *desc, const char *version)
{
    return strcmp(desc->version, version);
}

static struct component_desc *component_by_name(const char *name)
{
    for (size_t i = 0; i < ARRAY_SIZE(component_descs); i++) {
        struct component_desc *desc = &component_descs[i];

        if (strcmp(desc->name, name) == 0) {
            return desc;
        }
    }

    return NULL;
}

Downloaded firmware is written to flash directly, into second application slot:

static struct flash_img_context flash;

enum golioth_status write_fw(const struct golioth_ota_component *component,
                             uint32_t block_idx,
                             uint8_t *block_buffer,
                             size_t block_size,
                             bool is_last,
                             void *arg)
{
    const char *filename = component->package;
    int err;

    LOG_INF("Writing %s block idx %u", filename, (unsigned int) block_idx);

    if (block_idx == 0) {
        err = flash_img_prepare(&flash);
        if (err) {
            return GOLIOTH_ERR_FAIL;
        }
    }

    err = flash_img_buffered_write(&flash, block_buffer, block_size, is_last);
    if (err) {
        LOG_ERR("Failed to write to flash: %d", err);
        return GOLIOTH_ERR_FAIL;
    }

    if (is_last) {
        LOG_INF("Requesting upgrade");

        err = boot_request_upgrade(BOOT_UPGRADE_TEST);
        if (err) {
            LOG_ERR("Failed to request upgrade: %d", err);
            return GOLIOTH_ERR_FAIL;
        }
    }

    return GOLIOTH_OK;
}

The background image is written to file system using write_to_storage() callback:

enum golioth_status write_to_storage(const struct golioth_ota_component *component,
                                     uint32_t block_idx,
                                     uint8_t *block_buffer,
                                     size_t block_size,
                                     bool is_last,
                                     void *arg)
{
    const char *filename = component->package;
    struct fs_file_t fp = {};
    fs_mode_t flags = FS_O_CREATE | FS_O_WRITE;
    char path[32];
    int err;
    ssize_t ret;

    LOG_INF("Writing %s block idx %u", filename, (unsigned int) block_idx);

    if (block_idx == 0) {
        flags |= FS_O_TRUNC;
    }

    sprintf(path, "/storage/%s", filename);

    err = fs_open(&fp, path, flags);
    if (err) {
        LOG_ERR("Failed to open %s: %d", filename, err);

        return GOLIOTH_ERR_FAIL;
    }

    err = fs_seek(&fp, block_idx * CONFIG_GOLIOTH_BLOCKWISE_DOWNLOAD_BUFFER_SIZE, FS_SEEK_SET);
    if (err) {
        goto fp_close;
    }

    ret = fs_write(&fp, block_buffer, block_size);
    if (ret < 0) {
        err = ret;
        goto fp_close;
    }

fp_close:
    fs_close(&fp);

    if (err) {
        return GOLIOTH_ERR_FAIL;
    }

    return GOLIOTH_OK;
}

Displaying (updated) background

Firmware is updated automatically during next boot, so there is nothing more needed to start using it. Background image, on the other hand, needs to be loaded from file system in the application code:

static lv_img_dsc_t img_background;

static int background_show(void)
{
    char hash[32] = {};
    struct fs_dirent dirent;
    struct fs_file_t background_fp = {};
    lv_img_header_t *img_header;
    uint8_t *buffer;
    int err;
    ssize_t ret;

    err = fs_stat("/storage/background", &dirent);
    if (err) {
        if (err == -ENOENT) {
            LOG_WRN("No background image found on FS");
        } else {
            LOG_ERR("Failed to check/stat background image: %d", err);
        }

        return err;
    }

    LOG_INF("Background image file size: %zu", dirent.size);

    buffer = malloc(dirent.size);
    if (!buffer) {
        LOG_ERR("Failed to allocate memory");
        return -ENOMEM;
    }

    err = fs_open(&background_fp, "/storage/background", FS_O_READ);
    if (err) {
        LOG_WRN("Failed to load background: %d", err);
        goto buffer_free;
    }

    ret = fs_read(&background_fp, buffer, dirent.size);
    if (ret < 0) {
        LOG_ERR("Failed to read: %zd", ret);
        err = ret;
        goto background_close;
    }

    if (ret != dirent.size) {
        LOG_ERR("ret (%d) != dirent.size (%d)", (int) ret, (int) dirent.size);
        err = -EIO;
        goto background_close;
    }

    err = mbedtls_sha256(buffer, dirent.size, hash, 0);
    if (err) {
        LOG_ERR("Failed to get update sha256: %d", err);
        goto background_close;
    }

    LOG_HEXDUMP_INF(hash, sizeof(hash), "hash");

    component_hash_update(&component_descs[1], hash);

    img_header = (void *)buffer;
    img_background.header = *img_header;
    img_background.data_size = dirent.size - sizeof(*img_header);
    img_background.data = &buffer[sizeof(*img_header)];

    lv_obj_t * background = lv_img_create(lv_scr_act());
    lv_img_set_src(background, &img_background);
    lv_obj_align(background, LV_ALIGN_CENTER, 0, 0);

background_close:
    fs_close(&background_fp);

buffer_free:
    free(buffer);

    return err;
}

Note that besides loading the background image, there is also SHA256 calculation using mbedtls_sha256(). This is needed to compare with the SHA256 hash received from OTA service in order to decide whether the background image needs to be updated.

Testing with native_sim

The example-download-photo application is compatible with XIAO ESP32S3 with Seeed Studio XIAO Round Display. However it is possible to test with Native Simulator as well. To achieve that, the following command can be used:

# Build the example
west build -p -b native_sim/native/64 $(west topdir)/example-download-photo

# Run the example
west build -t run

Native Simulator uses the SDL library to emulate a display. On the first run it is blank because no background image is available. Now it is time to upload a background image as an OTA artifact and create a release. An example background image is included in the repository in background/Echo-Pose-Stand.bin. After rolling out an OTA release, this image is downloaded automatically to /storage/background file on the device, which is indicated with the following logs:

[00:00:01.310,007] <inf> example_download_photo: Received OTA manifest
...
[00:00:01.310,007] <inf> example_download_photo: component 0: package=background version=1.0.5 uri=/.u/c/[email protected] hash=6b4d243a362c0c4f63c535b2d2f7b8dfe4bcfbca69e7b2f8009f917458794c5e size=35716
[00:00:01.310,007] <inf> example_download_photo: Updating background package
[00:00:01.560,008] <inf> example_download_photo: Writing background block idx 0
[00:00:01.700,009] <inf> example_download_photo: Writing background block idx 1
...
[00:00:06.320,042] <inf> example_download_photo: Writing background block idx 34

Starting Native Simulator again shows the following screen: