Sending Audio for IoT to the Cloud

We’re on a roll with our showcase of examples that came out of Golioth’s AI Summer. Today I’m discussing an example that records audio on an IoT device and uploads the audio to the cloud.

Why is this useful? This example is a great approach to sending sensor data from your edge devices back to the cloud to use in a machine learning (ML) training set, or just a great way to collect data samples from your network. Either way, we’ve designed Golioth to efficiently handle data transfer between constrained devices and the cloud.

The full example code is open source and ready to use.

Overview

The bones of this example are the same as the image upload example we showcased earlier. The main components include:

1. An audio sample (or any other chunk of data you’d like to send).
2. A callback function to fill a buffer with blocks from your data source.
3. A function call to kick off the data upload.

That’s it for the device side of things. Sending large chunks of data is a snap for your firmware efforts.

The cloud side is very simple too, using Golioth Pipelines to route the data as desired. Today we’ll send the audio files to an Amazon S3 bucket.

1. An Audio Sample

The details of audio recording are not important for this example. WAV, MP3, FLAC…it’s all just 1’s and 0’s at the end of the day! The audio is stored in a buffer and all we need to know is the address of that buffer and its length.

If you really want to know more, this code is built to run on one of two M5 Stack boards: the Core2 or the CoreS3. Both have a built-in i2s microphone and an SD card that is used to store the recording. SD cards storage is a great choice for prototyping because you can easily pop out the card and access the file on your computer to confirm what you uploaded is identical. Full details are found in the audio.c file.

2. Callback function

To use block upload with Golioth, you need to supply a callback function to fill the data buffer. The Golioth Firmware SDK will call this function when preparing to send each block.

uint8_t audio_data[MAX_BUF_SIZE];
size_t audio_data_len;

/* Run some function to record data to buffer and set the length variable */

static enum golioth_status block_upload_read_chunk(uint32_t block_idx,
                                                   uint8_t *block_buffer,
                                                   size_t *block_size,
                                                   bool *is_last,
                                                   void *arg)
{
    size_t bu_offset = block_idx * bu_max_block_size;
    size_t bu_size = audio_data_len - bu_offset;
    if (bu_size <= block_size)
    {
        /* We run out of data to send after this block; mark as last block */
        *block_size = bu_size;
        *is_last = true;
    }
    /* Copy data to the block buffer */
    memcpy(block_buffer, audio_data + bu_offset, *block_size);
    return GOLIOTH_OK;
}

The above code is a very basic version of a callback. It assumes you have a global buffer audio_data[] where recorded audio is stored, and a variable audio_data_len to track the size of the data stored there. Each time the callback runs it reads from a different part of the source buffer by calculating the offset based on the supplied *block_size variable. The callback signals the final block by setting the *is_last variable to true, and updating the *block_size to indicate the actual number of bytes in the final block.

You can see the full callback function in the example app which includes full error checking and passes a file pointer as the user argument to access data on the SD card. The file handling APIs from the standard library are used, with a pointer to the file passed into the callback.

3. API call to begin upload

Now we start the upload by using the Stream API call, part of the Golioth Firmware SDK. Just provide the important details for your data source and the path to use when uploading.

int err = golioth_stream_set_blockwise_sync(client,
                                            "file_upload",
                                            GOLIOTH_CONTENT_TYPE_OCTET_STREAM,
                                            block_upload_read_chunk,
                                            NULL);

This API call includes four required parameters shown above:

• client is a pointer to the Golioth client that holds info like credentials and server address
• "file_upload" is the path at which the file should be uploaded (change this at will)
• GOLIOTH_CONTENT_TYPE_OCTET_STREAM is the data type (binary in this case)
• block_upload_read_chunk is the callback we wrote in the previous step

The final parameter is a user argument. In the audio sample app we use this to pass a pointer to read data from the file on the SD card.

Routing your data

The example includes a Golioth pipeline for routing your data.

filter:
  path: "/file_upload*"
  content_type: application/octet-stream
steps:
  - name: step0
    destination:
      type: aws-s3
      version: v1
      parameters:
        name: golioth-pipelines-test
        access_key: $AWS_S3_ACCESS_KEY
        access_secret: $AWS_S3_ACCESS_SECRET
        region: us-east-1

You can see the path in the pipeline matches the path we used in the API call of the previous step. This instructs Golioth to listen for binary data (octet-stream) on that path, and when found, route it to an Amazon S3 bucket. Once enabled, your audio file will automatically appear in your S3 bucket!

IoT data transfer shouldn’t be difficult

That’s worth saying twice: IoT data transfer shouldn’t be difficult. In fact, nothing in IoT should be difficult. And that’s why Golioth is here. It’s our mission to connect your fleet to the cloud, and make accessing, controlling, updating, and maintaining your fleet a great experience from day one. Take Golioth for a test drive now!