Recording Audio

This example is a simple application of the imp003/LBWA1ZV1CD’s hardware sampler — its integrated analog-to-digital converter (ADC) — to record a series of audio samples using the EVB’s microphone. Buffers of audio samples are sent to the agent as they are recorded. The agent writes a WAV header to the resulting file, allowing the audio to be opened and played back directly in a browser.

Peripherals Used

• Input Microphone, pre-amlifier, imp003/LBWA1ZV1CD ADC

APIs Used

• pin
• sampler

Setup

Create a Developer Account and add your EVB to impCentral™

1. Create a free Developer Account to access Electric Imp impCentral and log in.
2. Download the Electric Imp app from the App Store or Google Play store on your iOS or Android device, and log in using your Developer Account credentials.
3. On the Connect an Imp page, select your WiFi network.
   • The imp only operates using 2.4GHz 802.11b, g or n. 5GHz networks are not supported.
4. Use BlinkUp™ to configure your EVB for Internet access. Power up the EVB then press Send BlinkUp in the Electric Imp app. As soon as you hear the countdown tones, place your tablet or phone’s screen against the EVB’s imp003/LBWA1ZV1CD breakout board until the screen stops flashing and you hear a second set of tones.

If you are having problems getting the EVB online, please consult the BlinkUp Troubleshooting Guide.

A successful BlinkUp operation is indicated by a slowly flashing green LED on the EVB. The EVB will be listed in impCentral’s ‘My Development Devices’ list (accessed via the devices menu in the black bar at the top). The number presented in impCentral is the EVB’s unique Device ID. You can, if you wish, give it a more friendly name as follows:

1. Click ‘Settings’ in the ‘MANAGE’ column.
2. In the ‘Device Settings’ panel, enter a new name, eg. ‘LBWA1ZV1CD EVB’, ‘imp003 EVB’ or ‘My EVB’, and click ‘Update’.

Create a New Product and Device Group

1. Click on your account name in the grey bar at the top of the window to view a list of your Products — this will be empty initially. If not, select the “EVB” Product, click ‘Create New Device Group’ and go to step 4.
2. Click ‘Create New Product’.
3. Enter a name for the Product: “EVB”.
4. Enter a name for the first Device Group: “Audio”.

Add Code

1. Copy the device and agent code from our GitHub repo and paste it into the impCentral’s code editor ‘Device Code’ and ‘Agent Code’ panes, respectively.
2. Click ‘Build and Force Restart’ to transfer the device code to the imp003/LBWA1ZV1CD and the agent code to Electric Imp impCloud™, and to run them.

What’s Going On

This example works in essentially three steps:

1. The device waits for a button press. When the button press occurs, the device starts filling buffers with data from the sampler and passing them to the agent as they are filled, similar to a bucket brigade.
2. After a pre-set amount of time, the device stops recording. The agent takes all the data it has received, writes a WAV file header on it, and waits.
3. When an HTTP request arrives at the agent, the agent responds with the file it has assembled from the device.

What the Device Does

The device firmware is written in a pattern that is likely now becoming familiar: a class. The Recorder class wraps up all the code needed to run the sampler and send a short waveform over to the agents in small chunks.

After the Recorder class definition (covered in more detail in a moment), the device firmware defines a callback function, run when the user presses a button.

First, the callback function checks the current state of the button. If this is not done, the callback will be called every time the button changes state; once for press, and once for release.

function recordBtnCallback() {
    // if the button is currently pressed, start a recording
    if (btn.read()) {

If the button is currently pressed, the device starts a recording. The green LED is turned on to show the user that the mic is active, the microphone preamp is powered on, and the recorder class's start method is called:

// turn on the green LED
led_grn.write(0);
// enable the microphone
mic_en_l.write(0);
// start a recording; data is sent to the agent as it is recorded
recorder.start();

Before the recordBtnCallback returns, it schedules another callback to stop the sampler after RECORD_TIME seconds:

imp.wakeup(RECORD_TIME, function() {
    led_grn.write(1);
    recorder.stop();
    // disable microphone
    mic_en_l.write(1);
});

Lastly, we see each of the pins used in this example defined and configured, and then used to instantiate the Recorder class to create the recorder object (used in the recordBtnCallback function defined earlier):

server.log("Started. Free memory: "+imp.getmemoryfree());

mic         <- hardware.pinJ;
mic_en_l    <- hardware.pinT;
btn         <- hardware.pinU;
led_grn     <- hardware.pinF;

// configure button to start a recording
btn.configure(DIGITAL_IN, recordBtnCallback);
// configure LED for simple on/off
led_grn.configure(DIGITAL_OUT);
led_grn.write(1);
// mic enable pin
mic_en_l.configure(DIGITAL_OUT);
mic_en_l.write(1);

recorder <- Recorder(mic, RECORD_OPTS, SAMPLERATE, BUFFERSIZE);

Inside the Recorder Class

The recorder class itself doesn't contain much code. The are methods to start and stop recording, a finish method used to alert the agent when a recording has been completed, and a samplesReady callback, required by the sampler. The start and stop methods do just what they say: the call the sampler's start and stop methods. They also do a little bit of setup and teardown.

The start method configures the sampler before calling sampler.start. The sampler takes in several parameters:

• An ADC-capable pin to sample
• A sample rate in Hz
• An array of buffers to fill with data
• A callback function to call each time a buffer is filled, allowing the user to determine what is done with the data as it is made available
• Optional flags for compression

In this case, the Recorder class's samplesReady method is provided as a callback, with bindenv. The bindenv method is part of the Squirrel language; it means "when this function is called as a callback, call it in this environment". This allows the callback to be called from global scope even though it is part of the recorder object:

// helper: callback and buffers for the sampler
function samplesReady(buffer, length) {
    if (length > 0) {
        agent.send("push", buffer);
    } else {
        server.log("Sampler Buffer Overrun");
    }
}

// start recording audio
function start() {
    server.log("Staring Sampler");
    hardware.sampler.configure(mic, samplerate,
        [blob(buffersize),blob(buffersize),blob(buffersize)],samplesReady.bindenv(this),
            sampleroptions);
    hardware.sampler.start();
}

The stop method also uses bindenv to scope a callback. When the sampler is stopped, there is still data in the current buffer that hasn't been sent to the samplesReady callback yet. When sampler.stop is called, the sampler will immediately stop recording, and call the samplesReady callback with whatever is in the current buffer. After the samplesReady callback finishes with that buffer, squirrel will idle momentarily as the last call returns. The stop method schedules a function to run as soon as this idle occurs, using the imp.onidle method:

// stop recording audio
// the "finish" helper will be called to finish the process when the last buffer is ready
function stop() {
    hardware.sampler.stop();
    // the sampler will immediately call samplesReady to empty its last buffer
    // following samplesReady, the imp will idle, and finish will be called
    imp.onidle(finish.bindenv(this));
}

The finish method sends an alert to the agent that the entire audio waveform has been sent, finishing the process:

// helper: clean up after stopping the sampler
function finish() {
    // signal to the agent that we're ready to upload this new message
    // the agent will call back with a "pull" request, at which point we'll read the buffer out of flash and upload
    agent.send("done", 0);
    server.log("Sampler Stopped");
}

What the Agent Does

The agent has three jobs:

1. Receive chunks of audio data from the device and store them in a single buffer.
2. Write a WAV header onto the full buffer when all the chunks have been received from the device.
3. Respond to HTTP requests from the Internet

The agent handles interaction with the device through two device.on callbacks: one for the "push" event, and one for the "done" event, each sent from the device with agent.send.

The push event callback takes in buffers of audio data from the device and adds them to the end of a long buffer in the agent. The agent also keeps track of the total length of the message received with the message_len variable:

// take in chunks of data from the device during upload
device.on("push", function(buffer) {
    if (!recording) {
        recording = true;
        // reset to the beginning of the agent buffer
        // pre-allocate some space so we don't have to resize the buffer later for short messages
        agent_buffer = blob(AGENT_BUFFER_SIZE);
        agent_buffer.seek(HEADER_CHUNK_SIZE,'b');
    }

    message_len += buffer.len();
    agent_buffer.writeblob(buffer);
});

The done event callback takes in a dummy parameter, which is not used. This callback triggers the recording clean-up process: counters and flags are reset, and the agent writes the WAV header to the buffer with writeChunkHeaders:

// reset when the device indicates it is done recording
device.on("done", function(dummy) {
    message_ready = true;
    recording = false;
    // we now have the whole message and can write WAV headers at the beginning of the buffer
    writeChunkHeaders(agent_buffer, message_len);
    message_len = 0;
    agent_buffer.seek(0);
});

Last in the agent, the http.onrequest handler is defined to handle incoming HTTP requests. In this case, the agent has a very simple HTTP interface; it simply responds to all requests with the WAV file it has assembled, or responds with "No new messages" if the device has not yet sent any data.

The agent has access to a powerful http class which allows it a great deal of control over requests and responses. In this example, the agent sets a number of headers on the response. First, the Access-Control-Allow-Origin, Access-Control-Allow-Headers and Access-Control-Allow-Methods headers are set to prevent requests from being blocked due to restrictive browser rules.

Once the headers are set, the agent checks to see if a message is ready. If there is audio data ready to send, the agent sets one more header: Content-Type is set to audio/x-wav, which allows the browser to play the received file right in the open window, rather than download the file. Then, the response to the inbound request is sent.

http.onrequest(function(req, res) {
    // we need to set headers and respond to empty requests as they are usually preflight checks
    res.header("Access-Control-Allow-Origin", "*");
    res.header("Access-Control-Allow-Headers","Origin, X-Requested-With, Content-Type, Accept");
    res.header("Access-Control-Allow-Methods", "POST, GET, OPTIONS");

    server.log("Request received for latest recorded message.");
    if (message_ready) {
        server.log("Serving Audio Buffer, len "+agent_buffer.len());
        // set content-type header so audio will play in the browser
        res.header("Content-Type","audio/x-wav");
        res.send(200, agent_buffer);
    } else {
        res.send(200, "No new messages");
    }
});

For a more detailed investigation of the sampler, please see "Audio Waveforms and the Imp" in the Developer Center.

In the next section, we’ll use the imp003/LBWA1ZV1CD EVB’s fixed-frequency DAC to output audio downloaded from the Internet.