Skip to main content

Effective Internet-agent-device Communication

How imp-enabled devices interact with the outside world

Every type of imp has been designed to bring the benefits of Internet connectivity to a wide variety of devices. But imps don’t communicate with the cloud directly. Instead, this contact is delegated to their agents — per-device micro-servers running in the Electric Imp impCloud™. The imp API provides a full set of tools to manage this process.

Think of the agent as the imp’s ‘gopher’. It locates and accesses Internet-based resources on the imp’s behalf, and handles incoming commands and requests for information. This approach allows the imp to devote as little as possible of its runtime and, crucially, its power on Internet communications; instead, the agent does all the hard work.

As the developer of the agent code running in the impCloud and the device code running on the imp, you are the orchestrator of this dialogue between imp and agent, and of wider conversations with remote apps and websites, whether they are data sources in their own right, or being used to control the device the imp has brought Internet connectivity to.

So how do you use the imp’s Squirrel scripting language to define that communication?

Talking to the Web

Because it was designed specifically for the Internet, an agent speaks the cloud’s lingua franca, HTTP: the Hypertext Transfer Protocol. This means it can interact with any application that can issue standard HTTP requests, be it a web page — either local or server-hosted — or a mobile or desktop app written to support a specific operating system. As long as the user’s software can send out an HTTP request to a URL, it can talk to an imp by way of the imp’s agent.

HTTP is an open standard, so it is well defined and accessible. It is maintained by the W3C, which provides a full specification. The specification defines the HTTP message, of which there are two forms: the HTTP request and the HTTP response.

At its most basic, an HTTP request is a demand for data made by a client to a server, delivered by calling a single URL. However, it’s possible to scale that to complex requests that pass data and manage security.

Here we’ll focus on the basics of app-agent-imp interaction, which essentially follows this pathway: App -> impCloud -> device.

As we’ll see, the lines of communication also move up the chain, from device to impCloud to app.

Agent Communications

The hub of imp communication is the imp’s agent. This server-based code responds to incoming HTTP requests by analyzing what they contain and then, if necessary, triggering appropriate behavior in the imp itself. The specific chain of communication we’ll explore looks like this:

The foundation for the agent’s ability to deal with incoming HTTP requests — in this case made by a mobile app — is the imp API’s automatically instantiated http object, which provides the method http.onrequest(). This method registers the name of the function that will be called when the agent receives an HTTP request. The callback is only called if this event takes place, and is called every time such an event occurs. The http.onrequest() method’s only parameter is the callback’s name.

The callback function itself follows an established best-practice pattern: wrap the code which parses the HTTP request in Squirrel’s try... catch exception trap structure. This is done in order to provide a mechanism for the code to be notified about connectivity and other errors, and to deal with them while the code continues to run:

function requestHandler(request, response) {
    try {
        // "200: OK" is standard return message
        response.send(200, "OK"); 
    } catch (ex) {
        // Send 500 response if error occurred
        response.send(500, ("Agent Error: " + ex)); 
    }
}

// Register the callback function that will be triggered by incoming HTTP requests
http.onrequest(requestHandler);

The response.send() method is a member of the imp API’s httpresponse object. It takes two parameters: an integer holding the W3C-defined standard HTTP response status code, and a message string. The http.onrequest() method automatically generates an httpresponse object, and it passes this alongside the original incoming HTTP request to the request handler. Later, we’ll cache this response object — here accessed through a local variable, response, which has been primed to return data to the source of the request. The saved response will be used as a way of passing data back from the imp to the app.

The HTTP request itself — accessed within the handler through a local variable, request — is a Squirrel table containing the key elements of request packaged as a set of key/value pairs. The keys are derived from the standard HTTP request structure:

  • method — the HTTP request type: GET, PUT, POST, etc.
  • path — the path to the requested HTTP resource minus the agent URL.
  • query — a Squirrel table containing the request parameters as key/value pairs.
  • body — the body of the HTTP request.
  • headers — a Squirrel table containing the request headers as key/value pairs.

These elements in the imp object httprequest may themselves contain further tables: query, for instance, is a table comprising the URL-passed parameters as keys, along with their values. For example, if our agent is sent the following HTTP request from a browser’s URL entry field:

http://agent.electricimp.com/<AGENT_ID>?setmode=24&gmt=no&utc=+4

then request will have the following query table:

Key Value
setmode 24
gmt no
utc +4

This table-based organization of HTTP request data makes it easy to locate request parameters, read their values and invoke behavior in the imp accordingly. The subsidiary tables and their key/value components are accessed through standard dot notation. Using the request.query table, the value of one parameter, setmode, is read and, depending on its value, one of two possible actions are triggered. Squirrel’s in keyword is applied to find in the request the parameter we are interested in:

if ("setmode" in request.query) {
    device.send("clock.switch.mode", (request.query.setmode == "24" ? true : false));
    response.send(200, "Clock mode switched");
}

The device object represents the imp to the agent, and allows the agent to communicate with it. Here the device.send() method is invoked to trigger the issuing of a notification to which the imp may, or may not, respond — we will have to write device-side code to register our interest in the notification and to process it should one arrive. The first parameter in device.send() is the name of the notification, selected by the programmer. The second is a single data value. Here that data is an integer, but it’s possible to send any other Squirrel data type: float, boolean, string, array, table, or a blob of bytes.

Talking to the imp

How will the device use this data? In keeping with the imp’s event-driven programming model, the device, at start-up, registers handler functions for each of the notifications it is willing to deal with should they be sent by the agent. We’ve already seen this kind of event handling in action, in the way the function requestHandler() is called solely in response to an incoming HTTP request event.

Just as the agent has the object device to represent the imp, so the imp has an object, agent, to stand in for its server-side partner. The imp calls the agent.on() function for as many agent-sent notifications as it wants to be alerted to. Our example requires only one for now:

agent.on("clock.switch.mode", switchMode);

Like the other half of the pair, device.send(), agent.on() takes a notification name string which must match on both halves of the function pairing. Its other parameter is the name of the function which will be called upon the arrival of the named notification. The data value passed by device.send() is automatically relayed to the callback, which just needs a suitable local parameter variable to take it:

function switchMode(passedValue) {
    // This function is called when 12/24-hour modes are switched
    if (!passedValue && hr24Flag) {
        hr24Flag = false;
        server.log("Clock set to AM/PM");
    } else if (passedValue && !hr24Flag) {
        hr24Flag = true;
        server.log("Clock set 24-hour mode");
    }

    updateDisplay();
}

This function sets an imp-controlled digital clock to 24-hour mode or an AM/PM display according to the value passed. The command to do so originally came from a mobile app and, using an HTTP request, agent request handling code, imp messaging and imp message handling code, has been passed down the line to the entity that can obey it.

Serializable Squirrel

There a limits on what may and may not not be included in the data passed using the messaging methods we’ve discussed. The data must be serialized before it is transmitted, but not all Squirrel entities can be serialized. Integers, floats, bools and blobs may be serialized, as may arrays containing these values. Nested arrays can be serialized only if they too contain serializable data.

Similarly, tables, and nested tables, can be serialized only if string-type slot keys are themselves serializable: they are encoded in Ascii or UTF-8 and contain no embedded NUL (\0) characters. Data strings that are not slot keys are serializable, but those which are considered ‘unsafe’ — ie. are not encoded in Ascii or UTF-8, or do contain embedded NUL characters — are first converted to blobs.

Classes, class instances and functions are not serializable so can’t be passed from agent to imp or vice versa. For more details on how Squirrel serializes variables and other entities, see ‘Serializable Squirrel’.

Back up the Chain

What we have seen so far is a one-way communication, but because the agent automatically generates an httpresponse object whenever an inbound HTTP request arrives, the imp can use this to return information to the app. Once again, this process is mediated by the imp’s agent. The digital clock in the example should run continuously, but the app may not: it may be deliberately shut down by the user, or the phone’s operating system may close it in response to memory shortages. If the app is restarted, it will need to query the clock in order to update its UI so that the user can see immediately how the clock has been set.

For this, the developer might specify a second HTTP request, getmode, alongside setmode. As a request for data rather than a command that provides the imp with information, getmode has no incoming data parameters, so the agent’s HTTP request handler needs only look for the command’s presence in the request:

savedResponse <- null;

function requestHandler(request, response) {
    try {
        if ("getmode" in request.query) {
            device.send("get.info", 1);
            savedResponse = response;
            return;
        }

        if ("setmode" in request.query) {
            device.send("clock.switch.mode", (request.query.setmode == "24" ? true : false));
            response.send(200, "Clock mode switched");
        }
    } catch (ex) {
        response.send(500, ("Agent Error: " + ex));
    }
}

The code added to the agent’s request handler checks for the presence of the getmode command in the request query. If it finds the command, a second, alternative notification is sent to the device. The imp API requires device.send() to contain a data value. We don’t have one, so we send 1 as dummy data.

The next line stores the automatically generated httpresponse object into a global variable defined at the start of the program proper. This has to be done before the response variable, which is local to the function, goes out of scope. Squirrel stores global variables in a special table, so to create a global, a new key/value pair — called a ‘slot’ in Squirrel terminology — has to be added to this table. It is initially a key without a value:

savedResponse <- null;

The device needs to register its interest in the new notification by providing a suitable callback function:

agent.on("get.info", sendInfo);

This the callback itself:

function sendInfo(value) {
    // Responds to app request for the clock's 12/24 hour setting
    local data = hr24Flag ? "24" : "12";
    agent.send("new.info", data);
}

Just as an agent’s device.send() method needs to be paired with an agent.on() function on the device, so the imp’s agent.send() will need to be joined by a device.on() function within the agent code. Again, this registers a callback to be triggered in response to the arrival of the nominated notification:

device.on("new.info", relayData);

And, as before, the data passed by the notification posting method is automatically relayed to the callback:

function relayData(data) {
    // Relay current clock data to app.
    local dataString = data;
    savedResponse.send(200, dataString);
}

Here, the code makes use of the saved httpresponse object (referenced by savedResponse) to return a numeric status code to the app — ‘200’ is defined by the W3C as ‘OK’ — along with the data sent by the imp and relayed to this function. The data is sent in string form, but no conversion is required since the data the imp sent was already a string. The code calls the httpresponse object’s httpresponse.send() method to pass the string back to the app through the initial connection, which is kept alive until a response is received, or a time-out period is exceeded. It is up to the app to look for the presence of the extra clock-mode information and update its UI accordingly.

This code assumes the agent won’t receive another getmode HTTP request in short order — a not unreasonable assumption if the agent-device latency is low, as we might expect, and the app making the request isn’t able to fire off a second getmode quickly. That may not be the case, however. A second getmode request (or equivalent) coming before the savedResponse.send(200, passedValue); statement has executed will cause the value of savedResponse to be overwritten with a new httpresponse object. This means the original request will never be met.

This is potentially a minor problem for the clock app — it’s going to get the data it’s seeking from either httpresponse — but it leaves the agent with an open connection, a least for ten minutes until the process times out and the connection is closed automatically. When there are many incoming connections, this may lead to memory and access issues.

An alternative, safer approach is to save every newly generated httpresponse object in a table with a unique key, such as a datestamp or a random number. That key can be passed to device and back so that when the agent eventually calls response.send() it can select the specific httpresponse associated with the request it is responding to and send it.

Round Trip

This time the app-agent-imp communication path is longer, and circular:

This example uses the imp’s HTTP handling in a very rudimentary way, but one that is nonetheless very effective for the simple communication between web page or app and an imp-controlled device.

The httprequest object’s core properties — method, path, query, body and headers — provide a foundation for more complex communication. The data they contain allow the agent, with suitable parsing code, to establish whether the request came from a user’s web browser or their mobile app, perhaps to assess which of these methods is the most popular.

The httprequest’s methods field supports most, but not all of those methods defined by the W3C: only GET, POST, PUT and DELETE are supported; OPTIONS, HEAD, TRACE and CONNECT are not, but these are highly unlikely to be needed in imp-related communications.

The agent’s ability to encode and decode JSON (JavaScript Object Notation) allows app-agent-imp communication to incorporate rich data. Together these features allow developers to establish sophisticated APIs to media communication between imp devices and remote applications. The imp API provides the http.jsonencode() and http.jsondecode() methods to support the conversion of data object to and from JSON-format code.

Similarly, the imp, via its agent, has the tool to engage in rich communications with other web services’ APIs without the mediation of a smartphone or tablet. Binary data can be encoded and decoded using the http.base64encode() and http.base64decode() methods. More straightforward Squirrel variables can be encoded — and retrieved on return — using the http.urlencode() and http.urldecode() methods.

Rocky

While it is not difficult to build complex agent-hosted APIs using imp API calls alone, the process is much more straightforward if you make use of Electric Imp’s Rocky library. This allows you to define suitable API endpoints and set up handlers for requests made to those endpoints, using the standard HTTP verbs. For example:

#require "Rocky.class.nut:2.0.1"

// Set up the API that the agent will server
api = Rocky();

// GET at / returns the UI
api.get("/", function(context) {
    context.send(200, format(htmlString, http.agenturl()));
});

// GET at /current returns the current forecast as JSON:
// { "cast" : "<the forecast>",
//   "icon" : "<the weather icon name>" }
// If there is an error, the JSON will contain the key 'error'
api.get("/current", function(context) {
    local data = {};

    if (savedData != null) {
        data = savedData;
    } else {
        data.error <- "Weather data not yet received. Please try again shortly";
    }

    local loc = {};
    loc.long <- myLongitude;
    loc.lat <- myLatitude;
    data.location <- loc;
    data.angle <- settings.displayAngle.tostring();
    data.bright <- settings.displayBrightness;
    data.debug <- settings.debug;
    data.version <- appVersion.slice(0, 3);
    data = http.jsonencode(data);
    context.send(200, data);
});