Preserve Your Data In The impCloud™ Or Within The imp
Developers of Electric Imp applications often need to save state information to ensure that when their products are restarted those devices continue to operate as they had before the reboot. Batteries run flat; power cables may be accidentally pulled out of sockets; end-users may inadvertently catch and press power buttons. Out-of-specification circumstances, such a high environmental temperatures, may also trigger a device reset.
When a device goes into deep sleep, which it might do to minimize energy use in a battery powered product, the Squirrel VM running your application code will be shut down. This clears all stored data. You may therefore need to persist application state data across deep sleeps.
To cater for cases such as these, the imp API provides a number of tools to allow developers to cache small volumes of low-level data in storage located within the imp itself, or within the Electric Imp impCloud™.
There are three ways of saving data on the device itself. All of them have the benefit that access to the data does not depend on an Internet connection. Which method you choose will depend on the type and volume of information you need to cache, and which imp module your product is based upon.
The first of these methods makes use of the imp API’s nv object. This is recognized global variable which you can use to reference a Squirrel table. Once you add keys and values to nv they are retained under most circumstances. For example, data added to nv can be read back after the imp has been through a deep sleep cycle. But if the imp has been power-cycled, its OS was updated, its Squirrel application code was updated, or a Squirrel error occurred, the contents of nv will be lost.
The space available for nv is just under 4KB. It is primarily intended to cache variables across deep sleeps, rather than a true data persistence mechanism.
And the nv may not be available to you: it is not supported by all imps. Currently, the imp004m, imp005 and imp006 do not support nv, so for these modules you will need to consider the next alternative, the imp’s SPI flash.
Two areas with the imp’s SPI Flash storage are available for storing application data. The saved data will persist across all kinds of device restart (unlike the nv table) and is not dependent on an Internet connection. The data is also retained across application changes and the blessing process, which means content can be installed by your factory firmware and read back by the application in the field.
This storage is primarily intended to be used to to save data for use by the device during its lifetime. For example, you might write image data to the device’s Flash via your factory firmware so that it is ready to be displayed when an end-user starts the device up. You might also use it to store end-user settings which need to be available to the device upon a cold start.
Up to 4KB of data may be stored in SPI Flash, using the imp API method imp.setuserconfiguration(). The stored data is retrieved using the companion method imp.getuserconfiguration(). To clear the data, pass null into imp.setuserconfiguration(). In fact, this is the only way to clear the data.
Unlike nv, imp.setuserconfiguration() takes either a string or a blob as its argument, so your application will need to organize the data itself: for example, converting tables to serial data.
Another limitation to consider is that the imp’s Flash storage has a finite write capacity, placing a limit on the number of times user data can be written. However, impOS’ Flash management code provides a minimum lifespan of 60,000 writes, and users should have significantly more writes than this.
In addition to the above user configuration space, impOS makes available to the application all SPI flash capacity that it does not need itself. How much space is available will depend on the imp module in use and the size of the SPI flash chip on your board. Any extra space that can be made available to the application is accessed using the imp API’s Spiflash class.
For example, imp003-based designs requires an external SPI flash part of at least 4Mbit (512KB) in capacity. But the maximum size you can fit is 128Mbit (16MB). However much space is available from the address 0x70000 (448KB) can be used by your application: 64KB to 15,936KB, based on the supported size range of 512KB-16MB of SPI Flash.
Other imps have different ranges — check their design guides for details.
You may also build other storage devices into your design, but bear in mind these will introduce extra bill-of-materials costs and will not be supported directly through Squirrel. Storage parts can be added and accessed over the imp’s SPI and I2C buses as peripherals. Depending on which of the many options you select, there may be an Electric Imp Squirrel driver library to help you.Storage that requires file-system usage will need suitable support integrated into your application as impOS does not support any file systems natively.
Storage space in the impCloud is made available for preserving up to 64KB of device and/or agent data across restarts. Data persisted by the agent is available to it immediately upon restarts, but device data can only be returned to the device upon request and when the device is once more connected to the Internet.
Data is saved and retrieved by the agent using two API methods: server.save() and server.load().
The first of these methods, server.save(), takes a Squirrel table as its argument; you place all the data you want to be preserved into this table as key-value pairs.
The second method, server.load(), reads backs that data and returns it as a new table. Because only a single, unique table is stored, server.load() has no parameters. If no data has been persisted, server.load() returns an empty table. This provides the agent with a way to test whether a previous server.save() operation has taken place. For example:
// Load in any saved settings
local savedSettings = server.load();
// Retain the loaded settings, or get defaults if we've not saved anything yet
settings <- savedSetting.len() != 0 ? savedSettings : setDefaults();
Should the agent need to clear the saved data, all it has to do is write an empty table to the permanent storage:
server.save({});
This server-side data is available as long as the agent is running. The impCloud management system may move an agent from one host to another, and this migration will inevitably cause the agent to go offline for a brief period. During that time, the persisted data is not available, but is retained.
Agents are shut down if the device disconnects and then does not re-connect within 31 days. Again, any persisted data is retained. If the device subsequently reconnects, its agent will be restarted and will have access to all data previously preserved using server.save().
Persisted data will be lost if the agent is removed. This may happen if the end-user re-configures their device with your BlinkUp SDK-based mobile app: if your app is coded to request a new plan ID at BlinkUp, this will cause a new agent to be instantiated for the device. This agent will have a different ID than its predecessor, and will not be able to access the old agent’s data. If you wish to avoid this for your application, code your mobile app to request a new plan ID only at the end-user’s first BlinkUp. This plan ID should be stored locally and re-used if the same end-user ever re-configures their device. This will ensure the existing agent is retained and any preserved data continues to be accessible.
If the end-user configures a second device, the unit will always get its own agent and therefore have no access to the first device-agent pair’s persisted data. Similarly, if the end-user gives the first device to a second end-user, when that subsequent user configures the device, a new plan ID will be requested (because BlinkUp is being performed on a different mobile device) and so a new agent will be instantiated.
Data written using server.save() is serialized before it is stored. However, not all Squirrel entities can be serialized. Integers, floats, blobs and Boolean values may be serialized, as may arrays containing these data types. Nested arrays can be serialized only if they too contain serializable data.
Tables, and nested tables, can be serialized only if any keys which are strings are themselves serializable: they are encoded in Ascii or UTF-8 and contain no embedded NUL ("\0") characters. Such strings are said to be ‘safe’. Value strings are serializable; those which are considered ‘unsafe’ are first converted to blobs. This conversion will be performed automatically but your application will not be notified that this has taken place.
Classes, class instances and functions are not serializable so must not be included in tables that you expect to pass into server.save(); if they are, an exception will be thrown.