Tutorial: An Introduction To impWorks Builder

Explore Our Command Line Squirrel Code Preprocessor

The Electric Imp application architecture centers on two primary units of code: one which runs on the device and another which runs on the device’s cloud-based partner, called its agent. Electric Imp’s online IDE, impCentral™, gives you a means to enter and edit both of these closely connected programs alongside each other.

impCentral is a great tool for straightforward projects, but you may soon feel the need to organize your device and agent code in more sophisticated ways. For example, you might want to maintain separate files for code that’s unique to the application, for code that is the same on both the agent and the device, for code that is shared by a number of projects, and for non-Squirrel code that needs to be incorporated into the application, such as the HTML and JavaScript that define a web page served by the agent.

impCentral can’t help you manage all those files. So how can you combine their contents into the agent code and device code blocks that impCentral expects? You can use Builder, Electric Imp’s powerful source code preprocessor. It can generate a single source code file from a number of component files, and even perform logic during the generation process that affects the code it outputs. This tutorial will introduce you to Builder’s core functionality and show you how this can be achieved.

Builder really comes into its own when it’s paired with impt, Electric Imp’s tool for interacting with impCentral at the command line. For example, having generated your application code using Builder, you can use impt to upload the code to the impCloud and then to devices and agents. If you haven’t yet explored impt, we suggest you do so using our introductory tutorial. Since this Builder tutorial doesn’t require the use of impt, you can just continue here if you prefer. Either way, you will need an Electric Imp account to proceed. If you haven’t yet performed this basic task, please do so now.

Builder is run from the command line, so this tutorial assumes that you are familiar with accessing the command line on your development machine, and with running commands, creating directories and so forth.

1. Install Builder

Builder is written in JavaScript and runs under Node, which you’ll need to have installed on your system. Please check out the Node installation process for your computer at Nodejs.org. You will need to install Node 8 or above. When you’ve installed Node, run the following at the command line:

npm install -g Builder

Builder is invoked with the command pleasebuild. We’ll look at some of its options later; for now, it’s sufficient to know that you run Builder as follows:

pleasebuild {input_file} > {output_file}

2. Set Up The Files We Need

If you have set up an impt project directory, navigate to it now; if not, just create a new directory and jump into it instead. Create a plain text file called raw.device.nut. In each subsequent step, we’ll trigger Builder with the specific line:

pleasebuild raw.device.nut > device.nut

impt uses device.nut as its default device source file. Calling impt build run will transfer the code output by Builder into device.nut to your project’s devices, but this isn’t necessary to complete this tutorial.

For ongoing use, we recommend setting your impt projects to use a filename like compiled.device.nut and direct Builder’s output to that file. This allows you to add compiled.* to your .gitignore file to ensure that your compiled code isn’t committed to your source code repo. We’ll see shortly why you might not want it to be.

Note We’ll be working with device code in this tutorial, but Builder will handle agent code just as well — it’s simply a matter of calling pleasebuild again with different input and output files.

3. Include Local Source Files

Builder makes it very easy to assemble your application code from multiple sources. For example, if you’re creating an application whose agent serves a web-based UI, you might want to exclude the HTML source from your Squirrel code, both for clarity and so that you can easily open it in a browser for testing. You can then use Builder to insert the HTML file at build time.

How do you include other files in your core application source file in a way that Builder will recognize? You use the @include directive followed by the path to the file. You place the @include statement in your primary source file at the place where you want Builder to insert the referenced file’s contents. For example:

const HTML_STRING = @"
@include "app_ui.html"
";

This will insert the contents of the file app_ui.html into your main source file so that it will be interpreted by Squirrel as a verbatim string constant.

An easy mistake to make in this case is to forget to encode the strings in the HTML — and any JavaScript it includes — in an escaped form so that they are correctly interpolated into the Squirrel string. With Builder, you don’t need to worry: you can use its escape filter to process the incoming HTML and JavaScript so they are correctly escaped for you:

const HTML_STRING = @"
@{include("app_ui.html") | escape}
";

This uses the include() function and pipes the result (the imported text) through the escape filter. Finally, the escaped text is inserted in place of the @{...} statement.

4. Include Remote Source Files

Your HTML code file belongs to the application project, but what if you’re using code that is outside of it: a third-party library or your own code that’s common to multiple applications? For instance, you might have some generic code that stores log data to your server or to a cloud service. You use this code in all of your IoT products. If you modify the logging code, you simply use Builder to reassemble the agent code for each of your Products (and impt to upload it).

This time you provide Builder with a reference to the file in its remote location. The library could be in a GitHub repo, on a Bitbucket Server, in an Azure Repo, or a file on a website.

Let’s say the file, logging.nut, is in the ElectricImpSampleCode GitHub account, in a repo called BuilderDemo. Then the @include statement becomes:

@include "github:ElectricImpSampleCode/BuilderDemo/logging.nut"

Copy that line and paste it into your raw.device.nut file. At the command line, build the uploadable file:

pleasebuild raw.device.nut > device.nut

Now open device.nut — you’ll see the code has been downloaded and inserted into the file you would upload to impCentral.

You can see how the @include statement is structured: you provide a service identifier (github) followed by a colon and the account name, the repo name and the file name. This pulls the file from the head of the default branch, but you can specify a different branch by adding it to the end, prefixed with @:

@include "github:ElectricImpSampleCode/BuilderDemo/logging.nut@develop"

The same prefix can be used to specify a tag instead:

@include "github:ElectricImpSampleCode/BuilderDemo/logging.nut@v1.0.0"

You can try all of these statements yourself by adding each of the above lines to your raw.device.nut file and then running the pleasebuild command.

Other version control services accessed through Builder use the same syntax.

If you’re accessing a public repo, that’s all you need to do. For private repos, you will have to authenticate the request to download the code. This is done by using the --github-user and --github-token options with pleasebuild and passing in a GitHub username and personal access token, respectively. In fact, you may prefer to do this with public repos too: it lets you sidestep GitHub’s policy of rate-limiting anonymous accesses. There are similar options for passing in credentials for the other supported version control services.

5. Conditionally Include Code

Not all of the code you want to run is required in a given build. For example, during development, you might want to include extra code to help you debug your changes, but not include that code in the version that goes into production.

Builder provides variables and conditional branching to help you. Let’s see how.

Open raw.device.nut and change its contents to:

@if BUILD_TYPE == "debug"
    @include "github:ElectricImpSampleCode/BuilderSamples/logging.nut"
@endif

It’s a classic conditional statement; Builder also provides @else and @elseif if you need them:

@if BUILD_TYPE == "debug"
    @include "github:ElectricImpSampleCode/BuilderDemo/logging.nut"
@else
    @include "github:ElectricImpSampleCode/BuilderDemo/production.nut"
@endif

Here, BUILD_TYPE is a variable, capitalized for clarity, but that’s not mandatory. How do we set the value of BUILD_TYPE? There are two ways: you can use Builder’s @set directive at the top of your main source file, ie. @set BUILD_TYPE "debug", but here we’ll use it at the command line: the option -D to specify the value of BUILD_TYPE instead:

pleasebuild -DBUILD_TYPE debug raw.device.nut > device.nut

Note that there is no space between -D and a variable’s name.

If you inspect device.nut now, you’ll see that extra Squirrel code has been added. Re-run the command with BUILD_TYPE set to a different value, say release, or without the -D option, and the extra code is removed.

Another use-case for this functionality is to generate code that targets specific imps:

@if IMP == "6"
    i2c <- hardware.i2cLM;
@elseif IMP == "4"
    i2c <- hardware.i2cQP;
@else
    i2c <- hardware.i2c89;
@endif
i2c.configure(CLOCK_SPEED_400_KHZ);

We can now compile versions of the code for imp006, imp004m and imp001, reflecting the versions of imp used in each generation of our product. This example is trivial, but a real-world application might contain many of these statements to deliver functionality that later imps support but earlier ones do not.

We’ve seen how to set the value of the variable IMP when we make a build, but you might integrate into a shell script — one of the key benefits of command line tools — to automate the process:

imps=(6 4 1)
for imp in ${imps}; do
    pleasebuild -DIMP $imp source.nut > device-imp00$imp.nut
done

Other examples include managing multiple code files for multiple territories and/or languages,

6. Incorporate Confidential Data

If you include a cloud service integration in your code, you will typically need to provide some form of login credential, such as an API key or a username and password. Usually you don’t want those credentials appearing in your raw source files, especially if they are ever exposed to unauthorized viewers — your project is open source and your version control repo is public, for example.

Builder provides some handy ways to keep this confidential information out of the repo. One way is to set up all the confidential strings in your code as Builder variables:

api <- SuperCloudService("@{SCS_API_KEY}");

and then pass in the value of SCS_API_KEY at build time using the -D option, as we did earlier. When Builder encounters the variable written in the form above, ie. with the name inside curly brackets, it substitutes the expression for the variable’s value (or null if the variable hasn’t been defined).

Adding even one long variable value to pleasebuild can be cumbersome, so Builder allows you to store variables in JSON format in a file called directives.json and then load it in at build time.

Open your text editor, create a new file and enter the following JSON table:

{ "DUMMY_KEY_1": "rtfmrtfmrtfmrtfmrtfmrtfmrtfmrtfmrtfmrtfmrtfmrtfm",
  "DUMMY_KEY_2": "iouiouiouiouiouiouiouiouiouiouiouiouiouiouiouiou" }

Save the file on your hard drive as directives.json and in a location outside of your project directory; it should be well away from any version control repo. Now edit raw.device.nut and add the following lines:

function show(key) {
    return "Key: " + key;
}

server.log(show("@{DUMMY_KEY_1}"));
server.log(show("@{DUMMY_KEY_2}"));

Save the file and build it, but this time include the --use-directives option with the path to your directives.json file as its argument:

pleasebuild --use-directives ~/path/to/directives.json raw.device.nut > device.nut

If you now look at device.nut you’ll see the two strings inserted into the server.log() statements.

7. Use JavaScript To Update The Build Number

Builder can load and run JavaScript code — that’s one of the advantages of being written in that language. You can leverage this feature to interact with source files rather than simply include them. To show you how this works, we’re going to use a file to track application build numbers and preserve the latest build number for next time.

First, create a plain text file in your project directory called version. Add the following string to it and then save it:

1.0.0.0

This represents a standard semantic version number with an extra field at the end for the build number.

Now create a file called library.js and add the following code to it:

function build() {
    let fs = require('fs');
    let version = fs.readFileSync('version', 'utf-8');
    let parts = version.split('.');
    let build = parseInt(parts[3], 10);
    build += 1;
    Let newVersion = (parts[0] + "." +parts[1] + "." + parts[2] + "." + build.toString());
    fs.writeFileSync('version', newVersion);
    return build;
}

function year() {
    let d = new Date();
    return d.getFullYear();
}

module.exports = {
    update_build: build,
    get_year: year
}

What does the code do? It defines a couple of functions, build() and year(), and then exports those functions in a form Builder can call: update_build and get_year. The latter just returns the current year, but the first function is a little more complex. It reads in the file we created earlier, splits the file’s contents into parts, then increments the last part — the build number — and writes all the data back for next time. It then returns the current build number to the caller.

How do we use this? Open your raw.device.nut file and add the following line:

server.log("Build number @{update_build()}");

When Builder encounters this line, the curly brackets tell it to evaluate the expression between them. We did this in Step 5 to get the value of a Builder variable. Here, though, the expression is a function call; Builder calls the function and inserts the returned value in place of the @{...} statement.

Save raw.device.nut and build the code. This time, add the --lib option followed by the path to the JavaScript code file:

pleasebuild --lib library.js raw.device.nut > device.nut

If you look at the output, you’ll see the line you just added has become

server.log("Build number 1");

Run the pleasebuild command again a few times and you’ll see the numeric value in that line increase. Make sure you also take a look at the version file — it too will have changed with each build.

This is a great way to inject useful data into the application source code at build time. You might use this method to set a variable which the device Squirrel displays upon a host device’s OLED display to inform the end-user which code version their unit is running — handy in a support call. All you have to do is update version when you prepare a new version of your code; Builder itself will keep track of and update the build number for you.

We’ll leave it as an exercise for you to see how you might make use of the other function, get_year, or to think about what other functions you might add. How about a function to convert Ascii art graphics into byte values that the device code can use to set the correct pixels on that OLED display to render the images?

That’s All, Folks!

Well done, you’ve completed this introductory guide to Builder, Electric Imp’s source code file preprocessor. You’ve tried out its key functionality to:

  • Insert local files.
  • Insert files from a remote GitHub repo.
  • Use variables.
  • Conditionally insert code files based on the build level.
  • Pull in out-of-repo confidential data.
  • Extend Builder with JavaScript code to include updatable information.

There’s a lot more to try though. Builder has powerful expression evaluation and includes a macro capability. In addition to conditionals, it provides a set of program control and loop facilities to help you generate more data at build time.

Next Steps

  • If you haven’t done so already, work through our quick introduction to impt to learn how to upload compiled code to impCentral, manage your devices and monitor logs — all from the command line.
  • Check out the full Builder documentation to learn more about how to Builder’s advanced features, like macros and program control.
  • To find out more about the Electric Imp architecture and how it impacts application development, take a look at the Platform Overview.
  • Read The Squirrel Programming Guide to learn more about the easy-to-use language all imp applications are written in.
  • Work through the Introduction to Squirrel Apps code samples to learn how to use your device’s agent to process readings sent by the device and to communicate with Internet-connected services.

© 2024 KORE Wireless. All rights reserved. Terms of Service, Privacy Policy

Toggle