Skip to main content

impWorks Builder

Current Version: 2.4.0

Electric Imp’s impWorks™ Builder combines a code preprocessor with an expression language and advanced file import.

Builder Usage

Note Builder requires Node.js 4.0 and above.

It can be installed and used as an npm library:

npm i --save Builder

then

const builder = require('Builder');

// Provide GitHub credentials (optional)
builder.machine.readers.github.username = "<username>";
builder.machine.readers.github.token = "<personal_access_token>";

const output = builder.machine.execute(`@include "${inputFile}"`);

Alternatively, it can be used as a CLI, for which Builder provides the pleasebuild command when it is installed globally. For example:

npm i -g Builder
pleasebuild [-D<variable> <value>...] [-l] [--github-user <username> --github-token <token>]
    [-l] <input_file>

where:

  • -l — generate line control statements.
  • -D <variable> <value> — define a variable.
  • --github-user — GitHub username.
  • --github-token — GitHub personal access token or password (not recommended).

Including JavaScript Libraries

Builder can accept JavaScript libraries to add functionality to its global namespace. The library should export an object, the properties of which will be merged into the global namespace. For example, to include a function to convert strings to uppercase, define your library file like so:

module.exports = {
  upper: (s) => s.toUpperCase()
};

Include directives, such as the following example, in your input file:

@{upper("warning:")}
@{upper(include("warning.txt"))}

Run builder with the option --lib path/to/your/lib/file.

Binding the Context Object Correctly

Functions called by Builder will be called with their this argument set to a Builder context object. Within the context object, Builder variables like FILE, functions like max(), and other included library functions will be made available at the top level. Variables defined in your input code with @macro or @set will be available under the key globals.

Ignoring the binding of this may cause unexpected behavior, for example when calling methods on objects. Take the following example library:

class MyClass {
  constructor(str) {
    this._str = str;
  }

  getStr() {
    return this._str;
  }
}

myObject = MyClass("my text");

module.exports = {
  myObject
};

Attempting to use this library with the directive @{myObject.getStr()} will not deliver the expected behavior because this in getStr() will be set to a Builder context object and not to myObject. So when calling class methods, always ensure they have been bound to the correct value of this:

class MyClass {
  constructor(str) {
    this._str = str;
  }

  getStr() {
    return this._str;
  }
}

myObject = MyClass("my text");

module.exports = {
  getStr: myObject.getStr.bind(myObject)
};

Cache for Remote Includes

To reduce compilation time, Builder can optionally cache files included from a remote resource (GitHub or remote HTTP/HTTPs servers).

If this file cache is enabled, remote files are cached locally in the .builder-cache folder. Cached resources expire and are automatically invalidated 24 hours after their addition to the cache.

To turn the cache on, pass the --cache or -c option to Builder. If this option is not specified, Builder will not use the file cache even if the cached data exist and is valid — it will query remote resources on every execution.

To reset the cache use both the --cache and the --clear-cache options.

If a resource should never be cached, it needs to be added to the exclude-list.builder file. You can use wildcard characters to mask file names.

Wildcard pattern matching

Pattern matching syntax is a similar to that of .gitignore. A string is a wildcard pattern if it contains '?' or '*' characters. Empty strings or strings that starts with '#' are ignored.

A '?' symbol matches any single character. For example, bo?t.js matches boot.js and boat.js, but doesn't match bot.js.

A '*' matches any string, that is limited by slashes, including the empty string. For example, /foo/*ar matches /foo/bar, /foo/ar and /foo/foo-bar, but doesn't match /foo/get/bar or /foo/bar/get.

Two consecutive asterisks ** in patterns matched against full pathname may have special meaning:

  • A leading ** followed by a slash means match in all directories. For example, **/foo matches file or directory foo anywhere, the same as pattern foo. **/foo/bar matches file or directory bar anywhere that is directly under directory foo.
  • A trailing /** matches everything inside. For example, abc/** matches all files inside directory abc.
  • A slash followed by two consecutive asterisks then a slash matches zero or more directories. For example, a/**/b matches a/b, a/x/b, a/x/y/b and so on.
  • Other consecutive asterisks are considered invalid.

Example of 'exclude-list.builder'

# Avoid caching a specific file
github:electricimp/MessageManager/MessageManager.lib.nut

# Exclude all electricimp repos
github:electicimp/**

# Exclude all tagged files or files from the specific branches from the cache
github:*/**/*@*

Command Line Options

Option Name Short Version Description
--cache -c Turns on file cache for all files included from remote resources
--cache-exclude-list <path_to_file> Excludes the named file(s) from the cache
--clear-cache Clears the cache before Builder starts

Testing

SPEC_LOGLEVEL=<debug|info|warning|error> \
SPEC_GITHUB_USERNAME=<GitHub username> \
SPEC_GITHUB_TOKEN=<GitHub password/access token> \
npm test

Builder Directives

Directives start with the @ symbol.

@set

@set <variable:identifier> <value:expression>

@set <variable:identifier> = <value:expression>

This directive assigns the value of an expression to a variable. Variables are defined in a global context.

Example

Sets SOMEVAR to 1:

@set SOMEVAR min(1, 2, 3)

@macro

@macro <name>(<arguments>)
  <body>
@endmacro

@endmacro can be replaced with @end.

This directive defines a code block that can take its own parameters. Macros are declared in a global scope. Macro parameters are only available within the macro scope and override global variables with the same name (but do not affect them). Macros can be used via the @include directive:

@include macro(a, b, c)

Or they can be used inline:

@{macro(a, b, c)}

When macros are used inline:

  • No line-control statements are generated for the output inside the macro scope.
  • Trailing newlines are trimmed from the macro output.

Examples

@macro some_macro(a, b, c)
  Hello, @{a}!
  Roses are @{b},
  And violets are @{defined(c) ? c : "of undefined color"}.
@end

Then some_macro can be used as:

@include some_macro("username", "red")

This will produce:

Hello, username!
Roses are red,
And violets are of undefined color.

Here is the same macro used inline:

[[[ @{some_macro("username", "red", "blue")} ]]]

This will output:

[[[ Hello, username!
Roses are red,
And violets are blue. ]]]

@include

Use this directive to import local files, external sources, or macros into your code.

@include <source:expression>
  • Macro usage:

    @include some_macro("username", 123)
    
  • Import local files:

    @include "somefile.ext"
    
  • Import remote files:

    @include "http://example.com/file.ext"
    
    @include "https://example.com/file.ext"
    
  • Import source From GitHub:

    @include "github:<user>/<repo>/<path>[@<ref>]"
    
    • user is the user/organization name.
    • repo is the repository name.
    • ref is the git reference (branch name or tag, defaults to master). 

Single Line Comments — Any text after include, and between // and the end of the line, will be ignored by Builder and will not appear in the result output:

@include "https://example.com/file.ext" // Need update to file2.ext

GitHub Examples

Head of the default branch:

@include "github:electricimp/Promise/Promise.class.nut"

Head of the ‘develop’ branch:

@include "github:electricimp/Promise/Promise.class.nut@develop"

Tag ‘v2.0.0’:

@include "github:electricimp/Promise/Promise.class.nut@v2.0.0"

GitHub Authentication

When using GitHub @includes, authentication is optional. However, you should bear in mind that if you use authentication, the GitHub API provides much higher rate limits, and authentication is required to access private repositories.

Apart from a GitHub username, you need to provide either a personal access token or password (which is less secure and not recommended). More information on how to provide those parameters is included in the usage section.

@include once

@include once <source:expression>

This acts the same as @include but has no effect if source has already been included. Macros are always included.

@{...} (inline expressions/macros)

@{<expression>}

@{macro(a, b, c)}

This directive inserts the value of the enclosed expression or executes a macro.

Example

@set name "Someone"
Hello, @{name}, the result is: @{123 * 456}.

This results in the following output:

Hello, Someone, the result is: 56088.

@while

This invokes a while loop. You can access the loop variable in @while loops.

@while <test:expression>
  // 0-based iteration counter: @{loop.index}
  // 1-based iteration counter: @{loop.iteration}
@endwhile

@endwhile can be replaced with @end.

Example

@repeat

This invokes a loop that repeats over a certain number of iterations. You can access the loop variable in @repeat loops.

@repeat <times:expression>
  // 0-based iteration counter: @{loop.index}
  // 1-based iteration counter: @{loop.iteration}
@endrepeat

@endrepeat can be replaced with @end.

Example

@repeat 3
  loop.iteration: @{loop.iteration}
@end

This outputs:

  loop.iteration: 1
  loop.iteration: 2
  loop.iteration: 3

@if – @elseif – @else

This directive invokes conditional branching.

@if 
  // Consequent code
@elseif <test:expression>
  // else if #1 code
@elseif <test:expression>
  // else if #2 code
@else
  // Alternative code
@endif

@endif can be replaced with @end.

Example

@if FILE == 'abc.ext'
  // include something
@elseif FILE == 'def.ext'
  // include something else
@else
  // something completely different
@endif

@error

@error <message:expression>

Emits an error.

Example

@if PLATFORM == "platform1"
  // platform 1 code
@elseif PLATFORM == "platform2"
  // platform 2 code
@elseif PLATFORM == "platform3"
  // platform 3 code
@else
  @error "Platform is " + PLATFORM + " is unsupported"
@endif

@warning

@warning <message:expression>

Emits a warning.

Example

@if PLATFORM == "platform1"
  // platform 1 code
@elseif PLATFORM == "platform2"
  // platform 2 code
@elseif PLATFORM == "platform3"
  // platform 3 code
@else
  @warning "Building for default platform"
  // default platform code
@endif

Builder Filters

The filter operator, |, allows you to pass a value through any of the supported functions.

@{<expression> | <filter>}

This is equivalent to:

@{<filter>(<expression>)}

Example

// Include external HTML to a string
a = "@{include('index.html')|escape}"

// Include external binary file to a base64-encoded string b = "@{include('file.bin')|base64}"

Builder Expressions

Directives that take parameters allow the usage of expression syntax. For example:

  • @include <source:expression>
  • @set <variable:identifier> <value:expression>
  • @if <condition:expression>
  • @elseif <condition:expression>
  • @{<expression>} (inline expressions)

Types

The following types are supported in expressions:

  • numbers (eg. 1, 1E6, 1e-6, 1.567)
  • strings (eg. "abc", 'abc')
  • null
  • true
  • false

Operators

Binary

|| && == != < > <= >= + - * / %

Unary

+ - !

Member Expressions

  • somevar.member
  • somevar["member"]
  • ([1, 2, 3])[1]

Conditional Expressions

test ? consequent : alternate

Variables

  • Variables can be defined by -D MyVarName MyVarValue command line parameter, read from the runtime environment, or defined by @set statements.
  • Undefined variables are evaluated as null.
  • Variable names can contain $, _, latin letters and digits. They must not start with a digit.

Variable Definition Order

  1. When resolving a variable’s value, Builder first looks for its definition in the command line -D parameters (-D <variable name> <variable value>) passed to the pleasebuild command.
  2. If no such variable definition is found, Squirrel code is scanned for @set statements preceding the variable usage.
  3. If no variable definitions are found in the previous steps, Builder looks for it in the host environment variables.

__LINE__

Line number (relative to the file in which this variable appears).

Example

Hi from line @{__LINE__}!

__FILE__

Name of the file in which this variable appears.

Example

Hi from file @{__FILE__}!

__PATH__

Absolute path (not including file name) to the file where this variable appears. Can contain a URL for remote includes.

Example

Hi from file @{__PATH__}!

loop

Defined inside @while and @repeat loops. Contains information about the current loop:

  • loop.index — 0-indexed iteration counter
  • loop.iteration — 1-indexed iteration counter

Example

@set myvar = 12

@while myvar > 9 @set myvar = myvar - 1 var: @{myvar} loop.index: @{loop.index} @end

This outputs:

myvar: 11
loop.index: 0
myvar: 10
loop.index: 1
myvar: 9
loop.index: 2

Environment Variables

There is no special predicate required to make use of environment variables. Builder tries to resolve the macro from the context provided via the command line defines or from process environment variables. For example:

server.log("Host home path is @{HOME}");

will print the home directory path of the current user of the system where Builder was executed.

Builder Functions

  • defined(<variable_name>) — returns true if a variable is defined, false otherwise.
  • include(<source>) — includes external source.
  • escape(<value>) — escapes special characters in string (\b, \f, \n, \r, \t, \, ', ").
  • base64(<value>) — encodes value as base64.
  • min(<numbers>)
  • max(<numbers>)
  • abs(<number>)
  • String functions: the following string functions, based on the JavaScript methods of the same names, are available under the namespace S. The first argument to each function is always the string to be operated on. For documentation on the remaining arguments, please see the documentation for JavaScript string methods here:
    • S.concat()
    • S.endsWith()
    • S.includes()
    • S.repeat()
    • S.split()
    • S.startsWith()
    • S.substr()
    • S.substring()
    • S.toLowerCase()
    • S.toUpperCase()
    • S.trim()
    • S.trimLeft()
    • S.trimRight()

Comments

Lines starting with @ followed by space or a line break are treated as comments and not added to the output.

Example

@ something about platform #1
@set PLATFORM "platform1"

License

impWorks Builder is licensed under the MIT License.