WarpScript 101: About the syntax

WarpScript syntax can look alien at first. We discuss the thoughtful reasons for its design and see how easy it is to get used to it.

WarpScript is a powerful programming language that is used to interact with data from Warp 10.

Like FLoWS, WarpScript leverages WarpLib, an extendable library of more than 1000 functions that can be executed at query time. But unlike FLoWS, WarpScript has a particular syntax designed specifically for time series data manipulation.

The syntax of a programming language matters a lot

For WarpScript, that means more concise, idiomatic, and elegant code. Yet, the alien nature of it can be a barrier at first. FLoWS exists for this reason. But I can guarantee you that writing code to manipulate time series data will make you face much more difficult problems. If you got used to WarpScript syntax then it will be a strong asset to solve them, not a hindrance.

The syntax of WarpScript should not be a barrier to its adoption. For example, Clojure is the second most used language on the JVM, yet its syntax is not common and actually share some similarities with the syntax of WarpScript.

WarpScript is a powerful language designed specifically for time series data manipulation. Share on X

Learning WarpScript syntax is in fact very easy and can be done in the matter of a blog post … as you'll see if you read below.

Philosophy

Let's say you have a series of operations to apply to your data. In other programming languages, you'll typically write something like this:

x = operation-N( ... operation-2(operation-1($data)))

In comparison, in WarpScript, you can just write statements, one after the other, since everything in WarpScript is a function, and that each statement corresponds to the application of an operation.

So you would write something like this:

$data operation-1 operation-2 ... operation-N

With this syntax, it is very easy to see how operations are chained onto the data.

The first striking difference between the two syntaxes is that the notation of WarpScript is reversed. In fact, with this notation, operations are written exactly in the order they are applied. So that makes sense when thinking of processing data through a pipeline.

If a function has multiple arguments, they must all precede the function statement:

$arg1 $arg2 FUNCTION

Another difference is that in the first syntax, the result of the operations must necessarily be affected to a variable to be reused. In WarpScript, you can as well store this value into a variable, but you can also leave it as is, and pick it up later (the last value you left is the first you pick up).

This brings another feature of WarpScript syntax, in that the data structures you create for collecting objects can be inferred by just reading the code. For example, if you have multiple results, you can just enclose them with brackets, [ and ], to collect them into a list.

[ $data FUNCTION-1 $data FUNCTION-2 ]

Similarly, you can use curly brackets, { and }, to collect them into a map.

{
  'result-1' $data FUNCTION-1
  'result-2' $data FUNCTION-2
}

As you can see, the pipeline of operations as well as the data structure of the final result can be inferred just by reading the code in the same order.

Syntax

Comments

// comments are preceded by double slash
# comments can also be preceded by a sharp symbol
/* comments can be multiline in C-style */

Literal data

In WarpScript, everything is a function. So that goes as well for statements representing literal data. They are basically functions with 0 arguments that evaluate for themselves.

0 -15 999 // LONG
3.14 NAN PI E 1.234E-3 // DOUBLE
true false T F // BOOLEAN
'this is a string' 'this%20is%20a%20string' // STRING, note they are URL encoded
<'
This is
a multiline
string
'>
NULL // null object
<% 'hello world' %> // a macro, i.e. an object that can be evaluated (this is the equivalent of a function for other programming languages)

Symbolic data

42 'foo' STORE // affecting the value 42 to the symbol foo
$foo // loading the variable stored in foo to obtain 42.
'foo' LOAD // same as $foo
<% TOSTRING 'th' + %> 'bar' STORE // affecting a macro to the symbol bar
$foo @bar // evaluating the macro bar: obtaining '42th'
$bar EVAL // same as @bar

Examples of binary operations

Basic binary operations include + - * > < >= <= == || &&

For example:

// Both argument precede the function call, for example:
1 1 + // gives 2

Example of basic control and loop statements

// if-then syntax is: $boolean $thenMacro IFT
true <% 'hi' %> IFT // gives 'hi'

// if-then-else syntax is: $boolean $thenMacro $elseMacro IFTE
false <% 'hi' %> <% 'bye' %> IFTE // gives 'bye'

// for-loop syntax is: $from $to $macro FOR
1 1 5 <% * %> FOR // gives 5! = 1x2x3x4x5 = 120

// for-each syntax is: $container $macro FOREACH
[ [ 1 2 3 ] <% 2 * %> FOREACH ] // gives [ 2 4 6 ]

That is all there is to know (and more) for the syntax.

It's a bit different than for most programming languages, but it's quick to get used to.