Thinking in WarpScript – Functional & Data Flow

We are introducing a new series of posts about the WarpScript language.

They will cover various aspects of it and make you feel more comfortable and be more efficient when using Warp 10.

In this pilot post, we will introduce the overall philosophy behind the language, so your mind gets accustomed to the syntax and mode of operation.

Functional by nature

WarpScript is part of a wider family of programming languages called concatenative languages. Those languages are the ultimate ambassadors of functional style since everything is a function.

As every token in a WarpScript program is a function, successive tokens denote a composition of functions.

Functions take a certain number of operands, that number is called their arity. Functions which do not take arguments are called nullary. Those which take one are called unary, those that take two binary, then ternary, etc, etc.

And functions can also produce a certain number of results.

Consuming and producing data

WarpScript manages a common data pipeline which is initially empty. This data pipeline is passed from one function to the next with functions consuming their arguments off of it and adding their results at the end.

Depending on the sequence of functions being called, the pipeline will vary in size. It could very well become empty at times or contain thousands of elements at others.

Remember that all functions which either take arguments or produce results will interact with that data pipeline.

As an example, here is the evolution of the data pipeline along the execution of the following WarpScript code:

1 2 DUP 3 ->LIST

Initially the pipeline is empty

We symbolize this with gray boxes, assuming the pipeline has an initial capacity of 5 elements:

The first function is 1 which takes no argument and produces the numerical constant 1 as a result. After this function call, the pipeline contains a single element:

The next function is 2, a unary function which produces the numerical constant 2 as a result, the data pipeline now contains two elements:

The next function is DUP which DUPlicates the last element of the pipeline, it takes one argument, the last element of the pipeline, and produces two results, twice the element it consumed. The pipeline is shown before and after the function call. Highlighted in yellow is the argument of the function.

The function following DUP is 3 which produces the numerical constant 3. You have probably guessed what the pipeline looks like after this function is executed:

The last function of our minimalistic program is ->LIST (pronounced to-list). This function builds a list from elements in the data pipeline. It will consume a variable number of arguments off of the pipeline based on the value of the first element it consumes which is the number of elements to add to the list. In our case the first step of the function will consume 3 off of the pipeline and therefore ->LIST will then consume 3 elements from the pipeline to build the list. This is reflected in the two diagrams below:

The final content of the data pipeline is a single element, namely the list that ->LIST just created:

As you can see, the mode of operation of WarpScript is really simple to understand once you visualize the state of the data pipeline. This is a mental gym which comes really quick, especially when you read the documentation of the various functions and their signatures. They indicate how many arguments they take as input and how many results they produce. You can see the documentation for DUP and ->LIST.

Data Flow by extension

Since the data pipeline is passed from function to function, it is easy to consider the mode of operation of WarpScript as data flow, i.e. with data produced by one function flowing to the next. The minor difference is that the data pipeline may contain more than just the data produced by the previous function. So the operational mode is really an augmented data flow.

This data flow analogy makes it really straightforward to visualize the manipulation of Time Series data. Indeed, as Geo Time Series are first class objects in WarpScript, they too flow from function to function, each function transforming the series they take as arguments.

Next step

If you don't already have a Warp 10 instance deployed, you can use our sandbox or Docker Image and get up and running with WarpScript in no time.

Mathias Herberts

Co-Founder & Chief Technology Officer