Thinking in WarpScript: Variables in Macros

Macros and variables make the skeleton of all of your scripts, so this is important to understand those concepts well. A nice article already covers many useful aspects, but we will go one step further with registers and dereferencing. Those techniques can be used to speed up execution or to make your code more readable.

What are Variables?

Variables in WarpScript are like in any language: a way to associate an identifier to an object, and in our case, a reference to an object. As everything in WarpScript is a function, using variables also involves functions.

Associating a symbol to an object reference is done using the STORE function. Getting back the reference is usually done using $myvariable. In reality, this is just a convenient alias of "myvariable" LOAD. So STORE and LOAD are the main functions to interact with variables.

Internally, variables are stored using a HashMap. The complexity of HashMaps is O(1) for puts and gets, which means the number of variables does not impact the performance of STOREs and LOADs. However, the cost of each of these operations, while very low, can still have an impact on performance if you use them a lot.

Variables to Registers

To improve the performance of storing and loading variables, another mechanism is available: registers. Instead of using internally a HashMap, we use an array to store the references. Two function groups mimic the behavior of STORE and LOAD: POPRx and PUSHRx, with x a number:

1337 POPR0 PUSHR0

The drawback is that we cannot use Strings anymore to reference the objects because arrays are indexed using integers. This defeats the primary idea of variables, adding clarity. Hopefully, you don't have to handle registers yourself, the ASREGS function automatically converts variables to registers.

Since the 2.7.0 release, some improvements have been done to the ASREGS function. The list of variables is now optional and defaults to all variables which are STOREd, and it can now handle much more cases, especially the [ "a" "b" "c" ] STORE case.

<%  1337 'n' STORE $n %>
ASREGS
// Executing that will give you a macro containing
// 1337 POPR0 PUSHR0, exactly like above

Using ASREGS Correctly

So the idea is to write clean and readable code using variables and then automatically convert these variables to registers. You still have to be careful that none of the variables you use in the macro you ASREGSed "leaks" outside it, else it will break your code. Look for this code, for instance:

1337 'leaking_var' STORE
1 5
<%  
  $leaking_var * // This will fail
  'leaking_var' STORE
%>
ASREGS
FOR
$leaking_var

leaking_var will be converted to a register only inside the FOR macro, but will still be a variable outside. The call to $leaking_var inside the macro is converted to PUSHR0 and, as there is no link between variable and registers, it will return NULL, not 1337, making the code fail.

Another source of error is the nesting of ASREGSed macros:

<%
    [ 'str' 'n' ] STORE
    '' 1 $n <% $str + %> false FOR
%>
ASREGS // Code will work if you comment that line
'repeat' STORE

<%
  'b' STORE  
  $b 4 @repeat  
  ':' +  
  $b 2 @repeat +
%>
ASREGS // Code will work if you comment that line
'4:2' STORE

'foo'
@4:2

Why does that fail? Because both macros will use the 0 register (POPR0 and PUSHR0), ASREGS being done on the macros independently. Register values will get mixed resulting in an error.

To make sure ASREGS will work correctly on your macros, make sure to save the context. If you want to use it on your script, use ASREGS only on the top-level macro as it will recursively replace variables in nested macros. The best way to do that is to encapsulate your script in a <% ... %> ASREGS EVAL construct.

Dereferencing Variables

Sometimes your macro is only loading a variable and not modifying the referenced object. In that case, it can be considered as a constant, and your macro can be simplified.

Early Binding

We previously said that $myvariable is an alias of "myvariable" LOAD. When your script is executed, LOAD will look in the symbol table and get the object reference associated to "myvariable". This is called late binding.

But it is also possible to load the object reference when the macro is parsed, with the expression !$myvariable. This is done before the macro gets a chance of being executed, and for that reason, it is called early binding. In that case, the reference to the object in the symbol table will be directly added to the macro instead of a "myvariable" LOAD. In other terms, if you change the reference object of "myvariable" after the enclosing symbol %> of the macro, it won't change the value of !$myvariable that is in it.

You can find more examples and explanations in this article.

This mechanism is great when you want to define constants and optimize macros in your repository. Why? Because the .mc2 files that are in your repository are executed by your Warp10 platform at initialization to parse their macro and reference them for later use by incoming scripts. So every early binding in your macros will be computed only once at that time.

Take this macro:

/* some GeoJSON */
0.01 false
GEO.JSON
'shape' STORE
<% !$shape %>

If you put that macro in your macro repository, the shape computation, which could be very long, will be done only once. Calling the macro will be very fast because it will only return the reference to an already existing object.

But what if you want to place a constant in a macro, but its value depends on previous code? You can't do this:

<%
  NOW 'now' STORE
  <% !$now %>
%>

When the macro is parsed, "now" is not known, resulting in an error. There is a solution though.

The DEREF Function

The DEREF function takes a macro and a map associating symbols to values and replaces each of these symbol loads by their value. This mechanism is called dereferencing and can speed up your code and make it more readable. Since this substitution is done when DEREF is executed and not at macro parsing time, you can use it in nested macros. The script above can be corrected as:

<%
  <% $now %>  
  { 'now' NOW } DEREF
%>

Obviously, replacing the loading of a variable with a constant will make the code faster if a macro is executed numerous times because the symbol will only be looked once by DEREF. This is particularly useful for MACROMAPPERs which can be executed millions of times. Suppose for instance you want to create a mapper in your macro repository which translates all the latitudes and longitudes:

<%  
  SAVE '.context' STORE
  
  [ 'dlat' 'dlon' ] STORE  
  
  <%    
    'mw' STORE    
    $mw 0 GET // ts    
    $mw 4 GET 0 GET $dlat + // lat    
    $mw 5 GET 0 GET $dlon + // lon    
    $mw 6 GET 0 GET // elev    
    $mw 7 GET 0 GET // val  
  %>
  { 'dlat' $dlat 'dlon' $dlon } DEREF
  MACROMAPPER
  $.context RESTORE
%>

In that case, the dlat and dlon are replaced by constants when executing DEREF, so that will save some loading afterward.

Another nice use of the DEREF function is to replace the use of TEMPLATE when used with REXEC. If you want to create a script to be sent to another Warp 10 instance containing local parameters, you would usually do this:

<'  
  [    
    {{{token}}}    
    {{{class}}}    
    {{{labels}}}    
    {{{now}}}    
    {{{duration}}}  
  ]  FETCH
'>
{  
  'token' $token  
  'class' $class  
  'labels' $labels  
  'now' $now  
  'duration' $duration
}
TEMPLATE
'http://the.other.instance/api/v0/exec'
REXECZ

While this works well, you don't have any syntax highlighting and auto-completion in WarpStudio and VSCode because your script is a simple string. REXEC also accepts macros, so you can take advantage of that, plus DEREF to have local variable substitution, syntax highlighting, and auto-completion:

<%  
  [    
    $token    
    $class    
    $labels    
    $now    
    $duration 
  ]  FETCH
%>
{  
  'token' $token  
  'class' $class  
  'labels' $labels  
  'now' $now  
  'duration' $duration
}
DEREF
'http://the.other.instance/api/v0/exec'
REXECZ

Takeaways

In WarpScript, using variables is done through the STORE and LOAD functions. The $myvariable syntax is usually preferred over "myvariable" LOAD for readability.

Several ways are available to speed up the use of variables or make the code more readable. The first one is early binding with the !$myvariable syntax. It makes the definition of constant easy and efficient.

If your code relies heavily on variables, which is usually a good thing because it makes your code more readable, you can easily optimize it with registers. Instead of writing yourself the register logic, you can use the ASREGS functions which translate your code using variables to registers. There are several catches, however, so make sure to save the context and use ASREGS on the top-level macro.

Finally, you can directly replace a variable with a constant in a macro with the DEREF function. This can speed up your code and make a macro compatible with REXEC.

Read on: how to share your WarpScript macros?

Fabien Tencé

WarpScript™ Doctor