WarpScript audit tool for Warp 10 3.0

This post is dedicated to people who want to update their WarpScript code base for a smooth transition to Warp 10 3.0+.

Warp 10 3.0 comes with great new features and some breaking changes. There are therefore things to change in your WarpScript macros/runner/code, so we released a tool to spot them.

The first part of this article is for our customers, that will be able to run the audit tool on-premise. The second part is for our users, that can access an online tool for free.

Input example:

// AUTHENTICATE to raise limits

'token' AUTHENTICATE
10e6 TOLONG MAXOPS

{
  'url' 'test'
} HTTP

<%
  'test' SHMLOAD
  TOBITS 'r' STORE
%> 'test' MUTEX

$url WEBCALL

'http' CAPCHECK
<%
{
  'url' 'test'
} HTTP
%> 

WHAT IS THAT  // unknown functions

Output example:

The audit tool looks for removed functions (ERROR).

It also looks for functions that need capability, and check if CAPCHECK or CAPGET are used before (to check capability). It will raise a WARNING if there is no obvious capability management before or a simple INFO if CAPCHECK or CAPGET are present. The best practice is to check capability before calling functions that need them.

It will also spot other specific changes, for example, if you use MAP with 'ticks' options, or warn you if you use CALL without setting timeout option.

So, your goal is to only have INFO at the end!

Warp 10 with support contract

We will provide our customers warp10.ext.audit-1.0.2.jar. The goal of the tool is to spot what needs to be changed, taking into your configuration and your extensions. Here are the setup instructions.

You need to gather together:

• warp10-ext-audit-1.0.2.jar
• warp10-3.0.x.jar
• Your configuration (/opt/warp10/etc/conf.d)
• Your extensions (/opt/warp10/lib)

Example:

.
├── conf.d
│   ├── 00-secrets.conf
│   ├── 00-warp.conf
│   ├── 01-throttling.conf
│   ├── 10-directory.conf
│   ├── 10-egress.conf
│   ├── 10-fdb.conf
│   ├── 10-headers.conf
│   ├── 10-ingress.conf
│   ├── 10-leveldb.conf
│   ├── 10-runner.conf
│   ├── 20-datalog.conf
│   ├── 20-macros.conf
│   ├── 20-warpfleet.conf
│   ├── 20-warpscript.conf
│   ├── 30-in-memory.conf
│   ├── 30-ssl.conf
│   ├── 70--extensions.conf
│   ├── 80-http-plugin.conf
│   ├── 80-influxdb-plugin.conf
│   ├── 80--plugins.conf
│   ├── 80-sharding-plugin.conf
│   ├── 80-tcp-plugin.conf
│   ├── 80-udp-plugin.conf
│   └── 999-fdb.conf
├── lib
│   ├── warp10-ext-hfsproducer-2.1.1.jar
│   ├── warp10-ext-hfstore-2.1.1.jar
│   ├── warp10-ext-interpolation-0.0.13-49c0b27.jar
│   ├── warp10-plugin-bacnet.jar
│   └── warp10-plugin-mqtt-0fb32ce.jar
├── warp10-3.0.0.jar
└── warp10-ext-audit-1.0.2.jar

You can now run the audit command:

java -cp warp10-ext-audit-1.0.2.jar:warp10-3.0.0.jar:lib/* \
-Dwarp10.config="$(find ./conf.d -not -path '*/\.*' -name '*.conf' | sort | tr '\n' ' ')" \
io.warp10.ext.audit.AuditWarpScriptExtension audit3x \
/home/pierre/warpscripts/   >  $(date +"%Y-%m-%d-%Hh%M")-report.md

Your configuration may contain some obsolete lines that will cause Warp 10 to fail. For example, if you activated the debug extension (the one that provides STDOUT, STDERR…), you need to remove the line in your configuration, as these functions are now within WarpScript 3.0.0 core. The audit tool will propose a solution that will work with 3.0.0alpha2+:

The audit tool will also spot useless configurations and explain why there are useless.

Note you can pass several directories or files as final arguments.

Open the resulting markdown file with your favorite editor/plugin!

If any questions or problems, please open a ticket.

Warp 10 users without support contract

This will have a little cost… Your email! (No spam, gdpr compliant).

Contact us, and we will email you a token with audit capability. Once you have it, you can insert it in the following bash script:

#!/bin/bash

if [ "$#" -ne 1 ]; then
    echo "Usage: $0 myscript.mc2"
    exit
fi

MYSCRIPT=$(cat $1 | base64 -w0)

TOKEN="the token we send you"

SERVER="https://sandbox.senx.io/api/v0/exec"

WS=" '$TOKEN' CAPADD 
'$MYSCRIPT' B64-> 'utf8' BYTES->  // decode my warpscript
@tools/audit3x
JSONPRETTY // pretty print json
"

echo "###"
echo "### results for $1"
echo "###"
curl --data-binary "$WS" $SERVER 

You will get the same results in json. You can of course adapt this script to browse a directory recursively, or to present results differently.

Tool Limitations

The audit tool won't dive into strings… So, if you use WarpScript within a string to REXEC it to another server, the audit tool won't inspect this code. For example:

'AUTHENTICATE' EVAL
" 'token' AUTHENTICATE " REXEC

Last but not least, for Warp 10 standalone, keep in mind you must add the backend type in your configurations. So, to upgrade from 2.x to 3.x, add backend = leveldb to your /opt/warp10/etc/conf.d/99-custom.conf file. Otherwise, Warp 10 won't start!

We hope this tool will help you to migrate from Warp 10 2.x to 3.x. We will update it for beta and the final 3.0 release.

In the meantime, we would love to have your feedback about this first Alpha 3.0. To do so, join the community on the Warp 10 Lounge.

Pierre Papin

Electronics engineer, fond of computer science, embedded solution developer.