WarpScript audit tool for Warp 10 3.0

Audit your WarpScript codebase for a smooth transition to Warp 10 3.0!

This post is dedicated to people who want to update their WarpScript code base for a smooth transition to Warp 10 3.0+.

Warp 10 3.0 comes with great new features and some breaking changes. There are therefore things to change in your WarpScript macros/runner/code, so we released a tool to spot them.

The first part of this article is for our customers who will be able to run the audit tool on-premise. The second part is for our users who can access an online tool for free.

Introducing Warp 10 3.0!

Input example:

// AUTHENTICATE to raise limits

'token' AUTHENTICATE
10e6 TOLONG MAXOPS

{
  'url' 'test'
} HTTP

<%
  'test' SHMLOAD
  TOBITS 'r' STORE
%> 'test' MUTEX

$url WEBCALL

'http' CAPCHECK
<%
{
  'url' 'test'
} HTTP
%> 

WHAT IS THAT  // unknown functions

Output example:

The audit tool looks for removed functions (ERROR).

It also looks for functions that need capability, and checks if CAPCHECK or CAPGET are used before (to check capability). It will raise a WARNING if there is no obvious capability management before or a simple INFO if CAPCHECK or CAPGET are present. The best practice is to check capability before calling functions that require it.

It will also detect other specific changes, for example, if you use MAP with 'ticks' options, or warn you if you use CALL without setting timeout option.

So, your goal is to only have INFO at the end!

Warp 10 with support contract

We will provide our customers warp10.ext.audit-1.0.2.jar. The goal of this tool is to find out what needs to be changed in your configuration and your extensions. Here are the setup instructions.

You need to gather together:

warp10-ext-audit-1.0.2.jar
warp10-3.0.x.jar
Your configuration (/opt/warp10/etc/conf.d)
Your extensions (/opt/warp10/lib)

Example:

.
├── conf.d
│   ├── 00-secrets.conf
│   ├── 00-warp.conf
│   ├── 01-throttling.conf
│   ├── 10-directory.conf
│   ├── 10-egress.conf
│   ├── 10-fdb.conf
│   ├── 10-headers.conf
│   ├── 10-ingress.conf
│   ├── 10-leveldb.conf
│   ├── 10-runner.conf
│   ├── 20-datalog.conf
│   ├── 20-macros.conf
│   ├── 20-warpfleet.conf
│   ├── 20-warpscript.conf
│   ├── 30-in-memory.conf
│   ├── 30-ssl.conf
│   ├── 70--extensions.conf
│   ├── 80-http-plugin.conf
│   ├── 80-influxdb-plugin.conf
│   ├── 80--plugins.conf
│   ├── 80-sharding-plugin.conf
│   ├── 80-tcp-plugin.conf
│   ├── 80-udp-plugin.conf
│   └── 999-fdb.conf
├── lib
│   ├── warp10-ext-hfsproducer-2.1.1.jar
│   ├── warp10-ext-hfstore-2.1.1.jar
│   ├── warp10-ext-interpolation-0.0.13-49c0b27.jar
│   ├── warp10-plugin-bacnet.jar
│   └── warp10-plugin-mqtt-0fb32ce.jar
├── warp10-3.0.0.jar
└── warp10-ext-audit-1.0.2.jar

You can now run the audit command:

java -cp warp10-ext-audit-1.0.2.jar:warp10-3.0.0.jar:lib/* \
-Dwarp10.config="$(find ./conf.d -not -path '*/\.*' -name '*.conf' | sort | tr '\n' ' ')" \
io.warp10.ext.audit.AuditWarpScriptExtension audit3x \
/home/pierre/warpscripts/   >  $(date +"%Y-%m-%d-%Hh%M")-report.md

Your configuration may contain some obsolete lines that will cause Warp 10 to fail. For example, if you activated the debug extension (the one that provides STDOUT, STDERR…), you need to remove the line in your configuration, as these functions are now within WarpScript 3.0.0 core. The audit tool will propose a solution that will work with 3.0.0alpha2+:

The audit tool will also detect useless configurations and explain why they are useless.

Note that you can pass multiple directories or files as final arguments.

Open the resulting markdown file with your favorite editor/plugin!

If you have any questions or problems, please open a ticket.

Warp 10 users without support contract

This will have a small cost… Your email! (No spam, GDPR compliant).

Contact us, and we will send you an email with an audit token. Once you have it, you can insert it in the following bash script:

#!/bin/bash

if [ "$#" -ne 1 ]; then
    echo "Usage: $0 myscript.mc2"
    exit
fi

MYSCRIPT=$(cat $1 | base64 -w0)

TOKEN="the token we send you"

SERVER="https://sandbox.senx.io/api/v0/exec"

WS=" '$TOKEN' CAPADD 
'$MYSCRIPT' B64-> 'utf8' BYTES->  // decode my warpscript
@tools/audit3x
JSONPRETTY // pretty print json
"

echo "###"
echo "### results for $1"
echo "###"
curl --data-binary "$WS" $SERVER

You will get the same results in json. You can of course adapt this script to browse a directory recursively, or to present results differently.

Tool Limitations

The audit tool won't dive into strings… So, if you use WarpScript within a string to REXEC it to another server, the audit tool won't inspect this code. For example:

'AUTHENTICATE' EVAL
" 'token' AUTHENTICATE " REXEC

Last but not least, for Warp 10 standalone, keep in mind you must add the backend type in your configurations. So, to upgrade from 2.x to 3.x, add backend = leveldb to your /opt/warp10/etc/conf.d/99-custom.conf file. Otherwise, Warp 10 won't start!

We hope this tool will help you to migrate from Warp 10 2.x to 3.x. We will update it for the beta and the final 3.0 release.

In the meantime, we would love to have your feedback about this first Alpha 3.0. To do so, join the community on the Warp 10 Lounge.