Thinking in WarpScript – Protecting Secrets

The Warp 10 Platform allows you to factor out code in macros and to periodically run WarpScript code using a mechanism called runners. In both of those cases, the WarpScript code you write might need some secrets such as tokens, internal URLs, passwords or other sensitive information.

Storing those elements in your WarpScript code, whether in macros or in runner scripts, will work. But it will also expose those secrets. For example in your git repos, and will make it more cumbersome to change those elements when you need to.

In order to solve this tedious problem, WarpScript has a feature called macro config. It enables you to manage those secrets outside your macros and runner scripts, while still allowing those pieces of WarpScript code to access them.

If it's the first time that you hear about WarpScript, I suggest you read this post first.

Overview

The macro config feature works by setting values to specially formed configuration keys in the Warp 10 configuration files. You can then retrieve values from macros or runner scripts.

A hierarchical system allows you to set a value in a single key while still enabling multiple scripts or macros with a common prefix to access it.

If the warpscript.macroconfig.secret configuration is set, the SETMACROCONFIG function allows you to dynamically set or modify values of those specially formed keys.

Macro config in WarpScript macros

The format of the macro config keys for use in macros is key@path/to/macro.

Here path/to/macro can be a prefix to your actual full macro name.

When the WarpScript execution engine encounters a call to MACROCONFIG or MACROCONFIGDEFAULT, it will check in the configuration if it can find a value for one of the macro config keys matching the path to your macro.

So if your macro is called path/to/macro, and you are requesting the key key using MACROCONFIG, the execution engine will check for the following configuration keys, in this order:

key@path/to/macro
key@path/to
key@path

The value associated with the first existing key will be returned. If no matching key is found, then a call to MACROCONFIG will emit an error and a call to MACROCONFIGDEFAULT will return the specified default value.

This mechanism works for macros in your local macro repository, in the class path and in remote WarpFleet repositories (accessed via WarpFleet resolvers). It will not work for macros stored in variables, as this would allow you to access any secret stored in the configuration since you could craft any macro path.

Macro config in runner scripts

The format of the macro configs for use in runner scripts is key@/path/to/runner. Here path is the name of the directory under runner.root where your scripts reside, and to is the first directory below the periodicity. Note that you can ignore the periodicity, i.e. the keys are common to all periodicities. If you want to separate keys per periodicity, you need to create an additional subdirectory under the periodicity directory. It will be unique per periodicity.

The runner scripts are not necessarily executed in the local Warp 10 instance where they reside, so we need another mechanism than the use of MACROCONFIG to access the values associated with the macro config keys.

This mechanism uses a syntax similar to shell variable expansion, with a syntax of ${key} or ${key:default}. This syntax is searched throughout the script and replaced with values from macro config keys.

For a script top/10000/path/to/script requesting ${key} or ${key:default}, the following macro config keys will be checked in the configuration, in this order:

key@/top/path/to/script
key@/top/path/to
key@/top/path
key@/top

If one of those keys has an associated value, the ${key} or ${key:default} will be replaced with that value. If no value is found, the ${key} construct will be left as is and the ${key:default} construct will be replaced with default.

Those macro config keys can be set using SETMACROCONFIG just like in the case for macros.

Take aways

We have shown you how to access secrets from your WarpScript code in macros or in runner scripts. So now you know how not to expose those sensitive information except in the Warp 10 configuration.

This mechanism also enables you to distribute macros which need secrets (such as tokens) in WarpFleet repositories. Anyone willing to use them will only need to set the required macro config keys in their own Warp 10 configuration.

Mathias Herberts

Co-Founder & Chief Technology Officer