Learn how to access sensitive information from macros or runner scripts without exposing those secrets in your WarpScript source code
The Warp 10 Platform allows you to factor out code in macros and to periodically run WarpScript code using a mechanism called runners. In both of those cases, the WarpScript code you write might need some secrets such as tokens, internal URLs, passwords or other sensitive information.
Storing those elements in your WarpScript code, whether in macros or in runner scripts, will work. But it will also expose those secrets. For example in your git repos, and will make it more cumbersome to change those elements when you need to.
In order to solve this tedious problem, WarpScript has a feature called macro config. It enables you to manage those secrets outside your macros and runner scripts, while still allowing those pieces of WarpScript code to access them.
If it's the first time that you hear about WarpScript, I suggest you read this post first.
Overview
The macro config feature works by setting values to specially formed configuration keys in the Warp 10 configuration files. You can then retrieve values from macros or runner scripts.
A hierarchical system allows you to set a value in a single key while still enabling multiple scripts or macros with a common prefix to access it.
If the warpscript.macroconfig.secret
configuration is set, the SETMACROCONFIG function allows you to dynamically set or modify values of those specially formed keys.
Macro config in WarpScript macros
The format of the macro config keys for use in macros is key@path/to/macro
.
Here path/to/macro
can be a prefix to your actual full macro name.
When the WarpScript execution engine encounters a call to MACROCONFIG or MACROCONFIGDEFAULT, it will check in the configuration if it can find a value for one of the macro config keys matching the path to your macro.
So if your macro is called path/to/macro
, and you are requesting the key key
using MACROCONFIG
, the execution engine will check for the following configuration keys, in this order:
key@path/to/macro
key@path/to
key@path
The value associated with the first existing key will be returned. If no matching key is found, then a call to MACROCONFIG
will emit an error and a call to MACROCONFIGDEFAULT
will return the specified default value.
This mechanism works for macros in your local macro repository, in the class path and in remote WarpFleet repositories (accessed via WarpFleet resolvers). It will not work for macros stored in variables, as this would allow you to access any secret stored in the configuration since you could craft any macro path.
Macro config in runner scripts
The format of the macro configs for use in runner scripts is key@/path/to/runner
. Here path
is the name of the directory under runner.root
where your scripts reside, and to
is the first directory below the periodicity. Note that you can ignore the periodicity, i.e. the keys are common to all periodicities. If you want to separate keys per periodicity, you need to create an additional subdirectory under the periodicity directory. It will be unique per periodicity.
The runner scripts are not necessarily executed in the local Warp 10 instance where they reside, so we need another mechanism than the use of MACROCONFIG
to access the values associated with the macro config keys.
This mechanism uses a syntax similar to shell variable expansion, with a syntax of ${key}
or ${key:default}
. This syntax is searched throughout the script and replaced with values from macro config keys.
For a script top/10000/path/to/script
requesting ${key}
or ${key:default}
, the following macro config keys will be checked in the configuration, in this order:
key@/top/path/to/script
key@/top/path/to
key@/top/path
key@/top
If one of those keys has an associated value, the ${key}
or ${key:default}
will be replaced with that value. If no value is found, the ${key}
construct will be left as is and the ${key:default}
construct will be replaced with default
.
Those macro config keys can be set using SETMACROCONFIG
just like in the case for macros.
Take aways
We have shown you how to access secrets from your WarpScript code in macros or in runner scripts. So now you know how not to expose those sensitive information except in the Warp 10 configuration.
This mechanism also enables you to distribute macros which need secrets (such as tokens) in WarpFleet repositories. Anyone willing to use them will only need to set the required macro config keys in their own Warp 10 configuration.
Read more
All there is to know about Warp 10 Tokens
Warp 10 beyond OSS
Time Series Compliance and Auditability
Co-Founder & Chief Technology Officer