At SenX™ we firmly believe that the community around Warp 10™ will increasingly contribute add-ons to Warp 10™ more than modifications to the core of the project, which is good, because working with the Warp Core can be tough at times.
Warp 10™ has several types of add-ons, plugins, extensions and macro packages. SenX™ has put together an effort called WarpFleet™ to federate all those add-ons and make them easily discoverable and useable. In the upcoming weeks the WarpFleet™ effort as a whole will be unveiled. We are thrilled to know what you will think of it and how you will contribute to make Warp 10™ even more advanced.
In the mean time we want to introduce a new feature which appeared in Warp 10™ release 2.0.3, the WarpFleet™ Resolver. This feature allows your WarpScript™ code to use macros hosted on remote repositories called WarpFleet™ Macro Repositories.
The resolver is simply configured by setting a value to the property
warpfleet.macros.repos, this value should be a comma separated list of URL prefixes. When a macro call is performed in WarpScript™ code, the macro will be looked in various places, namely the symbol table, the local macro repository, the class path and in last resort, under all URL prefixes configured. This resolving mechanism allows you to host WarpScript™ macros outside of your Warp 10™ platform deployment thus increasing the flexibility of your set up since you can modify the macros without needing an access to the infrastructure hosting the platform.
As a prerequisite we will assume you have WarpScript™ macros in a GitHub repository and that you want to access those from code executing in a Warp 10™ instance. If you don’t have such a repository, you can use our demo repository senx/warpfleet-demo with URL
Configuring the WarpFleet™ Resolver
You can have access to the raw files from a GitHub repository via the Raw button from a file page but also via the direct URL:
This is the URL we will use for configuring the WarpFleet™ Resolver. You can either add
warpfleet.macros.repos list. If you chose the first URL prefix you are constraining a given branch or release. If you chose the second, the user will be able to specify the branch or release she or he wants to access.
Once you have added this configuration, you need to restart your Warp 10™ instance.
Using the WarpFleet™ Resolver
The use of the resolver is totally transparent. When the Warp 10™ Analytics Engine encounters a macro call that was not resolved locally it will attempt to retrieve the macro source code from the configured repositories.
Assuming the macro call encountered was
@foo/bar, and the configured prefix was
https://raw.githubusercontent.com/USER/REPO/BRANCH_OR_RELEASE, the resolver will attempt to retrieve the URL
https://raw.githubusercontent.com/USER/REPO/BRANCH_OR_RELEASE/foo/bar.mc2. Note that the
.mc2 suffix is added by the resolver, this is the official (and only supported) suffix for WarpScript™ files.
The resolver complete documentation describes the various caching mechanisms in place, for both positive and negative caching. Their description is out of scope of this hands on post.
Using import rules
Macros hosted on remote repositories may at time have names that become quite long, especially when the said repositories host many macros organized in directories and subdirectories. To make your WarpScript™ code easier to read, the WarpFleet™ Resolver comes with a companion function called IMPORT. This function allows to import macro prefixes under a different or simpler name.
At call time, the various configured import rules will be scanned and the longest matching one will be used.
Assuming you configured
https://raw.githubusercontent.com/USER/REPO as a WarpFleet™ Macro Repository, you could call
@BRANCH_OR_RELEASE/foo/bar to access a remote macro. Using import rules you could also import
BRANCH_OR_RELEASE/foo as simply
foo and call
@foo/bar like in the previous example. This is simply done with:
'BRANCH_OR_RELEASE/foo' 'foo' IMPORT
Before the call to the foo/bar macro. If at some point in the future you want to switch branch or release, you simply need to change the parameters of the call to
Using the senx/warpfleet-demo WarpFleet™ Macro Repository defined above, you can use the following WarpScript™ code to showcase the use of the WarpFleet™ Resolver.
// Explicitly use the master branch @master/wfdemo/HELLO // Use the 0.0.0 release @0.0.0/wfdemo/HELLO // Import the master branch as default 'master/wfdemo' 'senx' IMPORT @senx/HELLO
Now you know how to expose your WarpScript™ macros using GitHub and the WarpFleet™ Resolver. It is time to make your code available to the world. Let us know if you would like us to promote your work.
In this blog post, we review the essential frameworks available in WarpScript. They simplify greatly usual time series processing.
SHM is a Warp 10™ extension that allow to keep data in RAM. It speeds up data scientist job !
Co-Founder & Chief Technology Officer