On June 13th 2019 we held our first Warp 10™ Office Hours session. A small crowd gathered during an hour to ask questions to the SenX team about Warp 10™. This is a recap of this hour of Q&A.
Warp 10™ was not always Open Source. The change of licence became effective on January 29 2016. The product development started in summer of 2013 and the analytics language was then called Einstein. The name Einstein was chosen because Warp (not yet called Warp 10™) was enabling the management and analytics of spatio-temporal data and when we looked for a name we thought that Einstein was the best person to make sense of the spacetime continuum so we picked that name and quite naturally the
.mc2file extension came to mind and was chosen.
When we decided to go Open Source at the end of 2015, we addressed the associated legal issues. One of them was about trademarks. We then found that the Einstein name was actually a trademark of the Hebraic University of Jerusalem and therefore could not be used freely as we thought, we then changed the name to WarpScript™ which is a trademark of SenX but we kept the
.mc2 extension as a reminder of the Einstein days. We have registered this extension with the IANA, so it is the official WarpScript™ file extension you should use.
If you look into the Warp 10™ source code you might still find some references to Einstein here or there.
A fun fact about the Einstein story is that shortly after we changed the name from Einstein to WarpScript™, Salesforce announced its IoT/AI product line which it dubbed... Einstein, obviously if you have deep pockets you can use that name 🙂
As you know there are lots of functions in WarpScript™. The latest count is between 900 and 1000, so it can be intimidating and many people don't use most of the functions as their use cases might not require them. Nevertheless there are a few gems within that function forest, here is a short selection.
RTFM function is a fun one which returns the URL of the Warp 10™ doc site 🙂
The INFO function is used to document macros. It expects a MAP with fields describing various aspects of a macro, such as its name, its configuration, its description, the set of signatures and the description of parameters, a set of examples, a list of related macros and various other fields. You can view an example of such a parameter map in the
When encountered during the execution of WarpScript™ code, INFO will simply discard its parameter map. But if INFOMODE was previously called, INFO will stop the current execution, returning its parameter map as a result.
We encourage you to document your macros this way as we are working on a WarpStudio add-on to visualize a macro's documentation by executing the said macro after calling INFOMODE.
Encoder related functions
Encoders are low level structures which can store datapoints (timestamp, lat, lon, elevation, value). They are more compact than Geo Time Series™ when used inside of WarpScript™, can store values of multiple types, but are less flexible in terms of how you can access the datapoints inside them.
You can also have a look at the
NEWENCODER function which allows you to create a GTSEncoder inside of WarpScript™ code.
Snapshot related functions
SNAPSHOT function and its siblings are a very handy set of functions which will generate WarpScript™ code to reproduce the state of the stack when SNAPSHOT was called. The resulting WarpScript™ code can be persisted or transmitted. Later you can use it to recreate a stack in a given past state.
Little known fact, the WarpScript™ language contains functions which allow use to create images using the Processing syntax. Those functions, around a hundred, range from drawing points and lines to Bézier curves, stamping images or loading SVG content. This is a very useful set of functions to create custom visualizations. The Processing related functions all start with a P and are listed in their category page.
We hope to be able to release version 2.1 during the summer. We still have a set of pull requests we need to review and some dependencies we would like to update before we can ship it.
But we are eager to push 2.1 to GA as there are many improvements, both in terms of features and in terms of performance.
Among the new features is the ability to configure Warp 10™ natively to do SSL, and a new version of datalog with sharding and easy configuration for multiple targets. There is the ability to store multiple values and binary values in Geo Time Series™. You will find a whole set of new functions and performance improvements up to 300x for some common WarpScript™ patterns. And a lot more.
** UPDATE ** The 2.1 version is now available.
We have a branch with support for the Kafka 1.x API. is compatible with Kafka 2.x as we do not use any feature specific to Kafka 2. That branch was initially created as a joint effort to also support MapR Streams and MapR DB but the branch supports Apache Kafka 1.x+.
Note that during the Office Hours we said it was not possible to use that branch to use Pulsar instead of Kafka. But it seems recent versions of the Pulsar adaptor for Kafka are now fully compatible with the Kafka API and thus do not require the use of specific constructors.
We also have a branch for JDK11 compatibility. That branch includes changes we have done to the Warp 10™ code to make it compatible with JDK11. But there are some unsolved issues with a number of dependencies that Warp 10™ relies on which make them incompatible with JDK11. This is for example the case of dependencies which make use of Unsafe to access String internal fields for performance reasons, starting with JDK9, the String class no longer has a field of type
char but instead stores its content in a
byte, so those libraries accessing the
char all fail. Another issue is the way recent JDK report their version. Many libraries parse the version name expecting it to be 1.x where x is the JDK release (7,8,9, ...) but this is no longer the format used for the JDK version.
So we will keep working on that branch whenever we foresee solutions for some of the dependencies. Be sure that whenever we become JDK11 compatible we will let you know!
Next Office Hours
We intend to hold recurrent office hours sessions, follow our Twitter account to be notified of the next session. In the mean time you can ask questions on Stack Overflow, Twitter or the Warp 10™ Google Group.
Script optimization is critical to handle billions of datapoints. WarpScript comes with 4 functions to monitor execution time. Make your script run on NOS!
It is time go back to the basis. The FETCH function allows you to extract your data from Warp 10™ in different manners.
Co-Founder & Chief Technology Officer