July 2019: Warp 10 release 2.1.0

Our first feature rich release following 2.0 is out. Discover what we have packed into release 2.1 and how some of these features now probably make Warp 10™ also the fastest timeseries database!

Warp 10 release 2.1.0

Almost eight months after our major 2.0 release, the team at SenX is very happy to announce the release of version 2.1 of Warp 10, confirming it as the most advanced time series platform.

This release could almost have been a major one given how many changes it includes, but we decided to remain on the 2.x path, after all 2.1 is one twentieth of 42!

https://youtu.be/QAXRcLwHVP4

Discover a list of features and enhancements added to 2.1. We highly encourage you to migrate to 2.1 to benefit from all of them, performance improvements make the move way worth it.

Discover the latest version of Warp 10.

New features

Native SSL

All the external HTTP endpoints (ingress, egress, plasma) can now be configured with SSL support enabled. This requires that you have a certificate and associated private key available in a Java Key Store. You can use separate JKS and certificates for each endpoint.

Binary values

Geo Time Series can contain LONG, DOUBLE, BOOLEAN or STRING values. With release 2.1, STRING values can now either be standard UTF-8 encoded strings or ISO-8859-1 representations of raw byte arrays, we call this subtype BINARY. Encoding binary data in base64 is no longer required, thus leading to a 25% reduction of the storage space required for blobs. The FETCH and ->GTS functions have been modified to correctly return data that was stored as binary.

Multivariate values

The support of binary values enabled the automatic encoding of multivariate values. This new type of values enables Warp 10 to reach tremendous levels in terms of performance. This is the addition which made our record breaking Raspberry Pi 4 benchmark possible.

Multi Conf

We have modified the configuration mechanism for Warp 10 to support multiple configuration files. This makes it easier to store the configuration for a given extension in its own file, for example and allows for an overall better organization of your config.

Datalog++

We have enhanced the datalog mechanism to support forwarding to multiple targets from within a single Warp 10 instance, and to support sharding the dataset across targets. Coupled with a dedicated extension, you can configure a set of standalone Warp 10 instances to act as a mini-cluster, ensuring high availability (datalog replication) and scalability (sharding).

Warp Studio

We retired the aging quantum web interface to Warp 10 has been, and we replaced with a richer, less buggy and more performant web app dubbed Warp Studio. Warp Studio is also accessible online.

New functions

Release 2.1 has a certain number of new functions which were added in different domains to make you more efficient when working with WarpScript.

The set of operators working on Geo Time Series has been expanded and now includes comparison operators <, >, <=, >=, == and !=, bitwise operators <<, >>, >>>, & (AND), | (OR) and ^ (XOR) and boolean operators ! (NOT), || (OR) and && (AND).

Function for working with multidimensional arrays (i.e. tensor like structures) have been added: SHAPE, RESHAPE, CHECKSHAPE, HULLSHAPE and PERMUTE.

New timing functions have appeared to help you tune your WarpScript code and control its execution time: CHRONOSTART, CHRONOEND, CHRONOSTATS, TIMED and TIMEBOX.

New compression/decompression functions DEFLATE and INFLATE use the Zlib compression library.

Functions for working with controlling the WarpFleet Resolver: IMPORT, WFON and WFOFF.

Functions for dynamically configuring macros: MACROCONFIG, SETMACROCONFIG and MACROCONFIGSECRET.

Function to produce a summary of the Warp 10 instance configuration and status, useful when reporting a problem: REPORT.

Handy function to shuffle a list: SHUFFLE.

Functions for disabling/enabling data ingestion and deletion and metadata updates: UPDATEON UPDATEOFF METAON METAOFF DELETEON DELETEOFF.

Conversion function ->LONGBYTES to convert LONGs to byte arrays.

Macros can now be executed from registers via an enhanced RUN function.

Functions for working with multivariate value format: PARSEVALUE, ->MVSTRING, MVINDEXPLIT, MVTICKSPLIT, MVVALUES, MVTICKS, MVLOCATIONS, MVELEVATIONS, MVHHCODES, WRAPMV

Lastly, many functions operating on Geo Time Series can now also operate on Encoders.

Performance improvements

Bug fixes and code refactoring have made some operations way faster. As an example, APPLY and FILTER are now faster when called on several 10s of thousands of GTS. Also, all BUCKETIZE operations following a FETCH should see performance improvements and MAP operations were sped up thanks to faster subseries extraction.

On the storage engine side, the support for multiple values in the input format enables blazing-fast ingestion performance even on very limited hardware. A future blog post will show how this probably makes Warp 10 also the fastest time series platform!

How to migrate an existing Warp 10 instance to 2.1?

The 2.1 release of Warp 10 is compatible with previous releases, but slight changes to the layout of the configuration files requires that you proceed with the steps below. The configuration change is that Warp 10 can now use multiple configuration files sitting in the etc/conf.d directory, this enables you to create separate configuration files for separate components of Warp 10, thus making the configuration easier to read. This also makes it possible for the WarpFleet packages and plugins to have their own separate configuration file.

  1. Get the latest version of Warp 10: https://www.warp10.io/download
  2. Extract the jar (ie: warp10-2.1.0.jar) from the archive
  3. Extract the warp10-standalone.sh file from the archive
  4. Stop your current Warp 10 instance: ${WARP10_HOME}/bin/warp10-standalone.init stop
  5. Copy the new jar and new warp10-standalone.sh in your previous Warp 10 bin directory
  6. Adjust Java Heap size if needed
  7. Create a new directory ${WARP10_HOME}/etc/conf.d
  8. Move old configuration file to this new directory
  9. Start your new Warp 10 instance: ${WARP10_HOME}/bin/warp10-standalone.init start