The 2.9.0 version of Warp 10 adds even more customization and versatility functionalities as well as lots of new functions and bug fixes.
The 2.9.0 version of Warp 10 was released on the 22nd of October 2021, 5 months after the release of the 2.8.0 and 2.8.1 versions. Like the last minor version, this release delivers a lot of improvements and new ways of customizing the platform to your very needs! 110 Pull Requests have been merged since the 2.8.0, adding and extending functions, improving the performance, and fixing bugs on the platform.
You can now limit the duration of execution of all scripts submitted on
/exec. To do so, specify
egress.maxtime in your configuration file. To allow specific users to go over this limit either to some longer duration or remove the limit altogether, you can use token capabilities, see the part on
To further constrain what your users can do, you can limit
FETCH parameters using token attributes. You can restrict a user to access to a specific period of time or greatly reduce the number of points this user can fetch.
While it is very easy to dump data out of Warp 10, it can be less trivial to recover from a connection lost or to parallelize the process. With the two new parameters
FETCH, you can paginate your dump.
gcount will limit the number of returned GTS while
gskip will skip the given number of GTS before returning the result. Of course, this will only work if no new GTS is created during the process.
Would you need to rapidly generate a lot of realistic data for benchmarking purposes, for instance, you can now shift all the timestamps sent to
/update. By setting the
X-Warp10-Timeshift header, this shift will be applied to all the parsed timestamps. You can then easily submit the same file multiple time with different shifts to simulate a larger dataset.
This one is for advanced users only: it is possible to delete all points from a GTS without removing the metadata. This can be particularly useful if you want to clear a GTS just before repopulating it with fresh points. This is the counterpart of
When a server-side macro manipulates sensitive information you want to make sure that none leaks. The
GUARD function makes sure of that by cleaning variables, registers, stack and error messages.
To simplify the modification of Lists and Maps, in the same way
MMAP do, we introduced
MFILTER. As you may already have guessed, they allow the filtering of elements in Lists and Maps according to the value returned by a Macro.
The last added functions are variance and standard deviation aggregators.
bucketizer.var was missing so we exposed it. As the algorithm used for the computation of variance and standard deviation can suffer from numerical instability, especially when the variance is low compared to the mean, we added Welford's method to compute them. As it is 10% slower, we opted to expose the functions using a different name to leave the choice of the algorithm to you. Those new aggregators are:
You can now use
REMOVE to delete an element in a SET, in BYTES or in a STRING.
REDUCE will now keep common attributes in GTS partitions, in the same way it keeps common labels.
HHCODE.SOUTH.WEST can now work with odd HHCode resolutions. Although odd-resolution HHCodes are less practical because they cannot be represented using hexadecimal or used in GeoShapes, there are totally valid.
METASORT will now also use attributes to order GTSs, if class and labels are the same.
In case there are missing locations in a GTS,
MOTIONSPLIT will now ignore the points.
Capabilities can be specified using the
X-Warp10-Capabilities header and a comma-separated list of tokens on the
Access-Control-Allow-Private-Network to CORS headers and
Access-Control-Allow-Headers for RFC1918 have been added to allow browsers to connect to local instances.
A read timeout configuration can be set for the TCP plugin using the
socketReadTimeout key in the configuration Map.
The changelog is now much nicer to read being based on PRs. It can be found in the root of the Warp 10 release archive.
As always some optimizations and better error handling and messages.
The 2.9.0 fixes some bugs present in the 2.8.1 version. We strongly encourage you to update to the 2.9.0 version because of a memory leak which appears when doing a lot of deletes. Here is the list of fixes:
- Fix the closing of a batch which caused a memory leak.
- Fix the closing of a decoder in the accelerator preloading phase which could make some persisted data unavailable.
- Fix a bug where some
TCPClients could connect but were not handled instead of being immediately disconnected.
- Fix a NullPointerException when handling error messages which lead to vague error messages.
- Fix context handling in the Shadow extension.
- Fix macro TTL extraction for WarpFleet macro repositories.
- Fix bugs that made
STLfail with cryptic error messages.
DEVALshard modulus assignment.
Warp 10 2.9.0
WarpScript syntax can look alien at first. We discuss the thoughtful reasons for its design and see how easy it is to get used to it.
In this blog post, we review the essential frameworks available in WarpScript. They simplify greatly usual time series processing.
Having the ability to store and read files from Warp 10 is something often requested. Today we talk about a WarpScript extension which introduces func...