WarpScript is a handy toolkit build for time series manipulation. A common use case could be to detect a sequence of values lasting a given duration

This post describes how to detect in a series of values between 0 and 5, subsequences of value greater or equal to 3 lasting at least 5 seconds.
How to easily manipulate raw binary payloads in WarpScript |
First step
As a first step, we need a dataset. Here, we will generate a random GTS containing 300 values between 0 and 5 spaced by one second. Of course, you can use your own.
NEWGTS // Create an empty GTS
0 300 <% // for(i = 0; i < 300; i++)
'i' STORE // store the current index
NOW $i 1 s * + // tick ('now' + 'i seconds')
NaN NaN NaN // no geo coordinate
RAND 5 * ROUND // generate the random value
ADDVALUE // add the data point to gts
%> FOR // close FOR loop macro
'gts' STORE // store the result
Read more about:
NEWGTS, STORE, FOR, NOW, RAND, ROUND, ADDVALUE, DROP
The result is a random time series:

Second step
Now, we will build a sliding window that navigates across this series in order to detect 5 seconds of values greater or equal to 3:
[
$gts // our random series
<%
'data' STORE // store the current sliding window data
$data 0 GET // current timestamp
$data 4 GET 0 GET // current lat
$data 5 GET 0 GET // current long
$data 6 GET 0 GET // current elev
// in our 5 current points, we keep only those >= 3
// and verify we have 5 in a row
$data 7 GET <% 3 >= %> FILTERBY SIZE 5 >= // put true or false as value
%> MACROMAPPER // custom mapper
0 // pre window size
-5 s // 5 seconds of post window size
0 // occurrences
] MAP // Sliding window
RANGECOMPACT // keep only changing values
'result' STORE
Learn more about:
GET, FILTERBY, SIZE, MACROMAPPER, MAP, RANGECOMPACT
Here, we have a sequence of booleans.

Now, we will remove false values:
[ $result true mapper.eq 0 0 0 ] MAP 'result' STORE // retain only 'true'
Learn more about:
mapper.eq

Last step
Now, we have our original series and a second boolean series corresponding to subsequences starts:

Last thought
If you want to detect the end of subsequences, you have to invert pre- and post-window sizes of the mapper:
[
$gts
<%
'data' STORE
$data 0 GET // Current timestamp
$data 4 GET 0 GET // Current lat
$data 5 GET 0 GET // Current long
$data 6 GET 0 GET // Current elev
// in our 5 current points, we keep only >= 3 and verify we have 5 in a row
$data 7 GET <% 3 >= %> FILTERBY SIZE 5 >=
%> MACROMAPPER // custom mapper
-5 s
0
0
] MAP
You can find the final code and a running example of this sequence of values here: WarpStudio
Read more
WarpScript has support for Apache Arrow Format. Discover what this format is with examples using WarpScript with R, Python and Spark.
As part of the 2022 Ocean Hackathon, SenX provides the contestants with an AIS data set covering more than 2 years and containing 500 billion data poi...
Build a Santa tracker from scratch with Warp 10 and Discovery. Technics for asset tracking with a real-time monitoring dashboard.

Senior Software Engineer