Thinking in WarpScript – Detecting a sequence of values

WarpScript is a handy toolkit build for time series manipulation. A common use case could be to detect a sequence of values lasting a given duration

This post describes how to detect in a series of values between 0 and 5, subsequences of value greater or equal to 3 lasting at least 5 seconds.

How to easily manipulate raw binary payloads in WarpScript

First step

As a first step, we need a dataset. Here, we will generate a random GTS containing 300 values between 0 and 5 spaced by one second. Of course, you can use your own.

NEWGTS                       // Create an empty GTS
0 300 <%                     // for(i = 0; i < 300; i++)
  'i' STORE                  // store the current index
  NOW $i 1 s * +             // tick ('now' + 'i seconds')
  NaN NaN NaN                // no geo coordinate
  RAND 5 * ROUND             // generate the random value
  ADDVALUE                   // add the data point to gts
%> FOR                       // close FOR loop macro
'gts' STORE                  // store the result

Read more about:
NEWGTS, STORE, FOR, NOW, RAND, ROUND, ADDVALUE, DROP

The result is a random time series:

Second step

Now, we will build a sliding window that navigates across this series in order to detect 5 seconds of values greater or equal to 3:

[
  $gts // our random series
  <%
    'data' STORE      // store the current sliding window data
    $data 0 GET       // current timestamp
    $data 4 GET 0 GET // current lat
    $data 5 GET 0 GET // current long
    $data 6 GET 0 GET // current elev
    // in our 5 current points, we keep only those >= 3
    // and verify we have 5 in a row
    $data 7 GET <% 3 >= %> FILTERBY SIZE 5 >= // put true or false as value
  %> MACROMAPPER // custom mapper
  
  0 // pre window size
  -5 s // 5 seconds of post window size
  0 // occurrences
] MAP // Sliding window

RANGECOMPACT // keep only changing values
'result' STORE

Learn more about:
GET, FILTERBY, SIZE, MACROMAPPER, MAP, RANGECOMPACT

Here, we have a sequence of booleans.

Now, we will remove false values:

[ $result true mapper.eq 0 0 0 ] MAP 'result' STORE // retain only 'true'

Learn more about:
mapper.eq

Last step

Now, we have our original series and a second boolean series corresponding to subsequences starts:

Last thought

If you want to detect the end of subsequences, you have to invert pre- and post-window sizes of the mapper:

[
  $gts 
  <%
    'data' STORE
    $data 0 GET  // Current timestamp
    $data 4 GET 0 GET // Current lat
    $data 5 GET 0 GET // Current long
    $data 6 GET 0 GET // Current elev
    // in our 5 current points, we keep only >= 3 and verify we have 5 in a row
    $data 7 GET <% 3 >= %> FILTERBY SIZE 5 >=
  %> MACROMAPPER // custom mapper
  -5 s
  0
  0
] MAP

You can find the final code and a running example of this sequence of values here: WarpStudio

Discovery Tips&Tricks #2 - Reproduce the WarpStudio Plot view
Here is a simple way to reproduce the Plot dataview of WarpStudio in a Discovery dashboard.

Thinking in WarpScript - Multi-Value Support
Discover everything about the new Multi Value syntax introduced by release 2.1 of Warp 10.

Working with GEOSHAPEs: code contest results
Discover the results of the GEOSHAPE code contest about a car's GPS track on Route 66. Step by step, explore the data and see how to find the answers.