Upload Geo Time Series from a Warp 10 instance to another

How to migrate data from a Warp 10 instance to another in 2 easy steps.

GTS Warp 10

There are two main strategies to sync your data between two or more Warp 10 instances: you can use a datalog or define your own mechanism using WarpScript. Here, I would like to show you how to read high-frequency data stored on an embedded Warp 10, which is hosted on a computer full of sensors for instance, downsample and store it on a remote distributed instance.

In this example, we read all GTS with a one hour time span and downsample them. Then, we use REXECZ to upload them to the remote instance.

'123456' 'localReadToken' STORE
'456789' 'localWriteToken' STORE
'azerty' 'remoteWriteToken' STORE

// Define an upload macro
  'time' STORE
  $time 'start' STORE
  // While we have not reach present time
  <% NOW $start >= %>
    [ $localReadToken '~.*' {} $start 1 h ] FETCH 'gts' STORE
    // re-sample with the mean value every 15 minutes
    [ $gts bucketizer.mean NOW 15 m 0 ] BUCKETIZE
    'outputGTS' STORE
    // if we have data
    <%  $outputGTS SIZE 0 > %>
      // build a snapshot of the stack
      $outputGTS  SNAPSHOT 'wrapped' STORE
      // prepare the remote WarpScript
      $wrapped " <%25 DROP DUP NAME '%2B' + RENAME %25> LMAP '" + $remoteWriteToken + "' UPDATE" + 
      // store the new sync point
      [ $localReadToken 'upload.time' {} NOW -1 ] FETCH 
      $start NaN NaN NaN true ADDVALUE
      '%2B' RENAME  // Don't change name but allow UPDATE
      $localWriteToken UPDATE
      $start 1 h + 'start' STORE
    %> IFT     
  %>  WHILE
    'all updated'
%> 'upload' STORE

// First, check the last sync point
[ $localReadToken 'upload.time' {} NOW -1 ] FETCH 'uploadGTS' STORE

// if this is the first time
<% $uploadGTS SIZE 0 == %>
  // Create a new GTS
  NEWGTS 'upload.time' RENAME
  // and set a very first data point
  0 NaN NaN NaN true ADDVALUE
  $localWriteToken UPDATE
  0 @upload
  // else, we get the last sync time
  $uploadGTS 0 GET LASTTICK @upload
%> IFTE 

To go further, refer to this WarpScript Tutorial.
Discover a real world use case of this data sync technique.