Use motion to automatically split GTS

The easiest way to detect motion and to split a Geo Time Series accordingly. Split with distance between two timestamps, detect stopped/moving state etc.

Explicit Warning: if you use Warp 10 for data center monitoring, this article is not for you!

If you monitor things moving on the earth (boats, vehicles, planes, drones), this new function is for you!

Imagine you collect temperatures on a connected refrigerator truck (external temperature, internal temperature in several areas inside the truck). You are perfectly aware of Warp 10 2.0 multivalue feature, so your data model is really simple:

classname: temperatures
label: VIN
label: brand
label: model
data: multivalue payload (list of temperatures)

You want to study the temperatures when the truck is driving. Temperature, when doors are opened, are outliers of your study. When the truck is docked in, doors fully opened, or when the truck is stopped to deliver food at each restaurant, doors open.

Well-known WarpScript split functions

As a WarpScript expert, you already know TIMESPLIT. This useful function splits a GTS into several sub GTS. The trigger to split is a quiet period, a period without any data points. TIMESPLIT is perfect to split your temperature GTS in "missions": a mission start datalogger turns on, and ends when the datalogger stops.

That's fine… but the datalogger on your fleet has its own battery! So it never stops recording. TIMESPLIT won't help you.

VALUESPLIT is helpful for alerts or states, not for temperature.

DWTSPLIT is related to wavelet transforms. It won't help.

CHUNK is perfect to do regular chunks. Not for you.

CLIP is the surgery scalpel you use when you know exactly where to cut your time series.

So, you think about doing your own mapper to produce several GTS (that's possible with MACROMAPPER), with a state machine inside to detect movement, stops… That could be a solution, but it's not a trivial one!

WarpScript MOTIONSPLIT, since 2.3.0

I will explain MOTIONSPLIT with the following example:

Paris, its famous avenues, its well-known traffic jam, all you need for a good example. If you want to play with the same GTS, copy this snippet:


'60V7O5KgPMO_RbZQ.NV4S68pNqg5FIBlBKCH4V.L.0g..5Yo3WyA1..........KKb_RooRLCx6okslbJQFKl6eoKaepxLXfKGxeux3ILv0phxOUvLdMuxd2iiubqxJF979VJEl0i0jikoVF7VFA83.V6361_M.IkWp1.VX9388xzYiUaIUnjhTQHUStj7XHyj4YYjcC_lucSquboD1XUppqwaTLoRV6XUK8cFDovvWxfIoua5LHmuDcz3b5UA.RLTZPM6iKOJF0zfFiERFCK3hRjnzpqG2AOCwccZdxs7hVyLqAto2O1IkKo3VTlGbLFTzDlnaXxABgcuop33LPnFqub1ioxKp5_eZiBW84RGV4JtGgQEMLqmknNLoZ7JC0du68TvsEAniSOQeu8caad7m0pjrfUTW_jCqF8eKqsU.qgAnQ7KzynouLJHjmu9mEwWFN_7a_Rkajg7moL3Q_RQZAc2QzYIBMZWlNdazBIEYwwW48glgNxapwtNSnsxXgvkBe4edu_cAdx.TASsSiN0bnI.MlwvTjWUvmrQafgYlltNwXei5bM7HahQavkboRgSisRL9CubN0gxjYY5vW8dOt68O5OuC5_ZAQnjnNqOENjsaCc5K2FoysN2vMgfMT.tzOw2HEIjNZhay5PZrgjTPM1pb59asVjeLmO1oM28jUzEY7oGJgsuUM3eJVqF5BIbyDiYgUNWbOnF0ebGjWFNMhrzUyt1ETofNNmZQzHP5zwOIDyUFx_W335sXxIZx0MzK4SyTmzQY6K339FmNdC5Jzrqcnhv4Bh305vH324CcD1PvB5uxZ5M4P28qJ8E4.Rx1AEvvu9DsOg02U7IZETjwqia4ColbgEuaDnDdkGGSvxmnrRxmIEidEfQJAkOmVXFrTCsTM4zhHjY5AFcl4ZDxFmLROjY5P9V8qFJhgT4nChcogBxaOHOhoDWvoLz9iMEPdFhNuG2Gvc1yT1nxwPbfCyyht99ll7QL6EsfXFRDw5whub01Cm47WtHh9EkGIrUADBeSm_dZDXeHLux7po1WEA2lGBBomBnJ1a_rJ_oHVjeMbXy4QNdO8fTsVkIDqMI0RAjQxfyjjA0EQu3xf3YI_Ie2NzvzYbXcwtnxpasQ9ebmsN1dnb4aFYsIsLmDdEBxzBzHStexwRto.qwyGoamhrVrB5nvXUHqVA8BOCN_HLVq5xJQDkcMCA4CNRtzuOWLw0dfqSTyjeYRYyvO7U6Lf5xFqrBqyOjKV5Up3A.pCMFWucdFvQiHkI3vR6PeBTW35TniLxpkUGgRwmePy.Ij.DAv6RTnlPmln9qPFn8P7r91hDmpft23P8X.oWhtLA.QlTU6tBo_YXBLVZNHqOU5VHT6lHLIASdNdfFmX1btKOVc6ydQMyC_T5JKDST_oIEAo6TY_EMSiWSs4MlDY7rRPFggtZ0ncxvfhwACH_jiow7NA59tu6ImwzlDLBTOqZxacjsCLSzXBpf.HrI9LI0KqNIBMG_YhdLgH1Id0M36sXfMFiLH4BBoUeokJOKAUS.L4KVy66nZ4mpYaVPmOqBiMF6zRoSTx7f0BL5sy_LPbRq2VauJlVSv6L1Nl6aajRs1WcwinKROjw9QsPU8cr.bud_Fhnf6ta0HdDSHDdfGsdWU9hkkX0pC5sWS8NK1koUdwzFS7r3FTpiY1huqNMKQnzL6JSV4A1oBTwB.kdQHO_FnX75gbxd1Mk2YZBi6k9xec7g5EncPjWw5boOyHPx3gom5p3fj34LdncnnEDQm9JcxWeKLKnk4YTEEL1dBKxI0d7bOtPuNMPEUSDv2SOkgXSPDCH7hrVQaXTav6_jMsDcBjnIMjVwtMWUl6dp.ENReQS28_CFkSsolRm2y1YygJNJEfc3MF6aUQRE1OcyT7GfBS9bSyUqAPWA3nEG09umezVi6nUoWJM0jAWDdXeCeNtkIueyJCMYhsiCAz_2l2dPbrlnqzqI5wpUck9VxQ.CqjIcZDUPh6Fq35zEKv8GReZtHAG5jFIx6_SmazRYDoUrJd7hIy3RhZ_Gi7y5N00sNix1dsKJ4cpNFD3YfhwlnJz6mtKwYv3VZehd08rLdFAWkhvQ7dgOwH2K5A2c2a0yvQzod6s0sy1949Tlc6GeNShvzZYbMOm.GeJWq4jaquZfXAtp0nJIC39Ffz3_dYbdnYBohNdaqXaiFyYD81WMG_m..o7H34.YdtrFaevaL46Gx.P1gPTc_K8yS.yvvJZ4y5NW3cbk3gglUunnlzDS6zLxF59uHW4SJDcApFRAKevKPJYna2aW7uiI1wPB63muZh9BhO9Lpu7enrVcOVOxRQTrCNdeZzVsdhBoUWrNA46fEzkUH8UdW1zM79K7iVXjjOdp7OifNY6XTJzUsZ2vISWHu0VLXAS5gXKQTZ6c0sR8wBSb06WJyB3eKFMBL0zdlU3uQhP7BDiNsJqcAwD2.uS_RJu6VIRPu9c6my.pYyJDwqrNv0YC2bNoPw1Bb0vgTiTiExgMeLygaLVbulzL8TB7cZSgd1t6bxMh2pNF2zO7a0OKfidp1426DDzYGMS_rCBgf6LKWZ4Flujm27U6wMWZPGXNk8JvpVC5TfBrmALwIrZuW36yw4zP5OhvhUT0gbe9yBXcgAIc0D9ZasZces0pHdXOzgGd3IuAsvRyfZGGkLWIRIPD5xb4LV3NLjzRJOYRjrHuOgHwVhAbGMvbLQXUZC8xBFK_ABhIY.e9GhUbu41eSRGmlhebrs6EGgmdtnMs7YjBLzVHV3oQiWQhELsyAV_x.kYkQ7a9tWDUGzyE4rMVSAwMBkrZk9AGvF0XCvrmhz.lTuATGDK07DlW9Et1qP9pylI7HkadHOCs6tBozMSnklBQMe42efRs4WnOr.jGyzYAunIl4tpI6czIlryWznAFXuoZ2saTBRPCqnLRivqjFBB.hdACo7ddF2WjCXnD56jQTF_s4B_DuDsIT6rjuKOLlpJjeM8.5M9bE5flzZzsVuh2qVucG4Z89ZwYZZFcNQ6X_8YRxCz0L4Mx8jawCkIexM3vJiVkCbHvF_3TEE17pj8h9O6AnxH4jNqNET154wD8JAyYaF3GaeRjBMcK9HZn4VS0lcmiK95gUTkg6FLsR0swD0feWpXphKLUWAuBUoomo3IPp36QFb3fw7uJ4JJzov0yeEA1fQwFNDTp5lEtoCK6A0cZWUcneDbeoL20.tUl_N3bvXvDtTYHElUXsSgpXto0PoHuwQRBKOZS40C83LvAC6JevoUXjlztXbGToO1V..4e.13F.'
UNWRAP 'gts' STORE

MOTIONSPLIT overview

The function is able to split by time, by distance, to detect stopped state in a radius, exit the stopped state on several conditions.

It is available since Warp 10 2.3.0 release.

As the latest advanced function in WarpScript, MOTIONSPLIT takes a MAP for maximum flexibility of input parameters.

No operation

$gts { } MOTIONSPLIT

This won't break your WarpScript… It is just an inefficient CLONE!

Time split

$gts { 'timesplit' 10.95 s } MOTIONSPLIT

This performs a time split when there are more than 10.95 seconds between two data points.

Distance split

$gts
{
  'timesplit' 10.95 s
  'distance.split' 100
}
MOTIONSPLIT

If there are more than 100 meters between two positions, then split. I combined time split and distance split. Distance split could be very useful to remove outliers, such as GPS lost at (0 0) coordinates.

Add extra labels to output

$gts
{
  'timesplit' 10.95 s
  'distance.split' 100
  'label.split.number' 'split'
  'label.split.type' 'reason'
}
MOTIONSPLIT

That's the easiest way to filter out the GTS list later on, with filter.bylabels.

Stop detection split, simplest (time in an area)

$gts
1573135239233178 1160575178 TIMECLIP // cut the final stop
{
  'stopped.min.time'  60 s
  'stopped.max.radius' 80
  'label.stopped.time' 'stop_time' 
  'label.split.start' 'split'
  'label.split.type' 'reason'
}
MOTIONSPLIT

If the truck stays more than 60 seconds in an 80 meters radius, split. Note, the stop_time extra-label could also be used later to keep long stops… or not!

Discover how to detect a sequence of values in WarpScript

Stop detection split, with maximum speed to consider a stop

<% 3.6 / %> 'km/h' STORE
$gts
{
  'stopped.min.time'  60 s
  'stopped.max.speed' 10 @km/h
  'stopped.max.radius' 80
  'label.stopped.time' 'stop_time'
  'label.split.type' 'reason'
}
MOTIONSPLIT

Why? Because when driving again, there is this extra point in the proximity zone. The stopped.max.speed fix the problem:

Stop detection split, with a maximum mean speed in the proximity zone that prevents the split

$gts
{
  'stopped.min.time'  60 s
  'stopped.max.speed' 10 @km/h
  'stopped.max.radius' 80
  'stopped.max.mean.speed' 1 @km/h
  'label.stopped.time' 'stop_time'
  'label.split.number' 'split'
  'label.split.type' 'reason'
  }
MOTIONSPLIT

It will split only if there is a full stop during the given time and radius, and if the mean speed in the proximity zone is less than 1km/h

Combine everything!

$gts
{
  'timesplit' 10.95 s
  'distance.split' 100
  'stopped.min.time'  60 s
  'stopped.max.speed' 10 @km/h
  'stopped.max.radius' 80
  'stopped.max.mean.speed' 5 @km/h
  'label.stopped.time' 'stop_time'
  'label.split.number' 'split'
  'label.split.type' 'reason'
}
MOTIONSPLIT

When combining different split reasons, the split type priorities are always:

Stopped > distance split > time split > end of sequence

Conclusion

Isolate movements in a GTS could now be done in a few lines of WarpScript, with a maximum of flexibility. This function will evolve in future versions, with more parameters to address more use cases.