Discover how location is encoded in Warp 10 and how you can search your Geo Time Series.
One of the major features of the Warp 10 platform is its support for location data both in its core data model, Geo Time Series, and in the WarpScript data programming language. This one includes many functions for working with geo information.
This post will walk you through the location data support included in Warp 10. It will explain some of the advanced features you can use for solving your geo data problems.
|Discover why Time Series are the future of data.|
Storing latitude and longitude as floating-point numbers is straightforward. But unfortunately, it does not ease processes such as geofencing or spatial indexing. In order to be efficient for those operations, Warp 10 stores location data in a format called HHCode, for Helical Hyperspatial Code. HHCodes were invented in the 90s by Herman Varma for storing bathymetric information at the Canadian Hydrographic Service. Later they became the basis for Oracle’s SDO (Spatial Data Object) cartridge.
HHCodes can encode multiple dimensions, not just a location. At the core of HHCodes is a space-filling curve. A Z-order Curve, or Morton code depending on how you want to call it. The HHCode implementation used in Warp 10 uses a Z-order Curve which starts in the lower-left corner and uses the 16 even levels from 2 to 32.
At level 2, the space is divided into 16 cells. At level 4 into 256 levels (16×16), and at level N into 2^2N. So at finest level 32, space is divided into 2**64 cells, or 18,446,744,073,709,551,616 (roughly 18 billion of billions). We use a hexadecimal naming scheme for cells.
The benefit of this hexadecimal notation is that by using only P hexadecimal digits, you get the cell name at level 2P. You can also convert easily the name to a LONG.Discover how location is encoded in Warp 10 and how you can search your Geo Time Series. Click To Tweet
The following table gives approximate scales at the various HHCode levels (resolution). Scale in meters is the width of each cell at the equator. The height of each cell is half that and does not change with latitude.
+---------------------------+-----------+ | Resolution (HHCode level) | Scale | +---------------------------+-----------+ | 2 | 10,000 km | | 4 | 2,500 km | | 6 | 625 km | | 8 | 156 km | | 10 | 39 km | | 12 | 10 km | | 14 | 2.5 km | | 16 | 600 m | | 18 | 150 m | | 20 | 40 m | | 22 | 10 m | | 24 | 2.5 m | | 26 | 60 cm | | 28 | 15 cm | | 30 | 4 cm | | 32 | 1 cm | +---------------------------+-----------+
Working with areas
The next step in working with geo data is to work with zones covering a certain area of the Earth. In WarpScript, areas are called GEOSHAPE. A GEOSHAPE is a collection of HHCode cells and associated resolution. The resolution is stored as a prefix of the cells. So for this reason, GEOSHAPEs can only contain cells down to level 32 in order for the cells to be representable on 64bits.
You can then combine multiple GEOSHAPEs using set algebra with the help of the WarpScript functions
GEO.DIFFERENCE. This allows you to build complex shapes that you cannot easily express in either WKT or GeoJSON.
Indexing and searching Geo Time Series
You care for the location of your sensors and wish to be able to select them according to their position? So you need to index the associated Geo Time Series according to latitude and longitude.
In Warp 10, the most efficient way of doing this is by adding an attribute to your Geo Time Series with a value set to the HHCode of the location of the sensor. We recommend you use an attribute instead of a label. In this way, you don’t have to know the value of the HHCode to push data into the GTS. And you will be able to modify the location should your sensor move at a later time.
|Discover the easiest way to detect motion and to split a Geo Time Series accordingly.|
When your Geo Time Series have their location stored as an HHCode in an attribute, you can create a GEOSHAPE and use it as a selector for identifying matching Geo Time Series. The
GEO.REGEXP WarpScript function will emit a regular expression which will match all HHCodes included in the input GEOSHAPE, you can then use this regular expression in a label selector in a
FIND call, the matching attribute (or label if you did not follow our recommendation above).
You should now have a better understanding of how you can manage location in Warp 10. And you have learned how you can index and search Geo Time Series. In a future post, we will go beyond indexing the Geo Time Series by covering an advanced technique of spatio-temporal indexing of the content of your GTS.
Warp 10 is part of a rich Open Source ecosystem, but it also has some commercial modules available. Discover some of those modules and what benefits they provide.
Manipulation of raw binary payloads in WarpScript could be done easily with existing functions, no need to call external programs!