Get started

Are you getting started with the Warp 10 Platform? This blog and the documentation are here to guide you on your journey to become a Warp 10 expert.

Data reading the blog to become a Warp 10 expert

Warp 10 is an open source platform designed to collect, store and analyze sensor / IoT data and any kind of sequence data in a horizontal and industrial perspective.

If you haven’t done so already, we suggest you read the Getting Started page on the Warp 10 website.

Where to start?

Here are some resources that will help you to begin with.

Basic concepts
What is a Geo Time Series (GTS)?

Geo Time Series (GTS) is a time series with a location associated with the value. For example, timestamp / latitude / longitude / elevation / value.

The Warp 10 supported values are:

  • boolean (true/false)
  • long (ie: 1123456)
  • doubles (ie: 15.158)
  • strings
  • multivariates

Some metadata are associated with a GTS:

  • class name (ie: io.senx.temperature) which represents what is measured
  • labels: key/value pairs used to specify, for example, the data source (ie: {serial=123456,cutomerID=456789,unit=kelvin}). Labels are immutable during the lifetime of the series.
  • attributes are like labels, except they are mutable. (Warp 10 does not keep track of changes)

So it can be represented like that:

Infographic presenting what a geographic time series is
What is a Geo Time Series

When you push data, use the "compact input format":
TS/LAT:LON/ELEV NAME{LABELS}{ATTRIBUTES} VALUE

And when you fetch data, the output will be in JSON.

There could be some missing data for latitude, longitude, and elevation, this is supported by the platform (for example, while the GPS is not yet initialized).

Read on

  • Master GTS: follow this tutorial to learn how to retrieve, create, modify, store GTS.
  • Cyclones on Sandbox: follow this series of tutorials to learn how to upload and analyze real data on the Warp 10 Sandbox
  • Discover the list of articles dealing with geolocated data on the blog, with the Geo Time Series tag.
  • All about the Geo Time Series Model: understanding the GTS model is the first step in mastering the Warp 10 platform. This model is quite simple, yet powerful, and this post will go through most of its aspects
How to manipulate GTS and data?

The way to manipulate GTS is to use the WarpScript language:

WarpScript & FLoWS

WarpScript is a powerful programming language, data-flow oriented. FLoWS is a functional language. It is an alternative to WarpScript for users discovering the Warp 10 Platform.

WarpScript and FLoWS and can interact with the library of more than 1000 functions designed for time series analysis.

How to fetch data
Data manipulation
Frameworks and utilities

Discover the essential frameworks: we identified a few frameworks that allow performing all the usual time series processing, and of course, these frameworks are built into WarpScript. Note that we call framework a WarpScript function that has a function among its arguments.

Geographical manipulation
  • Working with Geo Data in Warp 10: Discover how location is encoded in Warp 10 and how you can search your Geo Time Series.
  • Find data in a specific area: Extract data within a geographical area.
  • Spatio-temporal indexation: Learn how to index spatio temporal data in Warp 10 so you can query your series by geographic area and time range.
  • Motion Split: The easiest way to detect motion and to split a Geo Time Series accordingly. Split with distance between two timestamps, detect stopped/moving state etc.
Usual manipulation functions
  • Find which timestamp a list of GTS have in common with COMMONTICKS.
  • Normalize a series with values between -1 and 1 or 0 and 1 with ISONORMALIZE and NORMALIZE
  • Smoothing transforms: WarpLib has a ton of functionalities. Among them are smoothing functions.
  • Forecasting: Present of the WarpScript extension for time series forecasting with an exemple on power consumption data and another one on air passenger data.
  • Synchronize timestamps: Learn how to synchronize timestamps of two Geo Time Series and copy the locations from one to the other.
  • Aggregate by calendar duration: How to aggregate by calendar duration that can be irregular (like months or years for example).
  • Manipulating Multivalue and how to create it. Creation of multivalue GTS from within WarpScript needs a good understanding of how multivalue works in Warp 10.
Pattern and outliers detection
  • Detecting a sequence of values: WarpScript is a handy toolkit build for time series manipulation. A common use case could be to detect a sequence of values lasting a given duration.
  • Compare data hour to hour, day to day: Comparing every sunday data in a given timezone is not trivial. With WarpScript language, this is a pretty straightforward time series analytics.
  • Compare with the past: We use the auto-correlation function. We see how to implement it in WarpScript, with an exact calculation and then with a fast and accurate approximation.
  • Detecting anomalies: WarpScript offers built-in functions to help you detecting anomalies.
Advanced usages
  • Variables in Macros: Variables and macros are the muscles and bones of your script. Here are concepts and functions to make the best use of variables in macros.
  • Speeding up your analytics with SHM: SHM is a Warp 10 extension that allow to keep data in RAM. It speeds up data scientist job!
Factorize & share your code
  • Author macros: Learn how WarpScript Macros can make your code more concise, more robust and shareable.
  • Share macros: How to configure and use WarpFleet resolver to share macros internally or globally with WarpScript developers.
  • WarpFleet Resolver: Using the WarpFleet Resolver you can expose your WarpScript macros, making them easily callable from any Warp 10 instance.
  • Publish on WarpFleet: This is a very detailed HowTo that covers maven/bintray publication, a necessary step to publish Warp 10 extensions.
Dataviz tricks
  • Compute a histogram of values with VALUEHISTOGRAM to feed a pie in your dataviz for example.
  • Use LTTB to reduce the number of data points to be displayed
  • Use WarpView to build quickly a dashboard.
Basic OPS and settings
  • Backup and restore: How you can perform backups and restores of data managed by Warp 10 (with both the standalone and distributed versions).
  • S3 archiving & automatic archiving: Learn how to manage the life cycle of your Time Series data by setting up automatic archiving of old data to an external system such as S3.
  • Transfer data between 2 Warp 10: How to migrate data from a Warp 10 instance to another in 2 easy steps.
  • Demystify LevelDB: LevelDB is very robust and rather simple. Learn how it works and how to read its LOG.
  • Adjusting limits: How to configure WarpScript limits and use the full power of Warp 10.
  • Warp 10 Accelerator: Introducing the Warp 10 Accelerator for blazing fast data access and improved analytics. Enabling unbelievable time series data applications.
  • All about tokens: Warp 10 uses cryptographic tokens to manage authentication and authorization, learn everything about this advanced mechanism in this comprehensive article.
  • Schedule Warp 10 tasks with runners: Schedule Warp 10 tasks with runners, like down-sampling, archiving, deletions, computations and so on.
Other tools to complete the Warp 10 Platform

To find more tutorials and documentation about the Warp 10 Platform, please visit warp10.io

Don't forget to download the WarpScript CheatSheet.

Need help?

Do you have questions or difficulties? Join the Warp 10 community on Slack. You can also ask your questions on Twitter or Stack Overflow.

SenX provides training sessions, PoCs, support, and more. Contact the sales team to learn more about our complete offering.