Adjusting WarpScript Limits to fit your needs

How to configure WarpScript limits and use the full power of Warp 10.

Limits mechanism is a strength of WarpScript that helps multiple users to use the same instance. While using WarpScript, you may have encountered some error messages when fetching data points or doing some computation on some time-series. Something like this:

Example of a maxops error

Those messages came from a safeguard mechanism intended to protect the user from consuming too many resources. Warp 10 is by design a multi-tenancy platform and needs to share resources between all its users. For instance, you don't want one user to use all the resources, while on your side, you are fetching a few points. You neither don't want to have your WarpScript running for an unreasonable amount of time.

The default limits are very low and a bit conservative for you to discover them.

You have to adjust them to your needs.

A ten billion MAXOPS is not a problem. A one billion max fetch, why not. If you need it… You will always be limited by your hardware, not by Warp 10™.

1 limit, 2 thresholds

For more flexibility, there are two thresholds for each limit: a "soft" threshold and a "hard" threshold. By default, one WarpScript must respect all the soft limits. For particular needs, it can override the soft limit up to the hard limit.

  • Hard limits cannot be overridden at run-time and are only modifiable by editing Warp 10 configuration files. This implies to have the permission to edit the configuration file and restart the Warp 10 instance.
  • Soft limits can be overridden at run-time and are modifiable in a WarpScript. A user can change its value up to the hard limit. This implies to be authenticated on the stack with a valid read token.

Overriding the limits

There are two ways to modify the Warp 10 restrictions.

By modifying the Warp 10 configuration files

You need to have the permission to modify the Warp 10 configuration file and be able to restart the instance (i.e. be a root or a warp10 user). If you have those requirements, you can modify either the soft and the hard limits.

Edit the file ${WARP10_HOME}/etc/conf.d/20-warpscript.conf and adapt the limits to your needs. This is a sample of the default configuration file:

Sample of the default limits configuration file

By modifying your WarpScript

You may not be allowed to modify the configuration files, or you want to keep the default settings. So there is another way to change some limits: by directly changing them in a WarpScript. However, you can only alter soft limits, and you need a valid read token.

If you are trying to set a soft limit above a hard limit, you will have an error message:

Example of a maxfetch error

List of configurable limits

To sum up, the following table describes each restriction. So it gives the configuration keys that set the soft and hard limits. In addition, it provides the associated WarpScript function.

RestrictionKey for soft limitKey for hard limitWarpScript function
Maximum number of datapoints that can be fetched during a WarpScript executionwarpscript.maxfetchwarpscript.maxfetch.hardLIMIT
Maximum number of GTS that can be retrieved from Directory during a WarpScript executionwarpscript.maxgtswarpscript.maxgts.hardMAXGTS
Maximum number of operations a single WarpScript execution can dowarpscript.maxopswarpscript.maxops.hardMAXOPS
Maximum number of buckets that can result from a call to BUCKETIZEwarpscript.maxbucketswarpscript.maxbuckets.hardMAXBUCKETS
Maximum depth of the stackwarpscript.maxdepthwarpscript.maxdepth.hardMAXDEPTH
Maximum time (in ms) that can be spent in a loopwarpscript.maxloopwarpscript.maxloop.hardMAXLOOP
Maximum level of recursion in macro callswarpscript.maxrecursionwarpscript.maxrecursion.hardMAXRECURSION
Maximum number of symbols that can be defined by a single WarpScript executionwarpscript.maxsymbolswarpscript.maxsymbols.hardMAXSYMBOLS
Maximum number of cells in geographic shapeswarpscript.maxgeocellswarpscript.maxgeocells.hardMAXGEOCELLS
Maximum number of pixels per PGraphics imagewarpscript.maxpixelswarpscript.maxpixels.hardMAXPIXELS
Maximum number of characters outputted by
->JSON
warpscript.maxjsonwarpscript.maxjson.hardMAXJSON

Takeaways

You can now modify the Warp 10 limits as a WarpScript user or as a Warp 10 administrator. As you can see, many limits can be adjusted.

Be careful if you remove the handbrake, with great power comes great responsibility!