Introducing BigDecimal within WarpScript

Warp 10 3.1.2 comes with BigDecimal support.

The purpose of this article is to introduce BigDecimal to familiarize yourself with this number representation and then to present its use in Warp 10.

BigDecimal, what is it?

Floating point representation has limits. The default floating point format used everywhere in Warp 10, is called DOUBLE (aka 64bit IEEE754). You can represent around 15 decimal digits exactly, no more, and an exponent. It is sufficient for most usage, but it is still a floating point in base 2. Every number that is not a power of 2 is represented with an error (1/2, 1/4, 1/8 is not a problem).

Here is the binary representation of some fractions:

In base 10, we know we have a problem with some rational numbers too in base 10. 1/3 cannot be exactly expressed in base 10, as you have an infinite number of "3". But if you look at the previous result, you see that 1/10 is a problem for a computer floating points representation. 1/10 is rounded by your CPU and FPU (the FPU is designed to help the CPU process floating points faster).

That is why you can have surprising results from very basic operations:

Both equals 0.1 for you. Not for a computer. This is why the "epsilon equal" operator exists (In WarpScript, see ~=). One of the most obvious operations in the decimal system is to divide by 10: As 10 is not a 2 power, try 0.2 0.1 + 10.0 /… It does not equal 0.03! And errors can cumulate.

There is a field where everything is expressed in base 10, where you do not tolerate such errors: finance. BigDecimal solves the financial calculation problem. It forces your computer to represent numbers in decimal: 12345.67 will be stored in memory as:

• An array of digits: [ 1 2 3 4 5 6 7 ]
• A scale (number of digits in the fraction): 2

BigDecimal comes with a cost: it will always be slower than a floating point operation. Operations between numbers with a different scale are even slower.

For interoperability with other languages, the safest compromise is to use the string representation. The fastest JSON parser does not support BigDecimal.

BigDecimal within Warp 10

Warp 10 3.1.2 introduces a new set of functions for BigDecimal operations:

• ->BD
• BD->
• BDABS
• BDADD
• BDCOMP
• BDDIV
• BDDIVINT
• BDMAX
• BDMIN
• BDMUL
• BDNEG
• BDPOW
• BDREM
• BDSCALE
• BDSTRIP
• BDSUB
• BDULP

Basics

First thing first, ->BD convert a STRING, a DOUBLE, or a LONG to BigDecimal. TYPEOF will return "BIGDECIMAL". All the BigDecimal operations do an implicit ->BD call on their parameters. So, the explicit conversion to BigDecimal is not mandatory:

The BD-> forces the conversion to a string. As previously explained, some JSON parsing libs do not support BigDecimal. In the following example, select "raw json" output after execution:

Arithmetic

Warp 10 arithmetic's operators cannot be used safely on BigDecimal (it will lead to unpredictable results), so you have to use these set of functions:

The division has two functions. Remember that BigDecimal cannot represent fractional numbers with infinite digits, such as 1/3. So, 1 3 BDDIV will throw this error: Non-terminating decimal expansion; no exact representable decimal result. BDDIVINT will return the integer part of the result, rounded down, just like the standard division operator applied on LONG numbers.

Comparisons

Warp 10 comparison operators will work out of the box on BigDecimal. You can safely use >, >=, <, <=, ==, !=. However, the standard operators do not do an implicit ->BD when needed, and are slower than BDCOMP.

Scale related operations

When casting from a string, scaling is obvious: counting the trailing zeroes defines the scale. But when casting from a DOUBLE, scaling is guessed. Remember that 0.1 is actually equal to 0.1000000000000000055511151231257(…) for your computer. ->BD conversion does its best, but the scale can be surprising.

So if you want to be sure to work with scale 2, as in the example below, you need to change the scale. SCALE function will return or set the scale, depending on its arguments.

You may also want to get rid of the trailing zeroes: BDSTRIP will optimize the scale for you. But beware, BDSTRIP will also optimize to a negative scale (for example, 320 could be represented with [ 3 2 ] and -1 scale.

The last function, BDULP, returns the minimum unit that can be added without changing scale. If scale is 2, it will return 0.01. If scale is -1, it will return 10.

Conclusion

So you may never use them for your business, but Warp 10 is designed as a horizontal solution that covers basic monitoring to advanced industrial datasets. Finance is an industry too! WarpScript now provides a way to handle BigDecimal easily!

If you want to go further to understand floating point limits, you can read the dedicated Wikipedia page.

A question about BigDecimal? Feel free to ask us on the Lounge, the Warp 10 community on Slack.

Pierre Papin

Electronics engineer, fond of computer science, embedded solution developer.