Precision and representation of number in Python

Posted on Edited on In ,

Preface

Python can handle precision of floating point numbers

That’s one of the common findings from Google when searching “Precision in Python”.

In one of my company’s projects, I need to either multiply a number or divide a number by 10^8, and return the result in string format. In primary school Math class, we can move the decimal point forward/backward and add/remove zero accordingly along the way. However, when I start doing this task with * or \ directly and apply str() to the result in Python, things began to get really complicated and interesting, results like "0.08353999999999999" (a precision issue) and "1e-08" (a representation issue) are returned.

Precision Issue

From Floating Point Arithmetic: Issues and Limitations

Computer hardware is base 2, hence for the fraction 1/3, it is approximately 0.3, 0.33 or 0.333, but no matter how many digits we’are willing to write down, the result will never be exactly 1/3, but will be an increasingly better approximation of 1/3.

Inaccurate floating point calculation

Consider this calculation in Python:

1
2
>>> 8.354/100
0.08353999999999999

Solution

Since we cannot store an infinite number in the memory due to how a base-10 number is represented in base-2, especially for floating point. The issue now becomes up to how many decimal points do we want to keep the accuracy.

1. With decimal

1
2
3
4
5
6
7
from decimal import Decimal, localcontext

with localcontext() as ctx:
    ctx.prec = 10 # or a even higher precision
    res = Decimal(value="0.854", context=ctx) / 100 # res is 0.00854 with high precision

res = +res # res is 0.00854 still, but rounded back to the default precision

I got this idea from ethereum currency conversion. Basically we set a sufficiently high precision and perform a high precision calculation (999 is used for the Ethereum currency conversion). Do note that in order to preserve the precision of the constructed decimal object, the value parameter is better to be str type then float.

(Try with Decimal(value=0.854, context=ctx) and have a look🤪)

I also did some further readings on Decimal library, it is mainly used for the banking and financial industry, since accuracy is very important. However, there is a tradeoff of slow performance by using this library. (I once read an old news, a bank employee wrote a code to transfer the tiny bits beyond the displayed balance to his own account, this employee made lots of money from it because of the large amount of daily transactions.)

2. With round()

1
2
d = 8.354 / 100
print(round(d, 10))

round() rounds a number to a given precision in decimal digits.

Discussion

I choose the Ethereum method over round(), as the input value of my conversion function could be string type. Converting a string type to a type that round() supports requires an extra step, which might cause some unexpected behaviours.

Representation Issue

In my case, I need to return the string type of the floating point result as it is, instead of scientific notation. This means result of 1 / 10 **8 should be returned as "0.00000001" not "1e-08".

Solution

1
2
3
4
5
6
7
8
9
from decimal import Decimal, localcontext

with localcontext() as ctx:
    ctx.prec = 10
    num = Decimal(value="1", context=ctx) / 10**10

print("%f" % num)          # 0.000000
print(f"{num:f}")          # 0.0000000001
print("{:f}".format(num))  # 0.0000000001

Note: .format() uses full precision while %f defaults to a precision of 6.

Discussion

Division in Python is tricky, as the result is always float type, even for integer division with no remainder, hence a tidy up step is neeed.

1
2
3
res = 100 / 10                    # res is 10.0   float type
res = str(res)                    # res is "10.0" str type
res = res.rstrip("0").rstrip(".") # res is "10"   str type

Other Methods

In primary school Math class, we can move the decimal point forward/backward and add/remove zero accordingly along the way.

To tackle ths problem, after discussing it with a friend, he mentioned the pure string manipulation solution just like how we do it in Math class, since the calculation involved is just multiplication or division by 10^x.

I did not implement this solution though, I just thought about it roughly.

For multiplication of 10 ^8

  1. If string does not contain any decimal points, just add 8 zeros at the back of the string, done

  2. If string contains 1 decimal point:

    1. Add 8 zeros at the back of the string
    2. Move "." from the current index to the current index + 8 position
    3. Tidy up the result by result.rstrip("0").rstrip(".")

    (Note: Do not do result.rstrip("0."), you may try "10.00".rstrip("0.")🤪)

  3. If string contains more than 1 decimal point, this is an invalid input❌

For division of 10 ^8

  1. Check string contains 0 or 1 ".", else it is an invalid input❌
  2. Add 8 zeros at the front of the string
  3. Move "." from the current index to the current index - 8 position
  4. Tidy up the result by result.lstrip("0")
  5. If index 0 of the result is ".", add a "0" in front of the result

Referrence

Floating Point Arithmetic: Issues and Limitations

Ethereum Utilities Currency Conversion Code

Stackoverflow: Converting a very small python Decimal into a non-scientific notation string