WinBatch Tech Support Home

Database Search

If you can't find the information using the categories below, post a question over in our WinBatch Tech Support Forum.

TechHome

Number Conversion

Can't find the information you are looking for here? Then leave a message over on our WinBatch Tech Support Forum.

Floating Point Arithmetic and Currency Calculations

Keywords: money rounding decimals	 

A problem exists in some floating point calculations when trying to apply the calculations to money. Internally, WinBatch keeps the numbers with a higher degree of precision than is actually displayed.

Background Info

In all computerized floating point calculations a given number is merely represented by an internal floating point number that is generally *very* close to the desired number. It is actually rare that a random number can be exactly represented. During the display process where the computer turns its internal version of the number into a human readable representation, rounding (to nine digits) occurs. This makes a number like:

   1.2499999999999		look like  1.25

This is generally fine. However in currency calculations, close is not always enough. It has to match the type of calculations banks, tax authorities, and accounting systems use. The idea is to reset the internal numbers after any floating poing calculation to the visible representation. See the following code. Especially notice the:

	c = %c%	   
construct. The construct forces WinBatch to make a human readable representation of the floating point number and then use that number instead of its internal version. Basically, it changes the .07599999999 stuff, so it looks like it's reset to what it outta be.

Use the %c% to adjust the number slightly after every floating point computation. %c% destroys precision. Doing this after every floating point operation keeps (mostly) currency computations on track.

decimals(2)   ; for US Currency calculations

;Method where floating point rounding affects answer: 
a=1.25
b=0.06	
c=a*b
d=a+c
Message("Expected answer=1.33","Computed answer=%d%")



; The fixed version:
a=1.25
b=0.06	
c=a*b

c=%c%

d=a+c

d=%d% 
Message("Expected answer=1.33","Computed answer=%d%")

Article ID:   W13033
Filename:   Floating Point and Currency Calculations.txt
File Created: 2011:05:31:11:55:36
Last Updated: 2011:05:31:11:55:36