WinBatch Tech Support Home

Database Search

If you can't find the information using the categories below, post a question over in our WinBatch Tech Support Forum.

TechHome

WinBatch
Functions

Can't find the information you are looking for here? Then leave a message over on our WinBatch Tech Support Forum.

Floor Function question

Question:

Does anyone know why I get different results with the last 2 lines of the following code? 

decimals(0)
message("Floor of 2.0",floor(2.0))
message("Floor of 2.00",floor(2.00))
message("Floor of 2.000",floor(2.000))
message("Floor of 2.0000",floor(2.0000))
message("Floor of 2.00000",floor(2.00000))
message("Floor of 2.000000",floor(2.000000))
The results for the last two lines return the FLOOR as 1 instead of 2.

Answer:

Good question. In general, the basic problem is that all floating point numbers are approximations. Thus, in theory, it is practically impossible to specify exactly the number you want. This is why you never ever want to compare two floating point numbers for equality.

A similar problem occurs here with the floor function. Various software algorithms are trying to convert different ASCII representations of 2.0 (2.00, 2.000, etc) to a usable IEEE format floating point number that the CPU's floating point processor can use. Apparently some of the approximations of 2.0 falls below the real 2.0, and this caused the Floor function to drop the result from 2 to 1.

However, the IEEE floating point format has been specially rigged so that for smaller numbers, it is supposed to be able it hit smaller integer values spot on.

So lets try a Floor/Ceiling test to check for deviations from the spot on expected of the algorithm... 

Decimals(-3)
a=2.0
Message(a,strcat(Floor(a),@crlf,Ceiling(a)))
a=2.00
Message(a,strcat(Floor(a),@crlf,Ceiling(a)))
a=2.000
Message(a,strcat(Floor(a),@crlf,Ceiling(a)))
a=2.0000
Message(a,strcat(Floor(a),@crlf,Ceiling(a)))
a=2.00000
Message(a,strcat(Floor(a),@crlf,Ceiling(a)))
a=2.000000
Message(a,strcat(Floor(a),@crlf,Ceiling(a)))
a=2.0000000
Message(a,strcat(Floor(a),@crlf,Ceiling(a)))
a=2.00000000
Message(a,strcat(Floor(a),@crlf,Ceiling(a)))
a=2.000000000
Message(a,strcat(Floor(a),@crlf,Ceiling(a)))
In the cases where both results are not equal to 2.0, it means there was some deviation from an exact 2.0 at some level perhaps in the 0.0000000000001 accuracy range. Close enough for most scientific work, but issues are caught by Floor/Ceiling/mod type functions.

In any event will forward to programmers. I suspect they are going to blame Microsoft.

Developers Reply:

Basically, 2.00000 is not equal to 2. And 0.00000 is not equal to 0.

A floating point number is 64 bits, and the more bits that are used to store the fractional part, the fewer bits available to store the whole part. So 2.00000 might be stored internally as 1.999999999999999.

If you want to round a floating point number, use the Int function. 

Article ID:   W16970

File Created: 2007:07:03:14:27:24
Last Updated: 2007:07:03:14:27:24