Division Precision

 Keywords: Order of Precedence Typeless Data Type Conversion Divide Division Integer Float Floating Point String 3057

Question:

Would you let me know if the results are as intended? I happened to stumble on this while trying to produce a ratio, and decided to experiment about how WinBatch would handle these possibilities (as you can find in the enclosed program).
a = 123456 b = 23456 ret1 = a / b ; 5 ret2 = (a:'.') / b ; 5.263302 ret3 = a / b:'.' ; 5. ret4 = a / (b:'.') ; 5.263302 ret5 = (a:'.') / (b:'.') ; 5.263302 ret6 = a:'.' / b ; error: 3057: Variable could not be converted to a valid number.

Answer:

Yes these are expected results. Let me see if I can explain....

What you are seeing is the result of a few different factors: WinBatch is a typeless language and attempts to convert data types based on their use and Order of Precedence in WinBatch.

The precedence of the operators affect the evaluation of operands in expressions. Operands associated with higher-precedence operators are evaluated before the lower-precedence operators.

The table below shows the precedence of the operators. Where operators have the same precedence, they are evaluated from left to right.

Operator	Description
( )	Parenthetical grouping
& *	Unary pointer operators
~ ! - +	Unary operators
**	Exponentiation
* / mod	Multiplication, Division & Modulo
+ -	Addition & Subtraction
<< >>	Shift operators
:	Concatenation
< <= == >= > != <>	Relational operators
& ^ \|	Bit manipulation operators
&& \|\|	Logical operators
=	Assignment Operator

a = 123456
b = 23456
 
;  If you take two integers, you will not necessarily get a floating point answer.  In WinBatch at least one of the numbers needs to be a Float to get a true Floating point answer.
ret1 = a / b 

; This example first interprets the data in the parentheses because of Order of Precedence. So (a:'.') results in a String "123456." Since this string is being used in a division 
; expression it gets converted to a Float, this will result in a Float result.   
ret2 = (a:'.') / b
 
; Because of Order of Precedence, Multiplication, Division & Modulo  will get process first. So a/b which will result in a integer result next the colon Concatenation operator gets 
; processed so the result is the integer 5 concatenated with the string '.', so the result will be a String.   
ret3 = a / b:'.'

; This example first interprets the data in the parentheses because of Order of Precedence. So (b:'.') results in a String "23456." Since this string is being used in a division 
; expression it gets converted to a Float, this will result in a Float result. 
ret4 = a / (b:'.')

; This example first interprets the data in the parentheses because of Order of Precedence. So (a:'.') results in a String "123456." and (b:'.') results in a String "23456." Since this string is being used in a division 
; expression it gets converted to a Float, this will result in a Float result. 
ret5 = (a:'.') / (b:'.')

; Because of Order of Precedence, Multiplication, Division & Modulo will get process Before Concatenation therefore causing the error:  3057: Variable could not be converted to a valid number. 
ret6 = a:'.' / b   ;Not Supported.

Try running the above code in WinBatch Studio in DEBUG mode, in the Watch window pay close attention to the data TYPE results. You will see integer, float and string data types.

Article ID:   W18218

Filename:   Division Precision.txt

File Created: 2014:01:17:13:30:46

Last Updated: 2014:01:17:13:30:46

Database Search

TechHome