5.15-2.55-2.6

General development discussion.

Moderators: Susan Smith, admin, Gabriel

John
Posts: 554
Joined: Sun Apr 26, 2009 8:27 am
Location: West Orange, NJ
Contact:

5.15-2.55-2.6

Postby John » Mon Jan 20, 2020 12:54 pm

Why does the answer to this:

5.15-2.55-2.6

not come out to 0?

try it in your console and see what you get?

I'm asking for a friend.

-John
John Bowman

John Carr
Posts: 3
Joined: Tue Jun 09, 2009 7:59 am

Re: 5.15-2.55-2.6

Postby John Carr » Mon Jan 20, 2020 1:06 pm

This is caused by the difference between decimal fraction and binary fraction values. IE-the binary values of the decimal numbers you picked don't EXACTLY equal the original decimal value. This is only a problem with some fractions. Whole numbers are exactly equal between decimal and binary. This is also why sometimes an apparent zero value will print as -0.00 or .00CR.

Gordon jump in if I have gotten it wrong.

Gabriel
Posts: 401
Joined: Sun Aug 10, 2008 7:37 am
Location: Arlington, TX
Contact:

Re: 5.15-2.55-2.6

Postby Gabriel » Mon Jan 20, 2020 6:44 pm

Very interesting.

I do note that if its printed in any kind of formatted way, or if its tested in an if statement, then it still works correctly.

Untitled.png
Untitled.png (2.25 KiB) Viewed 3389 times


Gabriel

gordon
Posts: 337
Joined: Fri Apr 24, 2009 6:02 pm

Re: 5.15-2.55-2.6

Postby gordon » Mon Jan 20, 2020 10:11 pm

Yes John. You stated it correctly. All programming languages that work with floating point numbers have the same problem. By using scientific notation internally the binary fractions used in floating point math do not precisely match decimal fractions. Floating point arithmetic necessarily and specifically accepts the notion that results will be infinitesimally imprecise.

For that reason when adding a long column of numbers it is a good idea to round the result every so often. This will erase any accumulating low level inaccuracy.

In BR the number of decimal places where rounding occurs during output formatting and comparisons, defaults to 6 decimal places. However the RD setting can be changed. See: http://brwiki2.brulescorp.com/index.php?title=RD

GomezL
Posts: 239
Joined: Wed Apr 29, 2009 5:51 am
Contact:

Re: 5.15-2.55-2.6

Postby GomezL » Tue Jan 21, 2020 7:30 am

You can better see the results using formated output:

Code: Select all

pr using "form n 30.28":5.15-2.55-2.6

Returns: 0.000000000000000444

You can see that there are 15 0's after the decimal place.

Loop 1 million times:

Code: Select all

00001   PRINT Newpage
00010   FOR Loop=1 TO 1000000
00020     LET A+=(5.15-2.55-2.6)
00030   NEXT Loop
00040   PRINT USING "FORM N 30.28": A


Returns: 0.0000000004440892098500630000

Not surprisingly, moves the decimal 6 places

Add the Round Command (to 15 digits):

Code: Select all

00001   PRINT Newpage
00010   FOR Loop=1 TO 1000000
00020     LET A+=Round(5.15-2.55-2.6,15)
00030   NEXT Loop
00040   PRINT USING "FORM N 30.28": A


Returns 0


Return to “General Development”

Who is online

Users browsing this forum: No registered users and 0 guests

cron