"SfR Fresh" - the SfR Freeware/Shareware Archive 
Member "docs/script_guide/floating_point.html" of archive TestMaker.zip:
<HTML>
<!--This file created 10/18/02 4:07 PM by Claris Home Page version 3.0 30 Day Trial-->
<HEAD>
<TITLE>TestMaker Script Guide - Floating Point Numbers</TITLE>
<META NAME=GENERATOR CONTENT="Claris Home Page 3.0 30 Day Trial">
<X-CLARIS-WINDOW TOP=162 BOTTOM=768 LEFT=24 RIGHT=554>
<X-CLARIS-TAGVIEW MODE=minimal>
</HEAD>
<BODY BGCOLOR="#FFFFFF">
<H1><FONT FACE="Arial"><TABLE BORDER=0 BGCOLOR="#0000CC" CELLSPACING=1 CELLPADDING=0 WIDTH="100%">
<TR>
<TD>
<P><FONT FACE="Arial"><TABLE BORDER=0 BGCOLOR="#FFFFCC" CELLSPACING=0 CELLPADDING=0 WIDTH="100%">
<TR>
<TD WIDTH=10>
<P><FONT FACE="Arial"><IMG SRC="../images/blank.gif" WIDTH=10 HEIGHT=1 X-CLARIS-USEIMAGEHEIGHT ALIGN=bottom></FONT></P>
</TD>
<TD WIDTH=120>
<P><A HREF="http://www.pushtotest.com"><FONT FACE="Arial"><IMG SRC="../images/ptt_logo_120w.gif" WIDTH=120 HEIGHT=32 X-CLARIS-USEIMAGEWIDTH X-CLARIS-USEIMAGEHEIGHT BORDER=0 ALIGN=bottom></FONT></A></P>
</TD>
<TD WIDTH=30>
<P><FONT FACE="Arial"><IMG SRC="../images/blank.gif" WIDTH=30 HEIGHT=1 X-CLARIS-USEIMAGEHEIGHT ALIGN=bottom></FONT></P>
</TD>
<TD>
<P><FONT FACE="Arial">TestMaker Script
Guide</FONT></P>
</TD>
<TD VALIGN=top ROWSPAN=3>
<P><A HREF="../index.html"><FONT SIZE="-1" FACE="Arial">Table
of Contents</FONT></A><FONT SIZE="-1" FACE="Arial"><BR>
<BR>
TestMaker Script Guide Chapters<BR>
</FONT><A HREF="script_index.html"><FONT SIZE="-1" FACE="Arial">Introduction</FONT></A><FONT SIZE="-1" FACE="Arial"><BR>
</FONT><A HREF="flow_control.html"><FONT SIZE="-1" FACE="Arial">Flow
Control</FONT></A><FONT SIZE="-1" FACE="Arial"><BR>
</FONT><A HREF="data_structures.html"><FONT SIZE="-1" FACE="Arial">Data
Structures</FONT></A><FONT SIZE="-1" FACE="Arial"><BR>
</FONT><A HREF="modules.html"><FONT SIZE="-1" FACE="Arial">Modules</FONT></A><FONT SIZE="-1" FACE="Arial"><BR>
</FONT><A HREF="input_output.html"><FONT SIZE="-1" FACE="Arial">Input
and Output</FONT></A><FONT SIZE="-1" FACE="Arial"><BR>
</FONT><A HREF="exceptions.html"><FONT SIZE="-1" FACE="Arial">Errors
and Exceptions</FONT></A><FONT SIZE="-1" FACE="Arial"><BR>
</FONT><A HREF="classes.html"><FONT SIZE="-1" FACE="Arial">Classes</FONT></A><FONT SIZE="-1" FACE="Arial"><BR>
</FONT><A HREF="floating_point.html"><FONT SIZE="-1" FACE="Arial">Floating
Point Numbers</FONT></A></P>
<P></P>
</TD>
</TR>
<TR>
<TD WIDTH=10>
<P><FONT FACE="Arial"><IMG SRC="../images/blank.gif" WIDTH=10 HEIGHT=1 X-CLARIS-USEIMAGEHEIGHT ALIGN=bottom></FONT></P>
</TD>
<TD WIDTH=120>
<P><FONT FACE="Arial"><IMG SRC="../images/blank.gif" WIDTH=120 HEIGHT=1 X-CLARIS-USEIMAGEHEIGHT ALIGN=bottom></FONT></P>
</TD>
<TD WIDTH=30>
<P><FONT FACE="Arial"><IMG SRC="../images/blank.gif" WIDTH=30 HEIGHT=1 X-CLARIS-USEIMAGEHEIGHT ALIGN=bottom></FONT></P>
</TD>
<TD VALIGN=top>
<P><FONT SIZE="-1" FACE="Arial">Published: October
21, 2002<BR>
Applies to: TestMaker 3.0</FONT></P>
</TD>
</TR>
<TR>
<TD WIDTH=10>
<P><FONT FACE="Arial"><IMG SRC="../images/blank.gif" WIDTH=10 HEIGHT=1 X-CLARIS-USEIMAGEHEIGHT ALIGN=bottom></FONT></P>
</TD>
<TD WIDTH=120>
<P><FONT FACE="Arial"><IMG SRC="../images/blank.gif" WIDTH=120 HEIGHT=1 X-CLARIS-USEIMAGEHEIGHT ALIGN=bottom></FONT></P>
</TD>
<TD WIDTH=30>
<P><FONT FACE="Arial"><IMG SRC="../images/blank.gif" WIDTH=30 HEIGHT=1 X-CLARIS-USEIMAGEHEIGHT ALIGN=bottom></FONT></P>
</TD>
<TD VALIGN=top>
<P><FONT FACE="Arial"><IMG SRC="../images/blank.gif" WIDTH=30 HEIGHT=1 X-CLARIS-USEIMAGEHEIGHT ALIGN=bottom></FONT></P>
</TD>
</TR>
</TABLE>
</FONT></P>
</TD>
</TR>
</TABLE>
<BR>
Floating Point Numbers</FONT></H1>
<P><FONT FACE="Arial">Floating-point numbers are represented in
computer hardware as base 2 (binary) fractions. For example, the
decimal fraction</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>0.125
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">has value 1/10 + 2/100 + 5/1000, and in the
same way the binary fraction</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>0.001
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">has value 0/2 + 0/4 + 1/8. These two fractions
have identical values, the only real difference being that the first
is written in base 10 fractional notation, and the second in base
2.</FONT></P>
<P><FONT FACE="Arial">Unfortunately, most decimal fractions cannot be
represented exactly as binary fractions. A consequence is that, in
general, the decimal floating-point numbers you enter are only
approximated by the binary floating-point numbers actually stored in
the machine.</FONT></P>
<P><FONT FACE="Arial">The problem is easier to understand at first in
base 10. Consider the fraction 1/3. You can approximate that as a
base 10 fraction:</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>0.3
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">or, better,</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>0.33
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">or, better,</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>0.333
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">and so on. No matter how many digits you're
willing to write down, the result will never be exactly 1/3, but will
be an increasingly better approximation to 1/3.</FONT></P>
<P><FONT FACE="Arial">In the same way, no matter how many base 2
digits you're willing to use, the decimal value 0.1 cannot be
represented exactly as a base 2 fraction. In base 2, 1/10 is the
infinitely repeating fraction</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>0.0001100110011001100110011001100110011001100110011...
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">Stop at any finite number of bits, and you get
an approximation. This is why you see things like:</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>>>> 0.1
0.10000000000000001
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">On most machines today, that is what you'll see
if you enter 0.1 at a Python prompt. You may not, though, because the
number of bits used by the hardware to store floating-point values
can vary across machines, and Python only prints a decimal
approximation to the true decimal value of the binary approximation
stored by the machine. On most machines, if Python were to print the
true decimal value of the binary approximation stored for 0.1, it
would have to display</FONT></P>
<P><FONT FACE="Arial"> </FONT></P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>>>> 0.1
0.1000000000000000055511151231257827021181583404541015625
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">instead! The Python prompt (implicitly) uses
the builtin <TT>repr()</TT> function to obtain a string version of
everything it displays. For floats,
<CODE>repr(<VAR>float</VAR>)</CODE> rounds the true decimal value to
17 significant digits, giving</FONT></P>
<P><FONT FACE="Arial"> </FONT></P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>0.10000000000000001
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial"><CODE>repr(<VAR>float</VAR>)</CODE> produces 17
significant digits because it turns out that's enough (on most
machines) so that <CODE>eval(repr(<VAR>x</VAR>)) ==
<VAR>x</VAR></CODE> exactly for all finite floats <VAR>x</VAR>, but
rounding to 16 digits is not enough to make that true.</FONT></P>
<P><FONT FACE="Arial">Note that this is in the very nature of binary
floating-point: this is not a bug in Python, it is not a bug in your
code either, and you'll see the same kind of thing in all languages
that support your hardware's floating-point arithmetic (although some
languages may not <I>display</I> the difference by default, or in all
output modes).</FONT></P>
<P><FONT FACE="Arial">Python's builtin <TT>str()</TT> function
produces only 12 significant digits, and you may wish to use that
instead. It's unusual for <CODE>eval(str(<VAR>x</VAR>))</CODE> to
reproduce <VAR>x</VAR>, but the output may be more pleasant to look
at:</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>>>> print str(0.1)
0.1
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">It's important to realize that this is, in a
real sense, an illusion: the value in the machine is not exactly
1/10, you're simply rounding the <I>display</I> of the true machine
value.</FONT></P>
<P><FONT FACE="Arial">Other surprises follow from this one. For
example, after seeing</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>>>> 0.1
0.10000000000000001
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">you may be tempted to use the <TT>round()</TT>
function to chop it back to the single digit you expect. But that
makes no difference:</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>>>> round(0.1, 1)
0.10000000000000001
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">The problem is that the binary floating-point
value stored for "0.1" was already the best possible binary
approximation to 1/10, so trying to round it again can't make it
better: it was already as good as it gets.</FONT></P>
<P><FONT FACE="Arial">Another consequence is that since 0.1 is not
exactly 1/10, adding 0.1 to itself 10 times may not yield exactly
1.0, either:</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>>>> sum = 0.0
>>> for i in range(10):
... sum += 0.1
...
>>> sum
0.99999999999999989
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">Binary floating-point arithmetic holds many
surprises like this. The problem with "0.1" is explained in precise
detail below, in the "Representation Error" section. See
</FONT><A HREF="http://www.lahey.com/float.htm" title="The Perils of Floating
Point"><FONT FACE="Arial"><EM>The Perils of Floating
Point</EM></FONT></A><FONT FACE="Arial"> for a more complete account
of other common surprises.</FONT></P>
<P><FONT FACE="Arial">As that says near the end, ``there are no easy
answers.'' Still, don't be unduly wary of floating-point! The errors
in Python float operations are inherited from the floating-point
hardware, and on most machines are on the order of no more than 1
part in 2**53 per operation. That's more than adequate for most
tasks, but you do need to keep in mind that it's not decimal
arithmetic, and that every float operation can suffer a new rounding
error.</FONT></P>
<P><FONT FACE="Arial">While pathological cases do exist, for most
casual use of floating-point arithmetic you'll see the result you
expect in the end if you simply round the display of your final
results to the number of decimal digits you expect. <TT>str()</TT>
usually suffices, and for finer control see the discussion of
Pythons's <CODE>%</CODE> format operator: the <CODE>%g</CODE>,
<CODE>%f</CODE> and <CODE>%e</CODE> format codes supply flexible and
easy ways to round float results for display.</FONT></P>
<P> </P>
<H2><FONT FACE="Arial" COLOR="#000099">
<HR>
Representation Error</FONT></H2>
<P><FONT FACE="Arial">This section explains the ``0.1'' example in
detail, and shows how you can perform an exact analysis of cases like
this yourself. Basic familiarity with binary floating-point
representation is assumed.</FONT></P>
<P><FONT FACE="Arial"><I>Representation error</I> refers to that some
(most, actually) decimal fractions cannot be represented exactly as
binary (base 2) fractions. This is the chief reason why Python (or
Perl, C, C++, Java, Fortran, and many others) often won't display the
exact decimal number you expect:</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>>>> 0.1
0.10000000000000001
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">Why is that? 1/10 is not exactly representable
as a binary fraction. Almost all machines today (November 2000) use
IEEE-754 floating point arithmetic, and almost all platforms map
Python floats to IEEE-754 "double precision". 754 doubles contain 53
bits of precision, so on input the computer strives to convert 0.1 to
the closest fraction it can of the form <VAR>J</VAR>/2**<VAR>N</VAR>
where <VAR>J</VAR> is an integer containing exactly 53 bits.
Rewriting</FONT></P>
<P><FONT FACE="Arial"> </FONT></P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim> 1 / 10 ~= J / (2**N)
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">as</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>J ~= 2**N / 10
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">and recalling that <VAR>J</VAR> has exactly 53
bits (is <CODE>>= 2**52</CODE> but <CODE>< 2**53</CODE>), the
best value for <VAR>N</VAR> is 56:</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>>>> 2L**52
4503599627370496L
>>> 2L**53
9007199254740992L
>>> 2L**56/10
7205759403792793L
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">That is, 56 is the only value for <VAR>N</VAR>
that leaves <VAR>J</VAR> with exactly 53 bits. The best possible
value for <VAR>J</VAR> is then that quotient rounded:</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>>>> q, r = divmod(2L**56, 10)
>>> r
6L
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">Since the remainder is more than half of 10,
the best approximation is obtained by rounding up:</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>>>> q+1
7205759403792794L
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">Therefore the best possible approximation to
1/10 in 754 double precision is that over 2**56, or</FONT></P>
<P><FONT FACE="Arial"> </FONT></P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>7205759403792794 / 72057594037927936
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">Note that since we rounded up, this is actually
a little bit larger than 1/10; if we had not rounded up, the quotient
would have been a little bit smaller than 1/10. But in no case can it
be <I>exactly</I> 1/10!</FONT></P>
<P><FONT FACE="Arial">So the computer never ``sees'' 1/10: what it
sees is the exact fraction given above, the best 754 double
approximation it can get:</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>>>> .1 * 2L**56
7205759403792794.0
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">If we multiply that fraction by 10**30, we can
see the (truncated) value of its 30 most significant decimal
digits:</FONT></P>
<P> </P>
<BLOCKQUOTE class=verbatim><PRE class=verbatim>>>> 7205759403792794L * 10L**30 / 2L**56
100000000000000005551115123125L
</PRE></BLOCKQUOTE>
<P><FONT FACE="Arial">meaning that the exact number stored in the
computer is approximately equal to the decimal value
0.100000000000000005551115123125. Rounding that to 17 significant
digits gives the 0.10000000000000001 that Python displays (well, will
display on any 754-conforming platform that does best-possible input
and output conversions in its C library -- yours may
not!).</FONT></P>
<H2> </H2>
<H2><FONT FACE="Arial" COLOR="#0033FF">
<HR>
What's Next?</FONT></H2>
<P><FONT FACE="Arial">Choose one of the other chapters in the
Scripting Guide:</FONT></P>
<OL>
<LI><A HREF="#Data"><FONT FACE="Arial">DataTypes</FONT></A></LI>
<LI><A HREF="flow_control.html"><FONT FACE="Arial">Flow Control
and Functions</FONT></A></LI>
<LI><A HREF="data_structures.html"><FONT FACE="Arial">Data
Structures</FONT></A></LI>
<LI><A HREF="modules.html"><FONT FACE="Arial">Modules</FONT></A></LI>
<LI><A HREF="input_output.html"><FONT FACE="Arial">Input and
Output</FONT></A></LI>
<LI><A HREF="exceptions.html"><FONT FACE="Arial">Errors and
Exceptions</FONT></A></LI>
<LI><A HREF="classes.html"><FONT FACE="Arial">Classes</FONT></A></LI>
<LI><A HREF="floating_point.html"><FONT FACE="Arial">Floating
Point Numbers</FONT></A></LI>
</OL>
<PRE>
<FONT FACE="Arial"> </FONT></PRE>
<P><FONT FACE="Arial"><TABLE BORDER=0 BGCOLOR="#0000CC" CELLSPACING=1 CELLPADDING=0 WIDTH="100%">
<TR>
<TD>
<P><FONT FACE="Arial"><TABLE BORDER=0 BGCOLOR="#FFFFCC" CELLPADDING=0 WIDTH="100%">
<TR>
<TD WIDTH=10>
<P><FONT FACE="Arial"><IMG SRC="../images/blank.gif" WIDTH=10 HEIGHT=1 X-CLARIS-USEIMAGEHEIGHT ALIGN=bottom></FONT></P>
</TD>
<TD WIDTH=120>
<P><A HREF="http://www.pushtotest.com"><FONT FACE="Arial"><IMG SRC="../images/ptt_logo_120w.gif" WIDTH=120 HEIGHT=32 X-CLARIS-USEIMAGEWIDTH X-CLARIS-USEIMAGEHEIGHT BORDER=0 ALIGN=bottom></FONT></A></P>
</TD>
<TD WIDTH=30>
<P><FONT FACE="Arial"><IMG SRC="../images/blank.gif" WIDTH=30 HEIGHT=1 X-CLARIS-USEIMAGEHEIGHT ALIGN=bottom></FONT></P>
</TD>
<TD>
<P><FONT SIZE="-1" FACE="Arial">Additional
documentation, product downloads and updates are at
</FONT><A HREF="http://www.pushtotest.com"><FONT SIZE="-1" FACE="Arial">www.PushToTest.com</FONT></A><FONT SIZE="-1" FACE="Arial">.
While the TestMaker software is distributed under
an open-source license, the documentation remains
(c) 2002 PushToTest. All rights
reserved.</FONT></P>
</TD>
</TR>
</TABLE>
</FONT></P>
</TD>
</TR>
</TABLE>
</FONT></P>
</BODY>
</HTML>