bug#22049: Test failure of ilmbase-2.2.0 on i686-linux (testBoxAlgo.cpp)

bug#22049: Test failure of ilmbase-2.2.0 on i686-linux (testBoxAlgo.cpp)

[Mark H Weaver]

[added 22049@debbugs.gnu.org to the CC list]

Leo Famulari <leo@famulari.name> writes:

> Greetings from Guix! [0]
>
> We're having trouble building ilmbase-2.2.0 for the i686 architecture on
> Linux, with gcc-4.9.3.
>
> The build process fails during testing. Specifically, it fails
> testBoxAlgo, like this:
>
> ImathTest: testBoxAlgo.cpp:892: void {anonymous}::boxMatrixTransform(): Assertion `b21 == b2' failed.
> /gnu/store/isxqjfaglyfsbcv75y8qbqbph8v28ykr-bash-4.3.39/bin/bash: line 5:  4565 Aborted                 ${dir}$tst
>
> On our mailing list, this was suggested as the nature of the problem
> [1]:
> On Mon, Nov 30, 2015 at 10:14:49PM +0200, Ludovic Courtès wrote:
>> Right.  This sounds very much like a rounding issue, whereby the
>> epsilon in floating-point number comparisons is to strict for 32-bit
>> machines.

Given that ilmbase builds successfully in Guix on x86_64, mips64el, and
armhf, and only fails on i686, I believe that Ludovic's suggestion is
right on the mark.

The issue is that the x87 instruction set (used on 32-bit Intel systems
without SSE) uses 80-bit double-extended precision internally.  When
these 80-bit results are later converted to 64-bit doubles, they are
rounded a second time.  This "double rounding" results in larger
round-off errors than would occur when rounding only once to 64-bit
doubles, as is done when using x86_64, SSE2, or other architectures.
For more on this, see:

  https://en.wikipedia.org/wiki/Rounding#Double_rounding

Quoting from that page:

  Some computer languages and the IEEE 754-2008 standard dictate that in
  straightforward calculations the result should not be rounded twice.
  This has been a particular problem with Java as it is designed to be
  run identically on different machines, special programming tricks have
  had to be used to achieve this with x87 floating point.[1][2]

  [1] Samuel A. Figueroa (July 1995). "When is double rounding
      innocuous?". ACM SIGNUM Newsletter (ACM) 30 (3):
      21–25. doi:10.1145/221332.221334.

  [2] Roger Golliver (October 1998). "Efficiently producing default
      orthogonal IEEE double results using extended IEEE
      hardware". Intel.
      <http://www.open-std.org/JTC1/SC22/JSG/docs/m3/docs/jsgn326.pdf>

Hope this helps,

      Mark