bug#22122: {python, python2}-statsmodels on i686: "SVD did not converge"

bug#22122: {python, python2}-statsmodels on i686: "SVD did not converge"

[Mark H Weaver]

Both python-statsmodels and python2-statsmodels sometimes fail on i686
during the "statsmodels.tsa.tests.test_arima.test_start_params_bug" test
with the error "LinAlgError: SVD did not converge".  Here's an excerpt
of one such error:

--8<---------------cut here---------------start------------->8---
statsmodels.tsa.tests.test_arima.test_compare_arma ... ok
statsmodels.tsa.tests.test_arima.test_reset_trend ... ok
statsmodels.tsa.tests.test_arima.test_start_params_bug ... ERROR

======================================================================
ERROR: statsmodels.tsa.tests.test_arima.test_start_params_bug
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/gnu/store/vp0l7cx9j7gzrd2kr0iass3nzvlc5yq3-python-nose-1.3.4/lib/python3.4/site-packages/nose-1.3.4-py3.4.egg/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/gnu/store/ny8ihp8q2wkffg978wwrzphxgr3s8xj5-python-statsmodels-0.6.1/lib/python3.4/site-packages/statsmodels-0.6.1-py3.4-linux-i686.egg/statsmodels/tsa/tests/test_arima.py", line 535, in test_start_params_bug
    res = ARMA(data, order=(4,1)).fit(disp=-1)
  File "/gnu/store/ny8ihp8q2wkffg978wwrzphxgr3s8xj5-python-statsmodels-0.6.1/lib/python3.4/site-packages/statsmodels-0.6.1-py3.4-linux-i686.egg/statsmodels/tsa/arima_model.py", line 932, in fit
    callback=callback, **kwargs)
  File "/gnu/store/ny8ihp8q2wkffg978wwrzphxgr3s8xj5-python-statsmodels-0.6.1/lib/python3.4/site-packages/statsmodels-0.6.1-py3.4-linux-i686.egg/statsmodels/base/model.py", line 425, in fit
    full_output=full_output)
  File "/gnu/store/ny8ihp8q2wkffg978wwrzphxgr3s8xj5-python-statsmodels-0.6.1/lib/python3.4/site-packages/statsmodels-0.6.1-py3.4-linux-i686.egg/statsmodels/base/optimizer.py", line 184, in _fit
    hess=hessian)
  File "/gnu/store/ny8ihp8q2wkffg978wwrzphxgr3s8xj5-python-statsmodels-0.6.1/lib/python3.4/site-packages/statsmodels-0.6.1-py3.4-linux-i686.egg/statsmodels/base/optimizer.py", line 382, in _fit_lbfgs
    **extra_kwargs)
  File "/gnu/store/1bx2plf9pmhrfvgb2d36321hr9grh2vg-python-scipy-0.16.0/lib/python3.4/site-packages/scipy/optimize/lbfgsb.py", line 188, in fmin_l_bfgs_b
    **opts)
  File "/gnu/store/1bx2plf9pmhrfvgb2d36321hr9grh2vg-python-scipy-0.16.0/lib/python3.4/site-packages/scipy/optimize/lbfgsb.py", line 320, in _minimize_lbfgsb
    f, g = func_and_grad(x)
  File "/gnu/store/1bx2plf9pmhrfvgb2d36321hr9grh2vg-python-scipy-0.16.0/lib/python3.4/site-packages/scipy/optimize/lbfgsb.py", line 266, in func_and_grad
    f = fun(x, *args)
  File "/gnu/store/1bx2plf9pmhrfvgb2d36321hr9grh2vg-python-scipy-0.16.0/lib/python3.4/site-packages/scipy/optimize/optimize.py", line 285, in function_wrapper
    return function(*(wrapper_args + args))
  File "/gnu/store/ny8ihp8q2wkffg978wwrzphxgr3s8xj5-python-statsmodels-0.6.1/lib/python3.4/site-packages/statsmodels-0.6.1-py3.4-linux-i686.egg/statsmodels/base/model.py", line 403, in <lambda>
    f = lambda params, *args: -self.loglike(params, *args) / nobs
  File "/gnu/store/ny8ihp8q2wkffg978wwrzphxgr3s8xj5-python-statsmodels-0.6.1/lib/python3.4/site-packages/statsmodels-0.6.1-py3.4-linux-i686.egg/statsmodels/tsa/arima_model.py", line 761, in loglike
    return self.loglike_kalman(params, set_sigma2)
  File "/gnu/store/ny8ihp8q2wkffg978wwrzphxgr3s8xj5-python-statsmodels-0.6.1/lib/python3.4/site-packages/statsmodels-0.6.1-py3.4-linux-i686.egg/statsmodels/tsa/arima_model.py", line 771, in loglike_kalman
    return KalmanFilter.loglike(params, self, set_sigma2)
  File "/gnu/store/ny8ihp8q2wkffg978wwrzphxgr3s8xj5-python-statsmodels-0.6.1/lib/python3.4/site-packages/statsmodels-0.6.1-py3.4-linux-i686.egg/statsmodels/tsa/kalmanf/kalmanfilter.py", line 649, in loglike
    R_mat, T_mat)
  File "statsmodels/tsa/kalmanf/kalman_loglike.pyx", line 342, in statsmodels.tsa.kalmanf.kalman_loglike.kalman_loglike_double (statsmodels/tsa/kalmanf/kalman_loglike.c:5245)
  File "statsmodels/tsa/kalmanf/kalman_loglike.pyx", line 74, in statsmodels.tsa.kalmanf.kalman_loglike.kalman_filter_double (statsmodels/tsa/kalmanf/kalman_loglike.c:2572)
  File "/gnu/store/mc60m1v7d0m7qdfsmhwf2k1qadhkdggm-python-numpy-1.9.1/lib/python3.4/site-packages/numpy/linalg/linalg.py", line 1585, in pinv
    u, s, vt = svd(a, 0)
  File "/gnu/store/mc60m1v7d0m7qdfsmhwf2k1qadhkdggm-python-numpy-1.9.1/lib/python3.4/site-packages/numpy/linalg/linalg.py", line 1327, in svd
    u, s, vt = gufunc(a, signature=signature, extobj=extobj)
  File "/gnu/store/mc60m1v7d0m7qdfsmhwf2k1qadhkdggm-python-numpy-1.9.1/lib/python3.4/site-packages/numpy/linalg/linalg.py", line 99, in _raise_linalgerror_svd_nonconvergence
    raise LinAlgError("SVD did not converge")
numpy.linalg.linalg.LinAlgError: SVD did not converge

----------------------------------------------------------------------
Ran 3206 tests in 468.902s

FAILED (SKIP=24, errors=1)
phase `check' failed after 498.0 seconds
--8<---------------cut here---------------end--------------->8---

      Mark

bug#22122: {python, python2}-statsmodels on i686: "SVD did not converge"

[Mark H Weaver]

I analyzed Hydra's database, and discovered that these "SVD did not
converge" test failures on i686 depend only on which machine attempts
the build, and are otherwise deterministic.

hydra.gnunet.org fails 100% of the time.  It has *never* passed this
test on i686.  chapters.gnu.org also fails 100% of the time.

guix.sjd.se consistently passes the test, since at least as far back as
July 2016.

       Mark

bug#22122: {python, python2}-statsmodels on i686: "SVD did not converge"

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> I analyzed Hydra's database, and discovered that these "SVD did not
> converge" test failures on i686 depend only on which machine attempts
> the build, and are otherwise deterministic.
>
> hydra.gnunet.org fails 100% of the time.  It has *never* passed this
> test on i686.  chapters.gnu.org also fails 100% of the time.
>
> guix.sjd.se consistently passes the test, since at least as far back as
> July 2016.

Weird.  Could it be that something in the stack uses ISA extensions?
(What could it be on i686?)

Ludo’.

bug#22122: {python, python2}-statsmodels on i686: "SVD did not converge"

[Maxim Cournoyer]

Hi Mark,

> Both python-statsmodels and python2-statsmodels sometimes fail on i686
> during the "statsmodels.tsa.tests.test_arima.test_start_params_bug" test
> with the error "LinAlgError: SVD did not converge".  Here's an excerpt
> of one such error:

As the test suite as been considered unreliable as a whole and disabled
in the package definition, I'm closing this issue.

Maxim