‘core-updates-frozen’: julia status

[zimoun <zimon.toutoune@gmail.com>]

Hi,

On Mon, 22 Nov 2021 at 15:13, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
> zimoun <zimon.toutoune@gmail.com> writes:

>> Julia is annoying,  Because the test suite sometimes passes, sometimes
>> not.  Well, I am not even sure it is the same part that fails.  I
>> notice that "guix build julia --without-tests=julia" then "guix build
>> julia" tends to works. :-)
>>
>> Any idea to debug this?
>
> No idea how to debug this, but when faced with nondeterministic tests,
> here's how I proceed:
>
> 1. Report them upstream.
> 2. Disable the problematic tests.  You may need to disable a whole test suite of
> similarly faulted tests, if they all behave the same.

After this hacking day, Julia builds deterministically* on
core-updates-frozen.  Along the way, now the test suite runs in parallel
which reduces a lot the feedback loop. :-) And a variant of the current
patch of julia source is upstream PR.

*deterministically: at least on 2 machines. ;-)

One or two failing tests randomly failing should be investigated and
probably reported upstream.

The test suite of julia-* packages supports also (more or less) some
parallelism.  From my understanding, the limitation comes from the
script 'test/runtests.jl' from each package; each implements more or
less some parallelism.  Now, the environment variable JULIA_CPU_THREADS
is set but Julia does not always honor it as its doc specifies – see the
patch fixing the parallelism of Julia test suite ;-).  Therefore, the
test suite of each package is launched with
'--procs=(parallel-job-count)' and Julia exploits at best the available
workers.

I am currently rebuilding all the julia-* packages.  It is high probable
some are now broken.  But it appears to me not a blocker for the merge.

On a side note, Julia is not currently reproducible [1] and
'julia-build-system' neither [2].  Therefore, probable non-reproducible
build introduced by parallelism is currently not the point, IMHO.



Thanks Maxim for the help and the good vibes to tackle that.

Cheers,
simon

1: <http://issues.guix.gnu.org/issue/22304>
2: <http://issues.guix.gnu.org/issue/47354>