From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:55018) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iuCns-00068i-9Y for guix-patches@gnu.org; Wed, 22 Jan 2020 05:00:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iuCnr-0001bb-0H for guix-patches@gnu.org; Wed, 22 Jan 2020 05:00:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:43056) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iuCnq-0001bU-TK for guix-patches@gnu.org; Wed, 22 Jan 2020 05:00:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1iuCnq-0002LU-PK for guix-patches@gnu.org; Wed, 22 Jan 2020 05:00:02 -0500 Subject: [bug#38546] [00/11] Update Julia, Fix precompilation, add HTTP.jl Resent-Message-ID: From: =?UTF-8?Q?Nicol=C3=B2?= Balzarotti In-Reply-To: References: <87v9qpvbyr.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> <87wo9ouaq5.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> <87d0bcly8f.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> Date: Wed, 22 Jan 2020 10:59:49 +0100 Message-ID: <877e1jdd62.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: zimoun Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 38546@debbugs.gnu.org Hi Simon, zimoun writes: > Hi Nicol=C3=B2, > > Cool that you figured out a source of non-reproducibility. > > > On Tue, 21 Jan 2020 at 14:45, Nicol=C3=B2 Balzarotti wrote: > >> Sorry, I forgot to send the dsfmt patch. Also, julia's >> SOURCE_DATE_EPOCH patch was named differently. I've fixed this in theatt= ached patches. You need to apply Add-dsfmt.patch, Update-to-1.3.1 >> and then julia-use-SOURCE_DATE_EPOCH. > > This patch 'julia-SOURCE_DATE_EPOCH-mtime.patch' is the one you > mentioned here [#], right? > > Could you send it as an upstream PR? > > [#] https://github.com/JuliaLang/julia/issues/34115#issuecomment-568171025 > > >> About reproducibility: if I'm not wrong, sys.so contains Base library >> precompiled ([1]). Precompilation is still non deterministic (here's >> [2] an issue on github). Something I did to check precompilation: > > I am not sure to well understand the source of non-determinism. > > Does the patch about SOURCE_DATE_EPOCH fix the issue of [1] and [2]? > Or is it something else? > The first patch (the one I mention in [#]) fixes _one source of_ non-determinism (the stored mtime inside the precompile cache). But others are present (precompiling the same file 2 times lead to differences even in file size, something I reported on [2]). What I'm working on is trying to find out what other sources are and patch them, too. I patched some c code (src/support/timefuncs.c) so that it follows SOURCE_DATE_EPOCH too. This finally lead to same-size files. I'm now working on parsing binary (.ji) files directly with julia built in function to have a textual representation of the differences. I can't send SOURCE_DATE_EPOCH patches upstream until [3] is merged, as patching timefuncts leads to an endless test suite (as they were using time() to check for passed time. If time() always returns 1, tests sleeps forever). Yesterday evening I completed that patch (and has been approved). My plans are on finding the hopefully last source of non-determinism, patch it and send everything upstream (hoping they care). > >> I could not get the same results twice (also, size differs). I'll work >> on this on some spare time (for example, there are other places where >> SOURCE_DATE_EPOCH can be used, but this [3] is a problem I need to >> solve first). > > Is the problem [3] not solved by 'julia-SOURCE_DATE_EPOCH-mtime.patch'? > > > >> Maybe 1.3.1 (when reviewed) can be merged, since we have >> the same problem with julia 1.1, but we can wait for the >> source-date-epoch and julia-xyz patches until we solve this. > > My opinion is: if a patch is floating around to fix the > source-date-epoch issue, let try to push it upstream. If it is > rejected, let talk later if Guix will include it or not. And in the > meantime, I will try to review the 1.3.1 because yes I agree that it > should be included even if we know it is not reproducible -- the > package Guitarix [@] is updated and not reproducible neither. > Thanks! I hope in a 1.4 release where everything is fixed ;) Nicol=C3=B2 > [@] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D21803 > > > Thank you for working on this. > > > All the best, > simon > > >> [1] https://docs.julialang.org/en/v1/devdocs/sysimg/ >> [2] https://github.com/JuliaLang/julia/issues/25900 >> [3] https://github.com/JuliaLang/julia/issues/34115