[bug#41167] Julia: fix current patches, upgrade to 1.4.1

[bug#41167] Julia: fix current patches, upgrade to 1.4.1

[Nicolò Balzarotti]

Hello guix!

In the following patches I:
- fixed current julia: I don't know how, the patch-url is missing
  ".patch" at the end, leading to a 404.  Don't know how it can build
  right now
- fixed the julia-SOURCE_DATE_EPOCH-mtime.patch: It's now a meta-patch,
  that creates the right patch file, so the current julia version is not
  patched, and precompile cache of packages installed with guix is
  broken.  Still no julia-package is in guix, that's why it went
  unnoticed
- Add libwhich (I added it to julia.scm but probably there's a better
  place for it)
- Update julia to latest release, v1.4.1.  I was finally able to
  unbundle the last pieces of bundled software (libwhich).  Now julia
  is based on llvm-8 (previously on llvm-6) and I measuread a speedup of
  ~17x on precompile times
- Add the first julia package, julia-compat.  If everything is fine, I'll
  then submit a big series of packages

Feedback welcome!

Nicolò

[bug#41167] Julia: fix current patches, upgrade to 1.4.1

[zimoun]

Hi Nicolò,

On Sun, 10 May 2020 at 10:08, Nicolò Balzarotti <anothersms@gmail.com> wrote:

> - fixed the julia-SOURCE_DATE_EPOCH-mtime.patch: It's now a meta-patch,
>   that creates the right patch file, so the current julia version is not
>   patched, and precompile cache of packages installed with guix is
>   broken.  Still no julia-package is in guix, that's why it went
>   unnoticed

Does this fix the unreproducible behaviour of Julia? As described in
this bug [1] and discussed at length for v1.3.

> - Update julia to latest release, v1.4.1.  I was finally able to
>   unbundle the last pieces of bundled software (libwhich).  Now julia
>   is based on llvm-8 (previously on llvm-6) and I measuread a speedup of
>   ~17x on precompile times

Because my machine (of lockdown) is not enough powerful, I cannot
rebuild Julia v1.4.1 with the option '--check'.
Could you report the status on the reproducibility of Julia with this
new version?


BTW, thank you for this important update!
Hope to play with it soon.


Cheers,
simon

ps:
hope that your PhD defense was fine?

[bug#41167] Julia: fix current patches, upgrade to 1.4.1

[Nicolò Balzarotti]

Hi Simon!

zimoun <zimon.toutoune@gmail.com> writes:

> Hi Nicolò,
>
> On Sun, 10 May 2020 at 10:08, Nicolò Balzarotti <anothersms@gmail.com> wrote:
>
>> - fixed the julia-SOURCE_DATE_EPOCH-mtime.patch: It's now a meta-patch,
>>   that creates the right patch file, so the current julia version is not
>>   patched, and precompile cache of packages installed with guix is
>>   broken.  Still no julia-package is in guix, that's why it went
>>   unnoticed
>
> Does this fix the unreproducible behaviour of Julia? As described in
> this bug [1] and discussed at length for v1.3.

Unfortunately, not (actually not tested, but no changes were made from
1.4.0).  I've not received feedback on the long issue I opened [1], so I
think I'll propose one fix at a time hoping this way they'll eventually
merge them.

>
>> - Update julia to latest release, v1.4.1.  I was finally able to
>>   unbundle the last pieces of bundled software (libwhich).  Now julia
>>   is based on llvm-8 (previously on llvm-6) and I measuread a speedup of
>>   ~17x on precompile times
>
> Because my machine (of lockdown) is not enough powerful, I cannot
> rebuild Julia v1.4.1 with the option '--check'.
> Could you report the status on the reproducibility of Julia with this
> new version?
>
>
> BTW, thank you for this important update!
> Hope to play with it soon.
A new update that adds tests on julia packages and adding more packages
is coming.  Please, let me know if you have any package you use
day-by-day so that I can add them to verify everything works.

>
>
[1] https://github.com/JuliaLang/julia/issues/34753

> Cheers,
> simon
>
> ps:
> hope that your PhD defense was fine?
My Thesis has been accepted, but due to Covid-related emergency, my
defense has been planned for the last day of this month.

Thanks, Nicolò

[bug#41167] Julia: fix current patches, upgrade to 1.4.1

[zimoun]

Hi Nicolò,

On Wed, 13 May 2020 at 12:13, Nicolò Balzarotti <anothersms@gmail.com> wrote:

> > Does this fix the unreproducible behaviour of Julia? As described in
> > this bug [1] and discussed at length for v1.3.
>
> Unfortunately, not (actually not tested, but no changes were made from
> 1.4.0).  I've not received feedback on the long issue I opened [1], so I
> think I'll propose one fix at a time hoping this way they'll eventually
> merge them.

Thank you for the explanations.
And please report on guix-devel or on 22304@debuugs.gnu.org or
personally the status of your proposals.

BTW, in the meantime of an hypothetical merge, does it make sense to
add a reproducible Julia package using your unmerged patches?

(define-public julia-reproducible
  (package
    (inherit julia)
    (name "julia-reproducible")
    (... patches ...))


> A new update that adds tests on julia packages and adding more packages
> is coming.  Please, let me know if you have any package you use
> day-by-day so that I can add them to verify everything works.

Nothing fancy.
When I am back to ideas of my previous job, I play with Krylov linear
solver stuff: LinearOperator.jl, IterativeSolvers.jl, Krylov.jl and
other.
Or time to time I am doing some experiments with Flux, FFTW, packages
in JuliaLinearAlgebra or JuliaMath.


> > hope that your PhD defense was fine?
> My Thesis has been accepted, but due to Covid-related emergency, my
> defense has been planned for the last day of this month.

Wish you all the best!


Cheers,
simon

[bug#41167] Julia: fix current patches, upgrade to 1.4.1

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1815 bytes --]

On Wed, May 13, 2020 at 02:11:49AM +0200, zimoun wrote:
> Hi Nicolò,
> 
> On Sun, 10 May 2020 at 10:08, Nicolò Balzarotti <anothersms@gmail.com> wrote:
> 
> > - fixed the julia-SOURCE_DATE_EPOCH-mtime.patch: It's now a meta-patch,
> >   that creates the right patch file, so the current julia version is not
> >   patched, and precompile cache of packages installed with guix is
> >   broken.  Still no julia-package is in guix, that's why it went
> >   unnoticed
> 
> Does this fix the unreproducible behaviour of Julia? As described in
> this bug [1] and discussed at length for v1.3.
> 
> > - Update julia to latest release, v1.4.1.  I was finally able to
> >   unbundle the last pieces of bundled software (libwhich).  Now julia
> >   is based on llvm-8 (previously on llvm-6) and I measuread a speedup of
> >   ~17x on precompile times
> 
> Because my machine (of lockdown) is not enough powerful, I cannot
> rebuild Julia v1.4.1 with the option '--check'.
> Could you report the status on the reproducibility of Julia with this
> new version?

guix build: error: derivation `/gnu/store/z21c12ijspavaclcins5pja8z1s8wbm0-julia-1.4.1.drv' may not be deterministic: output `/gnu/store/ri931fk8hf5wvagk76zhmwbphq2pwgnp-julia-1.4.1' differs from ?/gnu/store/ri931fk8hf5wvagk76zhmwbphq2pwgnp-julia-1.4.1-check?

I built it on bayfront twice. I'll get the diffoscope output once it
finally builds.

> 
> 
> BTW, thank you for this important update!
> Hope to play with it soon.
> 
> 
> Cheers,
> simon
> 
> ps:
> hope that your PhD defense was fine?
> 
> 
> 

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#22304: Julia v1.4.1 still not reproducible

[zimoun]

Hi Efraim,

On Wed, 13 May 2020 at 13:04, Efraim Flashner <efraim@flashner.co.il> wrote:

> guix build: error: derivation `/gnu/store/z21c12ijspavaclcins5pja8z1s8wbm0-julia-1.4.1.drv' may not be deterministic: output `/gnu/store/ri931fk8hf5wvagk76zhmwbphq2pwgnp-julia-1.4.1' differs from ?/gnu/store/ri931fk8hf5wvagk76zhmwbphq2pwgnp-julia-1.4.1-check?
>
> I built it on bayfront twice. I'll get the diffoscope output once it
> finally builds.

Thank you.

Cheers,
simon