[PATCH] Update Julia (and use smaller tarball)

[PATCH] Update Julia (and use smaller tarball)

[Ricardo Wurmus]

Hi Guix,

the attached patches update julia to the latest version.  The second of the
two patches makes a couple of changes to allow us to use the much smaller
release tarball.  This made it necessary to add a few tarballs to the inputs
manually.

I think this is nicer now, because we no longer download an unnecessarily
large bundle, and we see much clearer now what we still need to work on in
future updates.

~~ Ricardo

[PATCH 1/2] gnu: julia: Update to 0.4.5.

[Ricardo Wurmus]

* gnu/packages/julia.scm (julia): Update to 0.4.5.
(libuv-julia): Update commit.
---
 gnu/packages/julia.scm | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
index 4e6b871..d358a00 100644
--- a/gnu/packages/julia.scm
+++ b/gnu/packages/julia.scm
@@ -43,8 +43,8 @@
   #:use-module (ice-9 match))
 
 (define libuv-julia
-  (let ((commit "07730c4bd595b4d45a498a8ee0bcd53878ff7c10")
-        (revision "2"))
+  (let ((commit "efb40768b7c7bd9f173a7868f74b92b1c5a61a0e")
+        (revision "3"))
     (package (inherit libuv)
       (name "libuv-julia")
       (version (string-append "0.11.26." revision "-" (string-take commit 8)))
@@ -56,7 +56,7 @@
                 (file-name (string-append name "-" version "-checkout"))
                 (sha256
                  (base32
-                  "1r0d4wb41w8k3bi9sjr7kapk1ag131ss2fbqa4jz4gnrzvw94sqx"))))
+                  "16k6pm2jl0ymz5j4ldxn94imdimahqqfd2izgr3zf1vwyyay77w3"))))
       (build-system gnu-build-system)
       (arguments
        (substitute-keyword-arguments (package-arguments libuv)
@@ -68,7 +68,7 @@
 (define-public julia
   (package
     (name "julia")
-    (version "0.4.3")
+    (version "0.4.5")
     (source (origin
               (method url-fetch)
               (uri (string-append
@@ -76,7 +76,7 @@
                     version "/julia-" version "-full.tar.gz"))
               (sha256
                (base32
-                "1i8k847d8n9v37xg69grpl51dysx36p0phwf0d1qxpsqixdg579b"))))
+                "1nbi78fav5f4zj5332iwm4mfk0qhd5qh61z881q69rvp7b163wyb"))))
     (build-system gnu-build-system)
     (arguments
      `(#:test-target "test"
-- 
2.7.3

[PATCH 2/2] gnu: julia: Use unbundled release tarball.

[Ricardo Wurmus]

* gnu/packages/julia.scm (julia)[source]: Use smaller tarball without
bundled sources.
[inputs]: Add inputs for rmath-julia, suitesparse, objconv, dsfmt, and
virtualenv.
[arguments]: Add phase "prepare-deps" to copy tarballs to their expected
locations.
---
 gnu/packages/julia.scm | 62 +++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 59 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
index d358a00..d5293e3 100644
--- a/gnu/packages/julia.scm
+++ b/gnu/packages/julia.scm
@@ -73,10 +73,10 @@
               (method url-fetch)
               (uri (string-append
                     "https://github.com/JuliaLang/julia/releases/download/v"
-                    version "/julia-" version "-full.tar.gz"))
+                    version "/julia-" version ".tar.gz"))
               (sha256
                (base32
-                "1nbi78fav5f4zj5332iwm4mfk0qhd5qh61z881q69rvp7b163wyb"))))
+                "09gc6yf3v4in0qwhrbgjrjgvblp941di0mli4zax22mvf4dzc7s4"))))
     (build-system gnu-build-system)
     (arguments
      `(#:test-target "test"
@@ -92,6 +92,19 @@
        #:phases
        (modify-phases %standard-phases
          (delete 'configure)
+         (add-after 'unpack 'prepare-deps
+           (lambda* (#:key inputs #:allow-other-keys)
+             (copy-file (assoc-ref inputs "rmath-julia")
+                        "deps/Rmath-julia-0.1.tar.gz")
+             (copy-file (assoc-ref inputs "dsfmt")
+                        "deps/dsfmt-2.2.3.tar.gz")
+             (copy-file (assoc-ref inputs "objconv")
+                        "deps/objconv.zip")
+             (copy-file (assoc-ref inputs "suitesparse")
+                        "deps/SuiteSparse-4.4.2.tar.gz")
+             (copy-file (assoc-ref inputs "virtualenv")
+                        "deps/virtualenv-1.11.6.tar.gz")
+             #t))
          (add-after 'unpack 'hardcode-soname-map
           ;; ./src/ccall.cpp creates a map from library names to paths using the
           ;; output of "/sbin/ldconfig -p".  Since ldconfig is not used in Guix,
@@ -249,7 +262,50 @@
        ("mpfr" ,mpfr)
        ("wget" ,wget)
        ("which" ,which)
-       ("gmp" ,gmp)))
+       ("gmp" ,gmp)
+       ;; FIXME: The following inputs are downloaded from upstream to allow us
+       ;; to use the lightweight Julia release tarball.  Ideally, these inputs
+       ;; would eventually be replaced with proper Guix packages.
+       ("rmath-julia"
+        ,(origin
+           (method url-fetch)
+           (uri "https://api.github.com/repos/JuliaLang/Rmath-julia/tarball/v0.1")
+           ;;(file-name "rmath-julia-0.1.tar.gz")
+           (sha256
+            (base32
+             "0ai5dhjc43zcvangz123ryxmlbm51s21rg13bllwyn98w67arhb4"))))
+       ("suitesparse"
+        ,(origin
+           (method url-fetch)
+           (uri "http://faculty.cse.tamu.edu/davis/SuiteSparse/SuiteSparse-4.4.2.tar.gz")
+           (sha256
+            (base32
+             "1dg0qsv07n71nbn9cgcvn73933rgy1jnxw5bfqkwfq3bidk44cqc"))))
+       ("objconv"
+        ,(origin
+           (method url-fetch)
+           (uri "http://www.agner.org/optimize/objconv.zip")
+           (sha256
+            (base32
+             "1fi7qa2sd9vb35dvkgripjf0fayzg2qmff215f8agfqfiwd1g8qs"))))
+       ("dsfmt"
+        ,(origin
+           (method url-fetch)
+           (uri (string-append
+                 "http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/"
+                 "SFMT/dSFMT-src-2.2.3.tar.gz"))
+           (sha256
+            (base32
+             "03kaqbjbi6viz0n33dk5jlf6ayxqlsq4804n7kwkndiga9s4hd42"))))
+       ("virtualenv"
+        ,(origin
+           (method url-fetch)
+           (uri (string-append "https://pypi.python.org/packages/24/cc/"
+                               "a3cdf0a49ffcaef483b7e2511476aa520cf7260c199a6928fda6c43ba916/"
+                               "virtualenv-1.11.6.tar.gz"))
+           (sha256
+            (base32
+             "1xq4prmg25n9cz5zcvbqx68lmc3kl39by582vd8pzs9f3qalqyiy"))))))
     (native-inputs
      `(("perl" ,perl)
        ("patchelf" ,patchelf)
-- 
2.7.3

Re: [PATCH] Update Julia (and use smaller tarball)

[Ricardo Wurmus]

I should add that julia compiles with both patches applied.  I didn’t do
any checking beyond that.

~~ Ricardo

Re: [PATCH 2/2] gnu: julia: Use unbundled release tarball.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 4907 bytes --]

On Wed, May 25, 2016 at 05:25:24PM +0200, Ricardo Wurmus wrote:
> * gnu/packages/julia.scm (julia)[source]: Use smaller tarball without
> bundled sources.
> [inputs]: Add inputs for rmath-julia, suitesparse, objconv, dsfmt, and
> virtualenv.
> [arguments]: Add phase "prepare-deps" to copy tarballs to their expected
> locations.
> ---
>  gnu/packages/julia.scm | 62 +++++++++++++++++++++++++++++++++++++++++++++++---
>  1 file changed, 59 insertions(+), 3 deletions(-)
> 
> diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
> index d358a00..d5293e3 100644
> --- a/gnu/packages/julia.scm
> +++ b/gnu/packages/julia.scm
> @@ -73,10 +73,10 @@
>                (method url-fetch)
>                (uri (string-append
>                      "https://github.com/JuliaLang/julia/releases/download/v"
> -                    version "/julia-" version "-full.tar.gz"))
> +                    version "/julia-" version ".tar.gz"))
>                (sha256
>                 (base32
> -                "1nbi78fav5f4zj5332iwm4mfk0qhd5qh61z881q69rvp7b163wyb"))))
> +                "09gc6yf3v4in0qwhrbgjrjgvblp941di0mli4zax22mvf4dzc7s4"))))
>      (build-system gnu-build-system)
>      (arguments
>       `(#:test-target "test"
> @@ -92,6 +92,19 @@
>         #:phases
>         (modify-phases %standard-phases
>           (delete 'configure)
> +         (add-after 'unpack 'prepare-deps
> +           (lambda* (#:key inputs #:allow-other-keys)
> +             (copy-file (assoc-ref inputs "rmath-julia")
> +                        "deps/Rmath-julia-0.1.tar.gz")
> +             (copy-file (assoc-ref inputs "dsfmt")
> +                        "deps/dsfmt-2.2.3.tar.gz")
> +             (copy-file (assoc-ref inputs "objconv")
> +                        "deps/objconv.zip")
> +             (copy-file (assoc-ref inputs "suitesparse")
> +                        "deps/SuiteSparse-4.4.2.tar.gz")
> +             (copy-file (assoc-ref inputs "virtualenv")
> +                        "deps/virtualenv-1.11.6.tar.gz")
> +             #t))
>           (add-after 'unpack 'hardcode-soname-map
>            ;; ./src/ccall.cpp creates a map from library names to paths using the
>            ;; output of "/sbin/ldconfig -p".  Since ldconfig is not used in Guix,
> @@ -249,7 +262,50 @@
>         ("mpfr" ,mpfr)
>         ("wget" ,wget)
>         ("which" ,which)
> -       ("gmp" ,gmp)))
> +       ("gmp" ,gmp)
> +       ;; FIXME: The following inputs are downloaded from upstream to allow us
> +       ;; to use the lightweight Julia release tarball.  Ideally, these inputs
> +       ;; would eventually be replaced with proper Guix packages.
> +       ("rmath-julia"
> +        ,(origin
> +           (method url-fetch)
> +           (uri "https://api.github.com/repos/JuliaLang/Rmath-julia/tarball/v0.1")
> +           ;;(file-name "rmath-julia-0.1.tar.gz")
             ^^^
        did this not like being renamed?

> +           (sha256
> +            (base32
> +             "0ai5dhjc43zcvangz123ryxmlbm51s21rg13bllwyn98w67arhb4"))))
> +       ("suitesparse"
> +        ,(origin
> +           (method url-fetch)
> +           (uri "http://faculty.cse.tamu.edu/davis/SuiteSparse/SuiteSparse-4.4.2.tar.gz")
> +           (sha256
> +            (base32
> +             "1dg0qsv07n71nbn9cgcvn73933rgy1jnxw5bfqkwfq3bidk44cqc"))))
> +       ("objconv"
> +        ,(origin
> +           (method url-fetch)
> +           (uri "http://www.agner.org/optimize/objconv.zip")
> +           (sha256
> +            (base32
> +             "1fi7qa2sd9vb35dvkgripjf0fayzg2qmff215f8agfqfiwd1g8qs"))))
> +       ("dsfmt"
> +        ,(origin
> +           (method url-fetch)
> +           (uri (string-append
> +                 "http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/"
> +                 "SFMT/dSFMT-src-2.2.3.tar.gz"))
> +           (sha256
> +            (base32
> +             "03kaqbjbi6viz0n33dk5jlf6ayxqlsq4804n7kwkndiga9s4hd42"))))
> +       ("virtualenv"

we actually do have virtualenv packaged. I forgot when but at some point
they made a massive jump in version numbers.

> +        ,(origin
> +           (method url-fetch)
> +           (uri (string-append "https://pypi.python.org/packages/24/cc/"
> +                               "a3cdf0a49ffcaef483b7e2511476aa520cf7260c199a6928fda6c43ba916/"
> +                               "virtualenv-1.11.6.tar.gz"))
> +           (sha256
> +            (base32
> +             "1xq4prmg25n9cz5zcvbqx68lmc3kl39by582vd8pzs9f3qalqyiy"))))))
>      (native-inputs
>       `(("perl" ,perl)
>         ("patchelf" ,patchelf)
> -- 
> 2.7.3
> 
> 

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 2/2] gnu: julia: Use unbundled release tarball.

[Ricardo Wurmus]

Efraim Flashner <efraim@flashner.co.il> writes:

>> +       ;; FIXME: The following inputs are downloaded from upstream to allow us
>> +       ;; to use the lightweight Julia release tarball.  Ideally, these inputs
>> +       ;; would eventually be replaced with proper Guix packages.
>> +       ("rmath-julia"
>> +        ,(origin
>> +           (method url-fetch)
>> +           (uri "https://api.github.com/repos/JuliaLang/Rmath-julia/tarball/v0.1")
>> +           ;;(file-name "rmath-julia-0.1.tar.gz")
>              ^^^
>         did this not like being renamed?

Oops, this was not supposed to be commented.  There was a problem where
the build system complained about the hash of the tarball, and I could
only fix it by *not* overriding the name.  But this shouldn’t have been
part of the commit.  I’ll play with this again to see if it can stay or
needs to go.

>> +           (sha256
>> +            (base32
>> +             "0ai5dhjc43zcvangz123ryxmlbm51s21rg13bllwyn98w67arhb4"))))
>> +       ("suitesparse"
>> +        ,(origin
>> +           (method url-fetch)
>> +           (uri "http://faculty.cse.tamu.edu/davis/SuiteSparse/SuiteSparse-4.4.2.tar.gz")
>> +           (sha256
>> +            (base32
>> +             "1dg0qsv07n71nbn9cgcvn73933rgy1jnxw5bfqkwfq3bidk44cqc"))))
>> +       ("objconv"
>> +        ,(origin
>> +           (method url-fetch)
>> +           (uri "http://www.agner.org/optimize/objconv.zip")
>> +           (sha256
>> +            (base32
>> +             "1fi7qa2sd9vb35dvkgripjf0fayzg2qmff215f8agfqfiwd1g8qs"))))
>> +       ("dsfmt"
>> +        ,(origin
>> +           (method url-fetch)
>> +           (uri (string-append
>> +                 "http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/"
>> +                 "SFMT/dSFMT-src-2.2.3.tar.gz"))
>> +           (sha256
>> +            (base32
>> +             "03kaqbjbi6viz0n33dk5jlf6ayxqlsq4804n7kwkndiga9s4hd42"))))
>> +       ("virtualenv"
>
> we actually do have virtualenv packaged. I forgot when but at some point
> they made a massive jump in version numbers.

Yes, I’ve seen that we have it, but it looks like julia doesn’t make it
easy to use our packaged version (and it insists on this old version).
We should fix this eventually, but the goal of this patch was to make
this dependency explicit — before that it would just quietly take the
bundled “virtualenv” in the fat tarball.

~~ Ricardo

Re: [PATCH 1/2] gnu: julia: Update to 0.4.5.

[Ludovic Courtès]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> * gnu/packages/julia.scm (julia): Update to 0.4.5.
> (libuv-julia): Update commit.

LGTM.

Ludo’.

Re: [PATCH 2/2] gnu: julia: Use unbundled release tarball.

[Ludovic Courtès]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> Efraim Flashner <efraim@flashner.co.il> writes:
>
>>> +       ;; FIXME: The following inputs are downloaded from upstream to allow us
>>> +       ;; to use the lightweight Julia release tarball.  Ideally, these inputs
>>> +       ;; would eventually be replaced with proper Guix packages.
>>> +       ("rmath-julia"
>>> +        ,(origin
>>> +           (method url-fetch)
>>> +           (uri "https://api.github.com/repos/JuliaLang/Rmath-julia/tarball/v0.1")
>>> +           ;;(file-name "rmath-julia-0.1.tar.gz")
>>              ^^^
>>         did this not like being renamed?
>
> Oops, this was not supposed to be commented.  There was a problem where
> the build system complained about the hash of the tarball, and I could
> only fix it by *not* overriding the name.  But this shouldn’t have been
> part of the commit.  I’ll play with this again to see if it can stay or
> needs to go.
>
>>> +           (sha256
>>> +            (base32
>>> +             "0ai5dhjc43zcvangz123ryxmlbm51s21rg13bllwyn98w67arhb4"))))
>>> +       ("suitesparse"
>>> +        ,(origin
>>> +           (method url-fetch)
>>> +           (uri "http://faculty.cse.tamu.edu/davis/SuiteSparse/SuiteSparse-4.4.2.tar.gz")
>>> +           (sha256
>>> +            (base32
>>> +             "1dg0qsv07n71nbn9cgcvn73933rgy1jnxw5bfqkwfq3bidk44cqc"))))
>>> +       ("objconv"
>>> +        ,(origin
>>> +           (method url-fetch)
>>> +           (uri "http://www.agner.org/optimize/objconv.zip")
>>> +           (sha256
>>> +            (base32
>>> +             "1fi7qa2sd9vb35dvkgripjf0fayzg2qmff215f8agfqfiwd1g8qs"))))
>>> +       ("dsfmt"
>>> +        ,(origin
>>> +           (method url-fetch)
>>> +           (uri (string-append
>>> +                 "http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/"
>>> +                 "SFMT/dSFMT-src-2.2.3.tar.gz"))
>>> +           (sha256
>>> +            (base32
>>> +             "03kaqbjbi6viz0n33dk5jlf6ayxqlsq4804n7kwkndiga9s4hd42"))))
>>> +       ("virtualenv"
>>
>> we actually do have virtualenv packaged. I forgot when but at some point
>> they made a massive jump in version numbers.
>
> Yes, I’ve seen that we have it, but it looks like julia doesn’t make it
> easy to use our packaged version (and it insists on this old version).
> We should fix this eventually, but the goal of this patch was to make
> this dependency explicit — before that it would just quietly take the
> bundled “virtualenv” in the fat tarball.

This patch LGTM, but I agree that eventually it’d be nice if Julia could
use the packaged versions of these things, at least SuiteSparse and
virtualenv, but probably all of them.

Thanks!

Ludo’.