* [bug#45972] [PATCH] Add julia-json with dependencies @ 2021-01-18 23:47 Nicolò Balzarotti 2021-01-26 21:33 ` Ludovic Courtès 0 siblings, 1 reply; 14+ messages in thread From: Nicolò Balzarotti @ 2021-01-18 23:47 UTC (permalink / raw) To: 45972 [-- Attachment #1: Type: text/plain, Size: 144 bytes --] Hi guix! This patch series adds julia-json together with its dependencies. It also enable tests for julia packages. Thanks, Nicolò [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-guix-build-system-julia-Enable-tests.patch --] [-- Type: text/x-patch, Size: 3140 bytes --] From a374ec23346cca7ff6f28c8141102f8838db649a Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 23:56:54 +0100 Subject: [PATCH 1/9] guix: build-system (julia): Enable tests. * guix/build-system/julia.scm (julia-build): Set tests? default to #t. * guix/build/julia-build-system.scm (check): Respect tests? and fix julia invocation. (%standard-phases): Add check phase after install. --- guix/build-system/julia.scm | 2 +- guix/build/julia-build-system.scm | 27 +++++++++++++++++---------- 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/guix/build-system/julia.scm b/guix/build-system/julia.scm index 488fe9bb1d..d3cb41c054 100644 --- a/guix/build-system/julia.scm +++ b/guix/build-system/julia.scm @@ -75,7 +75,7 @@ (define* (julia-build store name inputs #:key source - (tests? #f) + (tests? #t) (phases '(@ (guix build julia-build-system) %standard-phases)) (outputs '("out")) diff --git a/guix/build/julia-build-system.scm b/guix/build/julia-build-system.scm index e8ebcf8ba0..61817e0b47 100644 --- a/guix/build/julia-build-system.scm +++ b/guix/build/julia-build-system.scm @@ -69,15 +69,22 @@ (string-append "pushfirst!(DEPOT_PATH, pop!(DEPOT_PATH)); using " package))) #t) -(define* (check #:key source inputs outputs #:allow-other-keys) - (let* ((out (assoc-ref outputs "out")) - (package (strip-store-file-name source)) - (builddir (string-append out "/share/julia/"))) - ;; With a patch, SOURCE_DATE_EPOCH is honored - (setenv "SOURCE_DATE_EPOCH" "1") - (setenv "JULIA_DEPOT_PATH" builddir) - (setenv "JULIA_LOAD_PATH" (string-append builddir "packages/")) - (invoke-julia (string-append "using Pkg;Pkg.test(\"" package "\")"))) +(define* (check #:key tests? source inputs outputs #:allow-other-keys) + (when tests? + (let* ((out (assoc-ref outputs "out")) + (package (strip-store-file-name source)) + (builddir (string-append out "/share/julia/"))) + ;; With a patch, SOURCE_DATE_EPOCH is honored + (setenv "SOURCE_DATE_EPOCH" "1") + (setenv "JULIA_DEPOT_PATH" builddir) + (setenv "JULIA_LOAD_PATH" + (string-append builddir "packages/" ":" + (or (getenv "JULIA_LOAD_PATH") + ""))) + (setenv "HOME" "/tmp") + (invoke "julia" + (string-append builddir "packages/" + package "/test/runtests.jl")))) #t) (define (julia-create-package-toml outputs source @@ -112,7 +119,7 @@ version = \"" version "\" (delete 'check) ; tests must be run after installation (replace 'install install) (add-after 'install 'precompile precompile) - ;; (add-after 'install 'check check) + (add-after 'install 'check check) ;; TODO: In the future we could add a "system-image-generation" phase ;; where we use PackageCompiler.jl to speed up package loading times (delete 'configure) -- 2.29.2 [-- Attachment #3: 0002-gnu-julia-compat-Update-to-3.25.0.patch --] [-- Type: text/x-patch, Size: 1428 bytes --] From 46cc28fe6d532c64006b6fea88604b198acec278 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 17:42:50 +0100 Subject: [PATCH 2/9] gnu: julia-compat: Update to 3.25.0. * gnu/packages/julia-xyz.scm (julia-compat): Update to 3.25.0. --- gnu/packages/julia-xyz.scm | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index f1da66a4be..9a4bf7c678 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2020 Nicolò Balzarotti <nicolo@nixo.xyz> +;;; Copyright © 2020, 2021 Nicolò Balzarotti <nicolo@nixo.xyz> ;;; ;;; This file is part of GNU Guix. ;;; @@ -25,7 +25,7 @@ (define-public julia-compat (package (name "julia-compat") - (version "3.9.1") + (version "3.25.0") (source (origin (method git-fetch) @@ -34,7 +34,7 @@ (commit (string-append "v" version)))) (file-name "Compat") (sha256 - (base32 "01vwjr2134bzgnaxwd67knbibbhnfgnqjw7gxrp29s6y2a6j3882")))) + (base32 "1m4r5i8mq29xjp3mllh6047n5a78sdyld57m15anrnsjgaapcgby")))) (build-system julia-build-system) (home-page "https://github.com/JuliaLang/Compat.jl") (synopsis "Compatibility across Julia versions") -- 2.29.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: 0003-gnu-Add-julia-orderedcollections.patch --] [-- Type: text/x-patch, Size: 1713 bytes --] From ac384ede7faccc6ce9ac6c97522c7dbd18200b4a Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 18:11:42 +0100 Subject: [PATCH 3/9] gnu: Add julia-orderedcollections. * gnu/packages/julia-xyz.scm (julia-orderedcollections): New variable. --- gnu/packages/julia-xyz.scm | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index 9a4bf7c678..d3c1f35996 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -43,3 +43,26 @@ between older and newer versions of the Julia language. The Compat package provides a macro that lets you use the latest syntax in a backwards-compatible way.") (license license:expat))) + +(define-public julia-orderedcollections + (package + (name "julia-orderedcollections") + (version "1.3.2") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaCollections/OrderedCollections.jl") + (commit (string-append "v" version)))) + (file-name "OrderedCollections") + (sha256 + (base32 "0sfip1ixghsz91q2s7d62rgzw3gppg42fg6bccxlplqa3hfmbycf")))) + (build-system julia-build-system) + (home-page "https://github.com/JuliaCollections/OrderedCollections.jl") + (synopsis "Julia implementation of associative containers that preserve +insertion order") + (description "This package implements OrderedDicts and OrderedSets, which +are similar to containers in base Julia. However, during iteration the +Ordered* containers return items in the order in which they were added to the +collection.") + (license license:expat))) -- 2.29.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #5: 0004-gnu-Add-julia-datastructures.patch --] [-- Type: text/x-patch, Size: 1724 bytes --] From 9736ab882b3348619cf3e74f4282404627dd2b64 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 18:17:50 +0100 Subject: [PATCH 4/9] gnu: Add julia-datastructures. * gnu/packages/julia-xyz.scm (julia-datastructures): New variable. --- gnu/packages/julia-xyz.scm | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index d3c1f35996..fee694b1fd 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -44,6 +44,30 @@ provides a macro that lets you use the latest syntax in a backwards-compatible way.") (license license:expat))) +(define-public julia-datastructures + (package + (name "julia-datastructures") + (version "0.18.8") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaCollections/DataStructures.jl") + (commit (string-append "v" version)))) + (file-name "DataStructures") + (sha256 + (base32 "1r4xswdvnnlbp5rfa7i7k6kzgkwizvqwyz7jscvyqjylcs6wdsnc")))) + (propagated-inputs + `(("julia-compat" ,julia-compat) + ("julia-orderedcollections" ,julia-orderedcollections))) + (build-system julia-build-system) + (home-page "https://github.com/JuliaCollections/DataStructures.jl") + (synopsis "Julia implementation of Data structures") + (description "This package implements a variety of data structures, +including, CircularBuffer, Queue, Stack, Accumulators, Linked Lists, Sorted +Dicts and many others.") + (license license:expat))) + (define-public julia-orderedcollections (package (name "julia-orderedcollections") -- 2.29.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #6: 0005-gnu-Add-julia-fixedpointnumbers.patch --] [-- Type: text/x-patch, Size: 2363 bytes --] From 79fa1af823688da6796a12d664352a4f0512d702 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 18:25:02 +0100 Subject: [PATCH 5/9] gnu: Add julia-fixedpointnumbers. * gnu/packages/julia-xyz.scm (julia-fixedpointnumbers): New variable. --- gnu/packages/julia-xyz.scm | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index fee694b1fd..f5c9e000db 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -68,6 +68,41 @@ including, CircularBuffer, Queue, Stack, Accumulators, Linked Lists, Sorted Dicts and many others.") (license license:expat))) +(define-public julia-fixedpointnumbers + (package + (name "julia-fixedpointnumbers") + (version "0.8.4") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaMath/FixedPointNumbers.jl") + (commit (string-append "v" version)))) + (file-name "FixedPointNumbers") + (sha256 + (base32 "0j0n40n04q9sk68wh9jq90m6c67k4ws02k41djjzkrqmpzv4rcdi")))) + (build-system julia-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'unpack 'disable-failing-test + (lambda* (#:key outputs #:allow-other-keys) + (substitute* "test/fixed.jl" + ;; A deprecation warning is not thrown + (("@test_logs.*:warn" all) (string-append "# " all))) + #t))))) + (propagated-inputs `(("julia-compat" ,julia-compat))) + (home-page "https://github.com/JuliaMath/FixedPointNumbers.jl") + (synopsis "Fixed point types for julia") + (description "@code{FixedPointNumbers.jl} implements fixed-point number +types for Julia. A fixed-point number represents a fractional, or +non-integral, number. In contrast with the more widely known floating-point +numbers, with fixed-point numbers the decimal point doesn't \"float\": +fixed-point numbers are effectively integers that are interpreted as being +scaled by a constant factor. Consequently, they have a fixed number of +digits (bits) after the decimal (radix) point.") + (license license:expat))) + (define-public julia-orderedcollections (package (name "julia-orderedcollections") -- 2.29.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #7: 0006-gnu-Add-julia-parsers.patch --] [-- Type: text/x-patch, Size: 1445 bytes --] From f4ddd59a14cb0fcc259efbf509c2a8073adf62fc Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 18:27:32 +0100 Subject: [PATCH 6/9] gnu: Add julia-parsers. * gnu/packages/julia-xyz.scm (julia-parsers): New variable. --- gnu/packages/julia-xyz.scm | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index f5c9e000db..b887832df3 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -125,3 +125,23 @@ are similar to containers in base Julia. However, during iteration the Ordered* containers return items in the order in which they were added to the collection.") (license license:expat))) + +(define-public julia-parsers + (package + (name "julia-parsers") + (version "1.0.15") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaData/Parsers.jl") + (commit (string-append "v" version)))) + (file-name "Parsers") + (sha256 + (base32 "16iffl6l28kspgqch48mhi1s8qhspr3cpqcwsph3rqi72lbfqygx")))) + (build-system julia-build-system) + (home-page "https://github.com/JuliaData/Parsers.jl") + (synopsis "Fast parsing machinery for basic types in Julia") + (description "@code{Parsers.jl} is a collection of type parsers and +utilities for Julia.") + (license license:expat))) -- 2.29.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #8: 0007-gnu-Add-julia-adapt.patch --] [-- Type: text/x-patch, Size: 1582 bytes --] From 358d115e218fc433b15474500a221861c9c82abb Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 23:46:51 +0100 Subject: [PATCH 7/9] gnu: Add julia-adapt. * gnu/packages/julia-xyz.scm (julia-adapt): New variable. --- gnu/packages/julia-xyz.scm | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index b887832df3..3601006976 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -22,6 +22,28 @@ #:use-module (guix git-download) #:use-module (guix build-system julia)) +(define-public julia-adapt + (package + (name "julia-adapt") + (version "3.1.0") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaGPU/Adapt.jl") + (commit (string-append "v" version)))) + (file-name "Adapt") + (sha256 + (base32 "1qcj7i75m0wwk0xcdgf89bjymnz2dipv2iwxys8wzpvrashi6wsp")))) + (build-system julia-build-system) + (home-page "https://github.com/JuliaGPU/Adapt.jl") + (synopsis "Package providing the @code{adapt} function, similar to @code{convert}") + (description "This julia package provides the @code{adapt(T, x)} function +acts like @code{convert(T, x)}, but without the restriction of returning a +T. This allows you to \"convert\" wrapper types like Adjoint to be GPU +compatible without throwing away the wrapper.") + (license license:expat))) + (define-public julia-compat (package (name "julia-compat") -- 2.29.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #9: 0008-gnu-Add-julia-offsetarrays.patch --] [-- Type: text/x-patch, Size: 1765 bytes --] From 06f2e7bbbff5b73493c9a0640ec4c9c6c20280c9 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Tue, 19 Jan 2021 00:01:28 +0100 Subject: [PATCH 8/9] gnu: Add julia-offsetarrays. * gnu/packages/julia-xyz.scm (julia-offsetarrays): New variable. --- gnu/packages/julia-xyz.scm | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index 3601006976..429ab0afad 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -148,6 +148,31 @@ Ordered* containers return items in the order in which they were added to the collection.") (license license:expat))) +(define-public julia-offsetarrays + (package + (name "julia-offsetarrays") + (version "1.5.1") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaArrays/OffsetArrays.jl") + (commit (string-append "v" version)))) + (file-name "OffsetArrays") + (sha256 + (base32 "0pd42an8kj2mgw3sv00ijfrsvmfjfgf12lfmpan6zkdckqa73jqz")))) + (build-system julia-build-system) + (propagated-inputs + `(("julia-adapt" ,julia-adapt))) + ;; CatIndices depends on OffsetArrays, introducing a recursive dependency + (arguments '(#:tests? #f)) + (home-page "https://juliaarrays.github.io/OffsetArrays.jl/stable/") + (synopsis "Fortran-like arrays with arbitrary, zero or negative indices") + (description "@code{OffsetArrays.jl} provides Julia users with arrays that +have arbitrary indices, similar to those found in some other programming +languages like Fortran.") + (license license:expat))) + (define-public julia-parsers (package (name "julia-parsers") -- 2.29.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #10: 0009-gnu-Add-julia-json.patch --] [-- Type: text/x-patch, Size: 1651 bytes --] From 3ebd72ef6a55ad46522fe2534b775e9b6d20130f Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Tue, 19 Jan 2021 00:06:34 +0100 Subject: [PATCH 9/9] gnu: Add julia-json. * gnu/packages/julia-xyz.scm (julia-json): New variable. --- gnu/packages/julia-xyz.scm | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index 429ab0afad..7aba08cdb1 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -125,6 +125,30 @@ scaled by a constant factor. Consequently, they have a fixed number of digits (bits) after the decimal (radix) point.") (license license:expat))) +(define-public julia-json + (package + (name "julia-json") + (version "0.21.1") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaIO/JSON.jl") + (commit (string-append "v" version)))) + (file-name "JSON") + (sha256 + (base32 "1f9k613kbknmp4fgjxvjaw4d5sfbx8a5hmcszmp1w9rqfqngjx9m")))) + (build-system julia-build-system) + (propagated-inputs + `(("julia-datastructures" ,julia-datastructures) + ("julia-fixedpointnumbers" ,julia-fixedpointnumbers) + ("julia-parsers" ,julia-parsers) + ("julia-offsetarrays" ,julia-offsetarrays))) + (home-page "https://github.com/JuliaIO/JSON.jl") + (synopsis "JSON parsing and printing") + (description "Parsing and printing JSON in pure Julia.") + (license license:expat))) + (define-public julia-orderedcollections (package (name "julia-orderedcollections") -- 2.29.2 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [bug#45972] [PATCH] Add julia-json with dependencies 2021-01-18 23:47 [bug#45972] [PATCH] Add julia-json with dependencies Nicolò Balzarotti @ 2021-01-26 21:33 ` Ludovic Courtès 2021-01-26 23:23 ` Nicolò Balzarotti 0 siblings, 1 reply; 14+ messages in thread From: Ludovic Courtès @ 2021-01-26 21:33 UTC (permalink / raw) To: Nicolò Balzarotti; +Cc: 45972 Hi Nicolò, Nicolò Balzarotti <anothersms@gmail.com> skribis: > This patch series adds julia-json together with its dependencies. Neat. I looked at it quickly and it LGTM. Nitpicks: > + (synopsis "Julia implementation of associative containers that preserve > +insertion order") Maybe remove “Julia implementation of”. > + (description "This package implements OrderedDicts and OrderedSets, which Ideally use @code for identifiers. > + (synopsis "Fixed point types for julia") s/julia/Julia/ > + (synopsis "JSON parsing and printing") > + (description "Parsing and printing JSON in pure Julia.") Please make that a full sentence. If you’re confident with the licenses and ‘guix lint’ is happy, please push! Nice to see more Julia packages. Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* [bug#45972] [PATCH] Add julia-json with dependencies 2021-01-26 21:33 ` Ludovic Courtès @ 2021-01-26 23:23 ` Nicolò Balzarotti 2021-01-27 22:40 ` Ludovic Courtès 0 siblings, 1 reply; 14+ messages in thread From: Nicolò Balzarotti @ 2021-01-26 23:23 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 45972 [-- Attachment #1: Type: text/plain, Size: 601 bytes --] I applied all your suggestions :) > If you’re confident with the licenses and ‘guix lint’ is happy, > please push! Almost all of Julia packages are under MIT (expat), I double checked and they are fine. Also, 3 packages (out of 8) have been updated since my submission, so I updated them now and guix lint does not complain anymore. > Nice to see more Julia packages. This is the first batch, I finally decided to submit in small batches the 100+ packages I have; you have been warned :D Here's the updated patchset (if somebody can push). Thanks Ludo for the review [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: v2-0001-guix-build-system-julia-Enable-tests.patch --] [-- Type: text/x-patch, Size: 3143 bytes --] From a374ec23346cca7ff6f28c8141102f8838db649a Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 23:56:54 +0100 Subject: [PATCH v2 1/9] guix: build-system (julia): Enable tests. * guix/build-system/julia.scm (julia-build): Set tests? default to #t. * guix/build/julia-build-system.scm (check): Respect tests? and fix julia invocation. (%standard-phases): Add check phase after install. --- guix/build-system/julia.scm | 2 +- guix/build/julia-build-system.scm | 27 +++++++++++++++++---------- 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/guix/build-system/julia.scm b/guix/build-system/julia.scm index 488fe9bb1d..d3cb41c054 100644 --- a/guix/build-system/julia.scm +++ b/guix/build-system/julia.scm @@ -75,7 +75,7 @@ (define* (julia-build store name inputs #:key source - (tests? #f) + (tests? #t) (phases '(@ (guix build julia-build-system) %standard-phases)) (outputs '("out")) diff --git a/guix/build/julia-build-system.scm b/guix/build/julia-build-system.scm index e8ebcf8ba0..61817e0b47 100644 --- a/guix/build/julia-build-system.scm +++ b/guix/build/julia-build-system.scm @@ -69,15 +69,22 @@ (string-append "pushfirst!(DEPOT_PATH, pop!(DEPOT_PATH)); using " package))) #t) -(define* (check #:key source inputs outputs #:allow-other-keys) - (let* ((out (assoc-ref outputs "out")) - (package (strip-store-file-name source)) - (builddir (string-append out "/share/julia/"))) - ;; With a patch, SOURCE_DATE_EPOCH is honored - (setenv "SOURCE_DATE_EPOCH" "1") - (setenv "JULIA_DEPOT_PATH" builddir) - (setenv "JULIA_LOAD_PATH" (string-append builddir "packages/")) - (invoke-julia (string-append "using Pkg;Pkg.test(\"" package "\")"))) +(define* (check #:key tests? source inputs outputs #:allow-other-keys) + (when tests? + (let* ((out (assoc-ref outputs "out")) + (package (strip-store-file-name source)) + (builddir (string-append out "/share/julia/"))) + ;; With a patch, SOURCE_DATE_EPOCH is honored + (setenv "SOURCE_DATE_EPOCH" "1") + (setenv "JULIA_DEPOT_PATH" builddir) + (setenv "JULIA_LOAD_PATH" + (string-append builddir "packages/" ":" + (or (getenv "JULIA_LOAD_PATH") + ""))) + (setenv "HOME" "/tmp") + (invoke "julia" + (string-append builddir "packages/" + package "/test/runtests.jl")))) #t) (define (julia-create-package-toml outputs source @@ -112,7 +119,7 @@ version = \"" version "\" (delete 'check) ; tests must be run after installation (replace 'install install) (add-after 'install 'precompile precompile) - ;; (add-after 'install 'check check) + (add-after 'install 'check check) ;; TODO: In the future we could add a "system-image-generation" phase ;; where we use PackageCompiler.jl to speed up package loading times (delete 'configure) -- 2.30.0 [-- Attachment #3: v2-0002-gnu-julia-compat-Update-to-3.25.0.patch --] [-- Type: text/x-patch, Size: 1431 bytes --] From 46cc28fe6d532c64006b6fea88604b198acec278 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 17:42:50 +0100 Subject: [PATCH v2 2/9] gnu: julia-compat: Update to 3.25.0. * gnu/packages/julia-xyz.scm (julia-compat): Update to 3.25.0. --- gnu/packages/julia-xyz.scm | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index f1da66a4be..9a4bf7c678 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2020 Nicolò Balzarotti <nicolo@nixo.xyz> +;;; Copyright © 2020, 2021 Nicolò Balzarotti <nicolo@nixo.xyz> ;;; ;;; This file is part of GNU Guix. ;;; @@ -25,7 +25,7 @@ (define-public julia-compat (package (name "julia-compat") - (version "3.9.1") + (version "3.25.0") (source (origin (method git-fetch) @@ -34,7 +34,7 @@ (commit (string-append "v" version)))) (file-name "Compat") (sha256 - (base32 "01vwjr2134bzgnaxwd67knbibbhnfgnqjw7gxrp29s6y2a6j3882")))) + (base32 "1m4r5i8mq29xjp3mllh6047n5a78sdyld57m15anrnsjgaapcgby")))) (build-system julia-build-system) (home-page "https://github.com/JuliaLang/Compat.jl") (synopsis "Compatibility across Julia versions") -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: v2-0003-gnu-Add-julia-orderedcollections.patch --] [-- Type: text/x-patch, Size: 1711 bytes --] From cecf5b2739226f06bfe9ff8df55207306e1db2b8 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 18:11:42 +0100 Subject: [PATCH v2 3/9] gnu: Add julia-orderedcollections. * gnu/packages/julia-xyz.scm (julia-orderedcollections): New variable. --- gnu/packages/julia-xyz.scm | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index 9a4bf7c678..58b8795eab 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -43,3 +43,25 @@ between older and newer versions of the Julia language. The Compat package provides a macro that lets you use the latest syntax in a backwards-compatible way.") (license license:expat))) + +(define-public julia-orderedcollections + (package + (name "julia-orderedcollections") + (version "1.3.2") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaCollections/OrderedCollections.jl") + (commit (string-append "v" version)))) + (file-name "OrderedCollections") + (sha256 + (base32 "0sfip1ixghsz91q2s7d62rgzw3gppg42fg6bccxlplqa3hfmbycf")))) + (build-system julia-build-system) + (home-page "https://github.com/JuliaCollections/OrderedCollections.jl") + (synopsis "Associative containers that preserve insertion order") + (description "This package implements @code{OrderedDicts} and +@code{OrderedSets}, which are similar to containers in base Julia. However, +during iteration the @code{Ordered*} containers return items in the order in +which they were added to the collection.") + (license license:expat))) -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #5: v2-0004-gnu-Add-julia-datastructures.patch --] [-- Type: text/x-patch, Size: 1776 bytes --] From ea24edc3e10695100a3c486736a2e4647316edf0 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 18:17:50 +0100 Subject: [PATCH v2 4/9] gnu: Add julia-datastructures. * gnu/packages/julia-xyz.scm (julia-datastructures): New variable. --- gnu/packages/julia-xyz.scm | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index 58b8795eab..aacc7ee96d 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -44,6 +44,30 @@ provides a macro that lets you use the latest syntax in a backwards-compatible way.") (license license:expat))) +(define-public julia-datastructures + (package + (name "julia-datastructures") + (version "0.18.9") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaCollections/DataStructures.jl") + (commit (string-append "v" version)))) + (file-name "DataStructures") + (sha256 + (base32 "0hdqp8ipsqdw5bqqkdvz4j6n67x80sj5azr9vzyxwjfsgkfbnk2l")))) + (propagated-inputs + `(("julia-compat" ,julia-compat) + ("julia-orderedcollections" ,julia-orderedcollections))) + (build-system julia-build-system) + (home-page "https://github.com/JuliaCollections/DataStructures.jl") + (synopsis "Julia module providing different data structures") + (description "This package implements a variety of data structures, +including, @code{CircularBuffer}, @code{Queue}, @code{Stack}, +@code{Accumulators}, @code{LinkedLists}, @code{SortedDicts} and many others.") + (license license:expat))) + (define-public julia-orderedcollections (package (name "julia-orderedcollections") -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #6: v2-0005-gnu-Add-julia-fixedpointnumbers.patch --] [-- Type: text/x-patch, Size: 2406 bytes --] From 431b0bb4d5b15c6b7121dcad56a81bef0f3c9aec Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 18:25:02 +0100 Subject: [PATCH v2 5/9] gnu: Add julia-fixedpointnumbers. * gnu/packages/julia-xyz.scm (julia-fixedpointnumbers): New variable. --- gnu/packages/julia-xyz.scm | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index aacc7ee96d..6b9ff1bc8d 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -68,6 +68,41 @@ including, @code{CircularBuffer}, @code{Queue}, @code{Stack}, @code{Accumulators}, @code{LinkedLists}, @code{SortedDicts} and many others.") (license license:expat))) +(define-public julia-fixedpointnumbers + (package + (name "julia-fixedpointnumbers") + (version "0.8.4") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaMath/FixedPointNumbers.jl") + (commit (string-append "v" version)))) + (file-name "FixedPointNumbers") + (sha256 + (base32 "0j0n40n04q9sk68wh9jq90m6c67k4ws02k41djjzkrqmpzv4rcdi")))) + (build-system julia-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'unpack 'disable-failing-test + (lambda* (#:key outputs #:allow-other-keys) + (substitute* "test/fixed.jl" + ;; A deprecation warning is not thrown + (("@test_logs.*:warn" all) (string-append "# " all))) + #t))))) + (propagated-inputs `(("julia-compat" ,julia-compat))) + (home-page "https://github.com/JuliaMath/FixedPointNumbers.jl") + (synopsis "Fixed point types for Julia") + (description "@code{FixedPointNumbers.jl} implements fixed-point number +types for Julia. A fixed-point number represents a fractional, or +non-integral, number. In contrast with the more widely known floating-point +numbers, with fixed-point numbers the decimal point doesn't \"float\": +fixed-point numbers are effectively integers that are interpreted as being +scaled by a constant factor. Consequently, they have a fixed number of +digits (bits) after the decimal (radix) point.") + (license license:expat))) + (define-public julia-orderedcollections (package (name "julia-orderedcollections") -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #7: v2-0006-gnu-Add-julia-parsers.patch --] [-- Type: text/x-patch, Size: 1453 bytes --] From 1814f61130c39eba91507fe5bde654e99d9c7421 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 18:27:32 +0100 Subject: [PATCH v2 6/9] gnu: Add julia-parsers. * gnu/packages/julia-xyz.scm (julia-parsers): New variable. --- gnu/packages/julia-xyz.scm | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index 6b9ff1bc8d..a35755d5da 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -124,3 +124,23 @@ digits (bits) after the decimal (radix) point.") during iteration the @code{Ordered*} containers return items in the order in which they were added to the collection.") (license license:expat))) + +(define-public julia-parsers + (package + (name "julia-parsers") + (version "1.0.15") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaData/Parsers.jl") + (commit (string-append "v" version)))) + (file-name "Parsers") + (sha256 + (base32 "16iffl6l28kspgqch48mhi1s8qhspr3cpqcwsph3rqi72lbfqygx")))) + (build-system julia-build-system) + (home-page "https://github.com/JuliaData/Parsers.jl") + (synopsis "Fast parsing machinery for basic types in Julia") + (description "@code{Parsers.jl} is a collection of type parsers and +utilities for Julia.") + (license license:expat))) -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #8: v2-0007-gnu-Add-julia-adapt.patch --] [-- Type: text/x-patch, Size: 1600 bytes --] From 985de9540ffc4ed5351f9b884e95fea920d9e325 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 23:46:51 +0100 Subject: [PATCH v2 7/9] gnu: Add julia-adapt. * gnu/packages/julia-xyz.scm (julia-adapt): New variable. --- gnu/packages/julia-xyz.scm | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index a35755d5da..58b45d3f15 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -22,6 +22,28 @@ #:use-module (guix git-download) #:use-module (guix build-system julia)) +(define-public julia-adapt + (package + (name "julia-adapt") + (version "3.1.1") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaGPU/Adapt.jl") + (commit (string-append "v" version)))) + (file-name "Adapt") + (sha256 + (base32 "1lks6k3a1gvwlplld47nh6xfy3nnlpc0vhkzg6zg0qn33qdmavrg")))) + (build-system julia-build-system) + (home-page "https://github.com/JuliaGPU/Adapt.jl") + (synopsis "Package providing the @code{adapt} function, similar to @code{convert}") + (description "This Julia package provides the @code{adapt(T, x)} function +acts like @code{convert(T, x)}, but without the restriction of returning a +@code{T}. This allows you to \"convert\" wrapper types like @code{Adjoint} to +be GPU compatible without throwing away the wrapper.") + (license license:expat))) + (define-public julia-compat (package (name "julia-compat") -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #9: v2-0008-gnu-Add-julia-offsetarrays.patch --] [-- Type: text/x-patch, Size: 1796 bytes --] From fed9e5fe47495120b49c35535bc9d26fdd065760 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Tue, 19 Jan 2021 00:01:28 +0100 Subject: [PATCH v2 8/9] gnu: Add julia-offsetarrays. * gnu/packages/julia-xyz.scm (julia-offsetarrays): New variable. --- gnu/packages/julia-xyz.scm | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index 58b45d3f15..70d7f1d718 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -147,6 +147,31 @@ during iteration the @code{Ordered*} containers return items in the order in which they were added to the collection.") (license license:expat))) +(define-public julia-offsetarrays + (package + (name "julia-offsetarrays") + (version "1.5.2") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaArrays/OffsetArrays.jl") + (commit (string-append "v" version)))) + (file-name "OffsetArrays") + (sha256 + (base32 "1y3fnssw2hzyghrk6jfcxslab0f8sjkjszh482snfq4k6mkrhy77")))) + (build-system julia-build-system) + (propagated-inputs + `(("julia-adapt" ,julia-adapt))) + ;; CatIndices depends on OffsetArrays, introducing a recursive dependency + (arguments '(#:tests? #f)) + (home-page "https://juliaarrays.github.io/OffsetArrays.jl/stable/") + (synopsis "Fortran-like arrays with arbitrary, zero or negative indices") + (description "@code{OffsetArrays.jl} provides Julia users with arrays that +have arbitrary indices, similar to those found in some other programming +languages like Fortran.") + (license license:expat))) + (define-public julia-parsers (package (name "julia-parsers") -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #10: v2-0009-gnu-Add-julia-json.patch --] [-- Type: text/x-patch, Size: 1723 bytes --] From aa1231bb89d1a7271181d437ebe36952ec8ccfc4 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Tue, 19 Jan 2021 00:06:34 +0100 Subject: [PATCH v2 9/9] gnu: Add julia-json. * gnu/packages/julia-xyz.scm (julia-json): New variable. --- gnu/packages/julia-xyz.scm | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index 70d7f1d718..a79b8d07b6 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -125,6 +125,31 @@ scaled by a constant factor. Consequently, they have a fixed number of digits (bits) after the decimal (radix) point.") (license license:expat))) +(define-public julia-json + (package + (name "julia-json") + (version "0.21.1") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaIO/JSON.jl") + (commit (string-append "v" version)))) + (file-name "JSON") + (sha256 + (base32 "1f9k613kbknmp4fgjxvjaw4d5sfbx8a5hmcszmp1w9rqfqngjx9m")))) + (build-system julia-build-system) + (propagated-inputs + `(("julia-datastructures" ,julia-datastructures) + ("julia-fixedpointnumbers" ,julia-fixedpointnumbers) + ("julia-parsers" ,julia-parsers) + ("julia-offsetarrays" ,julia-offsetarrays))) + (home-page "https://github.com/JuliaIO/JSON.jl") + (synopsis "JSON parsing and printing library for Julia") + (description "@code{JSON.jl} is a pure Julia module which supports parsing +and printing JSON documents.") + (license license:expat))) + (define-public julia-orderedcollections (package (name "julia-orderedcollections") -- 2.30.0 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [bug#45972] [PATCH] Add julia-json with dependencies 2021-01-26 23:23 ` Nicolò Balzarotti @ 2021-01-27 22:40 ` Ludovic Courtès 2021-01-28 0:30 ` Nicolò Balzarotti 0 siblings, 1 reply; 14+ messages in thread From: Ludovic Courtès @ 2021-01-27 22:40 UTC (permalink / raw) To: Nicolò Balzarotti; +Cc: 45972 Hi, Nicolò Balzarotti <anothersms@gmail.com> skribis: > Almost all of Julia packages are under MIT (expat), I double checked and > they are fine. Also, 3 packages (out of 8) have been updated since my > submission, so I updated them now and guix lint does not complain anymore. Great. > This is the first batch, I finally decided to submit in small batches > the 100+ packages I have; you have been warned :D Oh, fun! :-) So, actually, we have a problem: > +(define-public julia-orderedcollections > + (package > + (name "julia-orderedcollections") > + (version "1.3.2") > + (source > + (origin > + (method git-fetch) > + (uri (git-reference > + (url "https://github.com/JuliaCollections/OrderedCollections.jl") > + (commit (string-append "v" version)))) > + (file-name "OrderedCollections") I was surprised that ‘guix lint’ doesn’t complain about this file name, I thought it required the file name to match the package name and at least that’s the spirit (I’ll take a look). Anyway, I went ahead and replaced all these by (git-file-name name version) as is done elsewhere. But now the tests would fail like so: --8<---------------cut here---------------start------------->8--- starting phase `check' ERROR: LoadError: ArgumentError: Package Adapt not found in current path: - Run `import Pkg; Pkg.add("Adapt")` to install the Adapt package. --8<---------------cut here---------------end--------------->8--- My understanding is that the first patch expects the source file name to match the Julia package name. IMO, that shouldn’t be the case. Can we either extract the Julia package name from metadata that’s in the package itself (?), or otherwise pass it to all the phases via ‘julia-build-system’? Sorry for not noticing earlier! Thanks, Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* [bug#45972] [PATCH] Add julia-json with dependencies 2021-01-27 22:40 ` Ludovic Courtès @ 2021-01-28 0:30 ` Nicolò Balzarotti 2021-01-28 13:10 ` Ludovic Courtès 2021-01-30 14:46 ` [bug#45972] Julia importer? zimoun 0 siblings, 2 replies; 14+ messages in thread From: Nicolò Balzarotti @ 2021-01-28 0:30 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 45972 > My understanding is that the first patch expects the source file name to > match the Julia package name. That's correct, we use it in the build system. > IMO, that shouldn’t be the case. At first I wasn't sure it was ok, but it went on through the revision process when I submitted the first package (Compat) so I tought it was fine. > Can we either extract the Julia package name from metadata that’s in > the package itself (?), or otherwise pass it to all the phases via > ‘julia-build-system’? Sure, I'd just read it from Package.toml (nowadays almost all the packages have this file, and for when it's missing we also have the julia-create-package-toml procedure that creates it). The file is toml, but I don't see a toml parser in guix. So, I'd use a function like the following: #+begin_src scheme (define (package.toml->name file) (call-with-input-file file (lambda (in) (let loop ((line (read-line in 'concat))) (if (eof-object? line) #f ;What to do? (let ((m (string-match "name\\s*=\\s*\"(.*)\"" line))) (if m (match:substring m 1) (loop (read-line in 'concat))))))))) #+end_src As you can see this is very minimal/naive (takes the first name = "" occurrence, does not even consider comments, for which I'd add a negative lookahead "^(?!#)" which I can't get to work with ice-9 regex), but tested on a few packages it working. I don't know what to do when the match is not found (since it's something that might happen only during development, the #f fallback should not be too bad, btw, as the build will fail). The other way I think this is easily solvable is by asking julia directly, by reading the output of: (invoke-julia "using Pkg; Pkg.TOML.parsefile("Project.toml")["name"] |> println") doing something like cargo's manifest-target procedure does. But it'd go the other way if it's ok. Let me know! Once decided, I'll submit the updated patches > > Sorry for not noticing earlier! > np and thanks again! ^ permalink raw reply [flat|nested] 14+ messages in thread
* [bug#45972] [PATCH] Add julia-json with dependencies 2021-01-28 0:30 ` Nicolò Balzarotti @ 2021-01-28 13:10 ` Ludovic Courtès 2021-01-28 15:37 ` Nicolò Balzarotti 2021-01-30 14:46 ` [bug#45972] Julia importer? zimoun 1 sibling, 1 reply; 14+ messages in thread From: Ludovic Courtès @ 2021-01-28 13:10 UTC (permalink / raw) To: Nicolò Balzarotti; +Cc: 45972 Buon giorno! Nicolò Balzarotti <anothersms@gmail.com> skribis: >> My understanding is that the first patch expects the source file name to >> match the Julia package name. > > That's correct, we use it in the build system. > >> IMO, that shouldn’t be the case. > > At first I wasn't sure it was ok, but it went on through the revision > process when I submitted the first package (Compat) so I tought it was fine. If it was me it was probably an oversight, I’m sorry about that. >> Can we either extract the Julia package name from metadata that’s in >> the package itself (?), or otherwise pass it to all the phases via >> ‘julia-build-system’? > > Sure, I'd just read it from Package.toml (nowadays almost all the > packages have this file, and for when it's missing we also have the > julia-create-package-toml procedure that creates it). > > The file is toml, but I don't see a toml parser in guix. So, I'd use a > function like the following: > > #+begin_src scheme > (define (package.toml->name file) > (call-with-input-file file > (lambda (in) > (let loop ((line (read-line in 'concat))) > (if (eof-object? line) > #f ;What to do? > (let ((m (string-match "name\\s*=\\s*\"(.*)\"" line))) > (if m (match:substring m 1) > (loop (read-line in 'concat))))))))) > #+end_src Sounds reasonable to me. If I understand the toml format correctly, “name=.*” is guaranteed to be on a line on its own, so that looks safe. > As you can see this is very minimal/naive (takes the first name = "" > occurrence, does not even consider comments, for which I'd add a > negative lookahead "^(?!#)" which I can't get to work with ice-9 regex), > but tested on a few packages it working. I don't know what to do when > the match is not found (since it's something that might happen only > during development, the #f fallback should not be too bad, btw, as the > build will fail). Yes, sounds good. To be on the safe side, we can add a #:julia-package-name parameter to the build system; it would default to #f, in which case the name is extracted from the toml file. > The other way I think this is easily solvable is by asking julia > directly, by reading the output of: > > (invoke-julia "using Pkg; Pkg.TOML.parsefile("Project.toml")["name"] |> println") > > doing something like cargo's manifest-target procedure does. But it'd > go the other way if it's ok. Yeah, the basic “parser” seems to be good enough. Thanks! Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* [bug#45972] [PATCH] Add julia-json with dependencies 2021-01-28 13:10 ` Ludovic Courtès @ 2021-01-28 15:37 ` Nicolò Balzarotti 2021-01-30 14:10 ` bug#45972: " Ludovic Courtès 0 siblings, 1 reply; 14+ messages in thread From: Nicolò Balzarotti @ 2021-01-28 15:37 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 45972 [-- Attachment #1: Type: text/plain, Size: 813 bytes --] Ludovic Courtès <ludo@gnu.org> writes: > Buon giorno! > >>> IMO, that shouldn’t be the case. > If it was me it was probably an oversight, I’m sorry about that. No problem! It appeared also in the manual so I fixed it > >>> Can we either extract the Julia package name from metadata that’s in >>> the package itself (?), or otherwise pass it to all the phases via >>> ‘julia-build-system’? > > Sounds reasonable to me. If I understand the toml format correctly, > “name=.*” is guaranteed to be on a line on its own, so that looks > safe. exactly > > To be on the safe side, we can add a #:julia-package-name parameter to > the build system; it would default to #f, in which case the name is > extracted from the toml file. > Done, I attached the updated patch set! [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: v4-0001-guix-build-system-julia-Enable-tests.patch --] [-- Type: text/x-patch, Size: 3859 bytes --] From ea999a33f5b70bb98c890bdc92f6755ad56694a1 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 23:56:54 +0100 Subject: [PATCH v4 01/11] guix: build-system (julia): Enable tests. * guix/build-system/julia.scm (julia-build): Set tests? default to #t. * guix/build/julia-build-system.scm (check): Respect tests? and fix julia invocation. (%standard-phases): Add check phase after install. * doc/guix.texi (julia-build-system): Update accordingly. --- doc/guix.texi | 2 +- guix/build-system/julia.scm | 2 +- guix/build/julia-build-system.scm | 27 +++++++++++++++++---------- 3 files changed, 19 insertions(+), 12 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index dc41fe9aea..a7ae9b60a7 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -7619,7 +7619,7 @@ implements the build procedure used by @uref{https://julialang.org/, julia} packages, which essentially is similar to running @samp{julia -e 'using Pkg; Pkg.add(package)'} in an environment where @env{JULIA_LOAD_PATH} contains the paths to all Julia package inputs. -Tests are run with @code{Pkg.test}. +Tests are run by calling @code{/test/runtests.jl}. Julia packages require the source @code{file-name} to be the real name of the package, correctly capitalized. diff --git a/guix/build-system/julia.scm b/guix/build-system/julia.scm index 488fe9bb1d..d3cb41c054 100644 --- a/guix/build-system/julia.scm +++ b/guix/build-system/julia.scm @@ -75,7 +75,7 @@ (define* (julia-build store name inputs #:key source - (tests? #f) + (tests? #t) (phases '(@ (guix build julia-build-system) %standard-phases)) (outputs '("out")) diff --git a/guix/build/julia-build-system.scm b/guix/build/julia-build-system.scm index e8ebcf8ba0..61817e0b47 100644 --- a/guix/build/julia-build-system.scm +++ b/guix/build/julia-build-system.scm @@ -69,15 +69,22 @@ (string-append "pushfirst!(DEPOT_PATH, pop!(DEPOT_PATH)); using " package))) #t) -(define* (check #:key source inputs outputs #:allow-other-keys) - (let* ((out (assoc-ref outputs "out")) - (package (strip-store-file-name source)) - (builddir (string-append out "/share/julia/"))) - ;; With a patch, SOURCE_DATE_EPOCH is honored - (setenv "SOURCE_DATE_EPOCH" "1") - (setenv "JULIA_DEPOT_PATH" builddir) - (setenv "JULIA_LOAD_PATH" (string-append builddir "packages/")) - (invoke-julia (string-append "using Pkg;Pkg.test(\"" package "\")"))) +(define* (check #:key tests? source inputs outputs #:allow-other-keys) + (when tests? + (let* ((out (assoc-ref outputs "out")) + (package (strip-store-file-name source)) + (builddir (string-append out "/share/julia/"))) + ;; With a patch, SOURCE_DATE_EPOCH is honored + (setenv "SOURCE_DATE_EPOCH" "1") + (setenv "JULIA_DEPOT_PATH" builddir) + (setenv "JULIA_LOAD_PATH" + (string-append builddir "packages/" ":" + (or (getenv "JULIA_LOAD_PATH") + ""))) + (setenv "HOME" "/tmp") + (invoke "julia" + (string-append builddir "packages/" + package "/test/runtests.jl")))) #t) (define (julia-create-package-toml outputs source @@ -112,7 +119,7 @@ version = \"" version "\" (delete 'check) ; tests must be run after installation (replace 'install install) (add-after 'install 'precompile precompile) - ;; (add-after 'install 'check check) + (add-after 'install 'check check) ;; TODO: In the future we could add a "system-image-generation" phase ;; where we use PackageCompiler.jl to speed up package loading times (delete 'configure) -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: v4-0002-guix-julia-build-system-Don-t-rely-on-file-name-t.patch --] [-- Type: text/x-patch, Size: 5853 bytes --] From e7af058c7c162a2d2e755e5b9e39fd8016d2a1ae Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Thu, 28 Jan 2021 16:13:33 +0100 Subject: [PATCH v4 02/11] guix: julia-build-system: Don't rely on file-name to set module name. * guix/build/julia-build-system.scm (project.toml->name): New procedure. (precompile, check, julia-build): Accept new key argument #:julia-package-name. * guix/build-system/julia.scm (julia-build): ... add it. * doc/guix.texi (julia-build-system): Update julia-package-name accordingly. --- doc/guix.texi | 5 +++-- guix/build-system/julia.scm | 4 +++- guix/build/julia-build-system.scm | 35 ++++++++++++++++++++++++------- 3 files changed, 34 insertions(+), 10 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index a7ae9b60a7..64451c6de5 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -7621,8 +7621,9 @@ julia} packages, which essentially is similar to running @samp{julia -e @env{JULIA_LOAD_PATH} contains the paths to all Julia package inputs. Tests are run by calling @code{/test/runtests.jl}. -Julia packages require the source @code{file-name} to be the real name of the -package, correctly capitalized. +The Julia package name is read from the file @file{Project.toml}. This +value can be overridden by passing the argument @code{#:julia-file-name} +(which must be correctly capitalized). For packages requiring shared library dependencies, you may need to write the @file{/deps/deps.jl} file manually. It's usually a line of @code{const diff --git a/guix/build-system/julia.scm b/guix/build-system/julia.scm index d3cb41c054..63cb7cd864 100644 --- a/guix/build-system/julia.scm +++ b/guix/build-system/julia.scm @@ -82,6 +82,7 @@ (search-paths '()) (system (%current-system)) (guile #f) + (julia-package-name #f) (imported-modules %julia-build-system-modules) (modules '((guix build julia-build-system) (guix build utils)))) @@ -103,7 +104,8 @@ #:outputs %outputs #:search-paths ',(map search-path-specification->sexp search-paths) - #:inputs %build-inputs))) + #:inputs %build-inputs + #:julia-package-name ,julia-package-name))) (define guile-for-build (match guile diff --git a/guix/build/julia-build-system.scm b/guix/build/julia-build-system.scm index 61817e0b47..8f57045a8c 100644 --- a/guix/build/julia-build-system.scm +++ b/guix/build/julia-build-system.scm @@ -21,6 +21,8 @@ #:use-module ((guix build gnu-build-system) #:prefix gnu:) #:use-module (guix build utils) #:use-module (ice-9 match) + #:use-module (ice-9 regex) + #:use-module (ice-9 rdelim) #:export (%standard-phases julia-create-package-toml julia-build)) @@ -37,18 +39,34 @@ ;; subpath where we store the package content (define %package-path "/share/julia/packages/") -(define* (install #:key source inputs outputs #:allow-other-keys) +(define (project.toml->name file) + "Look for Julia package name in the TOML file FILE (usually named +Project.toml)." + (call-with-input-file file + (lambda (in) + (let loop ((line (read-line in 'concat))) + (if (eof-object? line) + #f + (let ((m (string-match "name\\s*=\\s*\"(.*)\"" line))) + (if m (match:substring m 1) + (loop (read-line in 'concat))))))))) + +(define* (install #:key source inputs outputs julia-package-name + #:allow-other-keys) (let* ((out (assoc-ref outputs "out")) (package-dir (string-append out %package-path - (strip-store-file-name source)))) + (or + julia-package-name + (project.toml->name "Project.toml"))))) (mkdir-p package-dir) (copy-recursively (getcwd) package-dir)) #t) -(define* (precompile #:key source inputs outputs #:allow-other-keys) +(define* (precompile #:key source inputs outputs julia-package-name + #:allow-other-keys) (let* ((out (assoc-ref outputs "out")) (builddir (string-append out "/share/julia/")) - (package (strip-store-file-name source))) + (package (or julia-package-name (project.toml->name "Project.toml")))) (mkdir-p builddir) ;; With a patch, SOURCE_DATE_EPOCH is honored (setenv "SOURCE_DATE_EPOCH" "1") @@ -69,10 +87,11 @@ (string-append "pushfirst!(DEPOT_PATH, pop!(DEPOT_PATH)); using " package))) #t) -(define* (check #:key tests? source inputs outputs #:allow-other-keys) +(define* (check #:key tests? source inputs outputs julia-package-name + #:allow-other-keys) (when tests? (let* ((out (assoc-ref outputs "out")) - (package (strip-store-file-name source)) + (package (or julia-package-name (project.toml->name "Project.toml"))) (builddir (string-append out "/share/julia/"))) ;; With a patch, SOURCE_DATE_EPOCH is honored (setenv "SOURCE_DATE_EPOCH" "1") @@ -127,9 +146,11 @@ version = \"" version "\" (delete 'patch-usr-bin-file) (delete 'build))) -(define* (julia-build #:key inputs (phases %standard-phases) +(define* (julia-build #:key inputs julia-package-name + (phases %standard-phases) #:allow-other-keys #:rest args) "Build the given Julia package, applying all of PHASES in order." (apply gnu:gnu-build #:inputs inputs #:phases phases + #:julia-package-name julia-package-name args)) -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: v4-0003-gnu-julia-xyz-julia-compat-Set-file-name-accordin.patch --] [-- Type: text/x-patch, Size: 921 bytes --] From 8512ee370db8d8aaf7457a28db3d57778210c484 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Thu, 28 Jan 2021 16:23:29 +0100 Subject: [PATCH v4 03/11] gnu: julia-xyz (julia-compat): Set file-name according to standards. * gnu/packages/julia-xyz.scm (): New variable. --- gnu/packages/julia-xyz.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index f1da66a4be..f00faf6223 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -32,7 +32,7 @@ (uri (git-reference (url "https://github.com/JuliaLang/Compat.jl") (commit (string-append "v" version)))) - (file-name "Compat") + (file-name (git-file-name name version)) (sha256 (base32 "01vwjr2134bzgnaxwd67knbibbhnfgnqjw7gxrp29s6y2a6j3882")))) (build-system julia-build-system) -- 2.30.0 [-- Attachment #5: v4-0004-gnu-julia-compat-Update-to-3.25.0.patch --] [-- Type: text/x-patch, Size: 1453 bytes --] From 0c02cfcca23a9e7a96edbbfcc78ed4637a6d0ec5 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 17:42:50 +0100 Subject: [PATCH v4 04/11] gnu: julia-compat: Update to 3.25.0. * gnu/packages/julia-xyz.scm (julia-compat): Update to 3.25.0. --- gnu/packages/julia-xyz.scm | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index f00faf6223..7836489e15 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2020 Nicolò Balzarotti <nicolo@nixo.xyz> +;;; Copyright © 2020, 2021 Nicolò Balzarotti <nicolo@nixo.xyz> ;;; ;;; This file is part of GNU Guix. ;;; @@ -25,7 +25,7 @@ (define-public julia-compat (package (name "julia-compat") - (version "3.9.1") + (version "3.25.0") (source (origin (method git-fetch) @@ -34,7 +34,7 @@ (commit (string-append "v" version)))) (file-name (git-file-name name version)) (sha256 - (base32 "01vwjr2134bzgnaxwd67knbibbhnfgnqjw7gxrp29s6y2a6j3882")))) + (base32 "1m4r5i8mq29xjp3mllh6047n5a78sdyld57m15anrnsjgaapcgby")))) (build-system julia-build-system) (home-page "https://github.com/JuliaLang/Compat.jl") (synopsis "Compatibility across Julia versions") -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #6: v4-0005-gnu-Add-julia-orderedcollections.patch --] [-- Type: text/x-patch, Size: 1721 bytes --] From cc301e31e2fead723bd3a1d2a47ac8d31012de5d Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 18:11:42 +0100 Subject: [PATCH v4 05/11] gnu: Add julia-orderedcollections. * gnu/packages/julia-xyz.scm (julia-orderedcollections): New variable. --- gnu/packages/julia-xyz.scm | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index 7836489e15..ab5bba1bda 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -43,3 +43,25 @@ between older and newer versions of the Julia language. The Compat package provides a macro that lets you use the latest syntax in a backwards-compatible way.") (license license:expat))) + +(define-public julia-orderedcollections + (package + (name "julia-orderedcollections") + (version "1.3.2") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaCollections/OrderedCollections.jl") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "0sfip1ixghsz91q2s7d62rgzw3gppg42fg6bccxlplqa3hfmbycf")))) + (build-system julia-build-system) + (home-page "https://github.com/JuliaCollections/OrderedCollections.jl") + (synopsis "Associative containers that preserve insertion order") + (description "This package implements @code{OrderedDicts} and +@code{OrderedSets}, which are similar to containers in base Julia. However, +during iteration the @code{Ordered*} containers return items in the order in +which they were added to the collection.") + (license license:expat))) -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #7: v4-0006-gnu-Add-julia-datastructures.patch --] [-- Type: text/x-patch, Size: 1790 bytes --] From 373e5eddc67703d3315c19dd0f26609bd5a7dbc2 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 18:17:50 +0100 Subject: [PATCH v4 06/11] gnu: Add julia-datastructures. * gnu/packages/julia-xyz.scm (julia-datastructures): New variable. --- gnu/packages/julia-xyz.scm | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index ab5bba1bda..bd7802b814 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -44,6 +44,30 @@ provides a macro that lets you use the latest syntax in a backwards-compatible way.") (license license:expat))) +(define-public julia-datastructures + (package + (name "julia-datastructures") + (version "0.18.9") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaCollections/DataStructures.jl") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "0hdqp8ipsqdw5bqqkdvz4j6n67x80sj5azr9vzyxwjfsgkfbnk2l")))) + (propagated-inputs + `(("julia-compat" ,julia-compat) + ("julia-orderedcollections" ,julia-orderedcollections))) + (build-system julia-build-system) + (home-page "https://github.com/JuliaCollections/DataStructures.jl") + (synopsis "Julia module providing different data structures") + (description "This package implements a variety of data structures, +including, @code{CircularBuffer}, @code{Queue}, @code{Stack}, +@code{Accumulators}, @code{LinkedLists}, @code{SortedDicts} and many others.") + (license license:expat))) + (define-public julia-orderedcollections (package (name "julia-orderedcollections") -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #8: v4-0007-gnu-Add-julia-fixedpointnumbers.patch --] [-- Type: text/x-patch, Size: 2417 bytes --] From c012a27c414e6568b4db8338b914862ed6da007b Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 18:25:02 +0100 Subject: [PATCH v4 07/11] gnu: Add julia-fixedpointnumbers. * gnu/packages/julia-xyz.scm (julia-fixedpointnumbers): New variable. --- gnu/packages/julia-xyz.scm | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index bd7802b814..b28906b546 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -68,6 +68,41 @@ including, @code{CircularBuffer}, @code{Queue}, @code{Stack}, @code{Accumulators}, @code{LinkedLists}, @code{SortedDicts} and many others.") (license license:expat))) +(define-public julia-fixedpointnumbers + (package + (name "julia-fixedpointnumbers") + (version "0.8.4") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaMath/FixedPointNumbers.jl") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "0j0n40n04q9sk68wh9jq90m6c67k4ws02k41djjzkrqmpzv4rcdi")))) + (build-system julia-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'unpack 'disable-failing-test + (lambda* (#:key outputs #:allow-other-keys) + (substitute* "test/fixed.jl" + ;; A deprecation warning is not thrown + (("@test_logs.*:warn" all) (string-append "# " all))) + #t))))) + (propagated-inputs `(("julia-compat" ,julia-compat))) + (home-page "https://github.com/JuliaMath/FixedPointNumbers.jl") + (synopsis "Fixed point types for Julia") + (description "@code{FixedPointNumbers.jl} implements fixed-point number +types for Julia. A fixed-point number represents a fractional, or +non-integral, number. In contrast with the more widely known floating-point +numbers, with fixed-point numbers the decimal point doesn't \"float\": +fixed-point numbers are effectively integers that are interpreted as being +scaled by a constant factor. Consequently, they have a fixed number of +digits (bits) after the decimal (radix) point.") + (license license:expat))) + (define-public julia-orderedcollections (package (name "julia-orderedcollections") -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #9: v4-0008-gnu-Add-julia-parsers.patch --] [-- Type: text/x-patch, Size: 1474 bytes --] From 1a116a54820f5e0434bdffb7bdb837035a76c89d Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 18:27:32 +0100 Subject: [PATCH v4 08/11] gnu: Add julia-parsers. * gnu/packages/julia-xyz.scm (julia-parsers): New variable. --- gnu/packages/julia-xyz.scm | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index b28906b546..9a615aba58 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -124,3 +124,23 @@ digits (bits) after the decimal (radix) point.") during iteration the @code{Ordered*} containers return items in the order in which they were added to the collection.") (license license:expat))) + +(define-public julia-parsers + (package + (name "julia-parsers") + (version "1.0.15") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaData/Parsers.jl") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "16iffl6l28kspgqch48mhi1s8qhspr3cpqcwsph3rqi72lbfqygx")))) + (build-system julia-build-system) + (home-page "https://github.com/JuliaData/Parsers.jl") + (synopsis "Fast parsing machinery for basic types in Julia") + (description "@code{Parsers.jl} is a collection of type parsers and +utilities for Julia.") + (license license:expat))) -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #10: v4-0009-gnu-Add-julia-adapt.patch --] [-- Type: text/x-patch, Size: 1602 bytes --] From 63d2bd90609ecddb25e9dd3c646739c1912e6e45 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Mon, 18 Jan 2021 23:46:51 +0100 Subject: [PATCH v4 09/11] gnu: Add julia-adapt. * gnu/packages/julia-xyz.scm (julia-adapt): New variable. --- gnu/packages/julia-xyz.scm | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index 9a615aba58..2e0efb0b5a 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -22,6 +22,28 @@ #:use-module (guix git-download) #:use-module (guix build-system julia)) +(define-public julia-adapt + (package + (name "julia-adapt") + (version "3.1.1") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaGPU/Adapt.jl") + (commit (string-append "v" version)))) + (file-name "Adapt") + (sha256 + (base32 "1lks6k3a1gvwlplld47nh6xfy3nnlpc0vhkzg6zg0qn33qdmavrg")))) + (build-system julia-build-system) + (home-page "https://github.com/JuliaGPU/Adapt.jl") + (synopsis "Package providing the @code{adapt} function, similar to @code{convert}") + (description "This Julia package provides the @code{adapt(T, x)} function +acts like @code{convert(T, x)}, but without the restriction of returning a +@code{T}. This allows you to \"convert\" wrapper types like @code{Adjoint} to +be GPU compatible without throwing away the wrapper.") + (license license:expat))) + (define-public julia-compat (package (name "julia-compat") -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #11: v4-0010-gnu-Add-julia-offsetarrays.patch --] [-- Type: text/x-patch, Size: 1812 bytes --] From 6e5c1ccfd8a805e8b0b16ddeb4d56fc8f738b078 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Tue, 19 Jan 2021 00:01:28 +0100 Subject: [PATCH v4 10/11] gnu: Add julia-offsetarrays. * gnu/packages/julia-xyz.scm (julia-offsetarrays): New variable. --- gnu/packages/julia-xyz.scm | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index 2e0efb0b5a..3dd731ef28 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -147,6 +147,31 @@ during iteration the @code{Ordered*} containers return items in the order in which they were added to the collection.") (license license:expat))) +(define-public julia-offsetarrays + (package + (name "julia-offsetarrays") + (version "1.5.2") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaArrays/OffsetArrays.jl") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "1y3fnssw2hzyghrk6jfcxslab0f8sjkjszh482snfq4k6mkrhy77")))) + (build-system julia-build-system) + (propagated-inputs + `(("julia-adapt" ,julia-adapt))) + ;; CatIndices depends on OffsetArrays, introducing a recursive dependency + (arguments '(#:tests? #f)) + (home-page "https://juliaarrays.github.io/OffsetArrays.jl/stable/") + (synopsis "Fortran-like arrays with arbitrary, zero or negative indices") + (description "@code{OffsetArrays.jl} provides Julia users with arrays that +have arbitrary indices, similar to those found in some other programming +languages like Fortran.") + (license license:expat))) + (define-public julia-parsers (package (name "julia-parsers") -- 2.30.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #12: v4-0011-gnu-Add-julia-json.patch --] [-- Type: text/x-patch, Size: 1747 bytes --] From 3da3028123253ebe69017607e3dc7c8825b47c51 Mon Sep 17 00:00:00 2001 From: nixo <nicolo@nixo.xyz> Date: Tue, 19 Jan 2021 00:06:34 +0100 Subject: [PATCH v4 11/11] gnu: Add julia-json. * gnu/packages/julia-xyz.scm (julia-json): New variable. --- gnu/packages/julia-xyz.scm | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index 3dd731ef28..34dba958ce 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -125,6 +125,31 @@ scaled by a constant factor. Consequently, they have a fixed number of digits (bits) after the decimal (radix) point.") (license license:expat))) +(define-public julia-json + (package + (name "julia-json") + (version "0.21.1") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaIO/JSON.jl") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "1f9k613kbknmp4fgjxvjaw4d5sfbx8a5hmcszmp1w9rqfqngjx9m")))) + (build-system julia-build-system) + (propagated-inputs + `(("julia-datastructures" ,julia-datastructures) + ("julia-fixedpointnumbers" ,julia-fixedpointnumbers) + ("julia-parsers" ,julia-parsers) + ("julia-offsetarrays" ,julia-offsetarrays))) + (home-page "https://github.com/JuliaIO/JSON.jl") + (synopsis "JSON parsing and printing library for Julia") + (description "@code{JSON.jl} is a pure Julia module which supports parsing +and printing JSON documents.") + (license license:expat))) + (define-public julia-orderedcollections (package (name "julia-orderedcollections") -- 2.30.0 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* bug#45972: [PATCH] Add julia-json with dependencies 2021-01-28 15:37 ` Nicolò Balzarotti @ 2021-01-30 14:10 ` Ludovic Courtès 0 siblings, 0 replies; 14+ messages in thread From: Ludovic Courtès @ 2021-01-30 14:10 UTC (permalink / raw) To: Nicolò Balzarotti; +Cc: 45972-done Hi, Nicolò Balzarotti <anothersms@gmail.com> skribis: > Done, I attached the updated patch set! Applied it all, thanks a lot! Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* [bug#45972] Julia importer? 2021-01-28 0:30 ` Nicolò Balzarotti 2021-01-28 13:10 ` Ludovic Courtès @ 2021-01-30 14:46 ` zimoun 2021-01-30 20:13 ` Nicolò Balzarotti 1 sibling, 1 reply; 14+ messages in thread From: zimoun @ 2021-01-30 14:46 UTC (permalink / raw) To: Nicolò Balzarotti, Ludovic Courtès; +Cc: 45972 Hi, On Thu, 28 Jan 2021 at 01:30, Nicolò Balzarotti <anothersms@gmail.com> wrote: > Sure, I'd just read it from Package.toml (nowadays almost all the > packages have this file, and for when it's missing we also have the > julia-create-package-toml procedure that creates it). > > The file is toml, but I don't see a toml parser in guix. So, I'd use a > function like the following: > > #+begin_src scheme > (define (package.toml->name file) > (call-with-input-file file > (lambda (in) > (let loop ((line (read-line in 'concat))) > (if (eof-object? line) > #f ;What to do? > (let ((m (string-match "name\\s*=\\s*\"(.*)\"" line))) > (if m (match:substring m 1) > (loop (read-line in 'concat))))))))) > #+end_src [...] > The other way I think this is easily solvable is by asking julia > directly, by reading the output of: > > (invoke-julia "using Pkg; Pkg.TOML.parsefile("Project.toml")["name"] |> println") With a bit more glue, could this be transformed into something like “julia->guix-package”? And so have a Julia package importer, even if it fails for some cases. WDYT? All the best, simon ^ permalink raw reply [flat|nested] 14+ messages in thread
* [bug#45972] Julia importer? 2021-01-30 14:46 ` [bug#45972] Julia importer? zimoun @ 2021-01-30 20:13 ` Nicolò Balzarotti 2021-01-30 21:42 ` Nicolò Balzarotti 2021-01-31 19:35 ` zimoun 0 siblings, 2 replies; 14+ messages in thread From: Nicolò Balzarotti @ 2021-01-30 20:13 UTC (permalink / raw) To: zimoun, Ludovic Courtès; +Cc: 45972 [-- Attachment #1: Type: text/plain, Size: 1108 bytes --] zimoun <zimon.toutoune@gmail.com> writes: > Hi, Hi Simon! >> >> (invoke-julia "using Pkg; Pkg.TOML.parsefile("Project.toml")["name"] |> println") > > With a bit more glue, could this be transformed into something like > “julia->guix-package”? And so have a Julia package importer, even if it > fails for some cases. Well, if you mean "Can we use Pkg.jl to generate package definitions for us?" the answer is "probably yes, but I never investigated this". That line uses just Julia Base TOML.jl, which for some reason is defined inside module Pkg.jl (and it's not top-level). If you instead meant "Can we have a Julia importer?" some time ago I wrote one in Julia, I've not used it in the last year, but it did work quite well back then. I attach it here for reference. But before digging into Pkg3, I need to push a new set of packages which contains julia-jllwrappers, which is needed for Julia packages which require binary distributions to work. After that, it will be possible to create an importer that work "ok" even with packages depending on .so libraries. [-- Attachment #2: julia import.jl --] [-- Type: text/plain, Size: 5358 bytes --] using Pkg using Pkg.TOML using LibGit2 const base = ["Base64", "CRC32c", "Dates", "DelimitedFiles", "Distributed", "FileWatching", "Future", "InteractiveUtils", "Libdl", "LibGit2", "LinearAlgebra", "Logging", "Markdown", "Mmap", "Pkg", "Printf", "Profile", "Random", "REPL", "Serialization", "SHA", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "SuiteSparse", "Test", "Unicode", "UUIDs"] const disabled = ["WinRPM", "Homebrew", "CMake", "Docile", "Color", "HTTPClient", "ICU", "Calendar", "LegacyStrings", "Nulls"] registrypath = expanduser("~/.julia-old/registries/General/") registry = joinpath(registrypath, "Registry.toml") const register = TOML.parse(join(readlines(registry), "\n")) jlpkgname(info) = "julia-$(lowercase(info.name))" function getrev(path) versions = TOML.parse(join(readlines(path), "\n")) versionv = findmax(map(x -> VersionNumber(x), keys(versions) |> collect)) (rev = versions[string(versionv[1])]["git-tree-sha1"], ver = versionv[1]) end function getpackagebyuuid(uuid) uuid in keys(register["packages"]) || return nothing path = register["packages"][uuid]["path"] getpath(x) = joinpath(registrypath, joinpath(path, x)) package = TOML.parse(join(readlines(getpath("Package.toml")), "\n")) deppath = getpath("Deps.toml") isfile(deppath) || return nothing deps = TOML.parse(join(readlines(deppath), "\n")) (name = package["name"], uuid = package["uuid"], repo = package["repo"], deps = deps, vers = getrev(getpath("Versions.toml"))) end function getpackage(wanted) uuid = findfirst(p -> lowercase(p["name"]) == lowercase(wanted), register["packages"]) uuid == nothing && return nothing return getpackagebyuuid(uuid) end function getdeps!(deps, vers, recursive, out) flat(arr::Array) = mapreduce(x -> isa(x, Array) ? flat(x) : x, append!, arr, init=[]) v = map(p -> VersionNumber.(split(p, '-')), keys(deps) |> collect) valid = findall(x -> length(x) == 1 || (x[2] == v"0" && x[1] <= vers) || x[1] <= vers <= x[2], v) f = flat(map(x -> values(x), values(collect(values(deps))[valid]))) push!.(Ref(out), f) # if recursive # push! # end nothing end function have(info) file = "/home/nixo/git/guix/gnu/packages/julia-xyz.scm" return "(name \"" * jlpkgname(info) * "\")" in strip.(readlines(file)) end function gethash(info) wd = mktempdir() if info.name in base || info.name in disabled || have(info) return "" end println(stderr, "Cloning $(info.name) in $wd") repo = LibGit2.clone(info.repo, wd) hash = cd(wd) do out = Pipe() try LibGit2.checkout!(repo, string(LibGit2.GitHash(LibGit2.peel(LibGit2.GitCommit, LibGit2.GitTag(repo, "v" * string(info.vers.ver)))))) catch e try LibGit2.checkout!(repo, string(LibGit2.GitHash(LibGit2.peel(LibGit2.GitCommit, LibGit2.GitCommit(repo, "v" * string(info.vers.ver)))))) catch e # FIXME: if this happens, return the commit too and use it in the package println(stderr, "Failed to checkout $(e), continuing") end end run(pipeline(`guix hash -rx .`, stdout=out)) readline(out) end rm(wd, recursive = true) hash end function makepackage(info; done = []) if info === nothing @warn "Could not find package (have you cloned the registry?)" return elseif info in done return "" end push!(done, info) deps = String[] getdeps!(info.deps, info.vers.ver, true, deps) # TODO: remove deps that are in base deps = filter(x -> x !== nothing, getpackagebyuuid.(deps)) deplist = join(map(name -> "(\"$name\" ,$name)", jlpkgname.(deps)), '\n') packagedeps = join(makepackage.(deps, done = done), "") hash = gethash(info) hash == "" && return "" """ $packagedeps (define-public $(jlpkgname(info)) (package (name "$(jlpkgname(info))") (version "$(info.vers.ver)") (source (origin (method git-fetch) (uri (git-reference (url "$(info.repo)") (commit (string-append "v" version)))) (file-name "$(info.name)") (sha256 (base32 "$hash")))) (propagated-inputs `($(deplist))) (build-system julia-build-system) (home-page "$(info.repo)") (synopsis "") (description "") (license license:expat))) """ end println.(makepackage.(getpackage.(ARGS))) ^ permalink raw reply [flat|nested] 14+ messages in thread
* [bug#45972] Julia importer? 2021-01-30 20:13 ` Nicolò Balzarotti @ 2021-01-30 21:42 ` Nicolò Balzarotti 2021-01-31 19:35 ` zimoun 1 sibling, 0 replies; 14+ messages in thread From: Nicolò Balzarotti @ 2021-01-30 21:42 UTC (permalink / raw) To: zimoun, Ludovic Courtès; +Cc: 45972 As a self reminder, I found this repo https://github.com/cdluminate/DistroHelper.jl which might contain something useful for our purposes ^ permalink raw reply [flat|nested] 14+ messages in thread
* [bug#45972] Julia importer? 2021-01-30 20:13 ` Nicolò Balzarotti 2021-01-30 21:42 ` Nicolò Balzarotti @ 2021-01-31 19:35 ` zimoun 2021-01-31 20:00 ` Nicolò Balzarotti 1 sibling, 1 reply; 14+ messages in thread From: zimoun @ 2021-01-31 19:35 UTC (permalink / raw) To: Nicolò Balzarotti, Ludovic Courtès; +Cc: 45972 Hi, On Sat, 30 Jan 2021 at 21:13, Nicolò Balzarotti <anothersms@gmail.com> wrote: > zimoun <zimon.toutoune@gmail.com> writes: >>> (invoke-julia "using Pkg; Pkg.TOML.parsefile("Project.toml")["name"] |> println") >> >> With a bit more glue, could this be transformed into something like >> “julia->guix-package”? And so have a Julia package importer, even if it >> fails for some cases. > > Well, if you mean "Can we use Pkg.jl to generate package definitions for > us?" the answer is "probably yes, but I never investigated this". That > line uses just Julia Base TOML.jl, which for some reason is defined > inside module Pkg.jl (and it's not top-level). I mean the correct way is to write a TOML parser, probably using PEG. As it is done with the other importers. It is a piece of work. Not especially hard but it could take some time. So instead, the question is, using a bit of Julia glue and Guile glue, is it possible to extract the necessary information to have a Guix package? Well, maybe these glue code is the same as writing a TOML parser. :-) > If you instead meant "Can we have a Julia importer?" some time ago I > wrote one in Julia, I've not used it in the last year, but it did work > quite well back then. I attach it here for reference. Fun! One thing is how to deal with ’Manifest.toml’. And pre-compiled substitutes do not make too much sense in the Julia world. Well, I do not know how these Manifest.toml overlap with how Guix works. Cheers, simon ^ permalink raw reply [flat|nested] 14+ messages in thread
* [bug#45972] Julia importer? 2021-01-31 19:35 ` zimoun @ 2021-01-31 20:00 ` Nicolò Balzarotti 2021-02-01 7:22 ` zimoun 0 siblings, 1 reply; 14+ messages in thread From: Nicolò Balzarotti @ 2021-01-31 20:00 UTC (permalink / raw) To: zimoun, Ludovic Courtès; +Cc: 45972 > I mean the correct way is to write a TOML parser, probably using PEG. > As it is done with the other importers. > It is a piece of work. Not especially hard but it could take some time. That wouldn't be too hard, but also it would not be enough. As you can see in [fn:1], there's just the list of dependencies, but no info on how to get them, so parsing the Julia General Registry [fn:2] is still required. > So instead, the question is, using a bit of Julia glue and Guile glue, > is it possible to extract the necessary information to have a Guix > package? > > Well, maybe these glue code is the same as writing a TOML parser. :-) This might be feasible, but as I said above, unfortunately I don't think that a TOML parser would be enough. But asking "Pkg3" to resolve dependencies for us, yes, this makes sense. > One thing is how to deal with ’Manifest.toml’. And pre-compiled > substitutes do not make too much sense in the Julia world. Well, I do > not know how these Manifest.toml overlap with how Guix works. WDYM? Julia uses .ji files (which are the analogoues of .go for guile and .pyc for python), if this is what you are referring to. I'm just ignoring Manifest.toml, which should be the same as venv in the python world. [fn:1] https://github.com/JuliaLang/TOML.jl/blob/master/Project.toml [fn:2] https://github.com/JuliaRegistries/General ^ permalink raw reply [flat|nested] 14+ messages in thread
* [bug#45972] Julia importer? 2021-01-31 20:00 ` Nicolò Balzarotti @ 2021-02-01 7:22 ` zimoun 0 siblings, 0 replies; 14+ messages in thread From: zimoun @ 2021-02-01 7:22 UTC (permalink / raw) To: Nicolò Balzarotti, Ludovic Courtès; +Cc: 45972 Hi, On Sun, 31 Jan 2021 at 21:00, Nicolò Balzarotti <anothersms@gmail.com> wrote: > That wouldn't be too hard, but also it would not be enough. As you can > see in [fn:1], there's just the list of dependencies, but no info on how > to get them, so parsing the Julia General Registry [fn:2] is still required. [...] > I'm just ignoring Manifest.toml, which should be the same as venv in the > python world. Sorry, you have right. I have misremembered an old presentation about Pkg3 by Stefan Karpinski. Well, thanks for the pointers. And I gave a look at the Pkg doc. ;-) Once a PEG parser for TOML files is written, all the information is in the Registry (by default this General repository), recursively. However, the “resolve” part is not straightforward, IMHO. It is what confused me in my previous email and I thought was “Manifest.toml”. Anyway. Instead of reinventing the wheel and reimplement Pkg.jl in Guix, maybe it is worth to have Julia code as you did that extracts the relevant information and then generates the relevant Guix packages. However, it should mean that “guix import julia” requires to have the package julia implicitly used. Why not. :-) Cheers, simon ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2021-02-01 7:32 UTC | newest] Thread overview: 14+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-01-18 23:47 [bug#45972] [PATCH] Add julia-json with dependencies Nicolò Balzarotti 2021-01-26 21:33 ` Ludovic Courtès 2021-01-26 23:23 ` Nicolò Balzarotti 2021-01-27 22:40 ` Ludovic Courtès 2021-01-28 0:30 ` Nicolò Balzarotti 2021-01-28 13:10 ` Ludovic Courtès 2021-01-28 15:37 ` Nicolò Balzarotti 2021-01-30 14:10 ` bug#45972: " Ludovic Courtès 2021-01-30 14:46 ` [bug#45972] Julia importer? zimoun 2021-01-30 20:13 ` Nicolò Balzarotti 2021-01-30 21:42 ` Nicolò Balzarotti 2021-01-31 19:35 ` zimoun 2021-01-31 20:00 ` Nicolò Balzarotti 2021-02-01 7:22 ` zimoun
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).