* Re: Guix build side - including modules (guix hash), (guix gcrypt): How to provide shared library?
2017-01-02 10:06 ` David Craven
@ 2017-01-02 13:51 ` Danny Milosavljevic
2017-01-02 14:31 ` David Craven
0 siblings, 1 reply; 5+ messages in thread
From: Danny Milosavljevic @ 2017-01-02 13:51 UTC (permalink / raw)
To: David Craven; +Cc: guix-devel
Hi David,
On Mon, 2 Jan 2017 11:06:27 +0100
David Craven <david@craven.ch> wrote:
> > how can I use (guix hash) on the build side? It requires the libgcrypt shared library and that's not available on the build side...
>
> Depends. For a one off package I'd add it to native-inputs. Otherwise
> you can create a new build-system that wraps the gnu-build-system and
> extend the gnu-build-system's implicit inputs with libgcrypt. Which is
> basically just automatically adding it to native-inputs for you.
Hmm. I tried both, also in host-inputs and native-inputs but it doesn't seem to work. What I mean is I want to use gcrypt in Guile on the build side. That's a lot more difficult than I thought...
Also, why is sha256 used via FFI via an external library? It's not exactly difficult to implement in Guile :P
Error messages:
@ build-started /gnu/store/3lfanl2xnmx3gr2j11hc51a3m7sp7ljp-module-import-compiled.drv - x86_64-linux /var/log/guix/drvs/3l//fanl2xnmx3gr2j11hc51a3m7sp7ljp-module-import-compiled.drv.bz2
Backtrace:
In ice-9/boot-9.scm:
2404: 19 [save-module-excursion #<procedure 1370870 at ice-9/boot-9.scm:3069:17 ()>]
3088: 18 [#<procedure 1370870 at ice-9/boot-9.scm:3069:17 ()>]
In unknown file:
?: 17 [primitive-load-path "guix/hash" ...]
In ice-9/eval.scm:
505: 16 [#<procedure 79b560 at ice-9/eval.scm:499:4 (exp)> (define-module # # ...)]
In ice-9/psyntax.scm:
1106: 15 [expand-top-sequence ((define-module (guix hash) #:use-module ...)) () ...]
989: 14 [scan ((define-module (guix hash) #:use-module ...)) () ...]
279: 13 [scan ((#(syntax-object let # ...) (#) (# #) ...)) () ...]
In ice-9/eval.scm:
411: 12 [eval # ()]
In ice-9/boot-9.scm:
2954: 11 [define-module* (guix hash) #:filename ...]
2929: 10 [resolve-imports ((#) (#) (#) (#) ...)]
2867: 9 [resolve-interface (guix gcrypt) #:select ...]
2792: 8 [#<procedure 852d00 at ice-9/boot-9.scm:2780:4 (name #:optional autoload version #:key ensure)> # ...]
3068: 7 [try-module-autoload (guix gcrypt) #f]
2404: 6 [save-module-excursion #<procedure 13840c0 at ice-9/boot-9.scm:3069:17 ()>]
3088: 5 [#<procedure 13840c0 at ice-9/boot-9.scm:3069:17 ()>]
In unknown file:
?: 4 [primitive-load-path "guix/gcrypt" ...]
In ice-9/eval.scm:
453: 3 [eval # ()]
411: 2 [eval # ()]
In unknown file:
?: 1 [dynamic-link "/gnu/store/4pnp5scrgmjp21flaihmgbckwrz6z4g3-libgcrypt-1.7.3/lib/libgcrypt"]
In ice-9/boot-9.scm:
109: 0 [#<procedure 983c80 at ice-9/boot-9.scm:100:6 (thrown-k . args)> misc-error ...]
ice-9/boot-9.scm:109:20: In procedure #<procedure 983c80 at ice-9/boot-9.scm:100:6 (thrown-k . args)>:
ice-9/boot-9.scm:109:20: In procedure dynamic-link: file: "/gnu/store/4pnp5scrgmjp21flaihmgbckwrz6z4g3-libgcrypt-1.7.3/lib/libgcrypt", message: "file not found"
here- 8w6yqqc5f0ny3yq5x5lnx7hhiw60p6l9-gcrypt.scm
note: keeping build directory `/tmp/guix-build-module-import-compiled.drv-0'
builder for `/gnu/store/3lfanl2xnmx3gr2j11hc51a3m7sp7ljp-module-import-compiled.drv' failed with exit code 1
@ build-failed /gnu/store/3lfanl2xnmx3gr2j11hc51a3m7sp7ljp-module-import-compiled.drv - 1 builder for `/gnu/store/3lfanl2xnmx3gr2j11hc51a3m7sp7ljp-module-import-compiled.drv' failed with exit code 1
cannot build derivation `/gnu/store/asr76labrpqq0kcg64yni0sqsxj0d30y-rust-gtk-0.1.1.drv': 1 dependencies couldn't be built
guix build: error: build failed: build of `/gnu/store/asr76labrpqq0kcg64yni0sqsxj0d30y-rust-gtk-0.1.1.drv' failed
My log file doesn't contain any matching files (I added debugging "display" statements).
Source (for important parts search for the text "HERE"):
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2016 David Craven <david@craven.ch>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix build cargo-build-system)
#:use-module ((guix build gnu-build-system) #:prefix gnu:)
#:use-module (guix build utils)
#:use-module (guix hash)
#:use-module (guix base32)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:use-module (rnrs bytevectors)
#:use-module (ice-9 binary-ports)
#:use-module (system foreign)
#:use-module ((guix build utils) #:select (dump-port))
#:use-module (srfi srfi-11)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-34)
#:use-module (srfi srfi-35)
#:use-module (srfi srfi-60)
#:use-module (rnrs bytevectors)
#:use-module (ice-9 vlist)
#:use-module (system foreign)
#:export (%standard-phases
cargo-build))
;; Commentary:
;;
;; Builder-side code of the standard Rust package build procedure.
;;
;; Code:
;; FIXME: Needs to be parsed from url not package name.
(define (package-name->crate-name name)
"Return the crate name of NAME."
(match (string-split name #\-)
(("rust" rest ...)
(string-join rest "-"))
(_ #f)))
(define* (configure #:key inputs #:allow-other-keys)
"Replace Cargo.toml [dependencies] section with guix inputs."
(system* "chmod" "+w" "Cargo.toml")
(system* "chmod" "+w" ".")
(let ((port (open-file "Cargo.toml.disable" "a" #:encoding "utf-8")))
(format port "~%[replace]~%")
(for-each
(match-lambda
((name . path)
(let ((crate (package-name->crate-name name)))
(when (and crate path)
(match (string-split (basename path) #\-)
((_ ... version)
(format port "\"~a:~a\" = { path = \"~a/rustsrc\" }~%"
crate version path)))))))
inputs)
(close-port port))
#t)
(define* (build #:key (cargo-build-flags '("--release")) #:allow-other-keys)
"Build a given Cargo package."
(and (zero? (apply system* `("cargo" "build" ,@cargo-build-flags))))
(zero? (system* "pwd"))
(zero? (system* "ls")))
(define* (check #:key tests? #:allow-other-keys)
"Run tests for a given Cargo package."
(when tests?
(zero? (system* "cargo" "test"))))
(define (generate-checksums dir-name)
(let* ((file-names (find-files dir-name "."))
(dir-prefix-name (string-append dir-name "/"))
(dir-prefix-name-len (string-length dir-prefix-name))
(checksums-file-name (string-append dir-name "/.cargo-checksum.json"))
(checksums-temp-file-name (string-append checksums-file-name ".tmp"))
(port (open-file checksums-temp-file-name "w")))
(display "{\"files\":{")
(for-each (lambda (file-name)
(let ((file-relative-name (string-drop file-name dir-prefix-name-len)))
(write file-relative-name)
(display ":")
(write (bytevector->nix-base32-string (file-sha256 file-name))) ; <========================================= HERE
(display ",") ; FIXME drop trailing comma?
(newline) ; FIXME remove
)) file-names)
(display "}, \"package\": ")
; FIXME add crate's sha256 checksum here
(display "}")
(newline)
(close-port port)
(rename-file checksums-temp-file-name checksums-file-name)))
(define* (install #:key inputs outputs #:allow-other-keys)
"Install a given Cargo package."
(let* ((out (assoc-ref outputs "out"))
(src (assoc-ref inputs "source"))
(bin (string-append out "/bin"))
(rsrc (string-append out "/rustsrc")))
(mkdir-p rsrc)
;; Rust doesn't have a stable ABI yet. Because of this
;; Cargo doesn't have a search path for binaries yet.
;; Until this changes we are working around this by
;; distributing crates as source and replacing
;; references in Cargo.toml with store paths.
(system* "ls" "/gnu/store")
(copy-recursively "src" (string-append rsrc "/src"))
(install-file "Cargo.toml" rsrc)
(system* "touch" (string-append rsrc "/.cargo-ok"))
(generate-checksums rsrc)
;; When the package includes executables we install
;; it using cargo install. This fails when the crate
;; doesn't contain an executable.
(system* "cargo" "install" "--root" out)
#f))
(define %standard-phases
;; 'configure' phase is not needed.
(modify-phases gnu:%standard-phases
(replace 'configure configure)
(replace 'build build)
(replace 'check check)
(replace 'install install)))
(define* (cargo-build #:key inputs (phases %standard-phases)
#:allow-other-keys #:rest args)
"Build the given Cargo package, applying all of PHASES in order."
(display "Source: ")
(write (assoc-ref inputs "source"))
(newline)
(apply gnu:gnu-build #:inputs inputs #:phases phases args))
;;; cargo-build-system.scm ends here
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2013, 2014, 2015, 2016 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2013 Andreas Enge <andreas@enge.fr>
;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
;;; Copyright © 2016 David Craven <david@craven.ch>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix build-system cargo)
#:use-module (guix search-paths)
#:use-module (guix store)
#:use-module (guix utils)
#:use-module (guix derivations)
#:use-module (guix packages)
#:use-module (guix build-system)
#:use-module (guix build-system gnu)
#:use-module (ice-9 match)
#:use-module (srfi srfi-26)
#:export (cargo-build-system
crate-url
crate-url?
crate-uri))
(define crate-url "https://crates.io/api/v1/crates/")
(define crate-url? (cut string-prefix? crate-url <>))
(define (crate-uri name version)
"Return a URI string for the crate package hosted at crates.io corresponding
to NAME and VERSION."
(string-append crate-url name "/" version "/download"))
(define (default-cargo)
"Return the default Cargo package."
;; Lazily resolve the binding to avoid a circular dependency.
(let ((rust (resolve-interface '(gnu packages rust))))
(module-ref rust 'cargo)))
(define (default-rustc)
"Return the default Rustc package."
;; Lazily resolve the binding to avoid a circular dependency.
(let ((rust (resolve-interface '(gnu packages rust))))
(module-ref rust 'rustc-bootstrap)))
(define (default-libgcrypt)
"Return the default Libgcrypt package."
;; Lazily resolve the binding to avoid a circular dependency.
(let ((gnupg (resolve-interface '(gnu packages gnupg))))
(module-ref gnupg 'libgcrypt)))
(define %cargo-build-system-modules
;; Build-side modules imported by default.
`((guix build cargo-build-system)
(guix hash)
(guix base32)
(guix gcrypt)
(guix config)
,@%gnu-build-system-modules))
(define* (cargo-build store name inputs
#:key
(tests? #t)
(test-target #f)
(configure-flags #f)
(phases '(@ (guix build cargo-build-system)
%standard-phases))
(outputs '("out"))
(search-paths '())
(system (%current-system))
(guile #f)
(imported-modules %cargo-build-system-modules)
(modules '((guix build cargo-build-system)
(guix build utils))))
"Build SOURCE using CARGO, and with INPUTS."
(define builder
`(begin
(use-modules ,@modules)
(cargo-build #:name ,name
#:source ,(match (assoc-ref inputs "source")
(((? derivation? source))
(derivation->output-path source))
((source)
source)
(source
source))
#:system ,system
#:test-target ,test-target
#:tests? ,tests?
#:phases ,phases
#:outputs %outputs
#:search-paths ',(map search-path-specification->sexp
search-paths)
#:inputs %build-inputs)))
(define guile-for-build
(match guile
((? package?)
(package-derivation store guile system #:graft? #f))
(#f ; the default
(let* ((distro (resolve-interface '(gnu packages commencement)))
(guile (module-ref distro 'guile-final)))
(package-derivation store guile system #:graft? #f)))))
(build-expression->derivation store name builder
#:inputs inputs
#:system system
#:modules imported-modules
#:outputs outputs
#:guile-for-build guile-for-build))
(define* (lower name
#:key source inputs native-inputs outputs system target
(cargo (default-cargo))
(rustc (default-rustc))
(libgcrypt (default-libgcrypt))
#:allow-other-keys
#:rest arguments)
"Return a bag for NAME."
(define private-keywords
'(#:source #:target #:cargo #:rustc #:inputs #:native-inputs))
(and (not target) ;; TODO: support cross-compilation
(bag
(name name)
(system system)
(target target)
(host-inputs `(,@(if source
`(("source" ,source))
'())
("libgcrypt" ,libgcrypt) ; <========================================= HERE
,@inputs
;; Keep the standard inputs of 'gnu-build-system'
,@(standard-packages)))
(build-inputs `(("cargo" ,cargo)
("rustc" ,rustc)
("libgcrypt" ,libgcrypt) ; <========================================= HERE
,@native-inputs))
(outputs outputs)
(build (if target cargo-cross-build cargo-build))
(arguments (strip-keyword-arguments private-keywords arguments)))))
(define cargo-build-system
(build-system
(name 'cargo)
(description
"Cargo build system, to build Rust crates")
(lower lower)))
^ permalink raw reply [flat|nested] 5+ messages in thread