[PATCH 1/7] build-system: Add cargo build system.

[PATCH 1/7] build-system: Add cargo build system.

[David Craven]

* guix/build-system/cargo.scm (default-cargo, default-rustc,
  %cargo-build-system-modules, cargo-build, lower, cargo-build-system):
  New variables.
* guix/build/cargo-build-system.scm (configure, build, check, install,
  %standard-phases, cargo-build): New variables.
---
 guix/build-system/cargo.scm       | 149 ++++++++++++++++++++++++++++++++++++++
 guix/build/cargo-build-system.scm |  81 +++++++++++++++++++++
 2 files changed, 230 insertions(+)
 create mode 100644 guix/build-system/cargo.scm
 create mode 100644 guix/build/cargo-build-system.scm

diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
new file mode 100644
index 0000000..84cb644
--- /dev/null
+++ b/guix/build-system/cargo.scm
@@ -0,0 +1,149 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2013, 2014, 2015, 2016 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2013 Andreas Enge <andreas@enge.fr>
+;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
+;;; Copyright © 2016 David Craven <david@craven.ch>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix build-system cargo)
+  #:use-module (guix search-paths)
+  #:use-module (guix store)
+  #:use-module (guix utils)
+  #:use-module (guix derivations)
+  #:use-module (guix packages)
+  #:use-module (guix build-system)
+  #:use-module (guix build-system gnu)
+  #:use-module (ice-9 match)
+  #:export (cargo-build-system
+            crate-uri
+            system->rust-platform))
+
+(define (crate-uri name version)
+  "Return a URI string for the crate package hosted at crates.io corresponding
+to NAME and VERSION."
+  (string-append "https://crates.io/api/v1/crates/" name "/" version "/download"))
+
+(define (system->rust-platform system)
+  (cond
+   ((string-prefix? "x86_64" system) "x86_64-unknown-linux-gnu")
+   ((string-prefix? "i686" system) "i686-unknown-linux-gnu")))
+
+(define (default-cargo)
+  "Return the default Cargo package."
+  ;; Lazily resolve the binding to avoid a circular dependency.
+  (let ((rust (resolve-interface '(gnu packages rust))))
+    (module-ref rust 'cargo-bootstrap)))
+
+(define (default-rustc)
+  "Return the default Rustc package."
+  ;; Lazily resolve the binding to avoid a circular dependency.
+  (let ((rust (resolve-interface '(gnu packages rust))))
+    (module-ref rust 'rustc-bootstrap)))
+
+(define %cargo-build-system-modules
+  ;; Build-side modules imported by default.
+  `((guix build cargo-build-system)
+    ,@%gnu-build-system-modules))
+
+(define* (cargo-build store name inputs
+                      #:key
+                      (tests? #t)
+                      (test-target #f)
+                      (configure-flags #f)
+                      (phases '(@ (guix build cargo-build-system)
+                                  %standard-phases))
+                      (outputs '("out"))
+                      (search-paths '())
+                      (system (%current-system))
+                      (guile #f)
+                      (imported-modules %cargo-build-system-modules)
+                      (modules '((guix build cargo-build-system)
+                                 (guix build utils))))
+  "Build SOURCE using CARGO, and with INPUTS."
+
+  (define builder
+    `(begin
+       (use-modules ,@modules)
+       (cargo-build #:name ,name
+                    #:source ,(match (assoc-ref inputs "source")
+                                (((? derivation? source))
+                                 (derivation->output-path source))
+                                ((source)
+                                 source)
+                                (source
+                                 source))
+                    #:system ,system
+                    #:test-target ,test-target
+                    #:tests? ,tests?
+                    #:phases ,phases
+                    #:outputs %outputs
+                    #:search-paths ',(map search-path-specification->sexp
+                                          search-paths)
+                    #:inputs %build-inputs)))
+
+  (define guile-for-build
+    (match guile
+      ((? package?)
+       (package-derivation store guile system #:graft? #f))
+      (#f                                         ; the default
+       (let* ((distro (resolve-interface '(gnu packages commencement)))
+              (guile  (module-ref distro 'guile-final)))
+         (package-derivation store guile system #:graft? #f)))))
+
+  (build-expression->derivation store name builder
+                                #:inputs inputs
+                                #:system system
+                                #:modules imported-modules
+                                #:outputs outputs
+                                #:guile-for-build guile-for-build))
+
+(define* (lower name
+                #:key source inputs native-inputs outputs system target
+                (cargo (default-cargo))
+                (rustc (default-rustc))
+                #:allow-other-keys
+                #:rest arguments)
+  "Return a bag for NAME."
+
+  (define private-keywords
+    '(#:source #:target #:cargo #:rustc #:inputs #:native-inputs))
+
+  (and (not target) ;; TODO: support cross-compilation
+       (bag
+         (name name)
+         (system system)
+         (target target)
+         (host-inputs `(,@(if source
+                              `(("source" ,source))
+                              '())
+                        ,@inputs
+
+                        ;; Keep the standard inputs of 'gnu-build-system'
+                        ,@(standard-packages)))
+         (build-inputs `(("cargo" ,cargo)
+                         ("rustc" ,rustc)
+                         ,@native-inputs))
+         (outputs outputs)
+         (build (if target cargo-cross-build cargo-build))
+         (arguments (strip-keyword-arguments private-keywords arguments)))))
+
+(define cargo-build-system
+  (build-system
+    (name 'cargo)
+    (description
+     "Cargo build system, to build Rust crates")
+    (lower lower)))
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
new file mode 100644
index 0000000..abe7e05
--- /dev/null
+++ b/guix/build/cargo-build-system.scm
@@ -0,0 +1,81 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2016 David Craven <david@craven.ch>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix build cargo-build-system)
+  #:use-module ((guix build gnu-build-system) #:prefix gnu:)
+  #:use-module (guix build utils)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 ftw)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:export (%standard-phases
+            cargo-build))
+
+;; Commentary:
+;;
+;; Builder-side code of the standard Rust package build procedure.
+;;
+;; Code:
+
+(define* (configure #:rest _)
+  "Replace Cargo.toml [dependencies] section with guix inputs."
+  ;; TODO
+  #t)
+
+(define* (build #:key (cargo-build-flags '("--release")) #:allow-other-keys)
+  "Build a given Cargo package."
+  (zero? (apply system* `("cargo" "build" ,@cargo-build-flags))))
+
+(define* (check #:rest _)
+  "Run tests for a given Cargo package."
+  (zero? (system* "cargo" "test")))
+
+(define* (install #:key inputs outputs #:allow-other-keys)
+  "Install a given Cargo package."
+  (let* ((out (assoc-ref outputs "out"))
+         (src (assoc-ref inputs "source"))
+         (bin (string-append out "/bin"))
+         (rsrc (string-append out "/rustsrc")))
+    (mkdir-p rsrc)
+    ;; Rust doesn't have a stable ABI yet. Because of this
+    ;; Cargo doesn't have a search path for binaries yet.
+    ;; Until this changes we are working around this by
+    ;; distributing crates as source and replacing
+    ;; references in Cargo.toml with store paths.
+    (copy-recursively "src" (string-append rsrc "/src"))
+    (install-file "Cargo.toml" rsrc)
+    ;; When the package includes executables we install
+    ;; it using cargo install. This fails when the crate
+    ;; doesn't contain an executable.
+    (system* "cargo" "install" "--root" bin)
+    #t))
+
+(define %standard-phases
+  ;; 'configure' phase is not needed.
+  (modify-phases gnu:%standard-phases
+    (replace 'configure configure)
+    (replace 'build build)
+    (replace 'check check)
+    (replace 'install install)))
+
+(define* (cargo-build #:key inputs (phases %standard-phases)
+                      #:allow-other-keys #:rest args)
+  "Build the given Cargo package, applying all of PHASES in order."
+  (apply gnu:gnu-build #:inputs inputs #:phases phases args))
+
+;;; cargo-build-system.scm ends here
-- 
2.9.0

[PATCH 2/7] import: Add importer for rust crates.

[David Craven]

* guix/import/crate.scm (crate-fetch, make-crate-sexp,
  crate->guix-package, guix-package->crate-name, string->license,
  crate-name->package-name): New variables.
* guix/scripts/import/crate.scm (%default-options, show-help, %options,
  guix-import-crate): New variables.
* guix/scripts/import.scm (importers): Add crate to list of importers.
* tests/crate.scm (test-json, test-source-hash,
  guix-package->crate-name, crate->guix-package): New variables.
---
 doc/guix.texi                 |  5 +++
 guix/import/crate.scm         | 93 ++++++++++++++++++++++++++++++++++++++++++
 guix/scripts/import.scm       |  2 +-
 guix/scripts/import/crate.scm | 94 +++++++++++++++++++++++++++++++++++++++++++
 tests/crate.scm               | 83 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 276 insertions(+), 1 deletion(-)
 create mode 100644 guix/import/crate.scm
 create mode 100644 guix/scripts/import/crate.scm
 create mode 100644 tests/crate.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index c159e12..785d6fb 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -4986,6 +4986,11 @@ identifier.  This is the default.
 identifier.
 @end itemize
 @end table
+
+@item crate
+@cindex crate
+Import metadata from the crates.io rust package repository
+@uref{https://crates.io, crates.io}.
 @end table
 
 The structure of the @command{guix import} code is modular.  It would be
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
new file mode 100644
index 0000000..db7f119
--- /dev/null
+++ b/guix/import/crate.scm
@@ -0,0 +1,93 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2016 David Craven <david@craven.ch>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import crate)
+  #:use-module (guix base32)
+  #:use-module (guix build-system cargo)
+  #:use-module ((guix download) #:prefix download:)
+  #:use-module (guix hash)
+  #:use-module (guix http-client)
+  #:use-module (guix import utils)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix packages)
+  #:use-module (guix upstream)
+  #:use-module (guix utils)
+  #:use-module (ice-9 match)
+  #:use-module (json)
+  #:use-module (srfi srfi-1)
+  #:export (crate->guix-package
+            guix-package->crate-name))
+
+(define (crate-fetch name)
+  "Return an alist representation of the crates.io metadata for the package NAME,
+or #f on failure."
+  (let* ((url (string-append "https://crates.io/api/v1/crates/" name))
+         (port (http-fetch url))
+         (result (json->scm port)))
+    (close-port port)
+    (hash-table->alist result)))
+
+;; TODO: Import inputs and native-inputs
+(define (make-crate-sexp name version home-page synopsis description license)
+  "Return the `package' s-expression for a rust package with the given NAME,
+VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
+  (let* ((port (http-fetch (crate-uri name version)))
+         (pkg `(package
+                 (name ,(crate-name->package-name name))
+                 (version ,version)
+                 (source (origin
+                           (method url-fetch)
+                           (uri (crate-uri ,name version))
+                           (file-name (string-append name "-" version ".tar.gz"))
+                           (sha256
+                            (base32
+                             ,(bytevector->nix-base32-string (port-sha256 port))))))
+                 (build-system cargo-build-system)
+                 (home-page ,home-page)
+                 (synopsis ,synopsis)
+                 (description ,(beautify-description description))
+                 (license ,(match license
+                             (() #f)
+                             ((license) license)
+                             (_ `(list ,@license)))))))
+    (close-port port)
+    pkg))
+
+(define (crate->guix-package crate-name)
+  "Fetch the metadata for CRATE-NAME from crates.io, and return the
+`package' s-expression corresponding to that package, or #f on failure."
+  (let ((crate (crate-fetch crate-name)))
+    (let ((name (assoc-ref* crate "crate" "name"))
+          (version (assoc-ref* crate "crate" "max_version"))
+          (home-page (assoc-ref* crate "crate" "homepage"))
+          (synopsis (assoc-ref* crate "crate" "description"))
+          (description (assoc-ref* crate "crate" "description"))
+          (license (string->license (assoc-ref* crate "crate" "license"))))
+      (make-crate-sexp name version home-page synopsis description license))))
+
+(define (guix-package->crate-name package)
+  "Return the crate name of PACKAGE."
+  (match (string-split (package-name package) #\-)
+    (("rust" rest ...)
+     (string-join rest "-"))))
+
+(define (string->license string)
+  (map spdx-string->license (string-split string #\/)))
+
+(define (crate-name->package-name name)
+  (string-append "rust-" name))
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index e54744f..c671686 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -73,7 +73,7 @@ rather than \\n."
 ;;; Entry point.
 ;;;
 
-(define importers '("gnu" "nix" "pypi" "cpan" "hackage" "elpa" "gem" "cran"))
+(define importers '("gnu" "nix" "pypi" "cpan" "hackage" "elpa" "gem" "cran" "crate"))
 
 (define (resolve-importer name)
   (let ((module (resolve-interface
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
new file mode 100644
index 0000000..4337a0b
--- /dev/null
+++ b/guix/scripts/import/crate.scm
@@ -0,0 +1,94 @@
+
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2014 David Thompson <davet@gnu.org>
+;;; Copyright © 2016 David Craven <david@craven.ch>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix scripts import crate)
+  #:use-module (guix ui)
+  #:use-module (guix utils)
+  #:use-module (guix scripts)
+  #:use-module (guix import crate)
+  #:use-module (guix scripts import)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-11)
+  #:use-module (srfi srfi-37)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 format)
+  #:export (guix-import-crate))
+
+\f
+;;;
+;;; Command-line options.
+;;;
+
+(define %default-options
+  '())
+
+(define (show-help)
+  (display (_ "Usage: guix import crate PACKAGE-NAME
+Import and convert the crate.io package for PACKAGE-NAME.\n"))
+  (display (_ "
+  -h, --help             display this help and exit"))
+  (display (_ "
+  -V, --version          display version information and exit"))
+  (newline)
+  (show-bug-report-information))
+
+(define %options
+  ;; Specification of the command-line options.
+  (cons* (option '(#\h "help") #f #f
+                 (lambda args
+                   (show-help)
+                   (exit 0)))
+         (option '(#\V "version") #f #f
+                 (lambda args
+                   (show-version-and-exit "guix import crate")))
+         %standard-import-options))
+
+\f
+;;;
+;;; Entry point.
+;;;
+
+(define (guix-import-crate . args)
+  (define (parse-options)
+    ;; Return the alist of option values.
+    (args-fold* args %options
+                (lambda (opt name arg result)
+                  (leave (_ "~A: unrecognized option~%") name))
+                (lambda (arg result)
+                  (alist-cons 'argument arg result))
+                %default-options))
+
+  (let* ((opts (parse-options))
+         (args (filter-map (match-lambda
+                            (('argument . value)
+                             value)
+                            (_ #f))
+                           (reverse opts))))
+    (match args
+      ((package-name)
+       (let ((sexp (crate->guix-package package-name)))
+         (unless sexp
+           (leave (_ "failed to download meta-data for package '~a'~%")
+                  package-name))
+         sexp))
+      (()
+       (leave (_ "too few arguments~%")))
+      ((many ...)
+       (leave (_ "too many arguments~%"))))))
diff --git a/tests/crate.scm b/tests/crate.scm
new file mode 100644
index 0000000..38ee8b8
--- /dev/null
+++ b/tests/crate.scm
@@ -0,0 +1,83 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2014 David Thompson <davet@gnu.org>
+;;; Copyright © 2016 David Craven <david@craven.ch>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (test-crate)
+  #:use-module (guix import crate)
+  #:use-module (guix base32)
+  #:use-module (guix hash)
+  #:use-module (guix tests)
+  #:use-module (ice-9 iconv)
+  #:use-module (ice-9 match)
+  #:use-module (srfi srfi-64))
+
+(define test-json
+  "{
+  \"crate\": {
+    \"max_version\": \"1.0.0\",
+    \"name\": \"foo\",
+    \"license\": \"MIT/Apache-2.0\",
+    \"description\": \"summary\",
+    \"homepage\": \"http://example.com\",
+  }
+}")
+
+(define test-source-hash
+  "")
+
+(test-begin "crate")
+
+(test-equal "guix-package->crate-name"
+  "rustc-serialize"
+  (guix-package->crate-name
+   (dummy-package "rust-rustc-serialize")))
+
+(test-assert "crate->guix-package"
+  ;; Replace network resources with sample data.
+  (mock ((guix http-client) http-fetch
+         (lambda (url)
+           (match url
+             ("https://crates.io/api/v1/crates/foo"
+              (open-input-string test-json))
+             ("https://crates.io/api/v1/crates/foo/1.0.0/download"
+              (set! test-source-hash
+                (bytevector->nix-base32-string
+                 (sha256 (string->bytevector "empty file\n" "utf-8"))))
+              (open-input-string "empty file\n"))
+             (_ (error "Unexpected URL: " url)))))
+    (match (crate->guix-package "foo")
+      (('package
+         ('name "rust-foo")
+         ('version "1.0.0")
+         ('source ('origin
+                    ('method 'url-fetch)
+                    ('uri ('crate-uri "foo" 'version))
+                    ('file-name ('string-append 'name "-" 'version ".tar.gz"))
+                    ('sha256
+                     ('base32
+                      (? string? hash)))))
+         ('build-system 'cargo-build-system)
+         ('home-page "http://example.com")
+         ('synopsis "summary")
+         ('description "summary")
+         ('license ('list 'license:expat 'license:asl2.0)))
+       (string=? test-source-hash hash))
+      (x
+       (pk 'fail x #f)))))
+
+(test-end "crate")
-- 
2.9.0

Re: [PATCH 2/7] import: Add importer for rust crates.

[Ludovic Courtès]

David Craven <david@craven.ch> skribis:

> * guix/import/crate.scm (crate-fetch, make-crate-sexp,
>   crate->guix-package, guix-package->crate-name, string->license,
>   crate-name->package-name): New variables.
> * guix/scripts/import/crate.scm (%default-options, show-help, %options,
>   guix-import-crate): New variables.
> * guix/scripts/import.scm (importers): Add crate to list of importers.
> * tests/crate.scm (test-json, test-source-hash,
>   guix-package->crate-name, crate->guix-package): New variables.

[...]

> +@item crate
> +@cindex crate
> +Import metadata from the crates.io rust package repository

s/rust/Rust/

Otherwise one might think the packages are rusty or something.  ;-)

OK with this change, thanks!

Ludo’.

[PATCH 3/7] import: crate: Add crate updater.

[David Craven]

* guix/import/crate.scm (crate-package?, latest-release,
  %crate-updater): New variables.
* guix/scripts/refresh.scm (%updaters): Add %crate-updater to list of
  updaters.
---
 guix/import/crate.scm    | 36 +++++++++++++++++++++++++++++++++++-
 guix/scripts/refresh.scm |  4 +++-
 2 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index db7f119..8fab200 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -31,7 +31,8 @@
   #:use-module (json)
   #:use-module (srfi srfi-1)
   #:export (crate->guix-package
-            guix-package->crate-name))
+            guix-package->crate-name
+            %crate-updater))
 
 (define (crate-fetch name)
   "Return an alist representation of the crates.io metadata for the package NAME,
@@ -91,3 +92,36 @@ VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
 
 (define (crate-name->package-name name)
   (string-append "rust-" name))
+
+(define (crate-package? package)
+  "Return true if PACKAGE is a Rust crate from crates.io."
+
+  (define (crate-url? url)
+    (string-prefix? "https://crates.io/" url))
+
+  (let ((source-url (and=> (package-source package) origin-uri))
+        (fetch-method (and=> (package-source package) origin-method)))
+    (and (eq? fetch-method download:url-fetch)
+         (match source-url
+           ((? string?)
+            (crate-url? source-url))
+           ((source-url ...)
+            (any crate-url? source-url))))))
+
+(define (latest-release package)
+  "Return an <upstream-source> for the latest release of PACKAGE."
+  (let* ((crate-name (guix-package->crate-name package))
+         (metadata (crate-fetch crate-name))
+         (version (assoc-ref* metadata "crate" "max_version"))
+         (url (crate-uri crate-name version)))
+    (upstream-source
+     (package (package-name package))
+     (version version)
+     (urls (list url)))))
+
+(define %crate-updater
+  (upstream-updater
+   (name 'crates)
+   (description "Updater for crates.io packages")
+   (pred crate-package?)
+   (latest latest-release)))
diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm
index 84e2a8f..eda10df 100644
--- a/guix/scripts/refresh.scm
+++ b/guix/scripts/refresh.scm
@@ -38,6 +38,7 @@
                           %xorg-updater))
   #:use-module (guix import elpa)
   #:use-module (guix import cran)
+  #:use-module (guix import crate)
   #:use-module (guix import hackage)
   #:use-module (guix gnupg)
   #:use-module (gnu packages)
@@ -206,7 +207,8 @@ unavailable optional dependencies such as Guile-JSON."
                  %hackage-updater
                  ((guix import pypi) => %pypi-updater)
                  ((guix import gem) => %gem-updater)
-                 ((guix import github) => %github-updater)))
+                 ((guix import github) => %github-updater)
+                 ((guix import crate) => %crate-updater)))
 
 (define (lookup-updater name)
   "Return the updater called NAME."
-- 
2.9.0

Re: [PATCH 3/7] import: crate: Add crate updater.

[Ludovic Courtès]

David Craven <david@craven.ch> skribis:

> * guix/import/crate.scm (crate-package?, latest-release,
>   %crate-updater): New variables.
> * guix/scripts/refresh.scm (%updaters): Add %crate-updater to list of
>   updaters.

Please add a line under “Invoking guix refresh” in guix.texi, like we
did for the other updaters.

OK with this change!

Ludo’.

[PATCH 4/7] upstream: Use a the first url from urls when find2 returns #f.

[David Craven]

* guix/upstream.scm (package-update): Use a url from the list when the
  find2 procedure doesn't find a url sig-url pair.
---
 guix/upstream.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/guix/upstream.scm b/guix/upstream.scm
index 1815737..ac3f72f 100644
--- a/guix/upstream.scm
+++ b/guix/upstream.scm
@@ -194,7 +194,7 @@ and 'interactive' (default)."
                              (string-suffix? archive-type url))
                            urls
                            (or signature-urls (circular-list #f)))))
-       (let ((tarball (download-tarball store url signature-url
+       (let ((tarball (download-tarball store (if url url (car urls)) signature-url
                                         #:key-download key-download)))
          (values version tarball))))
     (#f
-- 
2.9.0

Re: [PATCH 4/7] upstream: Use a the first url from urls when find2 returns #f.

[David Craven]

There is an example now. You can change the version of the rust-libc
package and then run guix refresh -u rust-libc on it.

Re: [PATCH 4/7] upstream: Use a the first url from urls when find2 returns #f.

[Ludovic Courtès]

David Craven <david@craven.ch> skribis:

> There is an example now. You can change the version of the rust-libc
> package and then run guix refresh -u rust-libc on it.

And what happens?  Could it happen with other updaters?

The concern I expressed at
<https://lists.gnu.org/archive/html/guix-devel/2016-09/msg01989.html>
remains.

Thanks,
Ludo’.

[PATCH 5/7] gnu: Add rustc-bootstrap.

[David Craven]

* gnu/packages/rust.scm (rustc-bootstrap, rust-bootstrap-x86_64-1.12.0):
  New variables.
---
 gnu/packages/rust.scm | 99 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 99 insertions(+)
 create mode 100644 gnu/packages/rust.scm

diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
new file mode 100644
index 0000000..2726348
--- /dev/null
+++ b/gnu/packages/rust.scm
@@ -0,0 +1,99 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2016 David Craven <david@craven.ch>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust)
+  #:use-module (gnu packages base)
+  #:use-module (gnu packages bootstrap)
+  #:use-module (gnu packages commencement)
+  #:use-module (gnu packages compression)
+  #:use-module (gnu packages elf)
+  #:use-module (gnu packages gcc)
+  #:use-module (guix build-system gnu)
+  #:use-module (guix download)
+  #:use-module (guix packages)
+  #:use-module ((guix licenses) #:prefix license:))
+
+(define rust-bootstrap-x86_64-1.12.0
+  (origin
+    (method url-fetch)
+    (uri (string-append
+          "https://static.rust-lang.org/dist/"
+          "rust-beta-x86_64-unknown-linux-gnu.tar.gz"))
+    (sha256
+     (base32
+      "1is1k93zarvxx0h7b57ga8vr9gj34b36l9la6zkph41x33gfgpvl"))))
+
+(define-public rustc-bootstrap
+  (package
+    (name "rustc-bootstrap")
+    (version "1.12.0")
+    (source rust-bootstrap-x86_64-1.12.0)
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("patchelf" ,patchelf)))
+    (inputs
+     `(("gcc-lib" ,gcc "lib")
+       ("gcc-toolchain-6" ,gcc-toolchain-6)
+       ("zlib" ,zlib)))
+    (arguments
+     `(#:tests? #f
+       #:strip-binaries? #f
+       #:phases
+       (modify-phases %standard-phases
+         (delete 'configure)
+         (delete 'build)
+         (replace 'install
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let ((out (assoc-ref outputs "out"))
+                   (gcc-lib (assoc-ref inputs "gcc-lib"))
+                   (toolchain (assoc-ref inputs "gcc-toolchain-6"))
+                   (zlib (assoc-ref inputs "zlib"))
+                   (platform ,(system->rust-platform (%current-system)))
+                   (ld-so (string-append (assoc-ref inputs "libc")
+                                         ,(glibc-dynamic-linker))))
+               (system* "bash" "install.sh"
+                        (string-append "--prefix=" out)
+                        (string-append "--components=rustc,"
+                                       "rust-std-" platform))
+               (for-each
+                (lambda (file)
+                  (system* "patchelf"
+                           "--set-rpath"
+                           (string-append out "/lib:" zlib "/lib:"
+                                          gcc-lib "/lib:" toolchain "/lib")
+                           file))
+                (cons* (string-append out "/bin/rustc")
+                       (string-append out "/bin/rustdoc")
+                       (find-files out "\\.so$")))
+               (for-each
+                (lambda (file)
+                  (system* "patchelf"
+                           "--set-interpreter" ld-so
+                           file))
+                (list (string-append out "/bin/rustc")
+                      (string-append out "/bin/rustdoc")))
+               ;; Rust requires a gcc toolchain for linking. It
+               ;; looks for a compiler named cc in it's path. This
+               ;; can probably be configured during the build.
+               (symlink (string-append toolchain "/bin/gcc")
+                        (string-append out "/bin/cc"))))))))
+    (home-page "https://www.rust-lang.org")
+    (synopsis "Rustc bootstrap")
+    (description "This package prepares the rustc binary for bootstrapping
+the rustc package.")
+    (license license:asl2.0)))
-- 
2.9.0

Re: [PATCH 5/7] gnu: Add rustc-bootstrap.

[David Craven]

gcc-toolchain is a function. We could use gcc-toolchain-4.9 if you prefer?

Re: [PATCH 5/7] gnu: Add rustc-bootstrap.

[David Craven]

> gcc-toolchain is a function. We could use gcc-toolchain-4.9 if you prefer?

ah see what you mean now, ignore this comment.

Re: [PATCH 5/7] gnu: Add rustc-bootstrap.

[Ludovic Courtès]

David Craven <david@craven.ch> skribis:

> * gnu/packages/rust.scm (rustc-bootstrap, rust-bootstrap-x86_64-1.12.0):
>   New variables.

I believe this is a followup to my comments at
<https://lists.gnu.org/archive/html/guix-devel/2016-09/msg01993.html>,
right?

> +;;; Copyright © 2016 David Craven <david@craven.ch>

And Eric Le Bihan?  Or is this independent from Eric’s patch?

> +(define rust-bootstrap-x86_64-1.12.0

Add a comment saying that this is a pre-built binary.

> +  (origin
> +    (method url-fetch)
> +    (uri (string-append
> +          "https://static.rust-lang.org/dist/"
> +          "rust-beta-x86_64-unknown-linux-gnu.tar.gz"))

This URL is unversioned, so the contents might change over time no?

> +               ;; Rust requires a gcc toolchain for linking. It
> +               ;; looks for a compiler named cc in it's path. This
> +               ;; can probably be configured during the build.

s/This can…/Thus, add a ‘cc’ symlink next to it./

> +    (home-page "https://www.rust-lang.org")
> +    (synopsis "Rustc bootstrap")

“Rust bootstrapping compiler”?

> +    (description "This package prepares the rustc binary for bootstrapping
> +the rustc package.")

“This package provides a pre-built @command{rustc} compiler, which can
in turn be used to build the final Rust compiler.”

Thanks,
Ludo’.

Re: [PATCH 5/7] gnu: Add rustc-bootstrap.

[ng0]

Ludovic Courtès <ludo@gnu.org> writes:

> David Craven <david@craven.ch> skribis:
>
>> * gnu/packages/rust.scm (rustc-bootstrap, rust-bootstrap-x86_64-1.12.0):
>>   New variables.
>
> I believe this is a followup to my comments at
> <https://lists.gnu.org/archive/html/guix-devel/2016-09/msg01993.html>,
> right?
>
>> +;;; Copyright © 2016 David Craven <david@craven.ch>
>
> And Eric Le Bihan?  Or is this independent from Eric’s patch?
>
>> +(define rust-bootstrap-x86_64-1.12.0
>
> Add a comment saying that this is a pre-built binary.
>
>> +  (origin
>> +    (method url-fetch)
>> +    (uri (string-append
>> +          "https://static.rust-lang.org/dist/"
>> +          "rust-beta-x86_64-unknown-linux-gnu.tar.gz"))
>
> This URL is unversioned, so the contents might change over time no?

I don't know what rust-beta-* is, but the rust package I created did not
use unversioned files. There are versioned files, but afaik not in
beta. For bootstrap I had this (though it could be that this is already
Jelle's work based on my work):

(define (archive-name version platform checksum)
  (string-append "rust-stage0-" version "-" platform "-" checksum ".tar.bz2"))

(define rust-stage0-bootstrap-x86_64-archive
  (archive-name "2016-02-17-4d3eebf" "linux-x86_64" "d29b7607d13d64078b6324aec82926fb493f59ba"))

(define rust-stage0-bootstrap-x86_64
  (origin
    (method url-fetch)
    (uri
     (string-append "https://static.rust-lang.org/stage0-snapshots/" rust-stage0-bootstrap-x86_64-archive))
    (sha256 
     (base32
      "0gk87rknijyirlhw3h34bjxzq98j0v0icp3l8flrxn5pgil8pswd"))))

(define rust-stage0-bootstrap-i386-archive
  (archive-name "2016-02-17-4d3eebf" "linux-i386" "5f194aa7628c0703f0fd48adc4ec7f3cc64b98c7"))

(define rust-stage0-bootstrap-i386
  (origin
    (method url-fetch)
    (uri
     (string-append "https://static.rust-lang.org/stage0-snapshots/" rust-stage0-bootstrap-i386-archive))
    (sha256
     (base32
      "16fd2hmli86g1q3fyicdhh2l4aqryzxcij7sk1pljig8dr2m8hg5"))))


>> +               ;; Rust requires a gcc toolchain for linking. It
>> +               ;; looks for a compiler named cc in it's path. This
>> +               ;; can probably be configured during the build.
>
> s/This can…/Thus, add a ‘cc’ symlink next to it./
>
>> +    (home-page "https://www.rust-lang.org")
>> +    (synopsis "Rustc bootstrap")
>
> “Rust bootstrapping compiler”?
>
>> +    (description "This package prepares the rustc binary for bootstrapping
>> +the rustc package.")
>
> “This package provides a pre-built @command{rustc} compiler, which can
> in turn be used to build the final Rust compiler.”
>
> Thanks,
> Ludo’.
>
>

-- 

Re: [PATCH 5/7] gnu: Add rustc-bootstrap.

[Ludovic Courtès]

ng0 <ngillmann@runbox.com> skribis:

> I don't know what rust-beta-* is, but the rust package I created did not
> use unversioned files.

Cool!  (Did you submit your own Rust package?)

> There are versioned files, but afaik not in beta. For bootstrap I had
> this (though it could be that this is already Jelle's work based on my
> work):
>
> (define (archive-name version platform checksum)
>   (string-append "rust-stage0-" version "-" platform "-" checksum ".tar.bz2"))
>
> (define rust-stage0-bootstrap-x86_64-archive
>   (archive-name "2016-02-17-4d3eebf" "linux-x86_64" "d29b7607d13d64078b6324aec82926fb493f59ba"))
>
> (define rust-stage0-bootstrap-x86_64
>   (origin
>     (method url-fetch)
>     (uri
>      (string-append "https://static.rust-lang.org/stage0-snapshots/" rust-stage0-bootstrap-x86_64-archive))
>     (sha256 
>      (base32
>       "0gk87rknijyirlhw3h34bjxzq98j0v0icp3l8flrxn5pgil8pswd"))))
>
> (define rust-stage0-bootstrap-i386-archive
>   (archive-name "2016-02-17-4d3eebf" "linux-i386" "5f194aa7628c0703f0fd48adc4ec7f3cc64b98c7"))
>
> (define rust-stage0-bootstrap-i386
>   (origin
>     (method url-fetch)
>     (uri
>      (string-append "https://static.rust-lang.org/stage0-snapshots/" rust-stage0-bootstrap-i386-archive))
>     (sha256
>      (base32
>       "16fd2hmli86g1q3fyicdhh2l4aqryzxcij7sk1pljig8dr2m8hg5"))))

That looks good.

I was also going to suggest adding support for i386, which could have
happened later, but I’m glad you’re providing guidance here!

Ludo’.

Re: [PATCH 5/7] gnu: Add rustc-bootstrap.

[ng0]

Ludovic Courtès <ludo@gnu.org> writes:

> ng0 <ngillmann@runbox.com> skribis:
>
>> I don't know what rust-beta-* is, but the rust package I created did not
>> use unversioned files.
>
> Cool!  (Did you submit your own Rust package?)

Yes, search the guix-devel archives for the first rust patches in
beginning of 2016, I went as far as I could at that point and that's how
probably everyone else started off based on that work.

>> There are versioned files, but afaik not in beta. For bootstrap I had
>> this (though it could be that this is already Jelle's work based on my
>> work):
>>
>> (define (archive-name version platform checksum)
>>   (string-append "rust-stage0-" version "-" platform "-" checksum ".tar.bz2"))
>>
>> (define rust-stage0-bootstrap-x86_64-archive
>>   (archive-name "2016-02-17-4d3eebf" "linux-x86_64" "d29b7607d13d64078b6324aec82926fb493f59ba"))
>>
>> (define rust-stage0-bootstrap-x86_64
>>   (origin
>>     (method url-fetch)
>>     (uri
>>      (string-append "https://static.rust-lang.org/stage0-snapshots/" rust-stage0-bootstrap-x86_64-archive))
>>     (sha256 
>>      (base32
>>       "0gk87rknijyirlhw3h34bjxzq98j0v0icp3l8flrxn5pgil8pswd"))))
>>
>> (define rust-stage0-bootstrap-i386-archive
>>   (archive-name "2016-02-17-4d3eebf" "linux-i386" "5f194aa7628c0703f0fd48adc4ec7f3cc64b98c7"))
>>
>> (define rust-stage0-bootstrap-i386
>>   (origin
>>     (method url-fetch)
>>     (uri
>>      (string-append "https://static.rust-lang.org/stage0-snapshots/" rust-stage0-bootstrap-i386-archive))
>>     (sha256
>>      (base32
>>       "16fd2hmli86g1q3fyicdhh2l4aqryzxcij7sk1pljig8dr2m8hg5"))))
>
> That looks good.
>
> I was also going to suggest adding support for i386, which could have
> happened later, but I’m glad you’re providing guidance here!
>
> Ludo’.
>

Do you want me to submit the entire file for context? It's just what
jelle ended up with, I no longer have this online in public searchable
format.
Okay turns out my old repo (moving to gnunet.org slowly) is still
existing and public accessible.
https://gitlab.com/secushare/guixpkgs/blob/master/n0is/packages/rust.scm

Re: [PATCH 5/7] gnu: Add rustc-bootstrap.

[David Craven]

I think Eric said he created the rustc package independently of
yours... Just saying...

Re: [PATCH 5/7] gnu: Add rustc-bootstrap.

[ng0]

David Craven <david@craven.ch> writes:

> I think Eric said he created the rustc package independently of
> yours... Just saying...
>

Sorry. My attempt to be short in communication leads to generalization.
Well it would be good if we had an overview of who's working on what so
that work on same packages can be avoided or aided for those who finish
and leave them for others to pick up. I really don't feel like reading
every single email of those circa 1000 / month to figure out details of
packages when I don't need them.
A bad temporary option while we lack something web based could be a
simple .org file in the repository.

I was working on opencolorio and only announced this on irc very
briefly, some days ago I gave my work to Leo as I happened to see that
Leo was working on it as well.

That being said, I do not try to claim anything here, I'm just proving
this as additional input so that we can fix up rust.
-- 

Re: [PATCH 5/7] gnu: Add rustc-bootstrap.

[Ludovic Courtès]

David Craven <david@craven.ch> skribis:

> +(define-public rustc-bootstrap
> +  (package
> +    (name "rustc-bootstrap")
> +    (version "1.12.0")
> +    (source rust-bootstrap-x86_64-1.12.0)

Also, add (supported-platforms '("x86_64-linux")).

Ludo’.

[PATCH 6/7] gnu: Add cargo-bootstrap.

[David Craven]

* gnu/packages/rust.scm (cargo-bootstrap): New variable.
---
 gnu/packages/rust.scm | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index 2726348..5b369fc 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -97,3 +97,46 @@
     (description "This package prepares the rustc binary for bootstrapping
 the rustc package.")
     (license license:asl2.0)))
+
+(define-public cargo-bootstrap
+  (package
+    (name "cargo-bootstrap")
+    (version "1.12.0")
+    (source rust-bootstrap-x86_64-1.12.0)
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("patchelf" ,patchelf)))
+    (inputs
+     `(("gcc-lib" ,gcc "lib")
+       ("gcc-toolchain-6" ,gcc-toolchain-6)))
+    (arguments
+     `(#:tests? #f
+       #:strip-binaries? #f
+       #:phases
+       (modify-phases %standard-phases
+         (delete 'configure)
+         (delete 'build)
+         (replace 'install
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let ((out (assoc-ref outputs "out"))
+                   (platform ,(system->rust-platform (%current-system)))
+                   (ld-so (string-append (assoc-ref inputs "libc")
+                                         ,(glibc-dynamic-linker))))
+               (system* "bash" "install.sh"
+                        (string-append "--prefix=" out)
+                        "--components=cargo")
+               (zero? (system* "patchelf"
+                               "--set-interpreter" ld-so
+                               "--set-rpath"
+                               (string-append
+                                out "/lib:"
+                                (assoc-ref inputs "gcc-lib") "/lib:"
+                                (assoc-ref inputs "gcc-toolchain-6") "/lib")
+                      (string-append out "/bin/cargo")))))))))
+    (home-page "https://www.rust-lang.org")
+    (synopsis "Cargo bootstrap")
+    (description "This package prepares the cargo binary for bootstrapping
+the cargo package and it's dependencies.  When rustc is build using the new
+rustbuild build system it also requires cargo.  The gnu build system is going
+to be deprecated.")
+    (license license:asl2.0)))
-- 
2.9.0

Re: [PATCH 6/7] gnu: Add cargo-bootstrap.

[Ludovic Courtès]

David Craven <david@craven.ch> skribis:

> * gnu/packages/rust.scm (cargo-bootstrap): New variable.

Please add (supported-platforms '("x86_64-linux")).

> +    (synopsis "Cargo bootstrap")
> +    (description "This package prepares the cargo binary for bootstrapping

@command{cargo}

> +the cargo package and it's dependencies.  When rustc is build using the new
       ^                   ^                      ^            ^
the @code{cargo} package and its dependencies.  When @command{rustc} is built

What about being more explicit here:

  This package provides a pre-built binary of the Cargo build tool,
  which in turn is used to build the Rust compiler and the final Cargo
  package.

?

> +rustbuild build system it also requires cargo.  The gnu build system is going

requires Cargo.

> +to be deprecated.")

The last sentence is hard to understand in the context of the
description of this package.  Probably best to remove it or to write it
as a comment with a longer explanation.

Otherwise LGTM, thanks!

Ludo’.

[PATCH 7/7] gnu: Add rust-libc.

[David Craven]

* gnu/packages/rust.scm (rust-libc): New variable.
---
 gnu/packages/rust.scm | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index 5b369fc..0ac09b3 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -23,6 +23,7 @@
   #:use-module (gnu packages compression)
   #:use-module (gnu packages elf)
   #:use-module (gnu packages gcc)
+  #:use-module (guix build-system cargo)
   #:use-module (guix build-system gnu)
   #:use-module (guix download)
   #:use-module (guix packages)
@@ -140,3 +141,22 @@ the cargo package and it's dependencies.  When rustc is build using the new
 rustbuild build system it also requires cargo.  The gnu build system is going
 to be deprecated.")
     (license license:asl2.0)))
+
+(define-public rust-libc
+  (package
+    (name "rust-libc")
+    (version "0.2.16")
+    (source (origin
+              (method url-fetch)
+              (uri (crate-uri "libc" version))
+              (file-name (string-append name "-" version ".tar.gz"))
+              (sha256
+               (base32
+                "139fl308mb5wxap5fyd5c7n779a60sc1fi8wgdv0zvihrv519020"))))
+    (build-system cargo-build-system)
+    (home-page "https://github.com/rust-lang/libc")
+    (synopsis
+     "Types and bindings to native C functions")
+    (description "This package provides a library for types and bindings to
+native C functions often found in libc or other common platform libraries.")
+    (license (list license:expat license:asl2.0))))
-- 
2.9.0

Re: [PATCH 1/7] build-system: Add cargo build system.

[Ricardo Wurmus]

David Craven <david@craven.ch> writes:

> * guix/build-system/cargo.scm (default-cargo, default-rustc,
>   %cargo-build-system-modules, cargo-build, lower, cargo-build-system):
>   New variables.
> * guix/build/cargo-build-system.scm (configure, build, check, install,
>   %standard-phases, cargo-build): New variables.

Since these are new files you can just write

* guix/build-system/cargo.scm: New file.
* guix/build/cargo-build-system.scm: New file.

But you should also add them to “gnu/local.mk”.

> +(define (system->rust-platform system)
> +  (cond
> +   ((string-prefix? "x86_64" system) "x86_64-unknown-linux-gnu")
> +   ((string-prefix? "i686" system) "i686-unknown-linux-gnu")))

Is there no rust for other systems?  How about adding a default case, or
are you ensuring that system can only have one of these two values?

> +(define* (cargo-build store name inputs
> +                      #:key
> +                      (tests? #t)
> +                      (test-target #f)

Is this correct?  What happens when test-target is not specified?  I see
below that the target is hard-coded to “test”.  Maybe set it to “test”
here and use it on the build side?

> diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
> new file mode 100644
> index 0000000..abe7e05
> --- /dev/null
> +++ b/guix/build/cargo-build-system.scm
> @@ -0,0 +1,81 @@
> +;;; GNU Guix --- Functional package management for GNU
> +;;; Copyright © 2016 David Craven <david@craven.ch>
> +;;;
> +;;; This file is part of GNU Guix.
> +;;;
> +;;; GNU Guix is free software; you can redistribute it and/or modify it
> +;;; under the terms of the GNU General Public License as published by
> +;;; the Free Software Foundation; either version 3 of the License, or (at
> +;;; your option) any later version.
> +;;;
> +;;; GNU Guix is distributed in the hope that it will be useful, but
> +;;; WITHOUT ANY WARRANTY; without even the implied warranty of
> +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +;;; GNU General Public License for more details.
> +;;;
> +;;; You should have received a copy of the GNU General Public License
> +;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
> +
> +(define-module (guix build cargo-build-system)
> +  #:use-module ((guix build gnu-build-system) #:prefix gnu:)
> +  #:use-module (guix build utils)
> +  #:use-module (ice-9 match)
> +  #:use-module (ice-9 ftw)
> +  #:use-module (srfi srfi-1)
> +  #:use-module (srfi srfi-26)
> +  #:export (%standard-phases
> +            cargo-build))
> +
> +;; Commentary:
> +;;
> +;; Builder-side code of the standard Rust package build procedure.
> +;;
> +;; Code:
> +
> +(define* (configure #:rest _)
> +  "Replace Cargo.toml [dependencies] section with guix inputs."
> +  ;; TODO
> +  #t)

I don’t understand this.  Does this mean that currently dependencies are
always bundled?

> +(define* (build #:key (cargo-build-flags '("--release")) #:allow-other-keys)
> +  "Build a given Cargo package."
> +  (zero? (apply system* `("cargo" "build" ,@cargo-build-flags))))
> +
> +(define* (check #:rest _)
> +  "Run tests for a given Cargo package."
> +  (zero? (system* "cargo" "test")))

Shouldn’t this respect “test-target”?

> +
> +(define* (install #:key inputs outputs #:allow-other-keys)
> +  "Install a given Cargo package."
> +  (let* ((out (assoc-ref outputs "out"))
> +         (src (assoc-ref inputs "source"))
> +         (bin (string-append out "/bin"))
> +         (rsrc (string-append out "/rustsrc")))
> +    (mkdir-p rsrc)
> +    ;; Rust doesn't have a stable ABI yet. Because of this
> +    ;; Cargo doesn't have a search path for binaries yet.
> +    ;; Until this changes we are working around this by
> +    ;; distributing crates as source and replacing
> +    ;; references in Cargo.toml with store paths.
> +    (copy-recursively "src" (string-append rsrc "/src"))
> +    (install-file "Cargo.toml" rsrc)

I’m not familiar with Rust so I don’t know what crates are.  Are they
actually source files?  Are they archives?

You write that we are replacing references in Cargo.toml with store
paths but I see no evidence of this.  Could you please clarify?

> +    ;; When the package includes executables we install
> +    ;; it using cargo install. This fails when the crate
> +    ;; doesn't contain an executable.
> +    (system* "cargo" "install" "--root" bin)
> +    #t))

Why only use “cargo install” in case there are executables?  Can we
detect this by looking at some description file of the package?  It
doesn’t seem right to unconditionally end on #t.

~~ Ricardo

Re: [PATCH 1/7] build-system: Add cargo build system.

[David Craven]

> Is this correct?  What happens when test-target is not specified?  I see
> below that the target is hard-coded to “test”.  Maybe set it to “test”
> here and use it on the build side?

> Shouldn’t this respect “test-target”?

There is no other build target than test. Functions that are tests
have an attribute #[cfg(test)] above them. Running cargo test tells
cargo to run all functions that are annotated like that.

> I’m not familiar with Rust so I don’t know what crates are.  Are they
> actually source files?  Are they archives?

A crate is a tar.gz file which contains the source.

> You write that we are replacing references in Cargo.toml with store
> paths but I see no evidence of this.  Could you please clarify?

These patches only build packages without dependencies. I have
packages importing/building with a simple dependency structure.

I'm refactoring the importer to allow recursive imports, and then I'll
try to package cargo with it.

> Why only use “cargo install” in case there are executables?  Can we
> detect this by looking at some description file of the package?  It
> doesn’t seem right to unconditionally end on #t.

So we installed the source. At this point the build system can use it. The
only time we install binaries is when the crate contains an executable or
a script. In those cases `cargo install` should work. If there isn't an
executable we don't care if it fails.

> I don’t understand this.  Does this mean that currently dependencies are
> always bundled?

No. This is to prevent cargo from fetching the missing dependencies from
crates.io and to use our crates. Rust/LLVM performs aggressive optimization,
executables include all their rust symbols. You can write a C
compatible library,
but most crates are meant for Rust development. They have their own binary
format called *.rlib that is like an *.a file. The main focus for now
is building
cargo. The situation should improve, but this is how other distros are packaging
crates currently.

Re: [PATCH 1/7] build-system: Add cargo build system.

[Ludovic Courtès]

David Craven <david@craven.ch> skribis:

> * guix/build-system/cargo.scm (default-cargo, default-rustc,
>   %cargo-build-system-modules, cargo-build, lower, cargo-build-system):
>   New variables.
> * guix/build/cargo-build-system.scm (configure, build, check, install,
>   %standard-phases, cargo-build): New variables.

[...]

> +(define* (install #:key inputs outputs #:allow-other-keys)
> +  "Install a given Cargo package."
> +  (let* ((out (assoc-ref outputs "out"))
> +         (src (assoc-ref inputs "source"))
> +         (bin (string-append out "/bin"))
> +         (rsrc (string-append out "/rustsrc")))
> +    (mkdir-p rsrc)
> +    ;; Rust doesn't have a stable ABI yet. Because of this
> +    ;; Cargo doesn't have a search path for binaries yet.
> +    ;; Until this changes we are working around this by
> +    ;; distributing crates as source and replacing
> +    ;; references in Cargo.toml with store paths.
> +    (copy-recursively "src" (string-append rsrc "/src"))

OK.  In
<https://lists.gnu.org/archive/html/guix-devel/2016-09/msg01991.html>, I
suggested using ‘share/rust-source’ or similar to store the source.
Would it be possible?

Last thing: Could you add a couple of lines in guix.texi under “Build
Systems”?

OK with these changes.

Thanks!

Ludo’.

Re: [PATCH 1/7] build-system: Add cargo build system.

[ng0]

Ludovic Courtès <ludo@gnu.org> writes:

> David Craven <david@craven.ch> skribis:
>
>> * guix/build-system/cargo.scm (default-cargo, default-rustc,
>>   %cargo-build-system-modules, cargo-build, lower, cargo-build-system):
>>   New variables.
>> * guix/build/cargo-build-system.scm (configure, build, check, install,
>>   %standard-phases, cargo-build): New variables.
>
> [...]
>
>> +(define* (install #:key inputs outputs #:allow-other-keys)
>> +  "Install a given Cargo package."
>> +  (let* ((out (assoc-ref outputs "out"))
>> +         (src (assoc-ref inputs "source"))
>> +         (bin (string-append out "/bin"))
>> +         (rsrc (string-append out "/rustsrc")))
>> +    (mkdir-p rsrc)
>> +    ;; Rust doesn't have a stable ABI yet. Because of this
>> +    ;; Cargo doesn't have a search path for binaries yet.
>> +    ;; Until this changes we are working around this by
>> +    ;; distributing crates as source and replacing
>> +    ;; references in Cargo.toml with store paths.
>> +    (copy-recursively "src" (string-append rsrc "/src"))
>
> OK.  In
> <https://lists.gnu.org/archive/html/guix-devel/2016-09/msg01991.html>, I
> suggested using ‘share/rust-source’ or similar to store the source.
> Would it be possible?
>
> Last thing: Could you add a couple of lines in guix.texi under “Build
> Systems”?
>
> OK with these changes.
>
> Thanks!
>
> Ludo’.
>
>
Hi, would it be possible to add some very short notes on why this
isn't complete from your (David) view?

Once you have some time to go into detail you could add further
explanations, for now an tl;dr would get great to help fixing
it up. If that's not possible just see this as an reminder to do
this when you have the time :)

Thanks for your work on this!
-- 
♥Ⓐ  ng0  | ng0.chaosnet.org

Re: [PATCH 1/7] build-system: Add cargo build system.

[Ludovic Courtès]

ng0 <ng0@libertad.pw> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> David Craven <david@craven.ch> skribis:
>>
>>> * guix/build-system/cargo.scm (default-cargo, default-rustc,
>>>   %cargo-build-system-modules, cargo-build, lower, cargo-build-system):
>>>   New variables.
>>> * guix/build/cargo-build-system.scm (configure, build, check, install,
>>>   %standard-phases, cargo-build): New variables.
>>
>> [...]
>>
>>> +(define* (install #:key inputs outputs #:allow-other-keys)
>>> +  "Install a given Cargo package."
>>> +  (let* ((out (assoc-ref outputs "out"))
>>> +         (src (assoc-ref inputs "source"))
>>> +         (bin (string-append out "/bin"))
>>> +         (rsrc (string-append out "/rustsrc")))
>>> +    (mkdir-p rsrc)
>>> +    ;; Rust doesn't have a stable ABI yet. Because of this
>>> +    ;; Cargo doesn't have a search path for binaries yet.
>>> +    ;; Until this changes we are working around this by
>>> +    ;; distributing crates as source and replacing
>>> +    ;; references in Cargo.toml with store paths.
>>> +    (copy-recursively "src" (string-append rsrc "/src"))
>>
>> OK.  In
>> <https://lists.gnu.org/archive/html/guix-devel/2016-09/msg01991.html>, I
>> suggested using ‘share/rust-source’ or similar to store the source.
>> Would it be possible?
>>
>> Last thing: Could you add a couple of lines in guix.texi under “Build
>> Systems”?
>>
>> OK with these changes.
>>
>> Thanks!
>>
>> Ludo’.
>>
>>
> Hi, would it be possible to add some very short notes on why this
> isn't complete from your (David) view?

Given the work that has gone into these Rust patches, I think it would
be nice to apply them and possibly document any shortcoming or future
work items.

WDYT, David?  :-)

Ludo’.

Re: [PATCH 1/7] build-system: Add cargo build system.

[David Craven]

Hi Ludo!

> Given the work that has gone into these Rust patches, I think it would
> be nice to apply them and possibly document any shortcoming or future
> work items.

I went over all your previous emails and fixed your previous comments.
Those aren't part of the new patch series yet...

In summary those changes are:
* add (supported-systems '("x86_64-linux")
* Improve rustc-bootstrap, cargo-bootstrap and rust-bootstrap synopsis
and description
* add crate updater to table in guix.texi
* add Eric to the rust.scm file


Some unanswered questions:

guix size of rust-bootstrap shows a size of zero. Are propagated
inputs not part of this measurement?

Now there are two different gcc:lib's in the dependency graph of
rustc-bootstrap. Can I use the system gcc:lib? Does gcc "lib" need to
be added to %final-inputs in commencement.scm?

To build rustc i686 and x86_64 with the same binary, rustc-bootstrap
needs to find the 32bit glibc dynamic linker. How do I get a 32bit
glibc for that?


What needs to be done:

* the updater patch needs to be looked at. it fixed the packages not
updating, I need to find out why that was and/or if it still is the
case.

* Crate names need to be saved as a property and propagated to the
build system when the crate name contains underscores.

* Put crate source in OUT/share/rust-source/PACKAGE-VERSION

* Cargo build system should not do anything if there isn't a
Cargo.lock file except copy the source to out.

* Finish the recursive importer. The recursive importer should use the
Cargo.lock file to get the right package versions as
rust-PACKAGENAME-VERSION. If a crate doesn't have a Cargo.lock only
import the latest version non-recursively as rust-PACKAGENAME.

* Get cargo to build

* Make the updater update package versions recursively

* Improve cargo bootstrapping story.



Hope I didn't forget anything...
David

Re: [PATCH 1/7] build-system: Add cargo build system.

[Ludovic Courtès]

Hi David!

David Craven <david@craven.ch> skribis:

>> Given the work that has gone into these Rust patches, I think it would
>> be nice to apply them and possibly document any shortcoming or future
>> work items.
>
> I went over all your previous emails and fixed your previous comments.
> Those aren't part of the new patch series yet...
>
> In summary those changes are:
> * add (supported-systems '("x86_64-linux")
> * Improve rustc-bootstrap, cargo-bootstrap and rust-bootstrap synopsis
> and description
> * add crate updater to table in guix.texi
> * add Eric to the rust.scm file

OK.

> Some unanswered questions:
>
> guix size of rust-bootstrap shows a size of zero. Are propagated
> inputs not part of this measurement?

No they’re not; it’s only the things that show in in ‘guix gc -R
rust-bootstrap’.

> Now there are two different gcc:lib's in the dependency graph of
> rustc-bootstrap. Can I use the system gcc:lib? Does gcc "lib" need to
> be added to %final-inputs in commencement.scm?

Hmm why is there a second one?  ‘gnu-build-system’ already provides gcc
and gcc:lib as implicit inputs, so maybe it’s just a matter of removing
‘gcc’ from Rust’s ‘inputs’?

> To build rustc i686 and x86_64 with the same binary, rustc-bootstrap
> needs to find the 32bit glibc dynamic linker. How do I get a 32bit
> glibc for that?

When you do ‘guix build rust -s i686-linux’, you get 32-bit packages.
Is that enough or am I missing something?  :-)

> What needs to be done:
>
> * the updater patch needs to be looked at. it fixed the packages not
> updating, I need to find out why that was and/or if it still is the
> case.
>
> * Crate names need to be saved as a property and propagated to the
> build system when the crate name contains underscores.
>
> * Put crate source in OUT/share/rust-source/PACKAGE-VERSION
>
> * Cargo build system should not do anything if there isn't a
> Cargo.lock file except copy the source to out.
>
> * Finish the recursive importer. The recursive importer should use the
> Cargo.lock file to get the right package versions as
> rust-PACKAGENAME-VERSION. If a crate doesn't have a Cargo.lock only
> import the latest version non-recursively as rust-PACKAGENAME.
>
> * Get cargo to build
>
> * Make the updater update package versions recursively
>
> * Improve cargo bootstrapping story.

OK, thanks for the detailed update!

Ludo’.

Re: [PATCH 1/7] build-system: Add cargo build system.

[David Craven]

> Hmm why is there a second one?  ‘gnu-build-system’ already provides gcc
> and gcc:lib as implicit inputs, so maybe it’s just a matter of removing
> ‘gcc’ from Rust’s ‘inputs’?

So we do have an implicit gcc:lib package as an input. But I have to patchelf
the rustc binaries and libraries so I need a way to retrieve the gcc lib output.
I can do (assoc-ref inputs "gcc") but how do I get a reference to the
lib output?
(assoc-ref* inputs "gcc" "lib") doesn't work... So I add ("gcc:lib"
,gcc "lib") explicitly
to the inputs so that I can retrieve it. This results in slightly
different versions
being included.

guix size rustc-bootstrap
/gnu/store/fafzikyzhwbryqxxqsbsdkkq47rfrd8z-rustc-bootstrap-1.12.1
414.6   240.4  58.0%
/gnu/store/y1g6991kxvdk4vxhsq07r5saww30v8dq-gcc-4.9.4
138.6    77.2  18.6%
/gnu/store/a64w9dq219a8d9k4mfz76mnzph9wsvfj-zlib-1.2.8
61.3     0.4   0.1%
/gnu/store/q4a1z2arssij8iq80i4k0zqallqlh2lv-gcc-4.9.4-lib
61.1    22.8   5.5%
/gnu/store/cdi08kw7r6r684w8mk0xq0dkgpjhfpmd-gcc-4.9.4-lib
61.0    22.7   5.5%
/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0
50.7     5.4   1.3%
/gnu/store/bm0gfw4jkw8gd0vpnnzrb6z0xncrbx3p-readline-7.0
45.3     1.3   0.3%
/gnu/store/hdrli1v7q3107w842s7di8rid82xlfvl-ncurses-6.0
44.0     5.7   1.4%
/gnu/store/vxdm2dqckv3yvwihr4hs6f886v6104az-zlib-1.2.8
38.6     0.4   0.1%
/gnu/store/iwgi9001dmmihrjg4rqhd6pa6788prjw-glibc-2.24
38.3    36.8   8.9%
/gnu/store/rvgmixpmsq5lqr9qflhkm70kg7a4rys2-bash-static-4.4.0
1.4     1.4   0.3%
total: 414.6 MiB

I also don't know why there are two zlib's in there.

guix gc --referrers /gnu/store/vxdm2dqckv3yvwihr4hs6f886v6104az-zlib-1.2.8
/gnu/store/vxdm2dqckv3yvwihr4hs6f886v6104az-zlib-1.2.8
/gnu/store/y1g6991kxvdk4vxhsq07r5saww30v8dq-gcc-4.9.4

guix gc --referrers /gnu/store/q4a1z2arssij8iq80i4k0zqallqlh2lv-gcc-4.9.4-lib
/gnu/store/59fld0ah99fn7fc5vfsn51dm4b7x401g-clang-3.8.1
/gnu/store/d5vl2fljy7vlhz51yccw8y76yw8rzjrc-cargo-bootstrap-0.13.0
/gnu/store/fafzikyzhwbryqxxqsbsdkkq47rfrd8z-rustc-bootstrap-1.12.1
/gnu/store/q4a1z2arssij8iq80i4k0zqallqlh2lv-gcc-4.9.4-lib

Maybe it doesn't matter too much, I was just curious that's all :)

> When you do ‘guix build rust -s i686-linux’, you get 32-bit packages.
> Is that enough or am I missing something?

The problem here is that we are patching precompiled binaries for i686 to get
them to run with guix. So I need to get a reference to a 32-bit glibc
from within the package definition. I think that we'd have to --enable-multilib.
I don't know if we can make it a separate output and how we can prevent it
from being added to %final-inputs by default. And then we need a way of
obtaining it from within a package definition which amounts to the same
problem as with gcc:lib.

Re: [PATCH 1/7] build-system: Add cargo build system.

[Ludovic Courtès]

David Craven <david@craven.ch> skribis:

>> Hmm why is there a second one?  ‘gnu-build-system’ already provides gcc
>> and gcc:lib as implicit inputs, so maybe it’s just a matter of removing
>> ‘gcc’ from Rust’s ‘inputs’?
>
> So we do have an implicit gcc:lib package as an input. But I have to patchelf
> the rustc binaries and libraries so I need a way to retrieve the gcc lib output.
> I can do (assoc-ref inputs "gcc") but how do I get a reference to the
> lib output?
> (assoc-ref* inputs "gcc" "lib") doesn't work... So I add ("gcc:lib"
> ,gcc "lib") explicitly
> to the inputs so that I can retrieve it. This results in slightly
> different versions
> being included.

OK, got it.

Then you should probably be able to do:

  (inputs `(("gcc:lib" ,(canonical-package gcc) "lib") …))

>> When you do ‘guix build rust -s i686-linux’, you get 32-bit packages.
>> Is that enough or am I missing something?
>
> The problem here is that we are patching precompiled binaries for i686 to get
> them to run with guix. So I need to get a reference to a 32-bit glibc
> from within the package definition. I think that we'd have to --enable-multilib.
> I don't know if we can make it a separate output and how we can prevent it
> from being added to %final-inputs by default. And then we need a way of
> obtaining it from within a package definition which amounts to the same
> problem as with gcc:lib.

We don’t currently --enable-multilib, but you could force a reference to
a 32-bit libc with this hack:

  (inputs `(("glibc32" ,(package (inherit glibc)
                          (arguments '(#:system "i686-linux" …))))))

That’s unnecessary when the system is already "i686-linux", of course.

Would that help?

Cheers,
Ludo’.

Re: [PATCH 1/7] build-system: Add cargo build system.

[David Craven]

> Would that help?

Awesome! Thank you.

The bootstrapping solution turned out much better. I added the
#:system argument to rustc-bootstrap and cargo-bootstrap and moved the
gcc -> cc symlink into rust-bootstrap, so that we get a native
toolchain. We can build rustc on i686 and x86_64 using the same
binary.

Re: [PATCH 1/7] build-system: Add cargo build system.

[Ludovic Courtès]

David Craven <david@craven.ch> skribis:

>> Would that help?
>
> Awesome! Thank you.
>
> The bootstrapping solution turned out much better. I added the
> #:system argument to rustc-bootstrap and cargo-bootstrap and moved the
> gcc -> cc symlink into rust-bootstrap, so that we get a native
> toolchain. We can build rustc on i686 and x86_64 using the same
> binary.

Cool, thanks!

Ludo'.