[bug#33465] [PATCH] gnu: rust: Don't depend on 'git'.

[bug#33465] [PATCH] gnu: rust: Don't depend on 'git'.

[Marius Bakke]

* gnu/packages/rust.scm (rust-1.19)[inputs]: Remove GIT.
(rust-1.20)[arguments]: Disable Cargo tests that require git.
(rust-1.26)[arguments]: Likewise.
---

Notes:
    Guix,
    
    The Rust toolchain is very expensive to build and needs less volatility.
    
    So far I have only built up to Rust 1.23 with this patch.  I suggest
    applying this on 'core-updates', and giving Rust in 'master' a
    git-2.19.1 input to the previous substitutes are valid again.
    
    WDYT?

 gnu/packages/rust.scm | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index a56faad07..ae41a7dd3 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -41,7 +41,6 @@
   #:use-module (gnu packages python)
   #:use-module (gnu packages ssh)
   #:use-module (gnu packages tls)
-  #:use-module (gnu packages version-control)
   #:use-module (gnu packages)
   #:use-module (guix build-system cargo)
   #:use-module (guix build-system gnu)
@@ -387,7 +386,6 @@ test = { path = \"../libtest\" }
        ("cmake" ,cmake)
        ("flex" ,flex) ; For the tests
        ("gdb" ,gdb)   ; For the tests
-       ("git" ,git)
        ("procps" ,procps) ; For the tests
        ("python-2" ,python-2)
        ("rustc-bootstrap" ,mrustc)
@@ -446,6 +444,13 @@ safety and thread safety guarantees.")
                  ;; i686-linux.
                  (substitute* "src/tools/cargo/tests/test.rs"
                    (("fn cargo_test_env") "#[ignore]\nfn cargo_test_env"))
+
+                 ;; These tests pull in a dependency on "git", which changes
+                 ;; too frequently take part in the Rust toolchain.
+                 (substitute* "src/tools/cargo/tests/new.rs"
+                   (("fn author_prefers_cargo") "#[ignore]\nfn author_prefers_cargo")
+                   (("fn finds_author_git") "#[ignore]\nfn finds_author_git")
+                   (("fn finds_local_author_git") "#[ignore]\nfn finds_local_author_git"))
                  #t))
              (add-after 'patch-cargo-tests 'ignore-glibc-2.27-incompatible-test
                ;; https://github.com/rust-lang/rust/issues/47863
@@ -678,6 +683,12 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
                  ;; i686-linux.
                  (substitute* "src/tools/cargo/tests/testsuite/test.rs"
                    (("fn cargo_test_env") "#[ignore]\nfn cargo_test_env"))
+
+                 ;; Avoid dependency on "git".
+                 (substitute* "src/tools/cargo/tests/new.rs"
+                   (("fn author_prefers_cargo") "#[ignore]\nfn author_prefers_cargo")
+                   (("fn finds_author_git") "#[ignore]\nfn finds_author_git")
+                   (("fn finds_local_author_git") "#[ignore]\nfn finds_local_author_git"))
                  #t))
              (add-after 'patch-cargo-tests 'disable-cargo-test-for-nightly-channel
                (lambda* _
-- 
2.19.1

[bug#33465] [PATCH] gnu: rust: Don't depend on 'git'.

[Julien Lepiller]

Le 2018-11-22 14:58, Marius Bakke a écrit :
> * gnu/packages/rust.scm (rust-1.19)[inputs]: Remove GIT.
> (rust-1.20)[arguments]: Disable Cargo tests that require git.
> (rust-1.26)[arguments]: Likewise.
> ---
> 
> Notes:
>     Guix,
> 
>     The Rust toolchain is very expensive to build and needs less 
> volatility.
> 
>     So far I have only built up to Rust 1.23 with this patch.  I 
> suggest
>     applying this on 'core-updates', and giving Rust in 'master' a
>     git-2.19.1 input to the previous substitutes are valid again.
> 
>     WDYT?

Hi, I don't really know how our rust packages are built, but I wonder if 
they have more than the bare minimum optional features for our purposes 
(apart from 1.24 used by icecat and the latest version, we probably 
don't need to build everything)? Maybe it's worth investigating if that 
can speed up the build of the whole chain. It seems we're two versions 
behind current rust too. Is anyone working on packaging them?

> 
>  gnu/packages/rust.scm | 15 +++++++++++++--
>  1 file changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
> index a56faad07..ae41a7dd3 100644
> --- a/gnu/packages/rust.scm
> +++ b/gnu/packages/rust.scm
> @@ -41,7 +41,6 @@
>    #:use-module (gnu packages python)
>    #:use-module (gnu packages ssh)
>    #:use-module (gnu packages tls)
> -  #:use-module (gnu packages version-control)
>    #:use-module (gnu packages)
>    #:use-module (guix build-system cargo)
>    #:use-module (guix build-system gnu)
> @@ -387,7 +386,6 @@ test = { path = \"../libtest\" }
>         ("cmake" ,cmake)
>         ("flex" ,flex) ; For the tests
>         ("gdb" ,gdb)   ; For the tests
> -       ("git" ,git)
>         ("procps" ,procps) ; For the tests
>         ("python-2" ,python-2)
>         ("rustc-bootstrap" ,mrustc)
> @@ -446,6 +444,13 @@ safety and thread safety guarantees.")
>                   ;; i686-linux.
>                   (substitute* "src/tools/cargo/tests/test.rs"
>                     (("fn cargo_test_env") "#[ignore]\nfn 
> cargo_test_env"))
> +
> +                 ;; These tests pull in a dependency on "git", which 
> changes
> +                 ;; too frequently take part in the Rust toolchain.
> +                 (substitute* "src/tools/cargo/tests/new.rs"
> +                   (("fn author_prefers_cargo") "#[ignore]\nfn
> author_prefers_cargo")
> +                   (("fn finds_author_git") "#[ignore]\nfn 
> finds_author_git")
> +                   (("fn finds_local_author_git") "#[ignore]\nfn
> finds_local_author_git"))
>                   #t))
>               (add-after 'patch-cargo-tests 
> 'ignore-glibc-2.27-incompatible-test
>                 ;; https://github.com/rust-lang/rust/issues/47863
> @@ -678,6 +683,12 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" 
> "\"
>                   ;; i686-linux.
>                   (substitute* 
> "src/tools/cargo/tests/testsuite/test.rs"
>                     (("fn cargo_test_env") "#[ignore]\nfn 
> cargo_test_env"))
> +
> +                 ;; Avoid dependency on "git".
> +                 (substitute* "src/tools/cargo/tests/new.rs"
> +                   (("fn author_prefers_cargo") "#[ignore]\nfn
> author_prefers_cargo")
> +                   (("fn finds_author_git") "#[ignore]\nfn 
> finds_author_git")
> +                   (("fn finds_local_author_git") "#[ignore]\nfn
> finds_local_author_git"))
>                   #t))
>               (add-after 'patch-cargo-tests
> 'disable-cargo-test-for-nightly-channel
>                 (lambda* _

[bug#33465] [PATCH] gnu: rust: Don't depend on 'git'.

[Pierre Langlois]

[-- Attachment #1: Type: text/plain, Size: 1302 bytes --]

Hi Julien and Marius,

Julien Lepiller writes:

> Le 2018-11-22 14:58, Marius Bakke a écrit:
>> * gnu/packages/rust.scm (rust-1.19)[inputs]: Remove GIT.
>> (rust-1.20)[arguments]: Disable Cargo tests that require git.
>> (rust-1.26)[arguments]: Likewise.
>> ---
>>
>> Notes:
>>     Guix,
>>
>>     The Rust toolchain is very expensive to build and needs less
>> volatility.
>>
>>     So far I have only built up to Rust 1.23 with this patch.  I
>> suggest
>>     applying this on 'core-updates', and giving Rust in 'master' a
>>     git-2.19.1 input to the previous substitutes are valid again.
>>
>>     WDYT?
>
> Hi, I don't really know how our rust packages are built, but I wonder
> if they have more than the bare minimum optional features for our
> purposes (apart from 1.24 used by icecat and the latest version, we
> probably don't need to build everything)? Maybe it's worth
> investigating if that can speed up the build of the whole chain.

One thing I've been wondering about would be to remove the 'check phase
when building a rust that will be used for bootstrapping only. Since the
tests are not ran in parallel, they take a huge amount of time.

See attached patched, I'm testing it now. It's currently building
rust@1.23. WDYT?

Thanks,
Pierre


[-- Attachment #2: 0001-gnu-rust-Do-not-run-tests-when-building-for-bootstra.patch --]
[-- Type: text/x-patch, Size: 1700 bytes --]

From 46233c5f6ced0ad5e535a848527ad35309535b97 Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois@gmx.com>
Date: Fri, 23 Nov 2018 11:58:06 +0000
Subject: [PATCH] gnu: rust: Do not run tests when building for bootstrapping.

* gnu/packages/rust.scm (rust-bootstrapped-package): Add 'arguments' field
that removes the check phase.
---
 gnu/packages/rust.scm | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index a56faad079..40160de05c 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -7,6 +7,7 @@
 ;;; Copyright © 2017 Efraim Flashner <efraim@flashner.co.il>
 ;;; Copyright © 2018 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2018 Danny Milosavljevic <dannym+a@scratchpost.org>
+;;; Copyright © 2018 Pierre Langlois <pierre.langlois@gmx.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -91,7 +92,15 @@
     (native-inputs
      (alist-replace "cargo-bootstrap" (list base-rust "cargo")
                     (alist-replace "rustc-bootstrap" (list base-rust)
-                                   (package-native-inputs base-rust))))))
+                                   (package-native-inputs base-rust))))
+    (arguments
+     (substitute-keyword-arguments (package-arguments base-rust)
+       ((#:phases phases)
+        `(modify-phases ,phases
+           ;; Tests take a long time to run, as they do not run in parallel
+           ;; for stability reasons. Disable them when building rust for
+           ;; bootstrapping.
+           (delete 'check)))))))
 
 (define-public mrustc
   (let ((rustc-version "1.19.0"))
-- 
2.19.1

[bug#33465] [PATCH] gnu: rust: Don't depend on 'git'.

[Pierre Langlois]

> From 46233c5f6ced0ad5e535a848527ad35309535b97 Mon Sep 17 00:00:00 2001
> From: Pierre Langlois <pierre.langlois@gmx.com>
> Date: Fri, 23 Nov 2018 11:58:06 +0000
> Subject: [PATCH] gnu: rust: Do not run tests when building for bootstrapping.
>
> * gnu/packages/rust.scm (rust-bootstrapped-package): Add 'arguments' field
> that removes the check phase.

Whoops, ignore that patch, it doesn't do what I wanted it to do. The
point was to skip the tests *only* for temporary packages used for
bootstrapping the final one. But here it's disabled the tests all the
time, we don't want that... my bad!  I'll another look when I have time.

Thanks,
Pierre

[bug#33465] [PATCH] gnu: rust: Don't depend on 'git'.

[Pierre Langlois]

[-- Attachment #1: Type: text/plain, Size: 370 bytes --]


Pierre Langlois writes:

> Whoops, ignore that patch, it doesn't do what I wanted it to do. The
> point was to skip the tests *only* for temporary packages used for
> bootstrapping the final one. But here it's disabled the tests all the
> time, we don't want that... my bad!  I'll another look when I have time.

Right, attached is what I meant to do.

Thanks!
Pierre


[-- Attachment #2: 0001-gnu-rust-Do-not-run-tests-when-building-for-bootstra.patch --]
[-- Type: text/x-patch, Size: 2620 bytes --]

From 326a4761b03c50481d44d5b485954d823006bbb8 Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois@gmx.com>
Date: Fri, 23 Nov 2018 11:58:06 +0000
Subject: [PATCH v2] gnu: rust: Do not run tests when building for bootstrapping.

* gnu/packages/rust.scm (rust-bootstrapped-package): Create a temporary
rust-bootstrap package that inherits from base-rust and removes the check
phase.  Then use it for the cargo-bootsrap and rustc-bootstrap native inputs.
---
 gnu/packages/rust.scm | 30 +++++++++++++++++++++---------
 1 file changed, 21 insertions(+), 9 deletions(-)

diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index a56faad079..7d416836aa 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -7,6 +7,7 @@
 ;;; Copyright © 2017 Efraim Flashner <efraim@flashner.co.il>
 ;;; Copyright © 2018 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2018 Danny Milosavljevic <dannym+a@scratchpost.org>
+;;; Copyright © 2018 Pierre Langlois <pierre.langlois@gmx.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -83,15 +84,26 @@
 (define* (rust-bootstrapped-package base-rust version checksum
                                     #:key (patches '()))
   "Bootstrap rust VERSION with source checksum CHECKSUM patched with PATCHES using BASE-RUST."
-  (package
-    (inherit base-rust)
-    (version version)
-    (source
-     (rust-source version checksum #:patches patches))
-    (native-inputs
-     (alist-replace "cargo-bootstrap" (list base-rust "cargo")
-                    (alist-replace "rustc-bootstrap" (list base-rust)
-                                   (package-native-inputs base-rust))))))
+  ;; Tests take a long time to run, as they do not run in parallel for
+  ;; stability reasons.  Disable them when building the rust used for
+  ;; bootstrapping.
+  (let ((rust-bootstrap
+         (package
+           (inherit base-rust)
+           (arguments
+            (substitute-keyword-arguments (package-arguments base-rust)
+              ((#:phases phases)
+               `(modify-phases ,phases
+                  (delete 'check))))))))
+    (package
+      (inherit base-rust)
+      (version version)
+      (source
+       (rust-source version checksum #:patches patches))
+      (native-inputs
+       (alist-replace "cargo-bootstrap" (list rust-bootstrap "cargo")
+                      (alist-replace "rustc-bootstrap" (list rust-bootstrap)
+                                     (package-native-inputs base-rust)))))))
 
 (define-public mrustc
   (let ((rustc-version "1.19.0"))
-- 
2.19.2

bug#33465: [PATCH] gnu: rust: Don't depend on 'git'.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 860 bytes --]

Pierre Langlois <pierre.langlois@gmx.com> writes:

> Pierre Langlois writes:
>
>> Whoops, ignore that patch, it doesn't do what I wanted it to do. The
>> point was to skip the tests *only* for temporary packages used for
>> bootstrapping the final one. But here it's disabled the tests all the
>> time, we don't want that... my bad!  I'll another look when I have time.
>
> Right, attached is what I meant to do.

Hello!

I don't have a strong opinion for or against disabling tests in the Rust
bootstrap toolchain.  But with Git removed, most (all?) of Rusts
dependencies are packages that do not change very frequently (i.e. only
on the 'staging' and 'core-updates' branches), so maybe it's not as
urgent?

Regardless, I've pushed the original patch to the 'core-updates'
branch.  Could you submit the other patch to a separate issue or to
guix-devel?  TIA!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#33465] [PATCH] gnu: rust: Don't depend on 'git'.

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 244 bytes --]

Hi,

I'd prefer if the git parts were removed in a new phase "remove-git-tests" and
that phase removed (ha) in the newest rust - otherwise we never test rust git
integration.

Then the substitution wouldn't need to be duplicated either.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[bug#33465] [PATCH] gnu: rust: Don't depend on 'git'.

[Pierre Langlois]

Hi Danny,

Danny Milosavljevic writes:

> Hi,
>
> I'd prefer if the git parts were removed in a new phase "remove-git-tests" and
> that phase removed (ha) in the newest rust - otherwise we never test rust git
> integration.
>
> Then the substitution wouldn't need to be duplicated either.

Alternatively, if we go with removing tests althogether when building
rust for bootstrapping, we can also remove non-essential native inputs
such as git and gdb.  And then we can keep all of them for the final
rust.

As with Marius, I don't have a strong opinion on this, I just thought
I'd mention it as a possibility to speedup rust's build process. I can
submit a patch in a separate ticket if you think it's a good idea.

Thanks!
Pierre