[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Janneke Nieuwenhuizen]

Hi!

So...updating mig using git-fetch created a circular dependency because
mig-boot0 re-used mig's source, and we cannot use git in commencement.

This patch series now uses a hack by Ludo (thanks!) as suggested on IRC
<https://logs.guix.gnu.org/guix/2023-07-17.log#233051> to not directly use
generated cgit tarballs from savannah.

The versions of gnumach-headers-boot0, mig-boot0, and hurd-headers-boot0 are
updated so that native builds work again.  Because upstream does not produce
release tarballs we now also need autoconf and automake in commencement.

Note that building natively won't get very far, as for many packages (guile,
coreutils, grep, findutils, diffutils, gettext-minimal, m4, openssl, python,
...) the `check' phase will fail.  See the `hurd-team' branch for updates on
this.

Now that mig-boot0 uses its own source definition, we can safely re-apply the
mig update.

Greetings,
Janneke

Janneke Nieuwenhuizen (6):
  gnu: commencement: Add autoconf-boot0.
  gnu: commencement: Add automake-boot0.
  gnu: commencement: gnumach-headers-boot0: Update to 1.8+git20221224.
  gnu: commencement: mig-boot0: Update to 1.8+git20230520.
  gnu: commencement: hurd-headers-boot0: Update to 0.9.git20230216.
  gnu: commencement: hurd-minimal-boot0: Update to 0.9.git20230216.

Josselin Poiret (1):
  gnu: mig: Update to 1.8+git20230520.

Ludovic Courtès (1):
  gnu: commencement: Add git-fetch-from-tarball utility.

 gnu/packages/commencement.scm | 200 ++++++++++++++++++++++++++--------
 gnu/packages/hurd.scm         |  20 ++--
 2 files changed, 160 insertions(+), 60 deletions(-)


base-commit: 94f83088d237ecd032c411bb94e2d647a2931990
-- 
2.40.1

[bug#64708] [PATCH 1/8] gnu: commencement: Add autoconf-boot0.

[Janneke Nieuwenhuizen]

* gnu/packages/commencement.scm (autoconf-boot0): New variable.
---
 gnu/packages/commencement.scm | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 9bd0c113c2..3ccd51ac52 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -33,6 +33,7 @@
 (define-module (gnu packages commencement)
   #:use-module (gnu packages)
   #:use-module (gnu packages bootstrap)
+  #:use-module (gnu packages autotools)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages c)
@@ -2602,6 +2603,15 @@ (define with-boot0
   (package-with-explicit-inputs %boot0-inputs
                                 %bootstrap-guile))
 
+(define autoconf-boot0
+  (with-boot0
+   (package
+     (inherit autoconf)
+     (name "autoconf-boot0")
+     (native-inputs (list m4-boot0 perl-boot0))
+     (inputs '())
+     (arguments (list #:tests? #f)))))
+
 (define gnumach-headers-boot0
   (with-boot0
    (package
-- 
2.40.1

[bug#64708] [PATCH 2/8] gnu: commencement: Add automake-boot0.

[Janneke Nieuwenhuizen]

* gnu/packages/commencement.scm (automake-boot0): New variable.
---
 gnu/packages/commencement.scm | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 3ccd51ac52..d0603e62c8 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2612,6 +2612,19 @@ (define autoconf-boot0
      (inputs '())
      (arguments (list #:tests? #f)))))
 
+(define automake-boot0
+  (with-boot0
+   (package
+     (inherit automake)
+     (name "automake-boot0")
+     (source (origin
+               (inherit (package-source automake))
+               (patches '())))        ;test are skipped anyway
+     (native-inputs (list autoconf-boot0 m4-boot0 perl-boot0))
+     (inputs '())
+     (arguments
+      (list #:tests? #f)))))
+
 (define gnumach-headers-boot0
   (with-boot0
    (package
-- 
2.40.1

[bug#64708] [PATCH 3/8] gnu: commencement: Add git-fetch-from-tarball utility.

[Janneke Nieuwenhuizen]

From: Ludovic Courtès <ludo@gnu.org>

* gnu/packages/commencement.scm (git-fetch-from-tarball): New procedure.
---
 gnu/packages/commencement.scm | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index d0603e62c8..1d69c2a80f 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -57,7 +57,10 @@ (define-module (gnu packages commencement)
   #:use-module (gnu packages xml)
   #:use-module (guix gexp)
   #:use-module (guix packages)
+  #:use-module ((guix store) #:select (%store-monad))
+  #:use-module (guix monads)
   #:use-module (guix download)
+  #:use-module ((guix git-download) #:select (git-reference git-file-name))
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system trivial)
   #:use-module ((guix licenses) #:prefix license:)
@@ -90,6 +93,34 @@ (define-module (gnu packages commencement)
 ;;;
 ;;; Code:
 
+(define* (git-fetch-from-tarball tarball)
+  (lambda* (url hash-algo hash
+                #:optional name
+                #:key (system (%current-system))
+                (guile %bootstrap-guile))
+    (mlet %store-monad ((guile (package->derivation guile system)))
+      (gexp->derivation
+       (or name "git-checkout")
+       (with-imported-modules '((guix build utils))
+         #~(begin
+             (use-modules (guix build utils)
+                          (ice-9 ftw)
+                          (ice-9 match))
+             (setenv "PATH"
+                     #+(file-append %bootstrap-coreutils&co "/bin"))
+             (invoke (string-append #+tar "/bin/tar")
+                     "xf" #$tarball)
+             (match (scandir ".")
+               (("." ".." directory)
+                (copy-recursively directory #$output)))))
+       #:recursive? #t
+       #:hash-algo hash-algo
+       #:hash hash
+       #:system system
+       #:guile-for-build guile
+       #:graft? #f
+       #:local-build? #t))))
+
 (define bootar
   (package
     (name "bootar")
-- 
2.40.1

[bug#64708] [PATCH 4/8] gnu: commencement: gnumach-headers-boot0: Update to 1.8+git20221224.

[Janneke Nieuwenhuizen]

* gnu/packages/commencement.scm (gnumach-headers-boot0)[name]: New field; give
proper name.
[source]: Update to 1.8+git20221224, using git-fetch-from-tarball.
[native-inputs]: Add autoconf-boot0, automake-boot0, texinfo-boot0.
[arguments]: Add 'patch-compat' phase to allow building hurd-minimal etc, with
our out-of-date bootstsrap binaries.
---
 gnu/packages/commencement.scm | 46 ++++++++++++++++++++++++++---------
 1 file changed, 34 insertions(+), 12 deletions(-)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 1d69c2a80f..b15a71feb3 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2660,18 +2660,40 @@ (define gnumach-headers-boot0
   (with-boot0
    (package
      (inherit gnumach-headers)
-     (version "1.8-116-g28b53508")
-     (source (bootstrap-origin
-              (origin
-                (method url-fetch)
-                (uri (list (string-append "mirror://gnu/guix/mirror/gnumach-"
-                                          version ".tar.gz")
-                           (string-append "https://lilypond.org/janneke/hurd/"
-                                          "gnumach-" version ".tar.gz")))
-                (sha256
-                 (base32
-                  "006i0zgwy81vxarpfm12vip4q6i5mgmi5mmy5ldvxp5hx9h3l0zg")))))
-     (native-inputs '()))))
+     (name "gnumach-headers-boot0")
+     (version "1.8+git20221224")
+     (source
+      (origin
+        (method
+         (git-fetch-from-tarball
+          (origin
+            (method url-fetch)
+            (uri (string-append
+                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
+                  "gnumach-" version ".tar.gz"))
+            (sha256
+             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
+        (uri (git-reference
+              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
+              (commit version)))
+        (file-name (git-file-name name version))
+        (sha256
+         (base32
+          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))
+     (native-inputs (list autoconf-boot0 automake-boot0 texinfo-boot0))
+     (arguments
+      (substitute-keyword-arguments (package-arguments gnumach-headers)
+        ((#:phases phases)
+         #~(modify-phases #$phases
+             (add-after 'unpack 'patch-compat
+               (lambda _
+                 (substitute* '("include/device/device_types.h"
+                                "include/mach_debug/slab_info.h"
+                                "include/mach_debug/vm_info.h")
+                   (("rpc_vm_size_t") "unsigned int")
+                   (("rpc_vm_offset_t") "unsigned int")
+                   (("rpc_long_natural_t") "unsigned long")
+                   (("long_natural_t") "unsigned long")))))))))))
 
 (define mig-boot0
   (let* ((mig (package
-- 
2.40.1

[bug#64708] [PATCH 5/8] gnu: commencement: mig-boot0: Update to 1.8+git20230520.

[Janneke Nieuwenhuizen]

* gnu/packages/commencement.scm (mig-boot0): Remove redundant
outer let.
[name]: New field, give proper name.
[version]: New field
[source]: Update to 1.8+git20230520, using git-fetch-from-tarball.
[native-inputs]: Add autoconf-boot0, automake-boot0.
[inputs]: Add gnumach-headers-boot0.
---
 gnu/packages/commencement.scm | 42 ++++++++++++++++++++++++++---------
 1 file changed, 31 insertions(+), 11 deletions(-)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index b15a71feb3..05a6badea0 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2696,17 +2696,37 @@ (define gnumach-headers-boot0
                    (("long_natural_t") "unsigned long")))))))))))
 
 (define mig-boot0
-  (let* ((mig (package
-                 (inherit (package-with-bootstrap-guile mig))
-                 (native-inputs `(("bison" ,bison-boot0)
-                                  ("flex" ,flex-boot0)))
-                 (inputs `(("flex" ,flex-boot0)))
-                 (arguments
-                  ;; TODO: On next rebuild cycle, reuse phases from 'mig'.
-                  `(#:configure-flags
-                    `(,(string-append "LDFLAGS=-Wl,-rpath="
-                                      (assoc-ref %build-inputs "flex") "/lib/")))))))
-    (with-boot0 mig)))
+  (with-boot0
+   (package
+     (inherit mig)
+     (name "mig-boot0")
+     (version "1.8+git20230520")
+     (source
+      (origin
+        (method
+         (git-fetch-from-tarball
+          (origin
+            (method url-fetch)
+            (uri (string-append
+                  "https://git.savannah.gnu.org/cgit/hurd/mig.git/snapshot/"
+                  "mig-" version ".tar.gz"))
+            (sha256
+             (base32 "1l1vfm4wap5yxylv91wssgpy7fnq22wp3akgd5nv995kychfa9jy")))))
+        (uri (git-reference
+              (url "https://git.savannah.gnu.org/git/hurd/mig.git")
+              (commit version)))
+        (file-name (git-file-name name version))
+        (sha256
+         (base32 "10r0fdjqjzqsy6ajb21rifvhw0wpjvrw6a1zdyliqlzqny5k0qlz"))))
+     (native-inputs (list autoconf-boot0 automake-boot0 bison-boot0 flex-boot0
+                          gnumach-headers-boot0))
+     (inputs (list flex-boot0 gnumach-headers-boot0))
+     (arguments
+      (list
+       #:configure-flags
+       #~(list (string-append "LDFLAGS=-Wl,-rpath="
+                              #$(this-package-native-input "flex")
+                              "/lib/")))))))
 
 (define hurd-version-boot0 "0.9-229-ga1efcee8")
 (define hurd-source-boot0
-- 
2.40.1

[bug#64708] [PATCH 6/8] gnu: commencement: hurd-headers-boot0: Update to 0.9.git20230216.

[Janneke Nieuwenhuizen]

* gnu/packages/commencement.scm (hurd-headers-boot0): Remove redundant outer
let and package-with-bootstrap-guile.
[name]: New field; give proper name.
[version]: New field.
[source]: Update to 0.9.git20230216, using git-fetch-from-tarball.
[inputs]: Add autoconf-boot0, automake-boot0, gnumach-headers-boot0.
---
 gnu/packages/commencement.scm | 31 +++++++++++++++++++++++++------
 1 file changed, 25 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 05a6badea0..86246c3b08 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2742,12 +2742,31 @@ (define hurd-source-boot0
         "0bq2q2jisxcy0kgcm6rz0z2fddwxxm7azsama7li28a2m08kdpzy")))))
 
 (define hurd-headers-boot0
-  (let ((hurd-headers (package (inherit hurd-headers)
-                               (version hurd-version-boot0)
-                               (source hurd-source-boot0)
-                               (native-inputs `(("mig" ,mig-boot0)))
-                               (inputs '()))))
-    (with-boot0 (package-with-bootstrap-guile hurd-headers))))
+  (with-boot0
+   (package
+     (inherit hurd-headers)
+     (name "hurd-headers-boot0")
+     (version "0.9.git20230216")
+     (source
+      (origin
+        (method
+         (git-fetch-from-tarball
+          (origin
+            (method url-fetch)
+            (uri (string-append
+                  "https://git.savannah.gnu.org/cgit/hurd/hurd.git/snapshot/"
+                  "hurd-v" version ".tar.gz"))
+            (sha256
+             (base32 "1f75nlkcl00dqnnrbrj1frvzs2qibfpygj3gwywqi85aldjl48y7")))))
+        (uri (git-reference
+              (url "https://git.savannah.gnu.org/git/hurd/hurd.git")
+              (commit version)))
+        (file-name (git-file-name name version))
+        (sha256
+         (base32 "0jm1dnqkx4kdwmby0z5w0yqp9m5qp4hbxd4jxlyhiqm8nkw9mkvv"))))
+     (native-inputs
+      (list autoconf-boot0 automake-boot0 mig-boot0))
+     (inputs '()))))
 
 (define hurd-minimal-boot0
   (let ((hurd-minimal (package (inherit hurd-minimal)
-- 
2.40.1

[bug#64708] [PATCH 7/8] gnu: commencement: hurd-minimal-boot0: Update to 0.9.git20230216.

[Janneke Nieuwenhuizen]

* gnu/packages/commencement.scm (hurd-minimal-boot0): Remove redundant let
and package-with-bootstrap-guile.
[name]: New field; give proper name.
[version]: New field.
[source]: Update to 0.9.git20230216, using hurd-headers-boot0' source.
[inputs]: Add autoconf-boot0, automake-boot0, gnumach-headers-boot0.
[native-inputs]: Add autoconf-boot0, automake-boot0, and
gnumach-headers-boot0.
* gnu/packages/commencement.scm (hurd-version-boot0, hurd-source-boot0):
Remove.
---
 gnu/packages/commencement.scm | 27 ++++++++-------------------
 1 file changed, 8 insertions(+), 19 deletions(-)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 86246c3b08..7e988129a5 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2728,19 +2728,6 @@ (define mig-boot0
                               #$(this-package-native-input "flex")
                               "/lib/")))))))
 
-(define hurd-version-boot0 "0.9-229-ga1efcee8")
-(define hurd-source-boot0
-  (let ((version hurd-version-boot0))
-    (origin
-      (method url-fetch)
-      (uri (list (string-append "mirror://gnu/guix/mirror/hurd-v"
-                                version ".tar.gz")
-                 (string-append "https://lilypond.org/janneke/hurd/"
-                                "hurd-v" version ".tar.gz")))
-      (sha256
-       (base32
-        "0bq2q2jisxcy0kgcm6rz0z2fddwxxm7azsama7li28a2m08kdpzy")))))
-
 (define hurd-headers-boot0
   (with-boot0
    (package
@@ -2769,12 +2756,14 @@ (define hurd-headers-boot0
      (inputs '()))))
 
 (define hurd-minimal-boot0
-  (let ((hurd-minimal (package (inherit hurd-minimal)
-                               (version hurd-version-boot0)
-                               (source hurd-source-boot0)
-                               (native-inputs `(("mig" ,mig-boot0)))
-                               (inputs '()))))
-    (with-boot0 (package-with-bootstrap-guile hurd-minimal))))
+  (with-boot0
+   (package
+     (inherit hurd-minimal)
+     (name "hurd-minimal-boot0")
+     (source (package-source hurd-headers-boot0))
+     (native-inputs
+      (list autoconf-boot0 automake-boot0 gnumach-headers-boot0 mig-boot0))
+     (inputs (list gnumach-headers-boot0)))))
 
 (define/system-dependent hurd-core-headers-boot0
   ;; Return the Hurd and Mach headers as well as initial Hurd libraries for
-- 
2.40.1

[bug#64708] [PATCH 8/8] gnu: mig: Update to 1.8+git20230520.

[Janneke Nieuwenhuizen]

From: Josselin Poiret <dev@jpoiret.xyz>

Now that mig-boot0 has been updated not to use mig's source, we can
safely update mig.

* gnu/packages/hurd.scm (mig)[source]: Update to 1.8+git20230520.
---
 gnu/packages/hurd.scm | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/gnu/packages/hurd.scm b/gnu/packages/hurd.scm
index 3785c47750..4064a9e2e8 100644
--- a/gnu/packages/hurd.scm
+++ b/gnu/packages/hurd.scm
@@ -91,24 +91,20 @@ (define-public gnumach-headers
 (define-public mig
   (package
     (name "mig")
-    (version "1.8+git20220827")
+    (version "1.8+git20230520")
     (source (origin
-              (method url-fetch)
-              ;; XXX: Version 2.35 of glibc can only be built with an
-              ;; unreleased version of MiG:
-              ;; <https://lists.gnu.org/archive/html/bug-hurd/2023-03/msg00025.html>.
-              ;; It cannot be fetched from Git though, as the extra dependency
-              ;; on Autoconf/Automake would complicate bootstrapping.
-              (uri (string-append "mirror://gnu/guix/mirror/mig-"
-                                  version ".tar.gz"))
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://git.savannah.gnu.org/git/hurd/mig.git")
+                    (commit (string-append "v" version))))
+              (file-name (git-file-name name version))
               (sha256
                (base32
-                "163d37s9lscd6zxyfng421m9nl857464mgjj90xsrcl5ykbng5p2"))
-              (patches (search-patches "mig-cpu.h-generation.patch"))))
+                "10r0fdjqjzqsy6ajb21rifvhw0wpjvrw6a1zdyliqlzqny5k0qlz"))))
     (build-system gnu-build-system)
     ;; Flex is needed both at build and run time.
     (inputs (list gnumach-headers flex))
-    (native-inputs (list flex bison))
+    (native-inputs (list autoconf automake flex bison))
     (arguments
      (list #:tests? #f
            #:phases
-- 
2.40.1

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Ludovic Courtès]

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

> From: Ludovic Courtès <ludo@gnu.org>
>
> * gnu/packages/commencement.scm (git-fetch-from-tarball): New procedure.

I guess this one is very weird so it deserves an explanation.

The problems we’re trying to solve are:

  1. We cannot use ‘git-fetch’ at this point because that’d introduce a
     circular dependency: <https://issues.guix.gnu.org/63331>.

  2. Downloading a cgit-generated tarball is unsafe: the tarball might
     be modified when cgit is updated or whatever, leading to a hash
     mismatch that we wouldn’t be able to resolve (well,
     disarchive.guix.gnu.org might be able to add an entry for that
     tarball, but then there’s currently no fallback in case of hash
     mismatch: <https://issues.guix.gnu.org/28659>).

So we need to obtain our Git checkouts by some other mean.  There are
further complications:

  3. We could use (web client), but without (gnutls) since it’s
     unavailable at this stage.

  4. We cannot use (guix swh) as is because Guile-JSON doesn’t build
     with Guile 2.0, which is what we have here.

The trick below is the only way I could think of to reconcile these
issues: it essentially stacks two fixed-output derivation.  The first
one downloads that auto-generated tarball (bad!), and the second one
extracts the contents of that tarball using ‘tar’ from
‘%bootstrap-coreutils&co’.  The output of the second one is identical to
what ‘git-fetch’ would give us (or ‘swh-download’).

That makes it more robust.  It also allows ‘guix lint -c archival’ to do
its work and similarly (well, almost) for ‘build-package-metadata.scm’,
the program that builds <https://guix.gnu.org/sources.json>, which SWH
periodically ingests.

Pfeww, long story.

> +(define* (git-fetch-from-tarball tarball)

Maybe add this docstring:

    "Return an <origin> method equivalent to 'git-fetch', except that it
  fetches the checkout from TARBALL, a tarball containing said
  checkout.

  The purpose of this procedure is to work around bootstrapping issues:
  'git-fetch' depends on Git, which is much higher in the dependency
  graph."

> +             (setenv "PATH"
> +                     #+(file-append %bootstrap-coreutils&co "/bin"))
> +             (invoke (string-append #+tar "/bin/tar")
> +                     "xf" #$tarball)

This should be: (invoke "tar" "xf" #$tarball).
Otherwise we’d get a cyclic dependency with ‘tar’.

(I didn’t notice because there’s a bug where ‘-s’ is ignored here:

  ./pre-inst-env guix build -e '(@@ (gnu packages commencement) mig-boot0)' \
    -s i586-gnu -Sd

)

OK with these changes!

Ludo’.

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Ludovic Courtès]

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

> +     (name "gnumach-headers-boot0")
> +     (version "1.8+git20221224")
> +     (source
> +      (origin
> +        (method
> +         (git-fetch-from-tarball
> +          (origin
> +            (method url-fetch)
> +            (uri (string-append
> +                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
> +                  "gnumach-" version ".tar.gz"))
> +            (sha256
> +             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
> +        (uri (git-reference
> +              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
> +              (commit version)))
> +        (file-name (git-file-name name version))
> +        (sha256
> +         (base32
> +          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))

Since this is the same revision as in hurd.scm, we can reduce
duplication like so:

  (origin
    (inherit (package-source gnumach-headers))
    (method (git-fetch-from-tarball …)))

Otherwise LGTM.

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Ludovic Courtès]

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

> +     (source
> +      (origin
> +        (method
> +         (git-fetch-from-tarball
> +          (origin
> +            (method url-fetch)
> +            (uri (string-append
> +                  "https://git.savannah.gnu.org/cgit/hurd/mig.git/snapshot/"
> +                  "mig-" version ".tar.gz"))
> +            (sha256
> +             (base32 "1l1vfm4wap5yxylv91wssgpy7fnq22wp3akgd5nv995kychfa9jy")))))
> +        (uri (git-reference
> +              (url "https://git.savannah.gnu.org/git/hurd/mig.git")
> +              (commit version)))
> +        (file-name (git-file-name name version))
> +        (sha256
> +         (base32 "10r0fdjqjzqsy6ajb21rifvhw0wpjvrw6a1zdyliqlzqny5k0qlz"))))

Likewise: (origin (inherit (package-source mig)) …)

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Ludovic Courtès]

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

> From: Josselin Poiret <dev@jpoiret.xyz>
>
> Now that mig-boot0 has been updated not to use mig's source, we can
> safely update mig.
>
> * gnu/packages/hurd.scm (mig)[source]: Update to 1.8+git20230520.

This one could be avoided, at least for now, due to
4e11cac6be68a674c16965dc74e7da54cd3168df and the tarball we’re hosting
on ftp.gnu.org.  (We could also avoid the corresponding change in
‘mig-boot0’.)

But really, either way is fine, as long as both cross-compilation and
native compilation work after that.  :-)

Ludo’.

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Ludovic Courtès]

Hi,

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

> So...updating mig using git-fetch created a circular dependency because
> mig-boot0 re-used mig's source, and we cannot use git in commencement.
>
> This patch series now uses a hack by Ludo (thanks!) as suggested on IRC
> <https://logs.guix.gnu.org/guix/2023-07-17.log#233051> to not directly use
> generated cgit tarballs from savannah.
>
> The versions of gnumach-headers-boot0, mig-boot0, and hurd-headers-boot0 are
> updated so that native builds work again.  Because upstream does not produce
> release tarballs we now also need autoconf and automake in commencement.

I sent minor suggestions, but overall LGTM!

As of 4e11cac6be68a674c16965dc74e7da54cd3168df, we have
cross-compilation to i586-pc-gnu working again.  After this series, we
should still have that + ‘-s i586-gnu’ (native compilation) working
(modulo test failures fixed by <https://issues.guix.gnu.org/64711>, of
course).  I haven’t checked whether this is the case but that’s the
idea, right?  :-)

Thanks again for this!

Ludo’.

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Janneke Nieuwenhuizen]

Ludovic Courtès writes:

Hi!

> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> +     (name "gnumach-headers-boot0")
>> +     (version "1.8+git20221224")
>> +     (source
>> +      (origin
>> +        (method
>> +         (git-fetch-from-tarball
>> +          (origin
>> +            (method url-fetch)
>> +            (uri (string-append
>> +                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
>> +                  "gnumach-" version ".tar.gz"))
>> +            (sha256
>> +             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
>> +        (uri (git-reference
>> +              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
>> +              (commit version)))
>> +        (file-name (git-file-name name version))
>> +        (sha256
>> +         (base32
>> +          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))
>
> Since this is the same revision as in hurd.scm, we can reduce
> duplication like so:
>
>   (origin
>     (inherit (package-source gnumach-headers))
>     (method (git-fetch-from-tarball …)))

Eh...no?  Because gnumach-headers uses method git-fetch now!

> Otherwise LGTM.

Thanks!

-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Janneke Nieuwenhuizen]

Ludovic Courtès writes:

Hi!

> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> From: Josselin Poiret <dev@jpoiret.xyz>
>>
>> Now that mig-boot0 has been updated not to use mig's source, we can
>> safely update mig.
>>
>> * gnu/packages/hurd.scm (mig)[source]: Update to 1.8+git20230520.
>
> This one could be avoided, at least for now, due to
> 4e11cac6be68a674c16965dc74e7da54cd3168df and the tarball we’re hosting
> on ftp.gnu.org.  (We could also avoid the corresponding change in
> ‘mig-boot0’.)

I believe we'd rather re-instate this change to build mig fram git
so that we can do --with-commit to track all the neat things that
upstream is doing, right Josselin?

> But really, either way is fine, as long as both cross-compilation and
> native compilation work after that.  :-)

Sure!

-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Ludovic Courtès]

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

>> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>>
>>> +     (name "gnumach-headers-boot0")
>>> +     (version "1.8+git20221224")
>>> +     (source
>>> +      (origin
>>> +        (method
>>> +         (git-fetch-from-tarball
>>> +          (origin
>>> +            (method url-fetch)
>>> +            (uri (string-append
>>> +                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
>>> +                  "gnumach-" version ".tar.gz"))
>>> +            (sha256
>>> +             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
>>> +        (uri (git-reference
>>> +              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
>>> +              (commit version)))
>>> +        (file-name (git-file-name name version))
>>> +        (sha256
>>> +         (base32
>>> +          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))
>>
>> Since this is the same revision as in hurd.scm, we can reduce
>> duplication like so:
>>
>>   (origin
>>     (inherit (package-source gnumach-headers))
>>     (method (git-fetch-from-tarball …)))
>
> Eh...no?  Because gnumach-headers uses method git-fetch now!

That’s the whole point: here we override the method with one that’s
close to the graph root, but the <git-reference> and hash are the same
as the one from ‘gnumach-headers’.  Inheriting makes the intent
explicit.

Which is pretty cool no?

Ludo’.

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Janneke Nieuwenhuizen]

Ludovic Courtès writes:

> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>>> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>>>
>>>> +     (name "gnumach-headers-boot0")
>>>> +     (version "1.8+git20221224")
>>>> +     (source
>>>> +      (origin
>>>> +        (method
>>>> +         (git-fetch-from-tarball
>>>> +          (origin
>>>> +            (method url-fetch)
>>>> +            (uri (string-append
>>>> +                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
>>>> +                  "gnumach-" version ".tar.gz"))
>>>> +            (sha256
>>>> +             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
>>>> +        (uri (git-reference
>>>> +              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
>>>> +              (commit version)))
>>>> +        (file-name (git-file-name name version))
>>>> +        (sha256
>>>> +         (base32
>>>> +          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))
>>>
>>> Since this is the same revision as in hurd.scm, we can reduce
>>> duplication like so:
>>>
>>>   (origin
>>>     (inherit (package-source gnumach-headers))
>>>     (method (git-fetch-from-tarball …)))

Ah, I didn't really read this properly; pretty cool!

>> Eh...no?  Because gnumach-headers uses method git-fetch now!
>
> That’s the whole point: here we override the method with one that’s
> close to the graph root, but the <git-reference> and hash are the same
> as the one from ‘gnumach-headers’.  Inheriting makes the intent
> explicit.

I still wonder a bit if it's a good idea to have commencement packages /
this commencement package inherit from hurd.scm; it means a world
rebuild whenever hurd.scm is updated (unless you put something back like
the verbose origin declaration above).

I'll take with your suggestion and go for the less code duplication
option right now, we can change it again when the need arises.

> Which is pretty cool no?

Yeah!  Sorry for the confusion :)

Greetings,
Janneke

-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

bug#64708: [PATCH 0/8] Fix cross build and native build for the Hurd.

[Janneke Nieuwenhuizen]

Ludovic Courtès writes:

Hello,

> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> The versions of gnumach-headers-boot0, mig-boot0, and hurd-headers-boot0 are
>> updated so that native builds work again.  Because upstream does not produce
>> release tarballs we now also need autoconf and automake in commencement.
>
> I sent minor suggestions, but overall LGTM!
>
> As of 4e11cac6be68a674c16965dc74e7da54cd3168df, we have
> cross-compilation to i586-pc-gnu working again.

That's right, someone beat me to it ;-) (thanks!!)

> After this series, we should still have that + ‘-s i586-gnu’ (native
> compilation) working (modulo test failures fixed by
> <https://issues.guix.gnu.org/64711>, of course).  I haven’t checked
> whether this is the case but that’s the idea, right?  :-)

Certainly!

> Thanks again for this!

That's always good to hear.

Pushed to master as 8831d3ba9f799b7342ab466d625097f1982513a9

Greetings,
Janneke

-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Josselin Poiret via Guix-patches via]

[-- Attachment #1: Type: text/plain, Size: 671 bytes --]

Hi Janneke and Ludo,

Janneke Nieuwenhuizen <janneke@gnu.org> writes:

> Ludovic Courtès writes:
>> This one could be avoided, at least for now, due to
>> 4e11cac6be68a674c16965dc74e7da54cd3168df and the tarball we’re hosting
>> on ftp.gnu.org.  (We could also avoid the corresponding change in
>> ‘mig-boot0’.)
>
> I believe we'd rather re-instate this change to build mig fram git
> so that we can do --with-commit to track all the neat things that
> upstream is doing, right Josselin?

Yes, that was the point, we don't want to upload a new tarball to the
ftp all the time (esp. because I can't do it myself :) ).

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Josselin Poiret via Guix-patches via]

[-- Attachment #1: Type: text/plain, Size: 1017 bytes --]

Hi Ludo and Janneke.

Ludovic Courtès <ludo@gnu.org> writes:

> The trick below is the only way I could think of to reconcile these
> issues: it essentially stacks two fixed-output derivation.  The first
> one downloads that auto-generated tarball (bad!), and the second one
> extracts the contents of that tarball using ‘tar’ from
> ‘%bootstrap-coreutils&co’.  The output of the second one is identical to
> what ‘git-fetch’ would give us (or ‘swh-download’).
>
> That makes it more robust.  It also allows ‘guix lint -c archival’ to do
> its work and similarly (well, almost) for ‘build-package-metadata.scm’,
> the program that builds <https://guix.gnu.org/sources.json>, which SWH
> periodically ingests.

I don't 100% understand what this resolves though, since this still
depends on the unstable tarball in the end!  In both cases, we'd rely on
CI to have the stable copy, or SWH providing it, since we wouldn't be
able to rebuild it.

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

[bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

[Janneke Nieuwenhuizen]

Josselin Poiret writes:

Hi Josselin and Ludo,

> Janneke Nieuwenhuizen <janneke@gnu.org> writes:
>
>> Ludovic Courtès writes:
>>> This one could be avoided, at least for now, due to
>>> 4e11cac6be68a674c16965dc74e7da54cd3168df and the tarball we’re hosting
>>> on ftp.gnu.org.  (We could also avoid the corresponding change in
>>> ‘mig-boot0’.)
>>
>> I believe we'd rather re-instate this change to build mig fram git
>> so that we can do --with-commit to track all the neat things that
>> upstream is doing, right Josselin?
>
> Yes, that was the point, we don't want to upload a new tarball to the
> ftp all the time (esp. because I can't do it myself :) ).

So, that's what I've done, I think we're OK!

Thanks for the confirmation,
Janneke

-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com