[PATCH] Replace static bash in grafted glibc.

[PATCH] Replace static bash in grafted glibc.

[Ricardo Wurmus]

[-- Attachment #1: Type: text/plain, Size: 1653 bytes --]

Hi Guix,

the two patches here are necessary to fix a problem with the grafted
glibc.

You may recall that we grafted the glibc packages to ensure that Guix
packages can still be used on CentOS 6, which has a kernel that reports
its version as 2.6.32 despite being compatible with the current glibc
version.  Since the glibc only checks for the version number and not for
kernel features it aborts with “FATAL: kernel too old”.  We added a
glibc graft to replace the glibc with a patched glibc.

Unfortunately, this was not enough.  The libc keeps a reference to a
statically linked bash for system(3).  That statically linked bash uses
features from an unpatched glibc, so that bash aborts with “FATAL:
kernel too old” on CentOS.  This is a problem, because it means that
Python’s “os.system” feature and R’s “system()” are broken.

The solution is to also replace the statically linked bash with a
variant that has been statically linked in the presence of a patched
glibc.  This is straight-forward for the most part, except that doing
this naively for “glibc-final” leads to a derivation loop, as Ludo
discovered.

The solution as suggested by Ludo is to model the replacement graph
after the bootstrap graph.  The result is attached to this email.

I have used this to build “r-minimal” and installed it on CentOS 6 –
“system("ls")” works fine now and doesn’t abort with “FATAL: kernel too
old”.  It also looks like this doesn’t cause a mass rebuild, so I’m
happy with the outcome.

I’d like to push this to master in a couple of hours, unless there are
objections.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-Use-patched-static-bash-in-glibc-replacement.patch --]
[-- Type: text/x-patch, Size: 2184 bytes --]

From 487ef5393cb2f93648b3912529385f30c43a603b Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Tue, 13 Mar 2018 22:34:43 +0100
Subject: [PATCH 1/2] gnu: Use patched static bash in glibc replacement.

* gnu/packages/base.scm (glibc-2.26-patched): Rename...
(glibc-2.26-patched-boot): ...to this.
(patched-static-bash, glibc-2.26-patched): New variables.
---
 gnu/packages/base.scm | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index e91f7d43e..56f79f580 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -841,7 +841,7 @@ GLIBC/HURD for a Hurd host"
 ;; Below are old libc versions, which we use mostly to build locale data in
 ;; the old format (which the new libc cannot cope with.)
 
-(define glibc-2.26-patched
+(define glibc-2.26-patched-boot
   (package
     (inherit glibc)
     (source (origin
@@ -849,6 +849,31 @@ GLIBC/HURD for a Hurd host"
               (patches (cons (search-patch "glibc-allow-kernel-2.6.32.patch")
                              (origin-patches (package-source glibc))))))))
 
+(define patched-static-bash
+  (package
+    (inherit static-bash)
+    (arguments
+     (substitute-keyword-arguments (package-arguments static-bash)
+       ((#:configure-flags flags '())
+        ;; Add a '-L' flag so that the pseudo-cross-ld of
+        ;; BINUTILS-BOOT0 can find libc.a.
+        `(append ,flags
+                 (list (string-append "LDFLAGS=-static -L"
+                                      (assoc-ref %build-inputs
+                                                 "libc-patched:static")
+                                      "/lib"))))))
+    (native-inputs
+     `(("libc-patched" ,glibc-2.26-patched-boot)
+       ("libc-patched:static" ,glibc-2.26-patched-boot "static")))))
+
+(define glibc-2.26-patched
+  (package
+    (inherit glibc-2.26-patched-boot)
+    (inputs `(("static-bash" ,patched-static-bash)
+              ,@(alist-delete
+                 "static-bash"
+                 (package-inputs glibc-2.26-patched-boot))))))
+
 (define-public glibc-2.25
   (package
     (inherit glibc)
-- 
2.16.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-gnu-Add-replacement-for-static-bash-for-glibc.patch --]
[-- Type: text/x-patch, Size: 3824 bytes --]

From f844a6254ebb90666be1cdb4f44bb57561f36faa Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Wed, 14 Mar 2018 00:09:03 +0100
Subject: [PATCH 2/2] gnu: Add replacement for static-bash-for-glibc.

This is needed to replace the static bash used in glibc-final.

* gnu/packages/commencement.scm (patched-glibc-final-with-bootstrap-bash,
patched-static-bash-for-glibc): New variables.
(static-bash-for-glibc)[replacement]: Use patched-static-bash-for-glibc as a
replacement.
---
 gnu/packages/commencement.scm | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index fe9fbebcc..94847b019 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -556,6 +556,14 @@ the bootstrap environment."
         ;; because we don't want to depend on bootstrap tools.
         ("static-bash" ,@(assoc-ref %boot0-inputs "bash")))))))
 
+(define patched-glibc-final-with-bootstrap-bash
+  (package
+    (inherit glibc-final-with-bootstrap-bash)
+    (source (origin
+              (inherit (package-source glibc-final-with-bootstrap-bash))
+              (patches (cons (search-patch "glibc-allow-kernel-2.6.32.patch")
+                             (origin-patches (package-source glibc-final-with-bootstrap-bash))))))))
+
 (define (cross-gcc-wrapper gcc binutils glibc bash)
   "Return a wrapper for the pseudo-cross toolchain GCC/BINUTILS/GLIBC
 that makes it available under the native tool names."
@@ -632,6 +640,40 @@ exec ~a/bin/~a-~a -B~a/lib -Wl,-dynamic-linker -Wl,~a/~a \"$@\"~%"
                    ("libc:static" ,glibc-final-with-bootstrap-bash "static")
                    ,@(fold alist-delete %boot1-inputs
                            '("gcc" "libc")))))
+    (let ((p (package-with-bootstrap-guile
+              (package-with-explicit-inputs bash inputs
+                                            (current-source-location)
+                                            #:guile %bootstrap-guile))))
+      (package (inherit p)
+               (replacement patched-static-bash-for-glibc)))))
+
+;; Same as static-bash-for-glibc, except that it uses a patched glibc for the
+;; "gcc" and "bash" packages.
+(define patched-static-bash-for-glibc
+  ;; A statically-linked Bash to be used by GLIBC-FINAL in system(3) & co.
+  (let* ((gcc  (cross-gcc-wrapper gcc-boot0 binutils-boot0
+                                  patched-glibc-final-with-bootstrap-bash
+                                  (car (assoc-ref %boot1-inputs "bash"))))
+         (bash (package
+                 (inherit static-bash)
+                 (arguments
+                  (substitute-keyword-arguments
+                      (package-arguments static-bash)
+                    ((#:guile _ #f)
+                     '%bootstrap-guile)
+                    ((#:configure-flags flags '())
+                     ;; Add a '-L' flag so that the pseudo-cross-ld of
+                     ;; BINUTILS-BOOT0 can find libc.a.
+                     `(append ,flags
+                              (list (string-append "LDFLAGS=-static -L"
+                                                   (assoc-ref %build-inputs
+                                                              "libc:static")
+                                                   "/lib"))))))))
+         (inputs `(("gcc" ,gcc)
+                   ("libc" ,patched-glibc-final-with-bootstrap-bash)
+                   ("libc:static" ,patched-glibc-final-with-bootstrap-bash "static")
+                   ,@(fold alist-delete %boot1-inputs
+                           '("gcc" "libc")))))
     (package-with-bootstrap-guile
      (package-with-explicit-inputs bash inputs
                                    (current-source-location)
-- 
2.16.2


[-- Attachment #4: Type: text/plain, Size: 12 bytes --]


--
Ricardo

Re: [PATCH] Replace static bash in grafted glibc.

[Ludovic Courtès]

Hi Ricardo,

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> I’d like to push this to master in a couple of hours, unless there are
> objections.

Go for it, thanks a lot!

Ludo’.

Re: [PATCH] Replace static bash in grafted glibc.

[Ricardo Wurmus]

Hi Ludo,

> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
>
>> I’d like to push this to master in a couple of hours, unless there are
>> objections.
>
> Go for it, thanks a lot!

Pushed to master with commit c4fb2b9f4.  Thanks for helping me
understand this problem!

-- 
Ricardo

Re: [PATCH] Replace static bash in grafted glibc.

[Ricardo Wurmus]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:

> Hi Ludo,
>
>> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
>>
>>> I’d like to push this to master in a couple of hours, unless there are
>>> objections.
>>
>> Go for it, thanks a lot!
>
> Pushed to master with commit c4fb2b9f4.  Thanks for helping me
> understand this problem!

Commit c4fb2b9f4 (the changes to commencement) seems to have broken
Guix.  On some derivations we now get “Uncaught Exception:” and nothing
else.  Reverting that one commit lets the problem disappear.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: [PATCH] Replace static bash in grafted glibc.

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:
>
>> Hi Ludo,
>>
>>> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
>>>
>>>> I’d like to push this to master in a couple of hours, unless there are
>>>> objections.
>>>
>>> Go for it, thanks a lot!
>>
>> Pushed to master with commit c4fb2b9f4.  Thanks for helping me
>> understand this problem!
>
> Commit c4fb2b9f4 (the changes to commencement) seems to have broken
> Guix.  On some derivations we now get “Uncaught Exception:” and nothing
> else.  Reverting that one commit lets the problem disappear.

We’re tracking progress on this one here:

  https://bugs.gnu.org/30820

Grafting may be broken in many cases, so we’ll have to rebuild and merge
‘core-updates’ sooner than we thought.

Ludo’.