From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Down with /bin/sh! Date: Thu, 03 Jan 2013 16:14:12 +0100 Message-ID: <87a9sq47uj.fsf@inria.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([208.118.235.92]:54965) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TqmUt-0000PH-Q9 for bug-guix@gnu.org; Thu, 03 Jan 2013 10:14:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TqmUq-0003Dq-Ur for bug-guix@gnu.org; Thu, 03 Jan 2013 10:14:19 -0500 Received: from mail4-relais-sop.national.inria.fr ([192.134.164.105]:48938) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TqmUq-0003DT-Nr for bug-guix@gnu.org; Thu, 03 Jan 2013 10:14:16 -0500 List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org To: bug-guix@gnu.org Cc: nix-dev Hello {N,Gu}ixers, and happy new year! The =E2=80=98core-updates=E2=80=99 branch of Guix now makes it possible to = build packages in a chroot lacking /bin/sh. It=E2=80=99s convenient to have /bin/sh in the chroot, because that=E2=80= =99s basically one of the files whose name is hardcoded in many places, from libc to shebangs. However, the problem with /bin/sh is that it=E2=80=99s out of control. On non-NixOS distros, it=E2=80=99s not necessarily Bash. Someone with Dash as /bin/sh may have different results than someone Bash as /bin/sh. That=E2= =80=99s an =E2=80=9Cimpurity=E2=80=9D, with practical consequences on build reprodu= cibility. Another issue on non-NixOS distros is that /bin/sh is typically a dynamically-linked executable. So adding /bin to the chroot is not enough; one typically needs to also add /lib* and /lib/*-linux-gnu to the chroot. At that point, there are many impurities, and a great potential for non-reproducibility=E2=80=93which defeats the purpose of the = chroot. So, here=E2=80=99s how this is solved in =E2=80=98core-updates=E2=80=99. =E2=80=A2 Right after unpacking a source tarball, all the source files go through =E2=80=98patch-shebang=E2=80=99, which replaces any #!/bin/sh a= nd similar with the right path. =E2=80=A2 Once configure has run, all makefiles that define the =E2=80=98= SHELL=E2=80=99 variable are patched similarly. The GNU build system supports =E2=80=98CONFIG_SHELL=E2=80=99 and =E2=80=98S= HELL=E2=80=99, which simplifies things (info "(autoconf) config.status Invocation"). Occasionally, packages have references to /bin/sh elsewhere, which need to be patched as well, notably: =E2=80=A2 the =E2=80=98system=E2=80=99 and =E2=80=98popen=E2=80=99 functi= ons in the GNU libc; =E2=80=A2 the =E2=80=98default_shell=E2=80=99 variable in GNU make; =E2=80=A2 io.c in GNU Awk; =E2=80=A2 ice-9/popen.scm in GNU Guile. For libc, referring to the compile-time Bash would retain a dependency on the bootstrap environment. So glibc is changed to contain a copy of a statically-linked Bash in its store path, which it can refer to (thanks to Shea and Llu=C3=ADs for the suggestion). The bootstrap libc and awk need a different trick, though, since they must be relocated (IOW, their path is not known in advance.) The trick is to search for =E2=80=98sh=E2=80=99 in $PATH, with these simple patches: http://git.savannah.gnu.org/cgit/guix.git/tree/distro/packages/patches/gl= ibc-bootstrap-system.patch?h=3Dcore-updates http://git.savannah.gnu.org/cgit/guix.git/tree/distro/packages/patches/ga= wk-shell.patch?h=3Dcore-updates (For security reasons, these patches are /not/ used in the final libc and awk.) So far, it seems that little or no manual tweaking is needed beyond the above, so hopefully it=E2=80=99ll scale. The main advantage is that it mak= es it possible to use Guix reliably on non-NixOS distros; even on a distro like NixOS, that=E2=80=99s one less impurity, and thus improved reproducibi= lity. Thanks, Ludo=E2=80=99.