'core-updates' spring 2018

'core-updates' spring 2018

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 14831 bytes --]

Guix,

'core-updates' has seen a lot of changes recently.  Some of the goodies
include HTTP/2 support in curl, binutils 2.30, glibc 2.27, reproducible
Python bytecode, gnu-build-system will automatically run 'autoreconf' if
necessary, warnings will be issued about snippets and phases not
returning #t, and much more.

Are there other things that should go in?

Let's set a date for "freezing" it, after which it will be started on
Hydra and only bugfixes are allowed.  How about Monday, April 2nd?

Full list of changes:

Alex Vong (1):
      gnu: tar: Update to 1.30.

Arun Isaac (3):
      gnu: boost: Update to 1.66.0.
      gnu: cmake: Update to 3.10.2.
      gnu: cmake: Re-indent.

Efraim Flashner (4):
      gnu: mpfr: Update to 4.0.1.
      gnu: libunistring: Update to 0.9.9.
      gnu: gawk: Update to 4.2.1.
      gnu: gawk: Enable parallel tests.

Leo Famulari (5):
      gnu: libgc: Update to 7.6.4.
      gnu: libatomic-ops: Update to 7.6.2.
      gnu: libgcrypt: Update to 1.8.2.
      gnu: gnutls: Update to 3.5.18.
      gnu: libtasn1: Update to 4.13.

Ludovic Courtès (7):
      gnu: sqlite: Fix source URL.
      gnu: tar: Skip faulty test.
      utils: Add 'false-if-file-not-found'.
      build-system/gnu: Add 'bootstrap' phase.
      build-system/gnu: Dump test suite logs upon 'check' failure.
      build-system/gnu: Fix typo.
      gnu: gcc: Adjust 'strmov' patch for string constants in variables.

Marius Bakke (66):
      gnu: libuv: Update to 1.19.1.
      gnu: util-linux: Update to 2.31.1.
      gnu: glib: Update to 2.54.3.
      gnu: dbus: Update to 1.12.4.
      gnu: sqlite: Update to 3.22.0.
      gnu: libbsd: Update to 0.8.7.
      gnu: gzip: Update to 1.9.
      gnu: gdbm: Update to 1.14.
      gnu: ncurses: Update to 6.1.
      gnu: mpc: Update to 1.1.0.
      gnu: less: Update to 530.
      gnu: libsigsegv: Update to 2.12.
      gnu: bash: Update to patch level 19.
      gnu: libxml2: Update to 2.9.8.
      gnu: graphite2: Update to 1.3.11.
      gnu: libuv: Update to 1.19.2.
      gnu: harfbuzz: Update to 1.7.6.
      gnu: dbus: Update to 1.12.6.
      gnu: util-macros: Update to 1.19.2.
      gnu: xcb-proto: Update to 1.13.
      gnu: libxcb: Update to 1.13.
      gnu: Remove gcc-for-libstdc++.
      gnu: patch: Update to 2.7.6.
      gnu: gdbm: Update to 1.14.1.
      gnu: automake: Update to 1.16.1.
      gnu: glib: Update to 2.56.0.
      gnu: gobject-introspection: Update to 1.56.0.
      gnu: pango: Update to 1.42.0.
      gnu: pango: Explicitly propagate some inputs.
      gnu: python: Update to 3.6.4.
      gnu: binutils: Update to 2.30.
      gnu: glibc: Don't apply upstreamed patch in newer glibc versions.
      gnu: Switch from xproto to xorgproto.
      gnu: Switch from randrproto to xorgproto.
      gnu: Switch from bigreqsproto to xorgproto.
      gnu: Switch from xextproto to xorgproto.
      gnu: Switch from fixesproto to xorgproto.
      gnu: Switch from compositeproto to xorgproto.
      gnu: Switch from kbproto to xorgproto.
      gnu: Switch from inputproto to xorgproto.
      gnu: Switch from fontsproto to xorgproto.
      gnu: Switch from glproto to xorgproto.
      gnu: Switch from dmxproto to xorgproto.
      gnu: Switch from damageproto to xorgproto.
      gnu: Switch from dri2proto to xorgproto.
      gnu: Switch from dri3proto to xorgproto.
      gnu: Switch from videoproto to xorgproto.
      gnu: Switch from windowswmproto to xorgproto.
      gnu: Switch from xcmiscproto to xorgproto.
      gnu: Switch from presentproto to xorgproto.
      gnu: Switch from renderproto to xorgproto.
      gnu: Switch from resourceproto to xorgproto.
      gnu: Switch from recordproto to xorgproto.
      gnu: Switch from scrnsaverproto to xorgproto.
      gnu: Switch from xf86driproto to xorgproto.
      gnu: Switch from xf86dgaproto to xorgproto.
      gnu: Switch from xf86bigfontproto to xorgproto.
      gnu: Switch from xf86vidmodeproto to xorgproto.
      gnu: Switch from xineramaproto to xorgproto.
      gnu: freetype: Update to 2.9.
      gnu: fontconfig: Update to 2.13.0.
      gnu: poppler: Update to 0.63.0.
      gnu: glibc: Update to 2.27.
      gnu: make: Fix compatibility with glibc 2.27.
      gnu: make: End phase on #t.
      gnu: util-linux: Don't build 'nologin'.

Mark H Weaver (162):
      packages: patch-and-repack: Use invoke instead of system*.
      gnu-dist: Use invoke instead of system*.
      build-system/gnu: Use invoke instead of system*.
      build-system/glib-or-gtk: Use invoke instead of system*.
      build-system/cmake: Use invoke instead of system*.
      build-system/perl: Use invoke instead of system*.
      build-system/python: Use invoke instead of system*.
      build-system/texlive: Use invoke instead of system*.
      build-system/scons: Use invoke instead of system*.
      bootstrap: Use invoke instead of system*.
      download: Use invoke instead of system*.
      gnu: linux-libre-headers: Use invoke instead of system*.
      gnu: glibc-utf8-locales: Use invoke instead of system*.
      gnu: glibc-locales: Use invoke instead of system*.
      gnu: glibc/hurd: Use invoke instead of system*.
      gnu: perl: Use invoke instead of system*.
      gnu: tcsh: Use invoke instead of system*.
      gnu: zlib: Use invoke instead of system*.
      gnu: unzip: Use invoke instead of system*.
      gnu: zip: Use invoke instead of system*.
      gnu: bzip2: Use invoke instead of system*.
      gnu: openssl: Use invoke instead of system*.
      gnu: eudev: Use invoke instead of system*.
      gnu: tzdata: Use invoke instead of system*.
      gnu: python-six: Use invoke instead of system*.
      cvs-download: Use invoke instead of system*.
      svn-download: Use invoke instead of system*.
      hg-download: Use invoke instead of system*.
      build: emacs-utils: Use invoke instead of system*.
      gnu: attr: Use invoke instead of system*.
      packages: patch-and-repack: Fix snippet handling.
      packages: Update copyright notice.
      build-system/gnu: Return a boolean from all phase procedures.
      gnu: autoconf-wrapper: Return #t from the builder.
      gnu: acl: Use invoke instead of system*.
      gnu: libcap: Return #t from all phases.
      gnu: gcc-stripped: Use invoke instead of system*.
      gnu: guile-static-stripped: Use invoke instead of system*.
      gnu: make-bootstrap: tarball-package: Use invoke instead of system*.
      gnu: linux-libre-headers: Update to 4.14.26.
      packages: Issue a warning unless the snippet returns #t.
      gnu-build: Issue a warning unless every phase returns #t.
      gnu: cross-base: Return #t from all phases.
      gnu: All snippets report errors using exceptions, else return #t.
      gnu: libstdc++: Return #t from chdir phase.
      gettext-boot0: Return #t from chdir phase.
      gnu: tar: Fix sparse file detection on Btrfs file systems.
      gnu: bash: Return #t from 'install-sh-symlink' phase.
      utils: invoke: Raise exceptions using SRFI-34 and SRFI-35.
      build-system/gnu: Tolerate errors during the 'strip' phase.
      gnu: Fix a few more snippets to return #t.
      gnu: unzip: Ungraft.
      gnu: curl: Ungraft.
      gnu: libtiff: Ungraft.
      gnu: freetype: Ungraft.
      gnu: libsndfile: Ungraft.
      gnu: libvorbis: Ungraft.
      gnu: guile-2.2/fixed: Remove vestigial (replacement #f).
      gnu: net-tools: Return #t from custom configure phase.
      gnu: openssl: Simplify calls to 'invoke'.
      gnu: gs-fonts: Return #t from install phase.
      gnu: python: Return #t from all phases.
      gnu: dbus: Detect failures in the install phase.
      gnu: giflib: Return #t from all phases.
      gnu: tk: Return #t from all phases.
      gnu: glib: Return #t from all phases.
      gnu: boost: Simplify phases.
      gnu: gobject-introspection: Disable parallel build.
      build-system/python: Return #t from all phases.
      build-system/python: Fix deletion of .egg-info dirs created by tests.
      gnu: cups: Return #t from all phases.
      gnu: guile-gdbm-ffi: Use 'invoke'.
      gnu: guile-gdbm-ffi: Add support for gdbm-1.14.
      gnu: qpdf: Return #t from all phases.
      gnu: font-dejavu: Return #t from all phases.
      gnu: orc: Return #t from all phases.
      gnu: webkitgtk: Return #t from all phases.
      gnu: itstool: Return #t from all phases.
      gnu: aspell: Return #t from all phases.
      gnu: tzdata-for-tests: Update to 2018c.
      gnu: glibc/linux: Return #t from pre-configure phase.
      gnu: apr-util: Use invoke.
      gnu: autoconf@2.13: Use invoke.
      gnu: curl: Use invoke.
      gnu: dejagnu: Return #t from all phases.
      gnu: elogind: Return #t from all phases.
      gnu: emacs: Return #t from all phases.
      gnu: extra-cmake-modules: Return #t from all phases.
      gnu: faad2: Use invoke.
      gnu: ffmpeg: Use invoke.
      gnu: fontconfig: Use invoke.
      gnu: gawk: Return #t from all phases.
      gnu: cross-base: Use invoke in (gnu build cross-toolchain).
      gnu: gdk-pixbuf+svg: Use invoke.
      gnu: geoclue: Return #t from all phases.
      gnu: gettext-minimal: Return #t from all phases.
      gnu: ghostscript: Use invoke.
      gnu: git: Return #t from all phases.
      gnu: glibc-hurd-headers: Return #t from all phases.
      gnu: gmime: Return #t from all phases.
      gnu: gnumach-headers: Use invoke.
      gnu: hurd-headers: Use invoke.
      gnu: hurd-minimal: Use invoke.
      gnu: gnupg@1: Return #t from all phases.
      gnu: gtk+@2: Return #t from all phases.
      gnu: guix: Use invoke.
      gnu: gnumach: Use invoke.
      gnu: icecat: Return #t from all phases.
      gnu: ijs: Use invoke.
      gnu: imagemagick: Return #t from all phases.
      gnu: perl-image-magick: Return #t from all phases.
      gnu: imake: Use invoke.
      gnu: libarchive: Use invoke.
      gnu: libelf: Use invoke.
      gnu: libsoup: Use invoke.
      gnu: libssh2: Use invoke.
      gnu: lua: Use invoke.
      gnu: lua5.1-expat: Use invoke.
      gnu: lua5.1-socket: Use invoke.
      gnu: man-db: Return #t from all phases.
      gnu: mariadb: Return #t from all phases.
      gnu: mesa-headers: Return #t from all phases.
      gnu: mesa-utils: Return #t from all phases.
      gnu: mozjs: Return #t from all phases.
      gnu: mozjs@24: Use invoke.
      gnu: mozjs@38: Use invoke.
      gnu: mysql: Return #t from all phases.
      gnu: nasm: Use invoke.
      gnu: nss: Use invoke.
      gnu: nss-certs: Use invoke.
      gnu: le-certs: Use invoke.
      gnu: openssh: Use invoke.
      gnu: pciutils: Use invoke.
      gnu: postgresql: Use invoke.
      gnu: subversion: Use invoke.
      gnu: tcl: Use invoke.
      gnu: texlive-bin: Use invoke.
      gnu: texlive-fonts-amsfonts: Use invoke.
      gnu: texlive-fonts-cm: Use invoke.
      gnu: texlive-fonts-ec: Remove vestigial error code plumbing.
      gnu: texlive-fonts-knuth-lib: Use invoke.
      gnu: texlive-fonts-latex: Use invoke.
      gnu: texlive-fonts-rsfs: Remove vestigial error plumbing.
      gnu: texlive-latex-base: Use invoke.
      gnu: texlive-metafont-base: Use invoke.
      gnu: texmaker: Use invoke.
      gnu: at-spi2-core: Use invoke.
      gnu: at-spi2-atk: Use invoke.
      gnu: clang: Return #t from all phases.
      gnu: bdb: Use invoke.
      gnu: binutils-vc4: Use invoke.
      gnu: mkfontdir: Return #t from all phases.
      gnu: pulseaudio: Fix build with glibc-2.27.
      gnu: shadow: Return #t from all phases.
      gnu: xf86-video-intel: Fix error during bootstrap.
      gnu: wireless-regdb: Inhibit gzip timestamp.
      gnu: asciidoc: Restore custom 'bootstrap' phase.
      gnu: crda: Inhibit gzip timestamp.
      gnu: make-bootstrap: Adapt glibc-bootstrap-system.patch for glibc-2.27.
      gnu: perl-boot0: Return #t from all phases.
      gnu: libva: Return #t from all phases.
      gnu: libselinux: Use invoke.

Ricardo Wurmus (19):
      gnu: Ungraft glibc.
      gnu: ghc-7: Fix build with ncurses 6.1.
      gnu: python-3.6: Disable failing test.
      gnu: python-3.6: Reset timestamps in pyc files.
      guix: python-build-system: Make bytecode compilation deterministic.
      gnu: libtool: Add libltdl to native-inputs.
      gnu: cunit: Replace bootstrap phase.
      gnu: boost: Remove bootstrap phase.
      gnu: Remove boost-1.66.
      gnu: acl: Fix build failure.
      gnu: kentutils: Build with mariadb.
      gnu: classpath-devel: Fix bootstrap phase.
      gnu: glib: Build reproducibly.
      gnu: qt: Build with mariadb.
      gnu: qtbase: Build with mariadb.
      gnu: python-2.7: Rename to "python2".
      gnu: python2-minimal: Rename to "python2-minimal".
      gnu: wayland: Disable failing tests.
      Revert "gnu: wayland: Disable failing tests."

Roel Janssen (1):
      gnu: bash: Don't specify default configuration filenames.

Sou Bunnbu (宋文武) (2):
      gnu: python-3.6: Restore name to "python".
      gnu: openssl: Fix the call of 'apply invoke'.

Tobias Geerinckx-Rice (38):
      gnu: lzip: Update to 1.20.
      gnu: bzip2: Use install-file.
      gnu: fftw: Update to 3.3.7.
      gnu: fftw: Update phase style.
      gnu: kmod: Update to 25.
      gnu: kmod: Update phase style.
      gnu: harfbuzz: Update to 1.7.5.
      gnu: lcms: Update to 2.9.
      gnu: pango: Update to 1.41.1.
      gnu: util-macros: Update phase style.
      gnu: gts: Update phase style.
      gnu: gcc@4.7: Update phase style.
      gnu: commencement: Update phase style.
      gnu: guile@2.0.14: Update phase style.
      gnu: glibc-hurd-headers: Update phase style.
      gnu: dblatex: Update phase style.
      gnu: libffi: Update phase style.
      gnu: make-bootstrap: Update phase style.
      gnu: intltool: Update phase style.
      gnu: indent: Update phase style.
      gnu: cross-base: Update phase style.
      gnu: patchelf: Update phase style.
      gnu: gnurl: Add HTTP/2 support.
      gnu: eudev: Update to 3.2.5.
      gnu: ncurses: Update phase style.
      gnu: m4: Update phase style.
      gnu: coreutils: Update to 8.29.
      Revert "gnu: gnurl: Add HTTP/2 support."
      gnu: automake: Update to 1.16.
      gnu: cups-filters: Update to 1.20.1.
      gnu: pcre2: Update to 10.31.
      gnu: help2man: Update to 1.47.6.
      gnu: fossil: Update to 2.5.
      gnu: curl: Add HTTP/2 support.
      gnu: r-minimal, r: Update to 3.4.4.
      gnu: perl-http-message: Update to 6.15.
      gnu: perl-net-ssleay: Update to 1.85.
      gnu: openldap: Update to 2.4.46.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: 'core-updates' spring 2018

[Ludovic Courtès]

Hello!

Marius Bakke <mbakke@fastmail.com> skribis:

> 'core-updates' has seen a lot of changes recently.  Some of the goodies
> include HTTP/2 support in curl, binutils 2.30, glibc 2.27, reproducible
> Python bytecode, gnu-build-system will automatically run 'autoreconf' if
> necessary, warnings will be issued about snippets and phases not
> returning #t, and much more.
>
> Are there other things that should go in?
>
> Let's set a date for "freezing" it, after which it will be started on
> Hydra and only bugfixes are allowed.  How about Monday, April 2nd?

Sounds good to me!  Save the date!  :-)

For those wondering how to keep the branch active in the meantime ;-), a
useful thing to do would be to put a few key packages on a diet.  For
instance, python and llvm are pretty big and it would be good to see how
we could make them smaller.

For Python specifically, it may be that we can make “python-minimal” the
new python, and build the sqlite, readline, tcl/tk stuff as separate
packages altogether.

Ludo’.

Re: 'core-updates' spring 2018

[Ricardo Wurmus]

Hi Marius,

> 'core-updates' has seen a lot of changes recently.  Some of the goodies
> include […] glibc 2.27 […]
>
> Are there other things that should go in?

I would really like to see a patch applied to glibc that ensures that
the “prlimit64” syscall is not used when running on the RHEL 6 kernel
(2.6.32).  The lack of this syscall on that kernel means that getrlimits
fails, which makes it impossible to start the JVM.

This problem appeared with the upgrade to glibc 2.26 already, and ever
since I’ve been trying to minimize the damage for RHEL 6 systems where
Guix is used as a package manager (such as the MDC).

A work-around for glibc 2.27 that makes things work fine with the RHEL 6
kernel would be very welcome!  I’ve started a branch “rhel6” where the
default glibc has been bumped back to version 2.25 but I really don’t
want it to be a long-lived branch; one of the reasons is that building
all packages for this old glibc version (even just on x86_64) puts our
build farms under extra stress that I would like to avoid.

Another thing that could go in is the new Guile wrapper for wrapped
scripts that I proposed a few months ago.  It requires a change in
build-side code to provide a “wrap-script” procedure (it wouldn’t be
used just yet).

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: 'core-updates' spring 2018

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 1818 bytes --]

2018-03-26 20:34 GMT+02:00 Ricardo Wurmus <rekado@elephly.net>:

>
> Hi Marius,
>
> > 'core-updates' has seen a lot of changes recently.  Some of the goodies
> > include […] glibc 2.27 […]
> >
> > Are there other things that should go in?
>
> I would really like to see a patch applied to glibc that ensures that
> the “prlimit64” syscall is not used when running on the RHEL 6 kernel
> (2.6.32).  The lack of this syscall on that kernel means that getrlimits
> fails, which makes it impossible to start the JVM.
>
>
Hello!
We already talked about this on #guix, and the patch removing code to
check for prlimit64 not implemented is a quite self contained commit on
the glibc tree. It is not a trivial one, and I did not try to revert that
yet, but
it might worth looking at.


> This problem appeared with the upgrade to glibc 2.26 already, and ever
> since I’ve been trying to minimize the damage for RHEL 6 systems where
> Guix is used as a package manager (such as the MDC).
>
> A work-around for glibc 2.27 that makes things work fine with the RHEL 6
> kernel would be very welcome!  I’ve started a branch “rhel6” where the
> default glibc has been bumped back to version 2.25 but I really don’t
> want it to be a long-lived branch; one of the reasons is that building
> all packages for this old glibc version (even just on x86_64) puts our
> build farms under extra stress that I would like to avoid.
>
> Another thing that could go in is the new Guile wrapper for wrapped
> scripts that I proposed a few months ago.  It requires a change in
> build-side code to provide a “wrap-script” procedure (it wouldn’t be
> used just yet).
>
> --
> Ricardo
>
> GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
> https://elephly.net
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2505 bytes --]

Re: 'core-updates' spring 2018

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 2456 bytes --]

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi Marius,
>
>> 'core-updates' has seen a lot of changes recently.  Some of the goodies
>> include […] glibc 2.27 […]
>>
>> Are there other things that should go in?
>
> I would really like to see a patch applied to glibc that ensures that
> the “prlimit64” syscall is not used when running on the RHEL 6 kernel
> (2.6.32).  The lack of this syscall on that kernel means that getrlimits
> fails, which makes it impossible to start the JVM.
>
> This problem appeared with the upgrade to glibc 2.26 already, and ever
> since I’ve been trying to minimize the damage for RHEL 6 systems where
> Guix is used as a package manager (such as the MDC).
>
> A work-around for glibc 2.27 that makes things work fine with the RHEL 6
> kernel would be very welcome!  I’ve started a branch “rhel6” where the
> default glibc has been bumped back to version 2.25 but I really don’t
> want it to be a long-lived branch; one of the reasons is that building
> all packages for this old glibc version (even just on x86_64) puts our
> build farms under extra stress that I would like to avoid.

Are you sure prlimit64 is the only syscall that needs to be adjusted?  A
quick grep through through the commit log reveals some other interfaces
that may need to be restored.

Here is the commit that removes the fallback code for missing prlimit64:
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=695d7d138eda449678a1650a8b8b58181033353f

And here are similar commits I found by grepping the log for '3.2':
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=e92030239abb4038d4f915d47021d6c037239309
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=1721145f0341d70a6d7807b172c5eb400b508fc0
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=9a45f54310573c190fa270e1f80d8307750305e9
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=e8f1225ca4d4afa4043c5267ae6dbe12268e2637

Since it's fairly late in the core-updates cycle, I think it would be
better to try restoring the prlimit64 fallback on the "rhel6" branch and
then revisit this during the next core-updates.

This got me thinking, perhaps it's possible to run Guix through a thin
hypervisor layer that uses the host virtualization facilities, or a Qemu
built against glibc 2.25.  This is similar to how "Docker" runs on
macOS, maybe it could be used for Guix in "hostile environments" too?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: 'core-updates' spring 2018

[Mark H Weaver]

Marius Bakke <mbakke@fastmail.com> writes:

> 'core-updates' has seen a lot of changes recently.  Some of the goodies
> include HTTP/2 support in curl, binutils 2.30, glibc 2.27, reproducible
> Python bytecode, gnu-build-system will automatically run 'autoreconf' if
> necessary, warnings will be issued about snippets and phases not
> returning #t, and much more.
>
> Are there other things that should go in?
>
> Let's set a date for "freezing" it, after which it will be started on
> Hydra and only bugfixes are allowed.  How about Monday, April 2nd?

Sounds good to me!

      Mark

Re: 'core-updates' spring 2018

[Ricardo Wurmus]

Hi Marius,

> Are you sure prlimit64 is the only syscall that needs to be adjusted?  A
> quick grep through through the commit log reveals some other interfaces
> that may need to be restored.

I’m not sure, but I looked at the RHEL6 kernel’s syscall table:

    curl http://vault.centos.org/6.9/os/Source/SPackages/kernel-2.6.32-696.el6.src.rpm | rpm2cpio - | cpio -idmv
    tar xf linux-*bz2
    less linux-*?/arch/x86/kernel/syscall_table_32.S

Looking at the table I see that the following syscalls are not
implemented and not marked as old or reserved:

        .long sys_ni_syscall    /* sys_vserver */
        .long sys_ni_syscall            /* 285 */ /* available */
        .long sys_ni_syscall    /* sys_fanotify_init */
        .long sys_ni_syscall    /* sys_fanotify_mark */
        .long sys_ni_syscall    /* sys_prlimit64  340 */
        .long sys_ni_syscall    /* sys_name_to_handle_at */
        .long sys_ni_syscall    /* sys_open_by_handle_at */

I don’t know what “available” is supposed to mean for number 285
(fallocate?) when the syscall is not implemented.  “vserver” is
not implemented on purpose according to man 2 unimplemented.

Also, I’ve been using a bunch of applications with the grafted glibc,
and the only thing I had problems with was Java’s use of prlimit64.  If
the other missing syscalls are problematic they don’t seem to affect the
software that is in use on the cluster.

Below I’ll refer to Linux 2.6.32-696.3.2.el6.x86_64, i.e. the current
RHEL6 kernel for x86_64.

> Here is the commit that removes the fallback code for missing prlimit64:
> https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=695d7d138eda449678a1650a8b8b58181033353f
>
> And here are similar commits I found by grepping the log for '3.2':
> https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=e92030239abb4038d4f915d47021d6c037239309

This is about accept4, which I cannot find in the syscall table, but
which is defined in include/linux/net.h and include/linux/syscalls.h.

> https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=1721145f0341d70a6d7807b172c5eb400b508fc0

This is about /proc/self/task/$PID/comm, which exists.

> https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=9a45f54310573c190fa270e1f80d8307750305e9

This is about recvmmsg, which is implemented.

> https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=e8f1225ca4d4afa4043c5267ae6dbe12268e2637

This is about the flags from statvfs.  I don’t know how to ascertain
whether this is implemented or not.

> Since it's fairly late in the core-updates cycle, I think it would be
> better to try restoring the prlimit64 fallback on the "rhel6" branch and
> then revisit this during the next core-updates.

I’m not too happy with this, because the longer we delay this the more
packages we have to build.  The longer the rhel6 branch has to be kept
alive the more urgent it becomes to build rhel6-core-updates and
duplicate efforts for the variant where glibc 2.25 is used.  I want to
keep this situation as short-lived as possible.

The change would only affect the 2.6.32 kernel (we shouldn’t just revert
that prlimit64 commit, but pick only the relevant parts).

> This got me thinking, perhaps it's possible to run Guix through a thin
> hypervisor layer that uses the host virtualization facilities, or a Qemu
> built against glibc 2.25.  This is similar to how "Docker" runs on
> macOS, maybe it could be used for Guix in "hostile environments" too?

I don’t want to spend time investigating this, but if someone wants to
work on this I’d be happy to test the results on my RHEL6 systems.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: 'core-updates' spring 2018

[Ricardo Wurmus]

Hi Gábor,

> We already talked about this on #guix, and the patch removing code to
> check for prlimit64 not implemented is a quite self contained commit on
> the glibc tree. It is not a trivial one, and I did not try to revert that
> yet, but
> it might worth looking at.

I would appreciate this a lot.  I don’t think we should just revert it.
It may be better to recover the alternative code for the case where
prlimit64 is not available and only use it when the kernel version is
reported as 2.6.32.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: 'core-updates' spring 2018

[Ludovic Courtès]

Hello!

Ricardo Wurmus <rekado@elephly.net> skribis:

>> Since it's fairly late in the core-updates cycle, I think it would be
>> better to try restoring the prlimit64 fallback on the "rhel6" branch and
>> then revisit this during the next core-updates.
>
> I’m not too happy with this, because the longer we delay this the more
> packages we have to build.  The longer the rhel6 branch has to be kept
> alive the more urgent it becomes to build rhel6-core-updates and
> duplicate efforts for the variant where glibc 2.25 is used.  I want to
> keep this situation as short-lived as possible.

I agree.  We said we’d freeze on April 2nd, so it’s still time to do it.

Ludo’.

glibc fallback code when ‘prlimit64’ is missing

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1737 bytes --]

Hello,

Ricardo Wurmus <rekado@elephly.net> skribis:

>> Here is the commit that removes the fallback code for missing prlimit64:
>> https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=695d7d138eda449678a1650a8b8b58181033353f

The attached patch appears to do the trick.

I checked the assembly with “objdump -S libc-2.27.so” and found that
getrlimit and setrlimit appear to include the fallback code (using
“printf '#include <sys/syscall.h>\n__NR_getrlimit' |gcc -E -” to check
the syscall numbers.)

That said it’d be even better if you could check on a system running
that kernel.  To do that, I think it’s enough to run:

  ./pre-inst-env guix build -e '(@@ (gnu packages commencement) static-bash-for-glibc)'

and then run the resulting bash with:

  …/bin/sh -c ulimit
  …/bin/sh -c 'ulimit -c 1000'

On my system I see:

--8<---------------cut here---------------start------------->8---
$ strace -e prlimit64,getrlimit,setrlimit sh -c ulimit
prlimit64(0, RLIMIT_NPROC, NULL, {rlim_cur=62722, rlim_max=62722}) = 0
prlimit64(0, RLIMIT_FSIZE, NULL, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
unlimited
+++ exited with 0 +++
$ strace -e prlimit64,getrlimit,setrlimit sh -c 'ulimit -c 1000'
prlimit64(0, RLIMIT_NPROC, NULL, {rlim_cur=62722, rlim_max=62722}) = 0
prlimit64(0, RLIMIT_CORE, NULL, {rlim_cur=0, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_CORE, NULL, {rlim_cur=0, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_CORE, {rlim_cur=500*1024, rlim_max=500*1024}, NULL) = 0
--8<---------------cut here---------------end--------------->8---

On 2.6.32, you should see ENOSYS followed by ‘setrlimit’ or ‘getrlimit’.

Thanks,
Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: the patch --]
[-- Type: text/x-patch, Size: 6466 bytes --]

From d33a6558a095debd806cf48e56ee41e5dae1c12c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludovic.courtes@inria.fr>
Date: Fri, 30 Mar 2018 11:18:47 +0200
Subject: [PATCH] gnu: glibc: Reinstate fallback code for systems lacking
 'prlimit64'.

* gnu/packages/patches/glibc-reinstate-prlimit64-fallback.patch: New file.
* gnu/packages/base.scm (glibc/linux)[source]: Use it.
* gnu/local.mk (dist_patch_DATA): Add it.
---
 gnu/local.mk                                       |   1 +
 gnu/packages/base.scm                              |   3 +-
 .../glibc-reinstate-prlimit64-fallback.patch       | 127 +++++++++++++++++++++
 3 files changed, 130 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/glibc-reinstate-prlimit64-fallback.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 7bb09fc7a..b99b40353 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -723,6 +723,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/glibc-locales.patch			\
   %D%/packages/patches/glibc-memchr-overflow-i686.patch		\
   %D%/packages/patches/glibc-o-largefile.patch			\
+  %D%/packages/patches/glibc-reinstate-prlimit64-fallback.patch	\
   %D%/packages/patches/glibc-vectorized-strcspn-guards.patch	\
   %D%/packages/patches/glibc-versioned-locpath.patch		\
   %D%/packages/patches/glibc-2.27-git-fixes.patch		\
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index e09af515d..1cbb7ae0e 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -568,7 +568,8 @@ store.")
                                      "glibc-2.27-git-fixes.patch"
                                      "glibc-hidden-visibility-ldconfig.patch"
                                      "glibc-versioned-locpath.patch"
-                                     "glibc-allow-kernel-2.6.32.patch"))))
+                                     "glibc-allow-kernel-2.6.32.patch"
+                                     "glibc-reinstate-prlimit64-fallback.patch"))))
    (build-system gnu-build-system)
 
    ;; Glibc's <limits.h> refers to <linux/limit.h>, for instance, so glibc
diff --git a/gnu/packages/patches/glibc-reinstate-prlimit64-fallback.patch b/gnu/packages/patches/glibc-reinstate-prlimit64-fallback.patch
new file mode 100644
index 000000000..ccc153c12
--- /dev/null
+++ b/gnu/packages/patches/glibc-reinstate-prlimit64-fallback.patch
@@ -0,0 +1,127 @@
+This patch reinstates fallback code when the 'prlimit64' system call is
+missing by reverting the relevant part of this upstream commit:
+
+  commit 695d7d138eda449678a1650a8b8b58181033353f
+  Author: Joseph Myers <joseph@codesourcery.com>
+  Date:   Tue May 9 14:05:09 2017 +0000
+
+      Assume prlimit64 is available.
+
+The fallback code is useful on systems that lack 'prlimit64', such as the
+2.6.32-on-steroid kernel found on RHEL 6:
+
+  <https://lists.gnu.org/archive/html/guix-devel/2018-03/msg00349.html>
+
+diff --git a/sysdeps/unix/sysv/linux/getrlimit64.c b/sysdeps/unix/sysv/linux/getrlimit64.c
+index 37c173286f..56af3c0646 100644
+--- b/sysdeps/unix/sysv/linux/getrlimit64.c
++++ a/sysdeps/unix/sysv/linux/getrlimit64.c
+@@ -35,7 +35,40 @@
+ int
+ __getrlimit64 (enum __rlimit_resource resource, struct rlimit64 *rlimits)
+ {
+-  return INLINE_SYSCALL_CALL (prlimit64, 0, resource, NULL, rlimits);
++#ifdef __NR_prlimit64
++  int res = INLINE_SYSCALL_CALL (prlimit64, 0, resource, NULL, rlimits);
++  if (res == 0 || errno != ENOSYS)
++    return res;
++#endif
++
++/* The fallback code only makes sense if the platform supports either
++   __NR_ugetrlimit and/or __NR_getrlimit.  */
++#if defined (__NR_ugetrlimit) || defined (__NR_getrlimit)
++# ifndef __NR_ugetrlimit
++#  define __NR_ugetrlimit __NR_getrlimit
++# endif
++# if __RLIM_T_MATCHES_RLIM64_T
++#  define rlimits32 (*rlimits)
++# else
++  struct rlimit rlimits32;
++# endif
++
++  if (INLINE_SYSCALL_CALL (ugetrlimit, resource, &rlimits32) < 0)
++    return -1;
++
++# if !__RLIM_T_MATCHES_RLIM64_T
++  if (rlimits32.rlim_cur == RLIM_INFINITY)
++    rlimits->rlim_cur = RLIM64_INFINITY;
++  else
++    rlimits->rlim_cur = rlimits32.rlim_cur;
++  if (rlimits32.rlim_max == RLIM_INFINITY)
++    rlimits->rlim_max = RLIM64_INFINITY;
++  else
++    rlimits->rlim_max = rlimits32.rlim_max;
++# endif /* !__RLIM_T_MATCHES_RLIM64_T */
++#endif /* defined (__NR_ugetrlimit) || defined (__NR_getrlimit)  */
++
++  return 0;
+ }
+ libc_hidden_def (__getrlimit64)
+ 
+diff --git a/sysdeps/unix/sysv/linux/setrlimit.c b/sysdeps/unix/sysv/linux/setrlimit.c
+index 01812ac355..8773c78236 100644
+--- b/sysdeps/unix/sysv/linux/setrlimit.c
++++ a/sysdeps/unix/sysv/linux/setrlimit.c
+@@ -34,6 +34,7 @@
+ int
+ __setrlimit (enum __rlimit_resource resource, const struct rlimit *rlim)
+ {
++# ifdef __NR_prlimit64
+   struct rlimit64 rlim64;
+ 
+   if (rlim->rlim_cur == RLIM_INFINITY)
+@@ -45,7 +46,11 @@
+   else
+     rlim64.rlim_max = rlim->rlim_max;
+ 
+-  return INLINE_SYSCALL_CALL (prlimit64, 0, resource, &rlim64, NULL);
++  int res = INLINE_SYSCALL_CALL (prlimit64, 0, resource, &rlim64, NULL);
++  if (res == 0 || errno != ENOSYS)
++    return res;
++# endif
++  return INLINE_SYSCALL_CALL (setrlimit, resource, rlim);
+ }
+ 
+ # if SHLIB_COMPAT (libc, GLIBC_2_0, GLIBC_2_2)
+diff --git a/sysdeps/unix/sysv/linux/setrlimit64.c b/sysdeps/unix/sysv/linux/setrlimit64.c
+index 2dd129d99e..db1960fc18 100644
+--- b/sysdeps/unix/sysv/linux/setrlimit64.c
++++ a/sysdeps/unix/sysv/linux/setrlimit64.c
+@@ -36,7 +36,36 @@
+ int
+ __setrlimit64 (enum __rlimit_resource resource, const struct rlimit64 *rlimits)
+ {
+-  return INLINE_SYSCALL_CALL (prlimit64, 0, resource, rlimits, NULL);
++  int res;
++
++#ifdef __NR_prlimit64
++  res = INLINE_SYSCALL_CALL (prlimit64, 0, resource, rlimits, NULL);
++  if (res == 0 || errno != ENOSYS)
++    return res;
++#endif
++
++/* The fallback code only makes sense if the platform supports
++   __NR_setrlimit.  */
++#ifdef __NR_setrlimit
++# if !__RLIM_T_MATCHES_RLIM64_T
++  struct rlimit rlimits32;
++
++  if (rlimits->rlim_cur >= RLIM_INFINITY)
++    rlimits32.rlim_cur = RLIM_INFINITY;
++  else
++    rlimits32.rlim_cur = rlimits->rlim_cur;
++  if (rlimits->rlim_max >= RLIM_INFINITY)
++    rlimits32.rlim_max = RLIM_INFINITY;
++  else
++    rlimits32.rlim_max = rlimits->rlim_max;
++# else
++#  define rlimits32 (*rlimits)
++# endif
++
++  res = INLINE_SYSCALL_CALL (setrlimit, resource, &rlimits32);
++#endif
++
++  return res;
+ }
+ weak_alias (__setrlimit64, setrlimit64)
+ 
-- 
2.16.2

Re: glibc fallback code when ‘prlimit64’ is missing

[Ludovic Courtès]

Hello,

ludovic.courtes@inria.fr (Ludovic Courtès) skribis:

> From d33a6558a095debd806cf48e56ee41e5dae1c12c Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludovic.courtes@inria.fr>
> Date: Fri, 30 Mar 2018 11:18:47 +0200
> Subject: [PATCH] gnu: glibc: Reinstate fallback code for systems lacking
>  'prlimit64'.
>
> * gnu/packages/patches/glibc-reinstate-prlimit64-fallback.patch: New file.
> * gnu/packages/base.scm (glibc/linux)[source]: Use it.
> * gnu/local.mk (dist_patch_DATA): Add it.

I went ahead and pushed it.

Ludo’.