Hurd bootstrap breaks guix building binutils-boot0@2.34

Hurd bootstrap breaks guix building binutils-boot0@2.34

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 1107 bytes --]

Hi!

I have been looking at the recent work of Efraim and others wrt the Hurd
bootstrap.  On the Hurd, make-boot0 v4.3 did not work for me so I
reverted to make-boot0 v4.1 for now.  Now I'm stuck on building
binutils-boot0

    ./pre-inst-env guix build -e '(@@ (gnu packages commencement) binutils-boot0)'

It almost fully builds but fails with this error

    phase `compress-documentation' succeeded after 0.4 seconds
    error: cannot kill processes for uid `999': Operation not permitted
    guix build: error: cannot kill processes for uid `999': failed with exit code 1

after which my Hurd setup is broken; I haven't been able to build a new
package and had to reinstall my Hurd VM.  This error is reproducible for
me however: I tried again and broke my Hurd setup in exact the same way.
Any ideas what may be going wrong and how to fix this?

I am now looking to revert bootstrap-guile to v2.0
(https://lists.gnu.org/archive/html/guix-devel/2019-09/msg00106.html)
and possibly binutils-boot0 to v2.32.

Find my notes attached for how I am setting up Hurd development for Guix.

Greetings,
janneke


[-- Attachment #2: INSTALL --]
[-- Type: application/octet-stream, Size: 5801 bytes --]

* Help with Hurd                                                  -*- mode: org -*-

In his talk @FOSDEM17

    https://fosdem.org/2017/schedule/event/guixhurd/
    https://fosdem.org/2017/schedule/event/guixhurd/attachments/slides/1850/export/events/attachments/guixhurd/slides/1850/guix_to_hurd_fosdem_2017.pdf

Manolis calls for help to get Hurd supported on Guix.  A lot of work
has been done.  Some bigger things remains, but there is also a lot of
small, simple things to be fixed.

On Guix Days 2020, Efraim and others resurrect building
bootstrap-binaries for the Hurd

    https://gitlab.com/Efraim/guix  @wip-hurd-bootstrap

* Setup Guix build environment on Debian/Hurd

** Get installed IMAGE!

wget http://cdimage.debian.org/cdimage/ports/latest/hurd-i386/debian-hurd.img.tar.gz
tar xf debian-hurd.img.tar debian-hurd.img.tar.gz

guix environment --ad-hoc qemu parted

parted debian-hurd-20200101.img -- print
qemu-img resize debian-hurd-20200101.img 32G
parted debian-hurd-20200101.img -- resizepart 2 100%
sudo losetup -o $((512*1953792)) /dev/loop0 debian-hurd-20200101.img
sudo resize2fs /dev/loop0
sudo losetup -d /dev/loop0

** Run the HURD
qemu-system-i386 -enable-kvm -drive file=debian-hurd-20200101.img,cache=writeback -m 4G -net user,hostfwd=tcp:127.0.0.1:2223-:22 -net nic

*** [@hurd] Create user

adduser janneke
adduser janneke sudo

*** [@Guix System]: Add to .ssh/config
Host hurd
   HostName localhost
   Port 2223
   ForwardAgent yes

ssh-keygen -R [localhost]:2223
ssh-copy-id hurd

*** [@Guix System] Build bootstrap binaries

**** Clone Guix git
    guix environment guix
    mkdir src
    cd src
    git config --global url."git+ssh://git.sv.gnu.org/srv/git/".insteadOf gnu:
    git clone gnu:guix
    cd guix

**** Checkout Hurd development (core-updates)
    git checkout core-updates
    git show HEAD | head -1
    commit 5afcb5caa53615c0a432e0c1781155398d747218
    ./bootstrap
    ./configure --localstatedir=/var --syscconfdir=/etc

    ./pre-inst-env guix build --target=i586-pc-gnu bootstrap-tarballs
    /gnu/store/yb6hbpsicy0akqs42q8bijjs5xz73i5z-bootstrap-tarballs-0

    ssh hurd mkdir -p var/hurd/gnu
    for i in /gnu/store/yb6hbpsicy0akqs42q8bijjs5xz73i5z-bootstrap-tarballs-0/*; do \
        echo $i; guix hash $i; done
    mkdir -p gnu/packages/bootstrap/i586-gnu
    tar xf /gnu/store/yb6hbpsicy0akqs42q8bijjs5xz73i5z-bootstrap-tarballs-0/static-binaries-0-i586-pc-gnu.tar.xz --strip=2 -C gnu/packages/bootstrap/i586-gnu ./bin/{xz,tar,mkdir,bash}
    for i in gnu/packages/bootstrap/i586-gnu/*; do \
        echo $i; guix hash -r $i; done
    scp -r /gnu/store/yb6hbpsicy0akqs42q8bijjs5xz73i5z-bootstrap-tarballs-0/  hurd:var/hurd/gnu/

*** [@Hurd] Prepare to build Guix

**** Build Guix from git
    sudo apt-get update
    sudo apt-get upgrade

**** Add Debian build dependencies
    sudo apt-get install build-essential dh-autoreconf dh-systemd autotools-dev git guile-2.2-dev guile-gcrypt guile-git guile-gnutls guile-json guile-sqlite3  graphviz help2man libbz2-dev libgcrypt20-dev libsqlite3-dev libssh-dev texinfo

**** Build missing dependencies
    mkdir src
    cd src
    apt-get source --download guile-ssh
    cd guile-ssh-0.11.3
    sed -i 's/dh_auto_test /true # dh_auto_test /' debian/rules
    dpkg-buildpackage -uc
    cd ..
    sudo dpkg -i *.deb

**** Checkout Hurd development (core-updates)
    git config --global url."git+ssh://git.sv.gnu.org/srv/git/".insteadOf gnu:
    git clone gnu:guix
    cd guix
    git remote add janneke git@gitlab.com:janneke/guix.git
    git fetch janneke
    git checkout -b wip-hurd-bootstrap janneke/wip-hurd-bootstrap
    git show HEAD | head -1
    commit 17d4ab57693bfcb777120215882ac8d64c9c6005

    ./bootstrap
    ./configure --with-courage --localstatedir=/var --sysconfdir=/etc
    make

#**** Add Hurd bootstrap binaries
     XXX this does not work anymore?  copy to web server (lilypond.org)
#    mkdir -p gnu/packages/bootstrap/i586-gnu
#    cp ~/var/hurd/gnu/yb6hbpsicy0akqs42q8bijjs5xz73i5z-bootstrap-tarballs-0/guile-static-stripped-2.2.6-i586-pc-gnu.tar.xz gnu/packages/bootstrap/i586-gnu/
#    tar xf ~/var/hurd/gnu/yb6hbpsicy0akqs42q8bijjs5xz73i5z-bootstrap-tarballs-0/static-binaries-0-i586-pc-gnu.tar.xz --strip=2 -C gnu/packages/bootstrap/i586-gnu ./bin/{xz,tar,mkdir,bash}

*** Prepare to start guix-daemon

     sudo groupadd --system guixbuild
     for i in `seq -w 1 10`;
       do
         sudo useradd -g guixbuild -G guixbuild      \
                 -d /var/empty -s `which nologin`    \
                 -c "Guix build user $i" --system    \
                 guixbuilder$i;
       done

     sudo ./pre-inst-env guix archive --authorize < etc/substitutes/ci.guix.gnu.org.pub
     sudo mkdir /var/guix

* Do it
** [@Guix System] Start qemu
    qemu-system-i386 -enable-kvm -drive file=debian-hurd-20170613.img,cache=writeback -m 4G -net user,hostfwd=tcp:127.0.0.1:2223-:22 -net nic

** [@Hurd] Start daemon
    sudo ./pre-inst-env guix-daemon --disable-chroot --build-users-group=guixbuild &

** [@Hurd] build package
    ./pre-inst-env guix build -e '(@@ (gnu packages commencement) gnu-make-boot0)'
    XXX fails: revert to make-4.1

    ./pre-inst-env guix build -e '(@@ (gnu packages commencement) diffutils-boot0)'
    ./pre-inst-env guix build -e '(@@ (gnu packages commencement) file-boot0)'
    ./pre-inst-env guix build -e '(@@ (gnu packages commencement) patch-boot0)'

    ./pre-inst-env guix build -e '(@@ (gnu packages commencement) binutils-boot0)'
    XXX fails
phase `compress-documentation' succeeded after 0.4 seconds
error: cannot kill processes for uid `999': Operation not permitted
guix build: error: cannot kill processes for uid `999': failed with exit code 1

    ./pre-inst-env guix build -e '(@@ (gnu packages commencement) gcc-boot0)'

[-- Attachment #3: Type: text/plain, Size: 152 bytes --]


-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: Hurd bootstrap breaks guix building binutils-boot0@2.34

[Svante Signell]

Hello,

I did build the bootstrap binaries some time ago. They were corrupt. As well as
the pre-built ones that were available at 
http://berlin.guix.gnu.org/guix/bootstrap/i586-gnu/20190508

Down-grading from guile-2.2 to guile-2.0 did not improve the situation.

Now I have a project to cross-build Hurd from a GNU/Linux box: hurd-cross, using
the latest versions of the gnumach, hurd, glibc, gcc, etc. The built Hurd boots
and runs fine in a VM :) The project will soon be available on Savannah. From
there I will again cross-build the bootstrap binaries for Hurd, in due time
though. (Other packages, like ones not yet available natively, can also be
cross-built. That's an interesting extension.)

See 
https://lists.gnu.org/archive/html/guix-devel/2019-08/msg00198.html
https://lists.gnu.org/archive/html/guix-devel/2019-09/msg00000.html
https://lists.gnu.org/archive/html/guix-devel/2019-09/msg00014.html
https://lists.gnu.org/archive/html/guix-devel/2019-09/msg00225.html
for some of the efforts to make Guix work on GNU/Hurd.

Thanks!
(and sorry for top-posting)

On Sat, 2020-02-29 at 10:36 +0100, Jan Nieuwenhuizen wrote:
> Hi!
> 
> I have been looking at the recent work of Efraim and others wrt the Hurd
> bootstrap.  On the Hurd, make-boot0 v4.3 did not work for me so I
> reverted to make-boot0 v4.1 for now.  Now I'm stuck on building
> binutils-boot0
> 
>     ./pre-inst-env guix build -e '(@@ (gnu packages commencement) binutils-
> boot0)'
> 
> It almost fully builds but fails with this error
> 
>     phase `compress-documentation' succeeded after 0.4 seconds
>     error: cannot kill processes for uid `999': Operation not permitted
>     guix build: error: cannot kill processes for uid `999': failed with exit
> code 1
> 
> after which my Hurd setup is broken; I haven't been able to build a new
> package and had to reinstall my Hurd VM.  This error is reproducible for
> me however: I tried again and broke my Hurd setup in exact the same way.
> Any ideas what may be going wrong and how to fix this?
> 
> I am now looking to revert bootstrap-guile to v2.0
> (https://lists.gnu.org/archive/html/guix-devel/2019-09/msg00106.html)
> and possibly binutils-boot0 to v2.32.
> 
> Find my notes attached for how I am setting up Hurd development for Guix.
> 
> Greetings,
> janneke
> 

Re: Hurd bootstrap breaks guix building binutils-boot0@2.34

[Jan Nieuwenhuizen]

Svante Signell writes:

Hi Svante,

> I did build the bootstrap binaries some time ago. They were corrupt.

Can you elaborate on that?  Do you know which packages/programs were
affected?

Ah, wait you mean:

> https://lists.gnu.org/archive/html/guix-devel/2019-09/msg00225.html

    I had to replace the following binaries in order to make
    ./pre-inst-env  guix build -K -e '(@@ (gnu packages commencement) gnu-make-
    boot0))' work:
    bash, cat, cp, gawk, grep, install, mv, rm, rmdir, tar, xz, cpp, guile, mkdir,

It seems these work for me.

For now it seems that the binaries I'm building from core-updates are
fine.

> Down-grading from guile-2.2 to guile-2.0 did not improve the situation.

Ah; yes I just found out as well :-)

> Now I have a project to cross-build Hurd from a GNU/Linux box: hurd-cross, using
> the latest versions of the gnumach, hurd, glibc, gcc, etc. The built Hurd boots
> and runs fine in a VM :) The project will soon be available on Savannah. From
> there I will again cross-build the bootstrap binaries for Hurd, in due time
> though. (Other packages, like ones not yet available natively, can also be
> cross-built. That's an interesting extension.)

Okay, that's a whole different route but starting from a working
situation can certainly help.  I'm going to try the Guix route some bit
more for now.

> See 
> https://lists.gnu.org/archive/html/guix-devel/2019-08/msg00198.html
> https://lists.gnu.org/archive/html/guix-devel/2019-09/msg00000.html
> https://lists.gnu.org/archive/html/guix-devel/2019-09/msg00014.html
> https://lists.gnu.org/archive/html/guix-devel/2019-09/msg00225.html
> for some of the efforts to make Guix work on GNU/Hurd.

Thanks for the pointers
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Hurd bootstrap breaks guix building binutils-boot0@2.34

[Rene]

[-- Attachment #1: Type: text/plain, Size: 147 bytes --]

Hello Jan,

Maybe it's related to this workaround:

https://github.com/Phant0mas/Guix-on-Hurd/commit/ba34da4c7fc811f486b429ff852dc6481fae337d

Rene

[-- Attachment #2: Type: text/html, Size: 549 bytes --]

Re: Hurd bootstrap breaks guix building binutils-boot0@2.34

[Jan Nieuwenhuizen]

Rene writes:

Hello Rene,

> Maybe it's related to this workaround:
>
> https://github.com/Phant0mas/Guix-on-Hurd/commit/ba34da4c7fc811f486b429ff852dc6481fae337d

Ah, I totally missed Phant0mas' wip-hurd-native branch...Yes, that's it

    phase `compress-documentation' succeeded after 0.5 seconds
    killing process 23791
    successfully built /gnu/store/za8qaq221bs231nlk12914334xmm1ilj-binutils-cross-boot0-2.34.drv
    /gnu/store/jk3kx5jwjs9m60svzk6sz79bf0w33l91-binutils-cross-boot0-2.34

Probably this is not a proper "fix" yet but at least it's identified and
has a workaround.  And there may be some other stuff there too; thanks!

Greetings,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com