Guix 1.4.0+i686: getting ghc substitutes?

Guix 1.4.0+i686: getting ghc substitutes?

[Denis 'GNUtoo' Carikli]

[-- Attachment #1: Type: text/plain, Size: 1432 bytes --]

Hi,

In GNU Boot we use Guix 1.4.0 instead of the latest revision because
we can easily point to the 1.4.0 manual and we don't need to update our
code following Guix changes.

We also use i686 (--system=i686-linux) to enable to build GNU Boot on
all the computers we support.

Until now that worked fine but we then hit an issue that we cannot
solve on our own.

We also want to use Guix 1.4.0 + --system=i686-linux for building our
website as well and this requires pandoc, which in turn requires ghc.

However even if ghc builds fine for i686, it takes a very long time to
build (more than 1 night of compilation on a ThinkPad X200). Building
on a way faster computer (a KGPE-D16) doesn't make things much faster.

Part of the issue is because tests are very long to run, and another
part is that we have ghc -> ghc@8.6 -> ghc@8.4 -> ghc@8.0 -> ghc@7.

Disabling tests makes the build an order of magnitude faster, but we
still have multiple hours just for building ghc, so it's not good
enough just to get pandoc to build a website.

Would it be possible to somehow trigger a build on one of the default
substitute server to fix this issue?

If not we could simply change the Guix revision and somehow publish the
corresponding manual as well, but using a release probably makes
things much easier for contributors. Not using Guix by default for
building the website is also an option.

Denis.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Guix 1.4.0+i686: getting ghc substitutes?

[Denis 'GNUtoo' Carikli]

[-- Attachment #1: Type: text/plain, Size: 2064 bytes --]

Hi,


On Fri, 18 Oct 2024 22:26:05 +0200
Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org> wrote:
[Missing pandoc / ghc substitutes for Guix 1.4.0 for i686]
[...]
> Would it be possible to somehow trigger a build on one of the default
> substitute server to fix this issue?
[...]
> If not we could simply change the Guix revision and somehow publish
> the corresponding manual as well, but using a release probably makes
> things much easier for contributors. Not using Guix by default for
> building the website is also an option.

I've found a better workaround but I'm unsure what is going on under
the hood:

Here the first substitute server has 0 substitutes:
> $ guix time-machine --commit=v1.4.0 -- weather -s i686-linux pandoc
> [...]
> computing 1 package derivations for i686-linux...
> looking for 3 store items on https://ci.guix.gnu.org...
> https://ci.guix.gnu.org ⛈
> 0.0% substitutes available (0 out of 3)

But then the same command also prints that:
> looking for 3 store items on https://bordeaux.guix.gnu.org...
> https://bordeaux.guix.gnu.org ☀
>  100.0% substitutes available (3 out of 3)

And so I end up being able to download these:
> $ guix build \
>   --substitute-urls=https://bordeaux.guix.gnu.org \
>   --system=i686-linux \
>   pandoc
> [...]
> substituting
> /gnu/store/aqxhlbiy3zfcv81y4f439b6x1pcmvynn-pandoc-2.19.2...
> downloading
> from https://bordeaux.guix.gnu.org/nar/lzip/aqxhlbiy3zfcv81y4f439b6x1pcmvynn-pandoc-2.19.2 ... pandoc-2.19.2  10.6MiB                                                                          3.2MiB/s 00:03 ▕██████████████████▏ 100.0%
> 
> /gnu/store/aqxhlbiy3zfcv81y4f439b6x1pcmvynn-pandoc-2.19.2

So this at least works.

But I've no idea why it doesn't fallback on bordeaux when substitutes
are unavailable on ci.guix.gnu.org (maybe there are good reasons for
that).

So at the end a workaround I could do here is to detect if the Bordeaux
substitute is enabled and if so force its use.

Denis.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Guix 1.4.0+i686: getting ghc substitutes?

[Andreas Enge]

Hello Denis,

Am Thu, Oct 24, 2024 at 04:32:33PM +0200 schrieb Denis 'GNUtoo' Carikli:
> And so I end up being able to download these:
> > $ guix build \
> >   --substitute-urls=https://bordeaux.guix.gnu.org \
> >   --system=i686-linux \
> >   pandoc
> > [...]
> > substituting

this looks as if you are using too old a Guix daemon; more recent versions
enable the Bordeaux build farm by default.

We should do a release :)

Andreas

Re: Guix 1.4.0+i686: getting ghc substitutes?

[Denis 'GNUtoo' Carikli]

[-- Attachment #1: Type: text/plain, Size: 3909 bytes --]

On Fri, 25 Oct 2024 11:17:12 +0200
Andreas Enge <andreas@enge.fr> wrote:

> Hello Denis,
Hi,

> Am Thu, Oct 24, 2024 at 04:32:33PM +0200 schrieb Denis 'GNUtoo'
> Carikli:
> > And so I end up being able to download these:
> > > $ guix build \
> > >   --substitute-urls=https://bordeaux.guix.gnu.org \
> > >   --system=i686-linux \
> > >   pandoc
> > > [...]
> > > substituting
> 
> this looks as if you are using too old a Guix daemon; more recent
> versions enable the Bordeaux build farm by default.
Before I did most of the tests on Guix system, and I only verified that
my script that detects the use of Bordeaux worked on both Guix system
and a fresh Trisquel 11 VM, with Guix installed through
guix-install.sh, without substitutes enabled.

But when doing more tests before finishing the patches for GNU Boot[1] I
found more issues.

Missing Bordeaux in older Debian packages:
------------------------------------------
In debian/rules of the Trisquel package, we have that (modified to fit
the ~70 lines limits of mails):
> override_dh_install:
>         dh_install
>         [...]
>         # Add /etc/default/acl with the default substitute server,
>         # with identical output as "guix archive --authorize"
>         mkdir -p debian/guix/etc/guix/
>         printf '(acl\n (entry\n' > \
>             debian/guix/etc/guix/acl
>         sed -e 's,^, ,g' -e 's, $$,,g' \ 
>             etc/substitutes/ci.guix.gnu.org.pub >> \
>             debian/guix/etc/guix/acl 
>         printf '  (tag\n   (guix import)\n )\n )\n )\n' >> \
>             debian/guix/etc/guix/acl

Bordeaux is added later on in the Debian package[2].

After testing on Trisquel 11 with the Guix package, as expected, 
'guix build --substitute-urls=https://bordeaux.guix.gnu.org' results in
ghc being built instead of downloaded.

So I'm unsure what to do here. I could ask to add Bordeaux in the
Trisquel package but that's probably not the best way to deal with that.

Potential issue with /etc/guix/acl
----------------------------------
My previous attempt to workaround the lack of substitutes was to detect
bordeaux and force its use if it's authorized. Here's my code (GPLv3+):
> (define bordeaux.guix.gnu.org
>   "(public-key
>      (ecc
>        (curve Ed25519)
>        (q
> #7D602902D3A2DBB83F8A0FB98602A754C5493B0B778C8D1DD4E0F41DE14DE34F#)))")
> 
> (if (authorized-key? (string->canonical-sexp bordeaux.guix.gnu.org))
>     (display "--substitute-urls=https://bordeaux.guix.gnu.org"))

But in some situations we have:
> $ guix repl force-bordeaux-substitute.scm
> guix repl: error: open-file: Permission denied: "/etc/guix/acl"

So under Trisquel 11 with the guix package we have:
> $ ls -la /etc/guix/acl 
> -rw------- 1 root root 355 Oct 26 18:06 /etc/guix/acl

With Guix system we have:
> $ ls -la /etc/guix/acl
> -r--r--r-- 1 root root 528 Oct 26 13:53 /etc/guix/acl

With 'sudo ./guix-install.sh' with substitutes enabled we have:
> $ ls -la /etc/guix/acl 
> -rw------- 1 root root 355 Oct 26 18:06 /etc/guix/acl

And with 'sudo ./guix-install.sh' without substitutes enabled there is
no issue since /etc/guix/acl doesn't exist so my detection of Bordeaux
works fine.

Is this a bug? Should the permissions be the same in all the
situations? Beside bugreporting / fixing it in the Debian package and
in guix-install.sh, it also brings the question of what to do for
previous installations.

References:
-----------
[1]Since GNU Boot wants to make it as easy as possible to contribute I
   test builds and changes in various environments (Trisquel 11 +
   guix-install.sh, guix system, Trisquel 11 + guix package, etc).

[2]It's added in the commit 2700105e8f ("debian/rules: Add "bordeaux"
   substitute server to /etc/guix/acl.") from the
   https://salsa.debian.org/debian/guix.git/ repository.

Denis.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]