bug#54292: Commit e8518c43 breaks guix pull on i686

bug#54292: Commit e8518c43 breaks guix pull on i686

[Diego Nicola Barbato]

Hi Guix,

Commit e8518c43 (gnu: Add stex) breaks guix pull, specifically the
package cache hook, on i686-linux.  I.e. the following command

--8<---------------cut here---------------start------------->8---
guix pull --commit=e8518c43 --system=i686-linux -p /tmp/guix
--8<---------------cut here---------------end--------------->8---

fails like this:

--8<---------------cut here---------------start------------->8---

[...]

building package cache...
/builder for `/gnu/store/f1j059k0x7hpqmhwgyar6761c6jrw4iw-guix-package-cache.drv' failed to produce output path `/gnu/store/0z272l1izcm45ynwfzz5vvnnhg4ik1ij-guix-package-cache'
build of /gnu/store/f1j059k0x7hpqmhwgyar6761c6jrw4iw-guix-package-cache.drv failed
View build log at '/var/log/guix/drvs/f1/j059k0x7hpqmhwgyar6761c6jrw4iw-guix-package-cache.drv.gz'.
cannot build derivation `/gnu/store/cm9wh5fv3a94cbpb839dhbf2wf31zg7r-profile.drv': 1 dependencies couldn't be built
guix pull: error: build of `/gnu/store/cm9wh5fv3a94cbpb839dhbf2wf31zg7r-profile.drv' failed
--8<---------------cut here---------------end--------------->8---

The build log looks like this:

--8<---------------cut here---------------start------------->8---
(repl-version 0 1 1)
Generating package cache for '/gnu/store/dzcm0fk5a41bwmhy74sv0v2bvmapx5qy-profile'...
(exception wrong-type-arg (value "string-ref") (value "Wrong type argument in position ~A (expecting ~A): ~S") (value (1 "string" #f)) (value (#f)))
--8<---------------cut here---------------end--------------->8---

This issue was introduced by commit e8518c43 and is still present on
current master (commit 6c3c4f70).

Regards,

Diego

bug#54292: Commit e8518c43 breaks guix pull on i686

[Liliana Marie Prikler]

Hi,

Am Montag, dem 07.03.2022 um 12:47 +0000 schrieb Diego Nicola Barbato:
> Hi Guix,
> 
> Commit e8518c43 (gnu: Add stex) breaks guix pull, specifically the
> package cache hook, on i686-linux.
This series also fails on CI in a rather peculiar manner [1,2,3]. 
However, it succeeded on bordeaux, even for i686 [4].  I don't think
this is an issue with the patch, we should start challenging berlin.

Cheers

[1] http://ci.guix.gnu.org/build/517560/log/raw
[2] http://ci.guix.gnu.org/build/517546/log/raw
[3] http://ci.guix.gnu.org/eval/130956/log/raw
[4]
https://bordeaux.guix.gnu.org/build/abde7656-0ced-4621-ad89-9a6bf9517f18

bug#54292: Commit e8518c43 breaks guix pull on i686

[Diego Nicola Barbato]

Hi Liliana,

Liliana Marie Prikler <liliana.prikler@ist.tugraz.at> writes:

> Hi,
>
> Am Montag, dem 07.03.2022 um 12:47 +0000 schrieb Diego Nicola Barbato:
>> Hi Guix,
>> 
>> Commit e8518c43 (gnu: Add stex) breaks guix pull, specifically the
>> package cache hook, on i686-linux.
> This series also fails on CI in a rather peculiar manner [1,2,3]. 
> However, it succeeded on bordeaux, even for i686 [4].

While these CI issues are certainly interesting and possibly deserving
of their own bug reports, they have little to do with this bug report: I
didn't try to build chez-nanopass [1],
chez-scheme-for-racket-bootstrap-bootfiles [2], or racket [4] itself and
I didn't experience any offloading errors [3].  This bug report is about
guix pull failing to build the package cache on i686-linux (the package
cache is one of the last things guix pull builds, it didn't have any
trouble building guix-packages-base, guix-packages, guix-system,
guix-home, guix-cli, etc.).

> I don't think this is an issue with the patch, we should start
> challenging berlin.

I do think this is an issue with commit e8518c43 because

--8<---------------cut here---------------start------------->8---
guix pull --commit=e8518c43 --system=i686-linux -p /tmp/guix
--8<---------------cut here---------------end--------------->8---

fails to build the package cache whereas 

--8<---------------cut here---------------start------------->8---
guix pull --commit=75f9f944 --system=i686-linux -p /tmp/guix
--8<---------------cut here---------------end--------------->8---

succeeds (75f9f944 being the parent commit of e8518c43).  I even ran
these with --substitute-urls=https://bordeaux.guix.gnu.org in a freshly
downloaded instance of the 1.3.0 QEMU image [5] to rule out corrupted
substitutes from berlin with the same result.

> Cheers

Regards,

Diego

> [1] http://ci.guix.gnu.org/build/517560/log/raw
> [2] http://ci.guix.gnu.org/build/517546/log/raw
> [3] http://ci.guix.gnu.org/eval/130956/log/raw
> [4]
> https://bordeaux.guix.gnu.org/build/abde7656-0ced-4621-ad89-9a6bf9517f18
[5] https://ftp.gnu.org/gnu/guix/guix-system-vm-image-1.3.0.x86_64-linux.qcow2

bug#54292: Commit e8518c43 breaks guix pull on i686

[Liliana Marie Prikler]

Hi Diego,

Am Montag, dem 07.03.2022 um 18:12 +0000 schrieb Diego Nicola Barbato:
> Hi Liliana,
> 
> Liliana Marie Prikler <liliana.prikler@ist.tugraz.at> writes:
> 
> > Hi,
> > 
> > Am Montag, dem 07.03.2022 um 12:47 +0000 schrieb Diego Nicola
> > Barbato:
> > > Hi Guix,
> > > 
> > > Commit e8518c43 (gnu: Add stex) breaks guix pull, specifically
> > > the package cache hook, on i686-linux.
> > This series also fails on CI in a rather peculiar manner [1,2,3]. 
> > However, it succeeded on bordeaux, even for i686 [4].
> 
> While these CI issues are certainly interesting and possibly
> deserving of their own bug reports, they have little to do with this
> bug report: I didn't try to build chez-nanopass [1],
> chez-scheme-for-racket-bootstrap-bootfiles [2], or racket [4] itself
> and I didn't experience any offloading errors [3].  This bug report
> is about guix pull failing to build the package cache on i686-linux
> (the package cache is one of the last things guix pull builds, it
> didn't have any trouble building guix-packages-base, guix-packages,
> guix-system, guix-home, guix-cli, etc.).
These are very much related in that they all belong to the same series.
We would not only have to revert that single commit, but all following
commits authored by Philip as well.

> > I don't think this is an issue with the patch, we should start
> > challenging berlin.
> 
> I do think this is an issue with commit e8518c43 because
> 
> --8<---------------cut here---------------start------------->8---
> guix pull --commit=e8518c43 --system=i686-linux -p /tmp/guix
> --8<---------------cut here---------------end--------------->8---
> 
> fails to build the package cache whereas 
> 
> --8<---------------cut here---------------start------------->8---
> guix pull --commit=75f9f944 --system=i686-linux -p /tmp/guix
> --8<---------------cut here---------------end--------------->8---
> 
> succeeds (75f9f944 being the parent commit of e8518c43).  I even ran
> these with --substitute-urls=https://bordeaux.guix.gnu.org in a
> freshly downloaded instance of the 1.3.0 QEMU image [5] to rule out
> corrupted substitutes from berlin with the same result.
For the sake of completeness, I'll be running this with --no-
substitutes and see what happens.  If you want to try the same without
rebuilding the world, I suggest first pulling
b5f654b238dd3dec43b0ee9e08b78981cf8de981 with substitutes -- that is
the last commit before the series. 

> 
Cheers

bug#54292: Commit e8518c43 breaks guix pull on i686

[Liliana Marie Prikler]

Am Dienstag, dem 08.03.2022 um 09:00 +0100 schrieb Liliana Marie
Prikler:
> > I do think this is an issue with commit e8518c43 because
> > 
> > --8<---------------cut here---------------start------------->8---
> > guix pull --commit=e8518c43 --system=i686-linux -p /tmp/guix
> > --8<---------------cut here---------------end--------------->8---
> > 
> > fails to build the package cache whereas 
> > 
> > --8<---------------cut here---------------start------------->8---
> > guix pull --commit=75f9f944 --system=i686-linux -p /tmp/guix
> > --8<---------------cut here---------------end--------------->8---
> > 
> > succeeds (75f9f944 being the parent commit of e8518c43).  I even ran
> > these with --substitute-urls=https://bordeaux.guix.gnu.org in a
> > freshly downloaded instance of the 1.3.0 QEMU image [5] to rule out
> > corrupted substitutes from berlin with the same result.
> For the sake of completeness, I'll be running this with --no-
> substitutes and see what happens.  If you want to try the same without
> rebuilding the world, I suggest first pulling
> b5f654b238dd3dec43b0ee9e08b78981cf8de981 with substitutes -- that is
> the last commit before the series. 
Okay, I now have the confirmation that this fails even "without any
substitutes" (I only had the guix package itself substituted to cut out
a little of the bootstrap chain).  I also have a full backtrace:

In gnu/packages.scm:
   437:11 19 (generate-package-cache _)
In srfi/srfi-1.scm:
   460:18 18 (fold #<procedure expand-cache expr> _ _)
In guix/packages.scm:
   518:21 17 (expand-cache . _)
  1260:17 16 (supported-package? #<package chez-fmt@0.8.11 gnu/pack…>
…)
In guix/memoization.scm:
    101:0 15 (_ #<hash-table a794e00 4898/7027> #<package chez-fmt@…>
…)
In guix/packages.scm:
  1230:12 14 (_)
In srfi/srfi-1.scm:
   460:18 13 (fold #<procedure ba84a30 at guix/packages.scm:1230:18…>
…)
In guix/packages.scm:
  1234:42 12 (_ _ ("x86_64-linux" "i686-linux" "armhf-linux" "aar…" …))
In guix/memoization.scm:
    101:0 11 (_ #<hash-table a794e00 4898/7027> #<package chez-sche…>
…)
In guix/packages.scm:
  1230:12 10 (_)
In srfi/srfi-1.scm:
   460:18  9 (fold #<procedure ba84960 at guix/packages.scm:1230:18…>
…)
In guix/packages.scm:
  1234:42  8 (_ _ ("x86_64-linux"))
In guix/memoization.scm:
    101:0  7 (_ #<hash-table a794e00 4898/7027> #<package stex@1.2.…>
…)
In guix/packages.scm:
  1238:37  6 (_)
  1498:16  5 (package->bag _ _ _ #:graft? _)
  1603:43  4 (thunk)
In gnu/packages/chez.scm:
   457:28  3 (arguments #<package stex@1.2.2-1.5405149 gnu/packages/…>)
    65:16  2 (chez-machine->threaded #f)
In unknown file:
           1 (string-ref #f 0)
In ice-9/boot-9.scm:
  1685:16  0 (raise-exception _ #:continuable? _)

The error appears to be that nix-system->chez-machine was rather poorly
coded and overlooked in review.  In particular, i686 should probably
also default to the i386 case.

Cheers

bug#54292: Commit e8518c43 breaks guix pull on i686

[Philip McGrath]

[-- Attachment #1: Type: text/plain, Size: 2170 bytes --]

On Tuesday, March 8, 2022 10:32:48 AM EST Liliana Marie Prikler wrote:
> 
> The error appears to be that nix-system->chez-machine was rather poorly
> coded and overlooked in review.  In particular, i686 should probably
> also default to the i386 case.
> 

I must for some reason not have been thinking about the fact that Guix 
distinguishes among "i[3456]86" when I ported `%nix-arch-to-chez-alist` from 
the `raco cross` codebase.

I can see (at least) two paths forward:

  1. `%nix-{arch,os}-to-chez-alist` could become many-to-one rather than
     one-to-one. Presumably we'd use the first applicable entry when going
     from Chez to Nix.

  2. We could give up on alists and use existing predicates from (guix utils)     
     like `target-x86-32?`, as Liliana had already suggested during review for
     `chez-upstream-features-for-system`.

I liked the alists because they made the bidirectional relationship obvious, 
but I wonder if there are other potential pitfalls analogous to this one. 
Maybe someone more familiar with non-x86_64 systems in Guix/Nix than I am 
should decide.

OTOH, while it's definitely a bug that
`(nix-system->chez-machine "i686-linux")` currently returns `#f`, it seems 
like the more immediate problem is from the `maybe-compile` phase of the stex-
bootstrap package, where this code:

                        (define machine
                          #$(chez-machine->threaded
                             (nix-system->chez-machine)))

assumes, apparently incorrectly, that it's only going to be run when the 
result of `nix-system->chez-machine` is non-false. In other words, even if we 
fix the bug in `nix-system->chez-machine` for i686, I think we'd still hit 
this problem for systems that Chez really doesn't support, e.g. ppc64le or 
MIPS. I don't know what the most correct way would be to write this code, but 
I think we could defer the error until someone attempts to build the package 
for the unsupported system by just writing something like:

                        (define machine
                          #$(and=> (nix-system->chez-machine)
                                   chez-machine->threaded))

-Philip

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#54292: Commit e8518c43 breaks guix pull on i686

[Liliana Marie Prikler]

Am Dienstag, dem 08.03.2022 um 13:33 -0500 schrieb Philip McGrath:
> I can see (at least) two paths forward:
> 
>   1. `%nix-{arch,os}-to-chez-alist` could become many-to-one rather
>      than one-to-one. Presumably we'd use the first applicable entry
> when going from Chez to Nix.
> 
>   2. We could give up on alists and use existing predicates from
>      (guix utils) like `target-x86-32?`, as Liliana had already
>      suggested during review for `chez-upstream-features-for-system`.
> 
> I liked the alists because they made the bidirectional relationship
> obvious, but I wonder if there are other potential pitfalls analogous
> to this one.  Maybe someone more familiar with non-x86_64 systems in
> Guix/Nix than I am should decide.
There is no 1:1 mapping to be found here as far as I can see. 
Likewise, the reverse mapping is currently unused and should in my
opinion be dropped so that people don't expect a round-trip to be well-
defined.

> OTOH, while it's definitely a bug that `(nix-system->chez-machine
> "i686-linux")` currently returns `#f`, it seems like the more
> immediate problem is from the `maybe-compile` phase of
> the stex-bootstrap package, where this code:
> 
>                         (define machine
>                           #$(chez-machine->threaded
>                              (nix-system->chez-machine)))
> 
> assumes, apparently incorrectly, that it's only going to be run when
> the result of `nix-system->chez-machine` is non-false. In other
> words, even if we fix the bug in `nix-system->chez-machine` for i686,
> I think we'd still hit this problem for systems that Chez really
> doesn't support, e.g. ppc64le or MIPS.  I don't know what the most
> correct way would be to write this code, but I think we could defer
> the error until someone attempts to build the package for the
> unsupported system by just writing something like:
> 
>                         (define machine
>                           #$(and=> (nix-system->chez-machine)
>                                    chez-machine->threaded))
I pushed that definition upstream, but a rewrite is still needed.  I
also think this logic should be a little decoupled from the question of
whether or not a given nix-system is supported.  While surely this
function returning #f means it's not, there are still other questions
to consider.

An implementation could look like the following

(define* (nix-system->chez-machine system #:key threaded?)
  (false-if-exception
   (string-append
    (if threads? "t" "")
    (cond
     ((target-x86_64? system) "a6")
     ...)
    (cond 
     ((target-linux? system) "le")
     ...))))

Cheers 

bug#54292: Commit e8518c43 breaks guix pull on i686

[Diego Nicola Barbato]

Hi Liliana,

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> Am Dienstag, dem 08.03.2022 um 13:33 -0500 schrieb Philip McGrath:

[...]

>> I don't know what the most correct way would be to write this code,
>> but I think we could defer the error until someone attempts to build
>> the package for the unsupported system by just writing something
>> like:
>> 
>>                         (define machine
>>                           #$(and=> (nix-system->chez-machine)
>>                                    chez-machine->threaded))
> I pushed that definition upstream, but a rewrite is still needed.

I can confirm that commit b8fc9169 (gnu: stex-bootstrap: Guard against
unsupported systems.) fixes this bug: The package cache builds and guix
pull succeeds on i686-linux.

[...]

Thanks,

Diego