bug#39352: guix pull failing on aarch64

bug#39352: guix pull failing on aarch64

[shtwzrd via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 3868 bytes --]

Hey there,

I've noticed that guix pull has started to hang consistently when building guix-system.drv, 55% of the way through.

I've narrowed the breakage down to this specific commit:

http://git.savannah.gnu.org/cgit/guix.git/commit/?id=8234fe653e61d0090138cbd4c48d877568355439

So it appears to be a side effect of the move from guile 2 to guile 3?

Here's the output of guix pull --debug=2
--

Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Building from this channel:
  guix      https://git.savannah.gnu.org/git/guix.git 8234fe6
Computing Guix derivation for 'aarch64-linux'... \
The following derivations will be built:
   /gnu/store/sl3qrdgf2xpb1fih5fjmc2sqssamyqb2-profile.drv
   /gnu/store/pl4qs7i5fn993kwk067hslxa876379gg-profile.drv
   /gnu/store/yavr7yp5qj142h4vjhjy6p2a66j09gsa-guix-8234fe653.drv
   /gnu/store/27j6f5bhygf0991mr8v17xx6qrpi3d8v-guix-daemon.drv
   /gnu/store/bjhxlz6ffvdhk7h1396qnz7cz5556rn3-guix-command.drv
   /gnu/store/5lvlhhgnmh65ci2c2z58dq26n4l26f05-guix-module-union.drv
   /gnu/store/9s93hj38x4x47ssmp743bymx5s84fx4k-guix-8234fe653-modules.drv
   /gnu/store/3kyk5hnkqj7yajl9n41g3ykwyw6jg58y-guix-cli-modules.drv
   /gnu/store/g8q6lv4mb548976a85shzfq6fj21lcm9-guix-cli.drv
   /gnu/store/mb32kmp1xkwdnpdnsl0j80c96iyp6hbx-guix-system.drv
   /gnu/store/9kmf6vlxax21jxccpv3m337czrsgs5la-guix-system-tests-modules.drv
   /gnu/store/6j1gig9p38bkk46lnr0ndgmjc2li7ij7-guix-system-tests.drv
   /gnu/store/vhiswqck3sq2fzy8wbl29m6wwsnmx2a9-guix-system-modules.drv
   /gnu/store/yvd6m5kmzccr5liwfz3r8wc6ax13zs4w-inferior-script.scm.drv
The following profile hooks will be built:
   /gnu/store/6lyi8ljzy2bjra3gfbg2iqpyfni468ly-guix-package-cache.drv
   /gnu/store/dfinz9wv0fcayq09qhrnzna4bzni694g-fonts-dir.drv
   /gnu/store/q166nphssmb4bhi348c0ml09r72agzqv-manual-database.drv
   /gnu/store/w9r4rkq7r7xkjq3iz20pxydngjxh794m-ca-certificate-bundle.drv
   /gnu/store/x4z5z1r2pg53wfc2ggjz6inhrkq171v0-info-dir.drv
building path(s) `/gnu/store/mw5nqp9nnsvg8jrvq12mznk4c8a8sfhd-guix-system'
|   @ build-started /gnu/store/mb32kmp1xkwdnpdnsl0j80c96iyp6hbx-guix-system.drv - aarch64-linux /var/log/guix/drvs/mb//32kmp1xkwdnpdnsl0j80c96iyp6hbx-guix-system.drv.bz2 3275
Backtrace:
          19 (primitive-load "/home/rock/.config/guix/current/bin/guix")
In guix/ui.scm:
  1813:12 18 (run-guix-command _ . _)
In ice-9/boot-9.scm:
    829:9 17 (catch _ _ #<procedure 7fa52ed430 at guix/ui.scm:657:2 (key c)> _)
    829:9 16 (catch _ _ #<procedure 7fa52ed448 at guix/ui.scm:780:6 (key pr?> ?)
    829:9 15 (catch _ _ #<procedure 7fa2d34e38 at guix/scripts/pull.scm:832?> ?)
In guix/store.scm:
   625:10 14 (call-with-store _)
In guix/status.scm:
    768:4 13 (call-with-status-report _ _)
In guix/scripts/pull.scm:
   872:24 12 (_)
In guix/build/syscalls.scm:
   1145:4 11 (call-with-file-lock/no-wait _ _ _)
In guix/store.scm:
  1848:24 10 (run-with-store #<store-connection 256.99 2290f960> _ # _ # _ # _)
In guix/scripts/pull.scm:
    440:4  9 (_ _)
In guix/store.scm:
  1736:38  8 (_ #<store-connection 256.99 249db240>)
In guix/scripts/package.scm:
   161:13  7 (build-and-use-profile #<store-connection 256.99 249db240> "/v?" ?)
In guix/store.scm:
  1224:15  6 (_ #<store-connection 256.99 249db240> _ _)
   720:13  5 (process-stderr _ _)
In unknown file:
           4 (display "@ build-log 3275 40\n\r[  0/154] loading...\t  0.0% ?" ?)
In guix/status.scm:
   713:19  3 (write! _ _ _)
In srfi/srfi-1.scm:
   466:18  2 (fold #<procedure 220ed200 at guix/status.scm:603:2 (event sta?> ?)
In guix/status.scm:
   604:15  1 (_ (build-log 3275 "\r[  0/154] loading...\t  0.0% of 77 fil?") #)
   180:23  0 (update-build #<<build-status> building: () downloading: () bu?> ?)

guix/status.scm:180:23: In procedure update-build:
In procedure struct_vtable: Wrong type argument in position 1 (expecting struct): #f

[-- Attachment #2: Type: text/html, Size: 5856 bytes --]

bug#39352: guix pull failing on aarch64

[Jesse Gibbons]

On Thu, 2020-01-30 at 00:51 +0000, shtwzrd via Bug reports for GNU Guix
wrote:
> Hey there,
> 
> I've noticed that guix pull has started to hang consistently when
> building guix-system.drv, 55% of the way through.
> 
> I've narrowed the breakage down to this specific commit:
> 
> http://git.savannah.gnu.org/cgit/guix.git/commit/?id=8234fe653e61d0090138cbd4c48d877568355439
> 
> So it appears to be a side effect of the move from guile 2 to guile
> 3?
> 
> Here's the output of guix pull --debug=2
> --
> ...
> 
> 
I've noticed most guix ops (including guix pull) hang on armhf as well,
but I don't have a log to provide.
Right now I'm fine with keeping guix at a stable (but potentially less
secure) commit until the JIT compiler in Guile 3 is fixed and enabled
on ARM.

bug#39352: guix pull failing on aarch64

[Ludovic Courtès]

Hello!

Jesse Gibbons <jgibbons2357@gmail.com> skribis:

> I've noticed most guix ops (including guix pull) hang on armhf as well,
> but I don't have a log to provide.
> Right now I'm fine with keeping guix at a stable (but potentially less
> secure) commit until the JIT compiler in Guile 3 is fixed and enabled
> on ARM.

Right.  The AArch64 bug is tracked at:

  https://issues.guix.gnu.org/issue/39266

I believe the problem is not deterministic, so it may be that retrying
‘guix pull’ will work…

Ludo’.

bug#39352: guix pull failing on aarch64

[shtwzrd via Bug reports for GNU Guix]

In my case on aarch64, it appears to be deterministic. I've tried over a dozen times and it always fails in the exact same place.

So perhaps aarch64 is a good target for debugging the underlying problem, if this and #39266 are related, as they appear to be.


‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Sunday, February 2, 2020 9:23 PM, Ludovic Courtès <ludo@gnu.org> wrote:

> Hello!
>
> Jesse Gibbons jgibbons2357@gmail.com skribis:
>
> > I've noticed most guix ops (including guix pull) hang on armhf as well,
> > but I don't have a log to provide.
> > Right now I'm fine with keeping guix at a stable (but potentially less
> > secure) commit until the JIT compiler in Guile 3 is fixed and enabled
> > on ARM.
>
> Right. The AArch64 bug is tracked at:
>
> https://issues.guix.gnu.org/issue/39266
>
> I believe the problem is not deterministic, so it may be that retrying
> ‘guix pull’ will work…
>
> Ludo’.

bug#39352: guix pull failing on aarch64

[Jan Nieuwenhuizen]

shtwzrd writes:

> I've noticed that guix pull has started to hang consistently when
> building guix-system.drv, 55% of the way through.

Ah; I'm seeing this too -- for me current master consistently hangs at
54%

janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

bug#39352: guix pull failing on aarch64

[shtwzrd via Bug reports for GNU Guix]

Ludo said:
> I believe the problem is not deterministic, so it may be that retrying
> ‘guix pull’ will work…

I take back what I said; I tried one more time today just out of curiosity, and it actually succeeded.

I tried four more times afterward though and it failed the same way as it has before.

So just as you said, it's not deterministic, but it fails much more often than not in my particular situation.

If there's any extra information I can give that could help people debug this, please let me know. Hunting down a hard to reproduce error is the worst. :)

bug#39352: guix pull failing on aarch64

[Ludovic Courtès]

Hello,

Ludovic Courtès <ludo@gnu.org> skribis:

> Jesse Gibbons <jgibbons2357@gmail.com> skribis:
>
>> I've noticed most guix ops (including guix pull) hang on armhf as well,
>> but I don't have a log to provide.
>> Right now I'm fine with keeping guix at a stable (but potentially less
>> secure) commit until the JIT compiler in Guile 3 is fixed and enabled
>> on ARM.
>
> Right.  The AArch64 bug is tracked at:
>
>   https://issues.guix.gnu.org/issue/39266

The issue was fixed two weeks ago.  Closing!

Ludo’.