bug#27157: Building Guile 2.2 "times out"

bug#27157: Building Guile 2.2 "times out"

[Maxim Cournoyer]

The build errors out with:

--8<---------------cut here---------------start------------->8---
  SNARF  weak-set.doc
  SNARF  weak-table.doc
  SNARF  weak-vector.doc
  SNARF  posix.doc
  SNARF  net_db.doc
  SNARF  socket.doc
  SNARF  regex-posix.doc
  CCLD     libguile-2.2.la
.libs/libguile_2.2_la-posix.o: In function `scm_tmpnam':
/tmp/guix-build-guile-2.2.2.drv-0/guile-2.2.2/libguile/posix.c:1574: warning: the use of `tmpnam' is dangerous, better use `mkstemp'
  CCLD     guile
  GEN      guile-procedures.texi
make[3]: Leaving directory '/tmp/guix-build-guile-2.2.2.drv-0/guile-2.2.2/libguile'
make[2]: Leaving directory '/tmp/guix-build-guile-2.2.2.drv-0/guile-2.2.2/libguile'
Making all in bootstrap
make[2]: Entering directory '/tmp/guix-build-guile-2.2.2.drv-0/guile-2.2.2/bootstrap'
  BOOTSTRAP GUILEC ice-9/eval.go
wrote `ice-9/eval.go'
  BOOTSTRAP GUILEC ice-9/psyntax-pp.go
  BOOTSTRAP GUILEC language/cps/intmap.go
building of `/gnu/store/i2p0gqxm378ydlh58qpy6jas7rdk79jg-guile-2.2.2.drv' timed out after 3600 seconds of silence
cannot build derivation
  `/gnu/store/c34w733549bjvcdixb0yj306ydhwv8c3-guile2.2-gnutls-3.5.9.drv':
  1 dependencies couldn't be built
--8<---------------cut here---------------end--------------->8---

I'm not sure where this timeout value can be configured, but clearly it
shouldn't be 1 hour since 2 cores of an i5 intel class processor can't
manage to build it under 1 hour.

Maxim

bug#27157: Building Guile 2.2 "times out"

[Ricardo Wurmus]

Hi Maxim,

> I'm not sure where this timeout value can be configured, but clearly it
> shouldn't be 1 hour since 2 cores of an i5 intel class processor can't
> manage to build it under 1 hour.

for the guile-2.2 package I see that this is part of the package
definition:

    (properties '((timeout . 72000)               ;20 hours
                  (max-silent-time . 10800)))     ;3 hours (needed on ARM)

We do this so that we can build it on Hydra.
You can override the max silent time:

    guix build --max-silent-time=9999999 guile

Does “guix build” not respect these build properties?

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#27157: Building Guile 2.2 "times out"

[Ludovic Courtès]

Hello,

Ricardo Wurmus <rekado@elephly.net> skribis:

>> I'm not sure where this timeout value can be configured, but clearly it
>> shouldn't be 1 hour since 2 cores of an i5 intel class processor can't
>> manage to build it under 1 hour.
>
> for the guile-2.2 package I see that this is part of the package
> definition:
>
>     (properties '((timeout . 72000)               ;20 hours
>                   (max-silent-time . 10800)))     ;3 hours (needed on ARM)
>
> We do this so that we can build it on Hydra.
> You can override the max silent time:
>
>     guix build --max-silent-time=9999999 guile
>
> Does “guix build” not respect these build properties?

It does not.  Only build-aux/hydra/gnu-system.scm does.

Ludo’.

bug#27157: Building Guile 2.2 "times out"

[Maxim Cournoyer]

Hi Ludo and Ricardor,

ludo@gnu.org (Ludovic Courtès) writes:

> Hello,
>
> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>>> I'm not sure where this timeout value can be configured, but clearly it
>>> shouldn't be 1 hour since 2 cores of an i5 intel class processor can't
>>> manage to build it under 1 hour.
>>
>> for the guile-2.2 package I see that this is part of the package
>> definition:
>>
>>     (properties '((timeout . 72000)               ;20 hours
>>                   (max-silent-time . 10800)))     ;3 hours (needed on ARM)
>>
>> We do this so that we can build it on Hydra.
>> You can override the max silent time:
>>
>>     guix build --max-silent-time=9999999 guile
>>
>> Does “guix build” not respect these build properties?
>
> It does not.  Only build-aux/hydra/gnu-system.scm does.

Apparently the problem was due to the build action being triggered as
part of a "guix environment" command. I could launch the command using
"guix build guile@2.2" and it completed successfully.

Guix build doesn't come with default option value for the
max-silent-time, but guix environment does, and it is set to 3600
s. This is defined under (guix scripts environment):

(define %default-options
  `((system . ,(%current-system))
    (substitutes? . #t)
    (graft? . #t)
    (max-silent-time . 3600)
    (verbosity . 0)))

If this is by design, I guess this issue can be closed.

Maxim

bug#27157: Building Guile 2.2 "times out"

[Ludovic Courtès]

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Guix build doesn't come with default option value for the
> max-silent-time, but guix environment does, and it is set to 3600
> s. This is defined under (guix scripts environment):
>
> (define %default-options
>   `((system . ,(%current-system))
>     (substitutes? . #t)
>     (graft? . #t)
>     (max-silent-time . 3600)
>     (verbosity . 0)))
>
> If this is by design, I guess this issue can be closed.

Good catch!

However, according to ‘set-build-options’ in (guix store),
#:max-silent-time #f is equivalent to #:max-silent-time 3600.

Perhaps the build remains silent for just about an hour and you were
lucky on your second try?

At any rate, it would be good to remove ‘max-silent-time’ from all the
‘%default-options’ variables, for consistency.

Ludo’.

bug#27157: Building Guile 2.2 "times out"

[Mark H Weaver]

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi Maxim,
>
>> I'm not sure where this timeout value can be configured, but clearly it
>> shouldn't be 1 hour since 2 cores of an i5 intel class processor can't
>> manage to build it under 1 hour.
>
> for the guile-2.2 package I see that this is part of the package
> definition:
>
>     (properties '((timeout . 72000)               ;20 hours
>                   (max-silent-time . 10800)))     ;3 hours (needed on ARM)

I think we should further increase the 'max-silent-time' value for
Guile-2.2.  I've had to restart that build several times on ARM.  3
hours might be enough on a machine that's not doing anything else, but
our build slaves are often building multiple packages.

I would increase it to 6 hours.  What do you think?

       Mark

bug#27157: Building Guile 2.2 "times out"

[Ricardo Wurmus]

Mark H Weaver <mhw@netris.org> writes:

> I think we should further increase the 'max-silent-time' value for
> Guile-2.2.  I've had to restart that build several times on ARM.  3
> hours might be enough on a machine that's not doing anything else, but
> our build slaves are often building multiple packages.
>
> I would increase it to 6 hours.  What do you think?

I have no objections.  We should assume busy build machines.  It’s
unfortunate to have no output whatever for up to six hours :-/

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#27157: [PATCH] Re: bug#27157: Building Guile 2.2 "times out"

[Maxim Cournoyer]

ludo@gnu.org (Ludovic Courtès) writes:

> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Guix build doesn't come with default option value for the
>> max-silent-time, but guix environment does, and it is set to 3600
>> s. This is defined under (guix scripts environment):
>>
>> (define %default-options
>>   `((system . ,(%current-system))
>>     (substitutes? . #t)
>>     (graft? . #t)
>>     (max-silent-time . 3600)
>>     (verbosity . 0)))
>>
>> If this is by design, I guess this issue can be closed.
>
> Good catch!
>
> However, according to ‘set-build-options’ in (guix store),
> #:max-silent-time #f is equivalent to #:max-silent-time 3600.

IIUC, this should only be true when (< (nix-server-minor-version
server) #x61), as can be seen in (guix store):

    (when (< (nix-server-minor-version server) #x61)
      (let ((max-build-jobs (or max-build-jobs 1))
            (xomax-silent-time (or max-silent-time 3600)))
        (send (integer max-build-jobs) (integer max-silent-time))))

Which seems to be matched with the following in nix-daemon.cc:

        if (GET_PROTOCOL_MINOR(clientVersion) < 0x61) {
            settings.set("build-max-jobs", std::to_string(readInt(from)));
            settings.set("build-max-silent-time", std::to_string(readInt(from)));
        }

Given that the the nix worker protocole PROTOCOL_VERSION was bumped to
0x161 in commit deac976d3d26c7b85b9c90efb424b0aa94f1027c last January, I
would *not* expect this condition to be evaluated.

By the way, I find it weird that these conditions be expressed in terms
of the minor version number only. The correctness of those is thus only
guaranteed as long as the major number version is not bumped; this seems
unwise.

> Perhaps the build remains silent for just about an hour and you were
> lucky on your second try?

See explanation above. I don't think this .

> At any rate, it would be good to remove ‘max-silent-time’ from all the
> ‘%default-options’ variables, for consistency.
>
> Ludo’.

I agree! In my view, interactive user commands should expect the users
to take action themselves (e.g.; user notice no output for 3 hours; hits
C-c. Or, user notice no output for 3 hours, check the process in top,
sees it's still doing something, decides to leave it running a couple
more hours).

Maxim

bug#27157: [PATCH] Re: bug#27157: Building Guile 2.2 "times out"

[Ludovic Courtès]

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>>
>>> Guix build doesn't come with default option value for the
>>> max-silent-time, but guix environment does, and it is set to 3600
>>> s. This is defined under (guix scripts environment):
>>>
>>> (define %default-options
>>>   `((system . ,(%current-system))
>>>     (substitutes? . #t)
>>>     (graft? . #t)
>>>     (max-silent-time . 3600)
>>>     (verbosity . 0)))
>>>
>>> If this is by design, I guess this issue can be closed.
>>
>> Good catch!
>>
>> However, according to ‘set-build-options’ in (guix store),
>> #:max-silent-time #f is equivalent to #:max-silent-time 3600.
>
> IIUC, this should only be true when (< (nix-server-minor-version
> server) #x61), as can be seen in (guix store):
>
>     (when (< (nix-server-minor-version server) #x61)
>       (let ((max-build-jobs (or max-build-jobs 1))
>             (xomax-silent-time (or max-silent-time 3600)))
>         (send (integer max-build-jobs) (integer max-silent-time))))

Oooh, indeed, sorry for the confusion.  So what happens when
#:max-silent-time is #f is that we use the daemon’s default value, which
is infinite (zero) by default (see libstore/globals.cc).

> Given that the the nix worker protocole PROTOCOL_VERSION was bumped to
> 0x161 in commit deac976d3d26c7b85b9c90efb424b0aa94f1027c last January, I
> would *not* expect this condition to be evaluated.
>
> By the way, I find it weird that these conditions be expressed in terms
> of the minor version number only. The correctness of those is thus only
> guaranteed as long as the major number version is not bumped; this seems
> unwise.

Historical baggage: the major number was never bumped, I think.  :-)
You’re right though.

>> At any rate, it would be good to remove ‘max-silent-time’ from all the
>> ‘%default-options’ variables, for consistency.
>>
>> Ludo’.
>
> I agree! In my view, interactive user commands should expect the users
> to take action themselves (e.g.; user notice no output for 3 hours; hits
> C-c. Or, user notice no output for 3 hours, check the process in top,
> sees it's still doing something, decides to leave it running a couple
> more hours).

Yes.  Would you like to prepare a patch to remove ‘max-silent-time’ from
all the ‘%default-options’?

In the meantime I’ll add --timeout and --max-silent-time as options to
guix-daemon.

Thanks!

Ludo’.

bug#27157: [PATCH] Re: bug#27157: Building Guile 2.2 "times out"

[Maxim Cournoyer]

ludo@gnu.org (Ludovic Courtès) writes:

[...]

>>> At any rate, it would be good to remove ‘max-silent-time’ from all the
>>> ‘%default-options’ variables, for consistency.
>>>
>>> Ludo’.
>>
>> I agree! In my view, interactive user commands should expect the users
>> to take action themselves (e.g.; user notice no output for 3 hours; hits
>> C-c. Or, user notice no output for 3 hours, check the process in top,
>> sees it's still doing something, decides to leave it running a couple
>> more hours).
>
> Yes.  Would you like to prepare a patch to remove ‘max-silent-time’ from
> all the ‘%default-options’?

I gave this a quick try, but got confused upon encountering the
guix-offload script; it has a max-silent-time option, but doesn't build
anything, it simply copies stuff IIUC.

Should we remove it there as well?

Maxim

bug#27157: [PATCH] Re: bug#27157: Building Guile 2.2 "times out"

[Ludovic Courtès]

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>>>> At any rate, it would be good to remove ‘max-silent-time’ from all the
>>>> ‘%default-options’ variables, for consistency.
>>>>
>>>> Ludo’.
>>>
>>> I agree! In my view, interactive user commands should expect the users
>>> to take action themselves (e.g.; user notice no output for 3 hours; hits
>>> C-c. Or, user notice no output for 3 hours, check the process in top,
>>> sees it's still doing something, decides to leave it running a couple
>>> more hours).
>>
>> Yes.  Would you like to prepare a patch to remove ‘max-silent-time’ from
>> all the ‘%default-options’?
>
> I gave this a quick try, but got confused upon encountering the
> guix-offload script; it has a max-silent-time option, but doesn't build
> anything, it simply copies stuff IIUC.
>
> Should we remove it there as well?

No: ‘guix offload’ gets max-silent-time systematically from its
command-line arguments (it is invoked by guix-daemon).  So this one can
remain as-is.

Ludo’.

bug#27157: Building Guile 2.2 "times out"

[Mark H Weaver]

Ricardo Wurmus <rekado@elephly.net> writes:

> Mark H Weaver <mhw@netris.org> writes:
>
>> I think we should further increase the 'max-silent-time' value for
>> Guile-2.2.  I've had to restart that build several times on ARM.  3
>> hours might be enough on a machine that's not doing anything else, but
>> our build slaves are often building multiple packages.
>>
>> I would increase it to 6 hours.  What do you think?
>
> I have no objections.  We should assume busy build machines.  It’s
> unfortunate to have no output whatever for up to six hours :-/

I pushed this as commit 1577a65809f8ef1883d18cf81095f632b65c042a.
I'm not sure whether to close this bug or not.  Can someone else do so
if appropriate?

      Mark

bug#27157: Building Guile 2.2 "times out"

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 607 bytes --]

Hi Mark,


> I pushed this as commit 1577a65809f8ef1883d18cf81095f632b65c042a.
> I'm not sure whether to close this bug or not.  Can someone else do so
> if appropriate?
>
>       Mark
>

I wanted to give another shot at removing the the default max-silent-time
options in our scripts, so that everything can be consistent, but haven't
got around it yet. I believe we shouldn't close this bug yet as it is still
possible to get into the problem by running "guix environment", for
example, which has a default max-silent-time value set to 1 hour.

Feel free to beat me to it if you have the bandwith.

Maxim

[-- Attachment #2: Type: text/html, Size: 987 bytes --]

bug#27157: [PATCH] Re: bug#27157: Building Guile 2.2 "times out"

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 218 bytes --]

Hi,

Here's a patch that fixes the problem originally reported (guile
was timing out while being built by 'guix environment guix', which was
capped to 3600 s of silent time.

I think the bug can now be closed.

Maxim


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ui-Remove-max-silent-time-from-the-default-options.patch --]
[-- Type: text/x-patch, Size: 3817 bytes --]

From ba078ca1b8efa17fb0b2ee7b4269611c6bb4c3d6 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Date: Wed, 30 Aug 2017 23:12:12 -0400
Subject: [PATCH] ui: Remove max-silent-time from the default options.

Having a finite default `max-silent-time' value for scripts such as 'guix
environment' could lead to timeouts when building subtitutes; this was
undesirable. This fixes bug #27157.

* guix/scripts/archive.scm (%default-options): Remove max-silent-time entry.
* guix/scripts/copy.scm (%default-options): Likewise.
* guix/scripts/environment.scm (%default-options): Likewise.
* guix/scripts/pack.scm (%default-options): Likewise.
* guix/scripts/package.scm (%default-options): Likewise.
* guix/scripts/pull.scm (%default-options): Likewise.
* guix/scripts/system.scm (%default-options): Likewise.
---
 guix/scripts/archive.scm     | 1 -
 guix/scripts/copy.scm        | 1 -
 guix/scripts/environment.scm | 1 -
 guix/scripts/pack.scm        | 1 -
 guix/scripts/package.scm     | 3 +--
 guix/scripts/pull.scm        | 1 -
 guix/scripts/system.scm      | 1 -
 7 files changed, 1 insertion(+), 8 deletions(-)

diff --git a/guix/scripts/archive.scm b/guix/scripts/archive.scm
index 5ea19784d..a569848ae 100644
--- a/guix/scripts/archive.scm
+++ b/guix/scripts/archive.scm
@@ -54,7 +54,6 @@
   `((system . ,(%current-system))
     (substitutes? . #t)
     (graft? . #t)
-    (max-silent-time . 3600)
     (verbosity . 0)))
 
 (define (show-help)
diff --git a/guix/scripts/copy.scm b/guix/scripts/copy.scm
index 32438b99d..9ffffe8cc 100644
--- a/guix/scripts/copy.scm
+++ b/guix/scripts/copy.scm
@@ -151,7 +151,6 @@ Copy ITEMS to or from the specified host over SSH.\n"))
   `((system . ,(%current-system))
     (substitutes? . #t)
     (graft? . #t)
-    (max-silent-time . 3600)
     (verbosity . 0)))
 
 \f
diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm
index 95ba199d9..0d6921833 100644
--- a/guix/scripts/environment.scm
+++ b/guix/scripts/environment.scm
@@ -179,7 +179,6 @@ COMMAND or an interactive shell in that environment.\n"))
   `((system . ,(%current-system))
     (substitutes? . #t)
     (graft? . #t)
-    (max-silent-time . 3600)
     (verbosity . 0)))
 
 (define (tag-package-arg opts arg)
diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index c269a1fef..21fea446a 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -263,7 +263,6 @@ the image."
     (system . ,(%current-system))
     (substitutes? . #t)
     (graft? . #t)
-    (max-silent-time . 3600)
     (verbosity . 0)
     (symlinks . ())
     (compressor . ,(first %compressors))))
diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
index fa45bd48a..9ec6950c4 100644
--- a/guix/scripts/package.scm
+++ b/guix/scripts/package.scm
@@ -377,8 +377,7 @@ ENTRIES, a list of manifest entries, in the context of PROFILE."
 
 (define %default-options
   ;; Alist of default option values.
-  `((max-silent-time . 3600)
-    (verbosity . 0)
+  `((verbosity . 0)
     (graft? . #t)
     (substitutes? . #t)))
 
diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm
index a1deec804..b1c87c870 100644
--- a/guix/scripts/pull.scm
+++ b/guix/scripts/pull.scm
@@ -96,7 +96,6 @@ Install it by running:
     (system . ,(%current-system))
     (substitutes? . #t)
     (graft? . #t)
-    (max-silent-time . 3600)
     (verbosity . 0)))
 
 (define (show-help)
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 8793c4092..773779318 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -864,7 +864,6 @@ Some ACTIONS support additional ARGS.\n"))
     (substitutes? . #t)
     (graft? . #t)
     (build-hook? . #t)
-    (max-silent-time . 3600)
     (verbosity . 0)
     (file-system-type . "ext4")
     (image-size . guess)
-- 
2.13.1

bug#27157: [PATCH] Re: bug#27157: Building Guile 2.2 "times out"

[Ludovic Courtès]

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> From ba078ca1b8efa17fb0b2ee7b4269611c6bb4c3d6 Mon Sep 17 00:00:00 2001
> From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
> Date: Wed, 30 Aug 2017 23:12:12 -0400
> Subject: [PATCH] ui: Remove max-silent-time from the default options.
>
> Having a finite default `max-silent-time' value for scripts such as 'guix
> environment' could lead to timeouts when building subtitutes; this was
> undesirable. This fixes bug #27157.
>
> * guix/scripts/archive.scm (%default-options): Remove max-silent-time entry.
> * guix/scripts/copy.scm (%default-options): Likewise.
> * guix/scripts/environment.scm (%default-options): Likewise.
> * guix/scripts/pack.scm (%default-options): Likewise.
> * guix/scripts/package.scm (%default-options): Likewise.
> * guix/scripts/pull.scm (%default-options): Likewise.
> * guix/scripts/system.scm (%default-options): Likewise.

Good catch!  It was actually a bug, because the doc said that the
daemon’s setting should be honored by default, which was obviously not
the case.

Pushed:

  https://git.savannah.gnu.org/cgit/guix.git/commit/?id=fe55199542590d16aa6ca6a8bfa0fda27903db5c

Thanks,
Ludo’.