bug#29826: nondeterministic Broken pipe

bug#29826: nondeterministic Broken pipe

[Alex Vong]

[-- Attachment #1: Type: text/plain, Size: 1571 bytes --]

Hello,

I get the following error when running ``guix --version | head -n 1''. I
can get similar after replacing ``--version'' with ``--help''. Also, the
error is nondeterministic. Any idea?


$ LC_ALL=C guix --version | head -n 1
guix (GNU Guix) 91213c384b62bc422896d39005efa922e4fa75f2
Backtrace:
In ice-9/boot-9.scm:
 160: 14 [catch #t #<catch-closure 5628460c6140> ...]
In unknown file:
   ?: 13 [apply-smob/1 #<catch-closure 5628460c6140>]
In ice-9/boot-9.scm:
  66: 12 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 432: 11 [eval # #]
In ice-9/boot-9.scm:
2404: 10 [save-module-excursion #<procedure 5628460e8900 at ice-9/boot-9.scm:4051:3 ()>]
4056: 9 [#<procedure 5628460e8900 at ice-9/boot-9.scm:4051:3 ()>]
1727: 8 [%start-stack load-stack ...]
1732: 7 [#<procedure 5628460fdb40 ()>]
In unknown file:
   ?: 6 [primitive-load "/usr/local/bin/guix"]
In guix/ui.scm:
 390: 5 [show-version-and-exit "guix"]
In ice-9/format.scm:
1593: 4 [format #t "Copyright ~a 2017 ~a" "(C)" "the Guix authors\n"]
 766: 3 [format:format-work "Copyright ~a 2017 ~a" ("(C)" "the Guix authors\n")]
 264: 2 [tilde-dispatch]
  73: 1 [format:out-obj-padded #f "the Guix authors\n" #f ()]
In unknown file:
   ?: 0 [display "the Guix authors\n" #<output: file 1>]

ERROR: In procedure display:
ERROR: In procedure scm_flush: Broken pipe
Backtrace:
In ice-9/boot-9.scm:
 160: 1 [catch #t #<catch-closure 562846cd3020> ...]
In unknown file:
   ?: 0 [apply-smob/1 #<catch-closure 562846cd3020>]

ERROR: In procedure apply-smob/1:
ERROR: In procedure scm_flush: Broken pipe


Cheers,
Alex

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#29826: nondeterministic Broken pipe

[Andreas Enge]

On Sun, Dec 24, 2017 at 04:23:44AM +0800, Alex Vong wrote:
> ERROR: In procedure display:
> ERROR: In procedure scm_flush: Broken pipe

Interesting! I get a different error message:

ERROR: In procedure display:
In procedure fport_write: Broken pipe

Andreas

bug#29826: nondeterministic Broken pipe

[Alex Vong]

[-- Attachment #1: Type: text/plain, Size: 383 bytes --]

Andreas Enge <andreas@enge.fr> writes:

> On Sun, Dec 24, 2017 at 04:23:44AM +0800, Alex Vong wrote:
>> ERROR: In procedure display:
>> ERROR: In procedure scm_flush: Broken pipe
>
> Interesting! I get a different error message:
>
> ERROR: In procedure display:
> In procedure fport_write: Broken pipe
>
> Andreas

Hmm... This may due to the fact I am using Guile 2.0 instead of 2.2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#29826: nondeterministic Broken pipe

[Mark H Weaver]

Alex Vong <alexvong1995@gmail.com> writes:

> I get the following error when running ``guix --version | head -n 1''. I
> can get similar after replacing ``--version'' with ``--help''. Also, the
> error is nondeterministic. Any idea?

Attempts to write to a pipe that has already been closed on the other
end results in EPIPE.  From the write(2) man page:

  EPIPE fd is connected to a pipe or socket whose reading end is closed.
        When this happens the writing process will also receive a
        SIGPIPE signal.  (Thus, the write return value is seen only if
        the program catches, blocks or ignores this signal.)

In this case, there's a race condition.  The result depends on whether
"head -n 1" closes its end of the pipe before or after "guix --version"
is finished writing all of its output.  If "head -n 1" closes the pipe
first, then "guix --version" will receive EPIPE while attempting to
write to it.

What normally happens is that the sending process receives SIGPIPE,
which simply causes it to exit prematurely without ever receiving this
error.  However, since Guix arranges to ignore SIGPIPE in
'initialize-guix' in guix/ui.scm, we receive EPIPE.

That's what's happening here.  I'll need to think on how best to fix it.

     Regards,
       Mark

bug#29826: nondeterministic Broken pipe

[Alex Vong]

[-- Attachment #1: Type: text/plain, Size: 1576 bytes --]

Mark H Weaver <mhw@netris.org> writes:

> Alex Vong <alexvong1995@gmail.com> writes:
>
>> I get the following error when running ``guix --version | head -n 1''. I
>> can get similar after replacing ``--version'' with ``--help''. Also, the
>> error is nondeterministic. Any idea?
>
> Attempts to write to a pipe that has already been closed on the other
> end results in EPIPE.  From the write(2) man page:
>
>   EPIPE fd is connected to a pipe or socket whose reading end is closed.
>         When this happens the writing process will also receive a
>         SIGPIPE signal.  (Thus, the write return value is seen only if
>         the program catches, blocks or ignores this signal.)
>
> In this case, there's a race condition.  The result depends on whether
> "head -n 1" closes its end of the pipe before or after "guix --version"
> is finished writing all of its output.  If "head -n 1" closes the pipe
> first, then "guix --version" will receive EPIPE while attempting to
> write to it.
>
> What normally happens is that the sending process receives SIGPIPE,
> which simply causes it to exit prematurely without ever receiving this
> error.  However, since Guix arranges to ignore SIGPIPE in
> 'initialize-guix' in guix/ui.scm, we receive EPIPE.
>
> That's what's happening here.  I'll need to think on how best to fix it.
>
>      Regards,
>        Mark

Nice explaination as always! I forget to mention that I reported a bug
of similar flavour before <http://bugs.gnu.org/27017>. I agree that
thought is needed to fix all instances of this type of bug.

Cheers,
Alex

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#29826: nondeterministic Broken pipe

[Ludovic Courtès]

Hi,

Alex Vong <alexvong1995@gmail.com> skribis:

> Mark H Weaver <mhw@netris.org> writes:
>
>> Alex Vong <alexvong1995@gmail.com> writes:
>>
>>> I get the following error when running ``guix --version | head -n 1''. I
>>> can get similar after replacing ``--version'' with ``--help''. Also, the
>>> error is nondeterministic. Any idea?
>>
>> Attempts to write to a pipe that has already been closed on the other
>> end results in EPIPE.  From the write(2) man page:
>>
>>   EPIPE fd is connected to a pipe or socket whose reading end is closed.
>>         When this happens the writing process will also receive a
>>         SIGPIPE signal.  (Thus, the write return value is seen only if
>>         the program catches, blocks or ignores this signal.)
>>
>> In this case, there's a race condition.  The result depends on whether
>> "head -n 1" closes its end of the pipe before or after "guix --version"
>> is finished writing all of its output.  If "head -n 1" closes the pipe
>> first, then "guix --version" will receive EPIPE while attempting to
>> write to it.
>>
>> What normally happens is that the sending process receives SIGPIPE,
>> which simply causes it to exit prematurely without ever receiving this
>> error.  However, since Guix arranges to ignore SIGPIPE in
>> 'initialize-guix' in guix/ui.scm, we receive EPIPE.
>>
>> That's what's happening here.  I'll need to think on how best to fix it.
>>
>>      Regards,
>>        Mark
>
> Nice explaination as always! I forget to mention that I reported a bug
> of similar flavour before <http://bugs.gnu.org/27017>. I agree that
> thought is needed to fix all instances of this type of bug.

Not sure!  We specifically ignore EPIPE in cases where it matters, such
as for the output of ‘guix package --search’, ‘guix package -A’, etc.
In other cases, it’s probably an error, so it’s worth reporting.

WDYT?

In C such errors are usually ignored, which is nice for shell hackery
but otherwise not so great.

Ludo’.

bug#29826: nondeterministic Broken pipe

[Alex Vong]

[-- Attachment #1: Type: text/plain, Size: 2485 bytes --]

Hello,

ludo@gnu.org (Ludovic Courtès) writes:

> Hi,
>
> Alex Vong <alexvong1995@gmail.com> skribis:
>
>> Mark H Weaver <mhw@netris.org> writes:
>>
>>> Alex Vong <alexvong1995@gmail.com> writes:
>>>
>>>> I get the following error when running ``guix --version | head -n 1''. I
>>>> can get similar after replacing ``--version'' with ``--help''. Also, the
>>>> error is nondeterministic. Any idea?
>>>
>>> Attempts to write to a pipe that has already been closed on the other
>>> end results in EPIPE.  From the write(2) man page:
>>>
>>>   EPIPE fd is connected to a pipe or socket whose reading end is closed.
>>>         When this happens the writing process will also receive a
>>>         SIGPIPE signal.  (Thus, the write return value is seen only if
>>>         the program catches, blocks or ignores this signal.)
>>>
>>> In this case, there's a race condition.  The result depends on whether
>>> "head -n 1" closes its end of the pipe before or after "guix --version"
>>> is finished writing all of its output.  If "head -n 1" closes the pipe
>>> first, then "guix --version" will receive EPIPE while attempting to
>>> write to it.
>>>
>>> What normally happens is that the sending process receives SIGPIPE,
>>> which simply causes it to exit prematurely without ever receiving this
>>> error.  However, since Guix arranges to ignore SIGPIPE in
>>> 'initialize-guix' in guix/ui.scm, we receive EPIPE.
>>>
>>> That's what's happening here.  I'll need to think on how best to fix it.
>>>
>>>      Regards,
>>>        Mark
>>
>> Nice explaination as always! I forget to mention that I reported a bug
>> of similar flavour before <http://bugs.gnu.org/27017>. I agree that
>> thought is needed to fix all instances of this type of bug.
>
> Not sure!  We specifically ignore EPIPE in cases where it matters, such
> as for the output of ‘guix package --search’, ‘guix package -A’, etc.
> In other cases, it’s probably an error, so it’s worth reporting.
>
> WDYT?
>
> In C such errors are usually ignored, which is nice for shell hackery
> but otherwise not so great.
>
> Ludo’.

Do you mean there are use-cases where the EPIPE signal really means
there is an error? What I think is that the 'guix' command is meant to
be used in a shell script, so it should work nice with other shell tools
in a pipe, including head & tail. But maybe it will cause other problems
if we always ignore EPIPE, I don't know... 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#29826: nondeterministic Broken pipe

[Mark H Weaver]

Hi,

ludo@gnu.org (Ludovic Courtès) writes:

> Alex Vong <alexvong1995@gmail.com> skribis:
>
>> Mark H Weaver <mhw@netris.org> writes:
>>
>>> Alex Vong <alexvong1995@gmail.com> writes:
>>>
>>>> I get the following error when running ``guix --version | head -n 1''. I
>>>> can get similar after replacing ``--version'' with ``--help''. Also, the
>>>> error is nondeterministic. Any idea?
>>>
>>> Attempts to write to a pipe that has already been closed on the other
>>> end results in EPIPE.  From the write(2) man page:
>>>
>>>   EPIPE fd is connected to a pipe or socket whose reading end is closed.
>>>         When this happens the writing process will also receive a
>>>         SIGPIPE signal.  (Thus, the write return value is seen only if
>>>         the program catches, blocks or ignores this signal.)
>>>
>>> In this case, there's a race condition.  The result depends on whether
>>> "head -n 1" closes its end of the pipe before or after "guix --version"
>>> is finished writing all of its output.  If "head -n 1" closes the pipe
>>> first, then "guix --version" will receive EPIPE while attempting to
>>> write to it.
>>>
>>> What normally happens is that the sending process receives SIGPIPE,
>>> which simply causes it to exit prematurely without ever receiving this
>>> error.  However, since Guix arranges to ignore SIGPIPE in
>>> 'initialize-guix' in guix/ui.scm, we receive EPIPE.
>>>
>>> That's what's happening here.  I'll need to think on how best to fix it.
>>>
>>>      Regards,
>>>        Mark
>>
>> Nice explaination as always! I forget to mention that I reported a bug
>> of similar flavour before <http://bugs.gnu.org/27017>. I agree that
>> thought is needed to fix all instances of this type of bug.
>
> Not sure!  We specifically ignore EPIPE in cases where it matters, such
> as for the output of ‘guix package --search’, ‘guix package -A’, etc.
> In other cases, it’s probably an error, so it’s worth reporting.
>
> WDYT?

I see from the comment in (guix ui) where SIGPIPE is ignored, the
rationale:

  ;; Ignore SIGPIPE.  If the daemon closes the connection, we prefer to be
  ;; notified via an EPIPE later.
  (sigaction SIGPIPE SIG_IGN)

Instead of unconditionally ignoring SIGPIPE here in (initialize-guix),
it might be better to ignore SIGPIPE only if we open a connection to the
daemon with the intent of mutating the store, and perhaps in some other
cases where we're mutating information on disk (e.g. switching
generations).  In those cases, we have a job to do that should ideally
be completed regardless of whether anyone is still listening to our
STDOUT.

However, in many other cases, we don't mutate anything on disk, and our
*only* job is printing information to the user, e.g. when showing
version/usage information, the list of available packages, the list of
generations, etc.  In those cases, I think it would be better to let
SIGPIPE kill us, because there is no reason to keep the 'guix' process
alive if its output is going nowhere.  These are also the cases where
it's most useful to pipe 'guix' output into other commands.

So, I think we should consider removing (sigaction SIGPIPE SIG_IGN) from
(initialize-guix), and instead putting it in various other selected
places.

What do you think?

      Mark

bug#29826: nondeterministic Broken pipe

[Ludovic Courtès]

Hello,

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:

[...]

>> Not sure!  We specifically ignore EPIPE in cases where it matters, such
>> as for the output of ‘guix package --search’, ‘guix package -A’, etc.
>> In other cases, it’s probably an error, so it’s worth reporting.
>>
>> WDYT?
>
> I see from the comment in (guix ui) where SIGPIPE is ignored, the
> rationale:
>
>   ;; Ignore SIGPIPE.  If the daemon closes the connection, we prefer to be
>   ;; notified via an EPIPE later.
>   (sigaction SIGPIPE SIG_IGN)
>
> Instead of unconditionally ignoring SIGPIPE here in (initialize-guix),
> it might be better to ignore SIGPIPE only if we open a connection to the
> daemon with the intent of mutating the store, and perhaps in some other
> cases where we're mutating information on disk (e.g. switching
> generations).  In those cases, we have a job to do that should ideally
> be completed regardless of whether anyone is still listening to our
> STDOUT.
>
> However, in many other cases, we don't mutate anything on disk, and our
> *only* job is printing information to the user, e.g. when showing
> version/usage information, the list of available packages, the list of
> generations, etc.  In those cases, I think it would be better to let
> SIGPIPE kill us, because there is no reason to keep the 'guix' process
> alive if its output is going nowhere.  These are also the cases where
> it's most useful to pipe 'guix' output into other commands.
>
> So, I think we should consider removing (sigaction SIGPIPE SIG_IGN) from
> (initialize-guix), and instead putting it in various other selected
> places.
>
> What do you think?

Why not.  An option would be to move (sigaction SIGPIPE SIG_IGN) to
‘open-connection’, though that’s not following “library design best
practices.”

If we do that, can we really remove the ‘leave-on-EPIPE’ uses that we
have in (guix scripts package) for instance?  At first sight they are in
‘process-query’, which corresponds to operations that don’t rely on the
store, so that should be safe.

There are a few other uses of ‘leave-on-EPIPE’ that happen while the
store is opened (in ‘guix size’, ‘guix challenge’).  We’d have to keep
these.

Thoughts?

Ludo’.

bug#29826: nondeterministic Broken pipe

[Oleg Pykhalov]

[-- Attachment #1: Type: text/plain, Size: 438 bytes --]

Hello Alex,

Alex Vong <alexvong1995@gmail.com> writes:

> I get the following error when running ``guix --version | head -n 1''. I
> can get similar after replacing ``--version'' with ``--help''. Also, the
> error is nondeterministic. Any idea?
>
> $ LC_ALL=C guix --version | head -n 1
> guix (GNU Guix) 91213c384b62bc422896d39005efa922e4fa75f2

As a workaround ‘LC_ALL=C head -n 1 <(guix --version)’.

[…]

Oleg.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]