bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Lars Ingebrigtsen]

If you eval the following, your Emacs will become unusable and you'll
have to kill it (so don't eval it):

(let ((process (start-process
		"foo" (get-buffer-create "*foo*")
		"bash" "-c" "while true; do echo foo; sleep 1; done")))
  (set-process-filter
   process
   (lambda (&rest _)
     (error))))

The reason for this is that when signalling an error from a process
filter, Emacs messages the error message and then seems to sleep for a
second?  I haven't investigated the code yet.

This is something that has bit me more than a few times when working
with process filters (i.e., making a syntax error and then having Emacs
blow up on me).

I'm not sure what solution would be best.  I see two obvious things we
could do: Remove the process filter, so that it doesn't trigger again.
Or -- remove the one-second sleep, which would allow the user to `M-x
list-processes' and kill the offending process.


In GNU Emacs 27.0.50 (build 6, x86_64-pc-linux-gnu, GTK+ Version 3.24.5)
 of 2019-11-01 built on marnie
Repository revision: eda98211e31ed969823c1048b3cde635e08eebe5
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
System Description: Debian GNU/Linux 10 (buster)


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 02 Nov 2019 18:53:57 +0100
> 
> (let ((process (start-process
> 		"foo" (get-buffer-create "*foo*")
> 		"bash" "-c" "while true; do echo foo; sleep 1; done")))
>   (set-process-filter
>    process
>    (lambda (&rest _)
>      (error))))
> 
> The reason for this is that when signalling an error from a process
> filter, Emacs messages the error message and then seems to sleep for a
> second?

Yes, see read_process_output_error_handler (it's actually 2-sec
sleep), and read_and_dispose_of_process_output, where we set up for
calling the process filter.

> I'm not sure what solution would be best.  I see two obvious things we
> could do: Remove the process filter, so that it doesn't trigger again.
> Or -- remove the one-second sleep, which would allow the user to `M-x
> list-processes' and kill the offending process.

I'd suggest to count the number of times a process filter errors out,
and disable it after some configurable number.  Doing that on the
first error sounds too drastic: it could be a one-time spurious error.

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Yes, see read_process_output_error_handler (it's actually 2-sec
> sleep), and read_and_dispose_of_process_output, where we set up for
> calling the process filter.

Ah, right.

> I'd suggest to count the number of times a process filter errors out,
> and disable it after some configurable number.  Doing that on the
> first error sounds too drastic: it could be a one-time spurious error.

Good idea.  Perhaps something time-based -- if it errors out at a rate
that would make Emacs unusable (so more than, say, ten errors per
minute (configurable), would disable the filter)?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 38035@debbugs.gnu.org
> Date: Sat, 02 Nov 2019 19:38:22 +0100
> 
> > I'd suggest to count the number of times a process filter errors out,
> > and disable it after some configurable number.  Doing that on the
> > first error sounds too drastic: it could be a one-time spurious error.
> 
> Good idea.  Perhaps something time-based -- if it errors out at a rate
> that would make Emacs unusable (so more than, say, ten errors per
> minute (configurable), would disable the filter)?

Works for me, thanks.

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

The special code to make errors in filters visible is an important
feature.  It prevents confusion, whereby users don't understand
what is failing because they don't see the error messages.

It also makes possible, as you've pointed out, one more way you can
screw yourself with a perverse Lisp program -- but is it worth trying
to fix that?

-- 
Dr Richard Stallman
Founder, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Lars Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

> It also makes possible, as you've pointed out, one more way you can
> screw yourself with a perverse Lisp program -- but is it worth trying
> to fix that?

This isn't about perverse Lisp programs -- it's about being able to
write code without any trivial error making Emacs blow up.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Sun, 03 Nov 2019 08:21:12 -0500
> Cc: 38035@debbugs.gnu.org
> 
> The special code to make errors in filters visible is an important
> feature.  It prevents confusion, whereby users don't understand
> what is failing because they don't see the error messages.
> 
> It also makes possible, as you've pointed out, one more way you can
> screw yourself with a perverse Lisp program -- but is it worth trying
> to fix that?

The current idea for a fix is to let the error be displayed several
times, until and unless it is clear that the error is repeatedly
signaled with a very high frequency, and if so, disable the filter for
that process.

I think this strikes the right balance between showing the error to
users and still leaving Emacs in a usable state.

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> The current idea for a fix is to let the error be displayed several
> times, until and unless it is clear that the error is repeatedly
> signaled with a very high frequency, and if so, disable the filter for
> that process.
>
> I think this strikes the right balance between showing the error to
> users and still leaving Emacs in a usable state.

I think that's the right solution, too.

Bu I do wonder about the need for the sleep when signalling errors from
a process filter in particular.  For instance

(run-at-time 1 1 (lambda () (error)))

doesn't pause Emacs at all, but just displays the error as normal, which
seems, well, more normal to me.  What's so special about filter errors
that you have to pause Emacs?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rms@gnu.org,  38035@debbugs.gnu.org
> Date: Sun, 03 Nov 2019 17:39:53 +0100
> 
> Bu I do wonder about the need for the sleep when signalling errors from
> a process filter in particular.  For instance
> 
> (run-at-time 1 1 (lambda () (error)))
> 
> doesn't pause Emacs at all, but just displays the error as normal, which
> seems, well, more normal to me.  What's so special about filter errors
> that you have to pause Emacs?

I guess the idea was to make sure the message is seen, not obscured
right away.

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The current idea for a fix is to let the error be displayed several
  > times, until and unless it is clear that the error is repeatedly
  > signaled with a very high frequency, and if so, disable the filter for
  > that process.

  > I think this strikes the right balance between showing the error to
  > users and still leaving Emacs in a usable state.

Maybe it will work well.  I won't argue against.

-- 
Dr Richard Stallman
Founder, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> Bu I do wonder about the need for the sleep when signalling errors from
>> a process filter in particular.  For instance
>> 
>> (run-at-time 1 1 (lambda () (error)))
>> 
>> doesn't pause Emacs at all, but just displays the error as normal, which
>> seems, well, more normal to me.  What's so special about filter errors
>> that you have to pause Emacs?
>
> I guess the idea was to make sure the message is seen, not obscured
> right away.

But we don't do this with errors that happen in other circumstances --
just the filter errors, I think?

(And some errors are so annoying to deal with that we disabled them
immediately if they happen.  For instance, if a function in
post-command-hook happens, we just remove the function immediately.)

So I think the sleep in the filter handling should be removed.  (This is
in addition to disabling the filter upon some threshold or other.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rms@gnu.org,  38035@debbugs.gnu.org
> Date: Fri, 08 Nov 2019 21:42:35 +0100
> 
> > I guess the idea was to make sure the message is seen, not obscured
> > right away.
> 
> But we don't do this with errors that happen in other circumstances --
> just the filter errors, I think?

Because reception of process output is not necessarily a frequent
event, so we want to be sure the user had enough time to see the
message.

> (And some errors are so annoying to deal with that we disabled them
> immediately if they happen.  For instance, if a function in
> post-command-hook happens, we just remove the function immediately.)

Post command hook is called much more frequently.

> So I think the sleep in the filter handling should be removed.  (This is
> in addition to disabling the filter upon some threshold or other.)

I see no reason.  If the mechanism of disabling such a filter will
work reasonably well, the sleep will annoy a few times, and then go
away.

I understand that it pissed you off, but the situation where it
happened will be resolved by the automatic disabling, so your past
annoyance doesn't need to affect anything else.  IOW, I think you are
over-reacting here.

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> But we don't do this with errors that happen in other circumstances --
>> just the filter errors, I think?
>
> Because reception of process output is not necessarily a frequent
> event, so we want to be sure the user had enough time to see the
> message.

But why filters in particular?  As I said, we don't do this with any
other errors, including run-at-time (which is usually not a frequent
thing at all).

> I understand that it pissed you off, but the situation where it
> happened will be resolved by the automatic disabling, so your past
> annoyance doesn't need to affect anything else.  IOW, I think you are
> over-reacting here.

I'm not pissed off?  :-)

I just think that Emacs should behave consistently.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rms@gnu.org,  38035@debbugs.gnu.org
> Date: Fri, 08 Nov 2019 21:54:57 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> But we don't do this with errors that happen in other circumstances --
> >> just the filter errors, I think?
> >
> > Because reception of process output is not necessarily a frequent
> > event, so we want to be sure the user had enough time to see the
> > message.
> 
> But why filters in particular?  As I said, we don't do this with any
> other errors, including run-at-time (which is usually not a frequent
> thing at all).

Maybe we should do the same with timer functions, indeed.

> I'm not pissed off?  :-)

Sorry.

> I just think that Emacs should behave consistently.

If doing the same with timers will contribute to that consistency, I
don't object.  I just think that we shouldn't remove an old feature
because of a single accident that should be quite rare.

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Maybe we should do the same with timer functions, indeed.

And post-command-hook and all the other errors?

>> I just think that Emacs should behave consistently.
>
> If doing the same with timers will contribute to that consistency, I
> don't object.  I just think that we shouldn't remove an old feature
> because of a single accident that should be quite rare.

What do you mean by "single accident"?  

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rms@gnu.org,  38035@debbugs.gnu.org
> Date: Sat, 09 Nov 2019 21:34:38 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Maybe we should do the same with timer functions, indeed.
> 
> And post-command-hook and all the other errors?

IMO no, because post-command-hook isn't asynchronous.

> >> I just think that Emacs should behave consistently.
> >
> > If doing the same with timers will contribute to that consistency, I
> > don't object.  I just think that we shouldn't remove an old feature
> > because of a single accident that should be quite rare.
> 
> What do you mean by "single accident"?  

The one that you cited, and which started this.

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> What do you mean by "single accident"?  
>
> The one that you cited, and which started this.

I posted a proof of concept.  I'm seeing this problem all the time when
working with filters.

Perhaps you just don't work with filters that much?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rms@gnu.org,  38035@debbugs.gnu.org
> Date: Thu, 14 Nov 2019 10:56:40 +0100
> 
> Perhaps you just don't work with filters that much?

Not with filters that signal errors, no.

But I thought the solution for the filters was already agreed upon?

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> Perhaps you just don't work with filters that much?
>
> Not with filters that signal errors, no.

Ah, somebody who writes perfect code on the first attempt.  That's
always nice.  :-)

> But I thought the solution for the filters was already agreed upon?

Sure.  I just want to remove the two-second sleep, too, but it's a
separate issue, really.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38035: 27.0.50; Trivial errors in process filters can render Emacs unusable

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> If you eval the following, your Emacs will become unusable and you'll
> have to kill it (so don't eval it):

This was fixed for bug#19457 by introducing a new variable.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no