[PATCH] Unconditional quit on SIGUSR2

[PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

[-- Attachment #1.1: Type: text/plain, Size: 1550 bytes --]

Sometimes Emacs gets stuck in places where quitting is suppressed,
such as during redisplay.  For most users, the only way to deal with
this situation is to abruptly terminate Emacs.

This patch allows users to recover from this situation by sending Emacs
a SIGUSR2 signal.  When we receive it, we set debug-on-quit to t,
inhibit-quit to nil, and quit-flag to t.  These operations will force
Emacs to stop at the next possible opportunity and display a backtrace.
 From here, users can save their work, and developers can figure out
what caused the hang.

This functionality cannot be achieved by simply modifying
special-event-map: event handled by this keymap are delivered too late
to cancel some loops.  For example, even after (define-key
special-event-map [sigusr2] #'debug), SIGUSR2 will not break out of a
(while t) loop.

Instead, we introduce a new sigusr2-action variable that controls the
operation of the signal handler at a lower level.  If this variable has
value 'quit, then when we receive SIGUSR2, we perform the operations we
described above and eat the signal.  Otherwise, we generate an event and
send it through special-event-map as before.  In order to be maximally
useful to users encountering unexpected infinite loops, we set
sigusr2-action to 'quit by default.

SIGUSR2 is chosen under the assumption that users would use SIGUSR1
instead of SIGUSR2 to implement any custom functionality based on
receiving signals through special-event-map, reducing the likelihood of
breaking existing code.


[-- Attachment #1.2: debugsig.patch --]
[-- Type: text/plain, Size: 4084 bytes --]

=== modified file 'doc/emacs/trouble.texi'
--- doc/emacs/trouble.texi	2011-01-25 04:08:28 +0000
+++ doc/emacs/trouble.texi	2011-03-28 06:53:07 +0000
@@ -812,6 +812,15 @@
 This backtrace is useful for debugging such long loops, so if you can
 produce it, copy it into the bug report.
 
+@vindex sigusr2-action
+If normal quitting does nothing and you are using a system that
+supports signals, then as a last resort, you can try sending Emacs the
+SIGUSR2 signal.  When Emacs receives this signal, it checks whether
+@code{sigusr2-action} is equal to @code{quit}.  If so, Emacs stops
+what it's doing, sets @code{debug-on-quit} to @code{t}, and quits just
+as if you had typed @kbd{C-g}.  This process happens even in places
+where quitting is normally not allowed.
+
 @item
 Check whether any programs you have loaded into the Lisp world,
 including your @file{.emacs} file, set any variables that may affect the

=== modified file 'doc/lispref/debugging.texi'
--- doc/lispref/debugging.texi	2011-01-25 04:08:28 +0000
+++ doc/lispref/debugging.texi	2011-03-28 06:54:07 +0000
@@ -185,6 +185,19 @@
 when you quit.  @xref{Quitting}.
 @end defopt
 
+@defopt sigusr2-action
+On systems with signal support, this variable determines what happens
+when Emacs receives SIGUSR2.  When its value is @code{quit}, Emacs
+will quit just as if @kbd{C-g} had been pressed, except that Emacs
+will first set @code{debug-on-quit} to @code{t} and
+@code{inhibit-quit} to @code{nil} so that the quit is not ignored.
+When @code{sigusr2-action} has any other value, Emacs will handle the
+signal as specified in @code{special-event-map} (@pxref{Active
+Keymaps}).  This feature is useful for terminating infinite loops in
+places where quits would normally be ignored, such as in code that
+runs during redisplay.
+@end defopt
+
 @node Function Debugging
 @subsection Entering the Debugger on a Function Call
 @cindex function call debugging

=== modified file 'lisp/cus-start.el'
--- lisp/cus-start.el	2011-03-27 10:55:07 +0000
+++ lisp/cus-start.el	2011-03-28 06:04:18 +0000
@@ -259,6 +259,10 @@
 	     (suggest-key-bindings keyboard (choice (const :tag "off" nil)
 						    (integer :tag "time" 2)
 						    (other :tag "on")))
+             (sigusr2-action debug
+			    (choice (const :tag "Handle SIGUSR2 normally")
+                                    (const :tag "Quit on SIGUSR2" quit))
+                            "24.1")
 
 ;; This is not good news because it will use the wrong
 ;; version-specific directories when you upgrade.  We need

=== modified file 'src/keyboard.c'
--- src/keyboard.c	2011-03-27 02:27:11 +0000
+++ src/keyboard.c	2011-03-28 06:46:50 +0000
@@ -7168,6 +7168,21 @@
 
   SIGNAL_THREAD_CHECK (sig);
 
+#ifdef SIGUSR2
+  if (sig == SIGUSR2)
+    {
+      /* We can be called at any time, but because variable assignment
+         is atomic, any races are harmless.  */
+      if (Vsigusr2_action == Qquit)
+        {
+          debug_on_quit = 1;
+          Vquit_flag = Qt;
+          Vinhibit_quit = Qnil;
+          return;
+        }
+    }
+#endif /* SIGUSR2 */
+
   for (p = user_signals; p; p = p->next)
     if (p->sig == sig)
       {
@@ -11356,6 +11371,10 @@
   poll_suppress_count = 1;
   start_polling ();
 #endif
+
+#ifdef SIGUSR2
+  Vsigusr2_action = Qquit;
+#endif
 }
 
 /* This type's only use is in syms_of_keyboard, to initialize the
@@ -12178,6 +12197,19 @@
 `deactivate-mark' call uses this to set the window selection.  */);
   Vsaved_region_selection = Qnil;
 
+#ifdef SIGUSR2
+  DEFVAR_LISP ("sigusr2-action",
+               Vsigusr2_action,
+               doc: /* Controls what Emacs does when it receives a
+SIGUSR2.  If 'quit, set `inhibit-quit' to nil and `quit-flag' to t,
+quitting unconditionally at the next opportunity. Any other value
+causes nothing to be done.
+
+In all cases, Emacs generates a `sigusr2' event as it normally
+would. See `special-event-map'.  */);
+  Vsigusr2_action = Qnil;
+#endif
+
   /* Create the initial keyboard. */
   initial_kboard = (KBOARD *) xmalloc (sizeof (KBOARD));
   init_kboard (initial_kboard);


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH] Unconditional quit on SIGUSR2

[Eli Zaretskii]

> Date: Mon, 28 Mar 2011 00:14:22 -0700
> From: Daniel Colascione <dan.colascione@gmail.com>
> 
> Sometimes Emacs gets stuck in places where quitting is suppressed,
> such as during redisplay.  For most users, the only way to deal with
> this situation is to abruptly terminate Emacs.
> 
> This patch allows users to recover from this situation by sending Emacs
> a SIGUSR2 signal.  When we receive it, we set debug-on-quit to t,
> inhibit-quit to nil, and quit-flag to t.  These operations will force
> Emacs to stop at the next possible opportunity and display a backtrace.
>  From here, users can save their work, and developers can figure out
> what caused the hang.

If the goal is to provide a way for the user to interrupt a stuck
Emacs in a way that allows to save edits or debug Emacs, then users
can already send SIGTERM and developers can already attach a debugger,
and that will achieve the same result.  Why do we need a new machinery
to do something that is already possible via essentially the same
sequence of actions, i.e. type "kill EMACS-PID" at the shell prompt?

Or am I missing something?

Btw, did you actually test the possibility to interrupt a stuck
redisplay using this patch?  I'm asking because I think redisplay
cannot be interrupted like that: it never checks the quit flag, and
for a good reason AFAIK -- you cannot interrupt it at an arbitrary
point without disastrous consequences.  If we want to make redisplay
interruptible, we need to add code to it in strategic places.  Again,
I don't see a need for doing this, unless someone shows a convincing
use case where such a feature would be useful.

Apologies if I missed something obvious, and thanks for working on
this.

Re: [PATCH] Unconditional quit on SIGUSR2

[Julien Danjou]

[-- Attachment #1: Type: text/plain, Size: 384 bytes --]

On Mon, Mar 28 2011, Eli Zaretskii wrote:

> Or am I missing something?

I can't judge the internal, but I already had Emacs hanging somewhere
for ever, and cannot do anything but SIGKILL it.
So if that code helps in that regards, it may be worth it.

But again, I don't know the internal. Just my user-dev point of view.

-- 
Julien Danjou
❱ http://julien.danjou.info

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] Unconditional quit on SIGUSR2

[Eli Zaretskii]

> From: Julien Danjou <julien@danjou.info>
> Cc: Daniel Colascione <dan.colascione@gmail.com>,  emacs-devel@gnu.org
> Date: Mon, 28 Mar 2011 14:32:02 +0200
> 
> I can't judge the internal, but I already had Emacs hanging somewhere
> for ever, and cannot do anything but SIGKILL it.

Maybe I'm mistaken, but AFAIK if SIGTERM cannot interrupt Emacs,
neither will SIGUSR2.

Re: [PATCH] Unconditional quit on SIGUSR2

[Julien Danjou]

[-- Attachment #1: Type: text/plain, Size: 364 bytes --]

On Mon, Mar 28 2011, Eli Zaretskii wrote:

> Maybe I'm mistaken, but AFAIK if SIGTERM cannot interrupt Emacs,
> neither will SIGUSR2.

My understanding was that this patch would help in this regard. Maybe
there's no need to add a different signal for that though.

Or maybe I misunderstood completely.

-- 
Julien Danjou
❱ http://julien.danjou.info

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

[-- Attachment #1.1: Type: text/plain, Size: 2977 bytes --]

On 3/28/11 4:48 AM, Eli Zaretskii wrote:
>> Date: Mon, 28 Mar 2011 00:14:22 -0700
>> From: Daniel Colascione <dan.colascione@gmail.com>
>>
>> Sometimes Emacs gets stuck in places where quitting is suppressed,
>> such as during redisplay.  For most users, the only way to deal with
>> this situation is to abruptly terminate Emacs.
>>
>> This patch allows users to recover from this situation by sending Emacs
>> a SIGUSR2 signal.  When we receive it, we set debug-on-quit to t,
>> inhibit-quit to nil, and quit-flag to t.  These operations will force
>> Emacs to stop at the next possible opportunity and display a backtrace.
>>  From here, users can save their work, and developers can figure out
>> what caused the hang.
> 
> If the goal is to provide a way for the user to interrupt a stuck
> Emacs in a way that allows to save edits or debug Emacs, then users
> can already send SIGTERM and developers can already attach a debugger,
> and that will achieve the same result.  Why do we need a new machinery
> to do something that is already possible via essentially the same
> sequence of actions, i.e. type "kill EMACS-PID" at the shell prompt?
> 
> Or am I missing something?

Killing Emacs destroys any transient state. The last-resort-quit
approach doesn't.  Attaching a debugger is well and good, but one isn't
always available, and even if one is, symbols for Emacs aren't always on
the system, as is the case for a typical Emacs distribution package.

Providing this facility (which does nothing unless you use it) can let
users recover work and provide more detailed bug reports, as well as
help us ferret out the cause of hard-to-reproduce lockups in font-lock
code without having to run under a debugger all the time.

> I'm asking because I think redisplay
> cannot be interrupted like that: it never checks the quit flag, and
> for a good reason AFAIK -- you cannot interrupt it at an arbitrary
> point without disastrous consequences.

Of course we can't quit just anywhere.  All the king's horses and all
the king's men couldn't safely break Emacs out of arbitrary C code ---
but we don't have to: the idea is to be able to quit in reasonable
places, e.g. where could insert a (debug) in Lisp.  This way, we can
hope to interrupt runaway font-lock code, and for this application, my
proposed approach does indeed work.  If I could have implemented this
feature without introducing additional machinery, I would have.

If you're modifying the core redisplay code and introduce a hang, you
probably have a debugger available.  OTOH, it's perfectly reasonable to
tweak font-lock code without one.

I've attached a new version of my patch that changes the action name
from "quit" to "debug" instead of "quit" to reflect what we actually do.
 I've also made the code attempt to regain control in another way.  This
version successfully breaks out of font-lock code running during
redisplay when C-g does not.

[-- Attachment #1.2: debugsig.patch --]
[-- Type: text/plain, Size: 4505 bytes --]

=== modified file 'doc/emacs/trouble.texi'
--- doc/emacs/trouble.texi	2011-01-25 04:08:28 +0000
+++ doc/emacs/trouble.texi	2011-03-28 13:04:49 +0000
@@ -812,6 +812,15 @@
 This backtrace is useful for debugging such long loops, so if you can
 produce it, copy it into the bug report.
 
+@vindex sigusr2-action
+If normal quitting does nothing and you are using a system that
+supports signals, then as a last resort, you can try sending Emacs the
+SIGUSR2 signal.  When Emacs receives this signal, it checks whether
+@code{sigusr2-action} is equal to @code{debug}.  If so, Emacs stops
+what it's doing, sets @code{debug-on-quit} to @code{t}, and tries to
+break into the debugger in a variety of ways.  This process happens
+even in places where quitting is normally not allowed.
+
 @item
 Check whether any programs you have loaded into the Lisp world,
 including your @file{.emacs} file, set any variables that may affect the

=== modified file 'doc/lispref/debugging.texi'
--- doc/lispref/debugging.texi	2011-01-25 04:08:28 +0000
+++ doc/lispref/debugging.texi	2011-03-28 13:06:25 +0000
@@ -185,6 +185,19 @@
 when you quit.  @xref{Quitting}.
 @end defopt
 
+@defopt sigusr2-action
+On systems with signal support, this variable determines what happens
+when Emacs receives SIGUSR2.  When its value is @code{debug}, Emacs
+will attempt to break into the Lisp debugger in a variety of ways,
+setting @code{debug-on-quit} to @code{t} as a side effect.
+
+When @code{sigusr2-action} has any other value, Emacs will handle the
+signal as specified in @code{special-event-map} (@pxref{Active
+Keymaps}).  This feature is useful for terminating infinite loops in
+places where quits would normally be ignored, such as in code that
+runs during redisplay.
+@end defopt
+
 @node Function Debugging
 @subsection Entering the Debugger on a Function Call
 @cindex function call debugging

=== modified file 'lisp/cus-start.el'
--- lisp/cus-start.el	2011-03-27 10:55:07 +0000
+++ lisp/cus-start.el	2011-03-28 13:12:30 +0000
@@ -259,6 +259,10 @@
 	     (suggest-key-bindings keyboard (choice (const :tag "off" nil)
 						    (integer :tag "time" 2)
 						    (other :tag "on")))
+             (sigusr2-action debug
+			    (choice (const :tag "Handle SIGUSR2 normally")
+                                    (const :tag "Quit on SIGUSR2" debug))
+                            "24.1")
 
 ;; This is not good news because it will use the wrong
 ;; version-specific directories when you upgrade.  We need

=== modified file 'src/keyboard.c'
--- src/keyboard.c	2011-03-27 02:27:11 +0000
+++ src/keyboard.c	2011-03-28 13:11:48 +0000
@@ -7168,6 +7168,22 @@
 
   SIGNAL_THREAD_CHECK (sig);
 
+#ifdef SIGUSR2
+  if (sig == SIGUSR2)
+    {
+      /* We can be called at any time, but because variable assignment
+         is atomic, any races are harmless.  */
+      if (Vsigusr2_action == Qdebug)
+        {
+          debug_on_next_call = 1;
+          debug_on_quit = 1;
+          Vquit_flag = Qt;
+          Vinhibit_quit = Qnil;
+          return;
+        }
+    }
+#endif /* SIGUSR2 */
+
   for (p = user_signals; p; p = p->next)
     if (p->sig == sig)
       {
@@ -11356,6 +11372,10 @@
   poll_suppress_count = 1;
   start_polling ();
 #endif
+
+#ifdef SIGUSR2
+  Vsigusr2_action = Qdebug;
+#endif
 }
 
 /* This type's only use is in syms_of_keyboard, to initialize the
@@ -12178,6 +12198,19 @@
 `deactivate-mark' call uses this to set the window selection.  */);
   Vsaved_region_selection = Qnil;
 
+#ifdef SIGUSR2
+  DEFVAR_LISP ("sigusr2-action",
+               Vsigusr2_action,
+               doc: /* Controls what Emacs does when it receives a
+SIGUSR2.  If 'quit, set `inhibit-quit' to nil and `quit-flag' to t,
+quitting unconditionally at the next opportunity. Any other value
+causes nothing to be done.
+
+In all cases, Emacs generates a `sigusr2' event as it normally
+would. See `special-event-map'.  */);
+  Vsigusr2_action = Qnil;
+#endif
+
   /* Create the initial keyboard. */
   initial_kboard = (KBOARD *) xmalloc (sizeof (KBOARD));
   init_kboard (initial_kboard);

=== modified file 'src/lisp.h'
--- src/lisp.h	2011-03-27 02:27:11 +0000
+++ src/lisp.h	2011-03-28 13:06:41 +0000
@@ -2814,7 +2814,7 @@
 
 /* Defined in eval.c */
 extern Lisp_Object Qautoload, Qexit, Qinteractive, Qcommandp, Qdefun, Qmacro;
-extern Lisp_Object Qinhibit_quit;
+extern Lisp_Object Qinhibit_quit, Qdebug;
 extern Lisp_Object Vautoload_queue;
 extern Lisp_Object Vsignaling_function;
 extern int handling_signal;


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH] Unconditional quit on SIGUSR2

[Eli Zaretskii]

> Date: Mon, 28 Mar 2011 06:38:39 -0700
> From: Daniel Colascione <dan.colascione@gmail.com>
> CC: emacs-devel@gnu.org
> 
> > If the goal is to provide a way for the user to interrupt a stuck
> > Emacs in a way that allows to save edits or debug Emacs, then users
> > can already send SIGTERM and developers can already attach a debugger,
> > and that will achieve the same result.  Why do we need a new machinery
> > to do something that is already possible via essentially the same
> > sequence of actions, i.e. type "kill EMACS-PID" at the shell prompt?
> > 
> > Or am I missing something?
> 
> Killing Emacs destroys any transient state. The last-resort-quit
> approach doesn't.

You originally said "save edits or debug Emacs", see above.

What transient state are we talking about?

> Attaching a debugger is well and good, but one isn't
> always available, and even if one is, symbols for Emacs aren't always on
> the system, as is the case for a typical Emacs distribution package.

I very much doubt that having a Lisp level backtrace in this situation
would help.  It would typically include illegible byte codes at
crucial places.

> Providing this facility (which does nothing unless you use it)

No feature costs nothing.  At the very least, we need to document it
(thanks for doing it as part of the patch, btw), maintain it, adapt it
to future platforms and changes in existing platforms, etc.  For a
highly platform-dependent feature such as SIGUSR2, these are
non-trivial efforts whose costs should be considered against the
gains.

> can let users recover work and provide more detailed bug reports, as
> well as help us ferret out the cause of hard-to-reproduce lockups in
> font-lock code without having to run under a debugger all the time.

I'd like to hear what work can be recovered this way that is not
recovered by auto-save at fatal signal time.

And if the main problem is with font-lock related lockups, perhaps we
should add something to font-lock to alleviate that.  That alternative
would have a benefit that it doesn't limit the solution to platforms
that support SIGUSR2.

> This way, we can hope to interrupt runaway font-lock code, and for
> this application, my proposed approach does indeed work.  If I could
> have implemented this feature without introducing additional
> machinery, I would have.

What prevents the implementation on the Lisp level of a feature that
could interrupt a runaway font-lock?

Re: [PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 4484 bytes --]

On 3/28/11 7:47 AM, Eli Zaretskii wrote:
>> Date: Mon, 28 Mar 2011 06:38:39 -0700
>> From: Daniel Colascione <dan.colascione@gmail.com>
>> CC: emacs-devel@gnu.org
>>
>>> If the goal is to provide a way for the user to interrupt a stuck
>>> Emacs in a way that allows to save edits or debug Emacs, then users
>>> can already send SIGTERM and developers can already attach a debugger,
>>> and that will achieve the same result.  Why do we need a new machinery
>>> to do something that is already possible via essentially the same
>>> sequence of actions, i.e. type "kill EMACS-PID" at the shell prompt?
>>>
>>> Or am I missing something?
>>
>> Killing Emacs destroys any transient state. The last-resort-quit
>> approach doesn't.
> 
> You originally said "save edits or debug Emacs", see above.
> 
> What transient state are we talking about?

I'm talking about any state not already written out to more permanent
storage --- that includes not only unsaved buffer contents, but tramp
sessions, window configurations, various histories, and so on.  All that
can be recreated, but losing this state and having to recreate it  is
very inconvenient.

>> Attaching a debugger is well and good, but one isn't
>> always available, and even if one is, symbols for Emacs aren't always on
>> the system, as is the case for a typical Emacs distribution package.
> 
> I very much doubt that having a Lisp level backtrace in this situation
> would help.  It would typically include illegible byte codes at
> crucial places.

Even "illegible byte codes" can be quite useful for narrowing the
problem down to the function level, and getting a detailed traceback
from a reproducible hang could be a matter of find-function and
eval-buffer --- steps that, if need be, can be taken by uninitiated
users without the installation of a debugger.

(Speaking of debugging bytecode, by the way: would it be feasible to
someday provide the ability to step through compiled functions
bytecode-by-bytecode?)

>> Providing this facility (which does nothing unless you use it)
> 
> No feature costs nothing.  At the very least, we need to document it
> (thanks for doing it as part of the patch, btw), maintain it, adapt it
> to future platforms and changes in existing platforms, etc.  For a
> highly platform-dependent feature such as SIGUSR2, these are
> non-trivial efforts whose costs should be considered against the
> gains.

Sure --- I'm merely pointing out that the feature does nothing *at
runtime* unless used.

>> can let users recover work and provide more detailed bug reports, as
>> well as help us ferret out the cause of hard-to-reproduce lockups in
>> font-lock code without having to run under a debugger all the time.
> 
> I'd like to hear what work can be recovered this way that is not
> recovered by auto-save at fatal signal time.

Auto-save doesn't run continuously, and not everyone has it turned on.

> And if the main problem is with font-lock related lockups, perhaps we
> should add something to font-lock to alleviate that.  That alternative
> would have a benefit that it doesn't limit the solution to platforms
> that support SIGUSR2.

I've long wished with-timeout would put a hard limit on execution time
(to within QUIT granularity).  This hard timeout capability would
provide a way to automatically cancel runaway fontification.  On the
other hand, this feature would be difficult to implement,
platform-dependent, and not applicable to more general cases.  How long
is "too long"?

>> This way, we can hope to interrupt runaway font-lock code, and for
>> this application, my proposed approach does indeed work.  If I could
>> have implemented this feature without introducing additional
>> machinery, I would have.
> 
> What prevents the implementation on the Lisp level of a feature that
> could interrupt a runaway font-lock?

What Lisp-level method do you propose for breaking out of (let
((inhibit-quit t)) (while t)) ?  ISTR a proposal that would have C-g C-g
C-g ignore inhibit-quit, but this approach would still allow a user
leaning on C-g to cancel legitimate background work.  AIUI, this problem
was the reason quit was disabled during fontification in the first
place.  A signal doesn't confuse the issue.

I agree that it'd be better to provide a portable way to break, but a
signal is a simple and relatively portable mechanism that doesn't depend
on the IO loop.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH] Unconditional quit on SIGUSR2

[Eli Zaretskii]

> Date: Mon, 28 Mar 2011 10:23:59 -0700
> From: Daniel Colascione <dan.colascione@gmail.com>
> CC: emacs-devel@gnu.org
> 
> >> Killing Emacs destroys any transient state. The last-resort-quit
> >> approach doesn't.
> > 
> > You originally said "save edits or debug Emacs", see above.
> > 
> > What transient state are we talking about?
> 
> I'm talking about any state not already written out to more permanent
> storage --- that includes not only unsaved buffer contents, but tramp
> sessions, window configurations, various histories, and so on.

Isn't that part of handling a fatal signal?

> (Speaking of debugging bytecode, by the way: would it be feasible to
> someday provide the ability to step through compiled functions
> bytecode-by-bytecode?)

I'm not sure I understand what you are suggesting here.  What I would
love is the ability to "disassemble" a Lisp backtrace, so that any
byte codes are converted into human-readable Lisp (or something
similar).  Perhaps there's already such a thing, who knows?

> >> can let users recover work and provide more detailed bug reports, as
> >> well as help us ferret out the cause of hard-to-reproduce lockups in
> >> font-lock code without having to run under a debugger all the time.
> > 
> > I'd like to hear what work can be recovered this way that is not
> > recovered by auto-save at fatal signal time.
> 
> Auto-save doesn't run continuously, and not everyone has it turned on.

I meant the automatic auto-save triggered by catching a fatal signal.
See emacs.c:fatal_error_signal.

> >> This way, we can hope to interrupt runaway font-lock code, and for
> >> this application, my proposed approach does indeed work.  If I could
> >> have implemented this feature without introducing additional
> >> machinery, I would have.
> > 
> > What prevents the implementation on the Lisp level of a feature that
> > could interrupt a runaway font-lock?
> 
> What Lisp-level method do you propose for breaking out of (let
> ((inhibit-quit t)) (while t)) ?

This isn't an interesting example.  I was talking about font-lock,
which I understand was the primary motivation for this patch.
Font-lock does not have such useless loops (I hope ;-).

I was asking about whatever you had in mind when you wished you could
implement such a feature without introducing additional machinery.
You tell me.

Re: [PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

On 3/28/2011 11:37 AM, Eli Zaretskii wrote:
>>>> Killing Emacs destroys any transient state. The last-resort-quit
>>>> approach doesn't.
>>> You originally said "save edits or debug Emacs", see above.
>>>
>>> What transient state are we talking about?
>> I'm talking about any state not already written out to more permanent
>> storage --- that includes not only unsaved buffer contents, but tramp
>> sessions, window configurations, various histories, and so on.
> Isn't that part of handling a fatal signal?

Right --- but with a patch like mine, a lockup in font-lock or some 
other obscure corner of the codebase isn't fatal anymore.

>> (Speaking of debugging bytecode, by the way: would it be feasible to
>> someday provide the ability to step through compiled functions
>> bytecode-by-bytecode?)
> I'm not sure I understand what you are suggesting here.  What I would
> love is the ability to "disassemble" a Lisp backtrace, so that any
> byte codes are converted into human-readable Lisp (or something
> similar).  Perhaps there's already such a thing, who knows?

There isn't, to my knowledge.  What I'd like to see is the ability to 
view and step through Emacs disassembled bytecode the same way we can 
view and step through disassembled machine code in gdb.  I don't know 
how  complex the implementation of such a feature would be, but we do 
have a disassembled in Emacs to start.

>>>> can let users recover work and provide more detailed bug reports, as
>>>> well as help us ferret out the cause of hard-to-reproduce lockups in
>>>> font-lock code without having to run under a debugger all the time.
>>> I'd like to hear what work can be recovered this way that is not
>>> recovered by auto-save at fatal signal time.
>> Auto-save doesn't run continuously, and not everyone has it turned on.
> I meant the automatic auto-save triggered by catching a fatal signal.
> See emacs.c:fatal_error_signal.

Ah, I didn't know about that.  It'd still be nice to be able to recover 
a hung Emacs process instead of killing it, and my point about having to 
restore other transient state still applies.

>> What Lisp-level method do you propose for breaking out of (let
>> ((inhibit-quit t)) (while t)) ?
> This isn't an interesting example.  I was talking about font-lock,
> which I understand was the primary motivation for this patch.
> Font-lock does not have such useless loops (I hope ;-).

It shouldn't, but there are and always will be bugs that result in 
essentially identical behavior.

> I was asking about whatever you had in mind when you wished you could
> implement such a feature without introducing additional machinery.
> You tell me.

We want to be able to interrupt code running in a tight loop in 
situations when quit would normally be disabled, such as during 
redisplay.  Quit is disabled during background work for a good reason, 
so we shouldn't just rely on the normal quit mechanism.  Thus, we need 
some other way of signaling Emacs.  If events aren't being delivered, as 
in a tight loop, then commands won't be run and anything bound normally 
to some keymap will be useless.  One way of delivering an event to Emacs 
that does not depend on the command loop is the signal mechanism.  There 
is a facility to bind a command to a signal via `special-event-map'.  
This facility is not useful for our purposes here because it works by 
having signals post events to the command loop, which, as we've already 
established, is not running.  Except for the quit mechanism itself, 
there is no way to configure Emacs to react asynchronously to some event 
when the command loop will not run, and so there is no way to break into 
a wedged Emacs.  Any implementation of this functionality will require 
changes to the C core.

We have a few options for the implementation strategy:

1) Bind (debug) to a signal in special-event-map, and send Emacs that 
signal to interrupt it.  We would have to change the core such that it 
would run commands associated with pending signals during QUIT.

2) Create a quit-like mechanism in all the ports that would listen for a 
special character.  This character would be interpreted like C-g today, 
except that it would unconditionally break into the debugger.

3) Provide an explicit way to configure the C core to break in response 
to a signal, then send that signal to interrupt Emacs.  This is the 
approach I've implemented.

Option 1 wont work because code today doesn't expect to arbitrary Lisp 
code to run wherever there is a QUIT, and running a command implies just 
that.  Option 2 is more invasive, steals a key, and doesn't add any 
expressive power.  I've implemented option 3, and it works well enough 
so far.

Re: [PATCH] Unconditional quit on SIGUSR2

[Eli Zaretskii]

> Date: Mon, 28 Mar 2011 12:29:29 -0700
> From: Daniel Colascione <dan.colascione@gmail.com>
> CC: emacs-devel@gnu.org
> 
> On 3/28/2011 11:37 AM, Eli Zaretskii wrote:
> >>>> Killing Emacs destroys any transient state. The last-resort-quit
> >>>> approach doesn't.
> >>> You originally said "save edits or debug Emacs", see above.
> >>>
> >>> What transient state are we talking about?
> >> I'm talking about any state not already written out to more permanent
> >> storage --- that includes not only unsaved buffer contents, but tramp
> >> sessions, window configurations, various histories, and so on.
> > Isn't that part of handling a fatal signal?
> 
> Right --- but with a patch like mine, a lockup in font-lock or some 
> other obscure corner of the codebase isn't fatal anymore.

We are miscommunicating: I meant that delivering a SIGTERM will end up
in fatal_error_signal, which will save all that's worth saving, before
Emacs commits suicide.  Your patch achieves the same goal, as far as
saving unsaved work is concerned, except it uses SIGUSR2.

> We want to be able to interrupt code running in a tight loop in 
> situations when quit would normally be disabled, such as during 
> redisplay.  Quit is disabled during background work for a good reason, 
> so we shouldn't just rely on the normal quit mechanism.

If it is safe to interrupt font-lock with the method used by your
patch, it should be safe to enable quitting when redisplay calls
font-lock, right?  So maybe we should simply enable quite when
redisplay calls font-lock and disable it when font-lock returns back
to redisplay code which called it.  Then C-g will be able to interrupt
it.  Would that solve the problem with font-lock that gets stuck?

Re: [PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

On 3/28/2011 12:41 PM, Eli Zaretskii wrote:
> We are miscommunicating: I meant that delivering a SIGTERM will end up
> in fatal_error_signal, which will save all that's worth saving, before
> Emacs commits suicide.  Your patch achieves the same goal, as far as
> saving unsaved work is concerned, except it uses SIGUSR2.

My patch does not do the same thing.  Instead, my patch returns the user 
to an *interactive* context where he can continue using Emacs normally 
after resolving whatever was causing the infinite loop (perhaps by 
closing a buffer, or setting some configuration variables).  It is not 
an unconditional quit the same way SIGTERM is.

>> We want to be able to interrupt code running in a tight loop in
>> situations when quit would normally be disabled, such as during
>> redisplay.  Quit is disabled during background work for a good reason,
>> so we shouldn't just rely on the normal quit mechanism.
> If it is safe to interrupt font-lock with the method used by your
> patch, it should be safe to enable quitting when redisplay calls
> font-lock, right?  So maybe we should simply enable quite when
> redisplay calls font-lock and disable it when font-lock returns back
> to redisplay code which called it.  Then C-g will be able to interrupt
> it.  Would that solve the problem with font-lock that gets stuck?

It would solve that problem but introduce another: users have no way of 
knowing went font-lock happens.  Innocently typing C-g at the wrong time 
can terminate font lock and leave parts of the buffer unfontified.  It's 
inherently racy.  We need an alternate mechanism that is not as easy to 
trigger.  Signals provide such a mechanism.

Re: [PATCH] Unconditional quit on SIGUSR2

[Lennart Borgman]

On Mon, Mar 28, 2011 at 9:49 PM, Daniel Colascione
<dan.colascione@gmail.com> wrote:
>>
>> If it is safe to interrupt font-lock with the method used by your
>> patch, it should be safe to enable quitting when redisplay calls
>> font-lock, right?  So maybe we should simply enable quite when
>> redisplay calls font-lock and disable it when font-lock returns back
>> to redisplay code which called it.  Then C-g will be able to interrupt
>> it.  Would that solve the problem with font-lock that gets stuck?
>
> It would solve that problem but introduce another: users have no way of
> knowing went font-lock happens.  Innocently typing C-g at the wrong time can
> terminate font lock and leave parts of the buffer unfontified.  It's
> inherently racy.  We need an alternate mechanism that is not as easy to
> trigger.  Signals provide such a mechanism.

But can't that be taken care of by letting font lock run again?

Re: [PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

On 3/28/2011 12:52 PM, Lennart Borgman wrote:
> But can't that be taken care of by letting font lock run again?

It depends on how fontification is being run.  If we're using jit-lock, 
then I believe we'll try again automatically.  If we're using some other 
approach, then fontification will likely just be canceled until the next 
time that portion of the buffer is modified.  There's also the issue of 
runaway process filters which I believe also run with quit disabled.

Re: [PATCH] Unconditional quit on SIGUSR2

[Lennart Borgman]

On Mon, Mar 28, 2011 at 9:56 PM, Daniel Colascione
<dan.colascione@gmail.com> wrote:
> On 3/28/2011 12:52 PM, Lennart Borgman wrote:
>>
>> But can't that be taken care of by letting font lock run again?
>
> It depends on how fontification is being run.  If we're using jit-lock, then
> I believe we'll try again automatically.  If we're using some other
> approach, then fontification will likely just be canceled until the next
> time that portion of the buffer is modified.  There's also the issue of
> runaway process filters which I believe also run with quit disabled.

Should not all new code use jit-lock for fontification (or the a bit
misname fontification-functions)? Is there a lot of old code in Emacs
not using jit-lock?

Re: [PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

On 3/28/2011 1:06 PM, Lennart Borgman wrote:
> On Mon, Mar 28, 2011 at 9:56 PM, Daniel Colascione
> <dan.colascione@gmail.com>  wrote:
>> On 3/28/2011 12:52 PM, Lennart Borgman wrote:
>>>
>>> But can't that be taken care of by letting font lock run again?
>>
>> It depends on how fontification is being run.  If we're using jit-lock, then
>> I believe we'll try again automatically.  If we're using some other
>> approach, then fontification will likely just be canceled until the next
>> time that portion of the buffer is modified.  There's also the issue of
>> runaway process filters which I believe also run with quit disabled.
>
> Should not all new code use jit-lock for fontification (or the a bit
> misname fontification-functions)? Is there a lot of old code in Emacs
> not using jit-lock?

Until we remove everything besides jit-lock, we have to account for its 
absence.  And it still doesn't address the issue of other background 
work that runs with quit inhibited.

Also, I strongly discourage anyone using fontification-functions 
directly.  Using this function bypass font-lock entirely, breaking 
font-lock-mode, add-font-lock-keywords, highlight-phrase, and other 
Emacs features.  It also adds no expressive power over 
font-lock-keywords, and IMHO, fontification-functions should be 
documented as being for internal use only.

Re: [PATCH] Unconditional quit on SIGUSR2

[Lennart Borgman]

On Mon, Mar 28, 2011 at 10:12 PM, Daniel Colascione
<dan.colascione@gmail.com> wrote:
>>>>
>>>> But can't that be taken care of by letting font lock run again?
>>>
>>> It depends on how fontification is being run.  If we're using jit-lock,
>>> then
>>> I believe we'll try again automatically.  If we're using some other
>>> approach, then fontification will likely just be canceled until the next
>>> time that portion of the buffer is modified.  There's also the issue of
>>> runaway process filters which I believe also run with quit disabled.
>>
>> Should not all new code use jit-lock for fontification (or the a bit
>> misname fontification-functions)? Is there a lot of old code in Emacs
>> not using jit-lock?
>
> Until we remove everything besides jit-lock, we have to account for its
> absence.

I really would not worry about fontification code that does not yet
use jit-lock. Such code has caused me a lot of trouble and moving it
to jit-lock would be very good. (Maybe jit-lock need some enhancement
for some cases, but let us worry about that instead then.)

> And it still doesn't address the issue of other background work
> that runs with quit inhibited.

True of course.

> Also, I strongly discourage anyone using fontification-functions directly.
>  Using this function bypass font-lock entirely, breaking font-lock-mode,
> add-font-lock-keywords, highlight-phrase, and other Emacs features.  It also
> adds no expressive power over font-lock-keywords, and IMHO,
> fontification-functions should be documented as being for internal use only.

Thanks, I misremembered. I am using jit-lock-register.

In praise of font-lock (Was: Re: [PATCH] Unconditional quit on SIGUSR2)

[Daniel Colascione]

On 3/28/2011 1:45 PM, Lennart Borgman wrote:
>> Also, I strongly discourage anyone using fontification-functions directly.
>>   Using this function bypass font-lock entirely, breaking font-lock-mode,
>> add-font-lock-keywords, highlight-phrase, and other Emacs features.  It also
>> adds no expressive power over font-lock-keywords, and IMHO,
>> fontification-functions should be documented as being for internal use only.
>
> Thanks, I misremembered. I am using jit-lock-register.

Well, thanks for not blowing away font-lock.  Still, why did you decide 
to use jit-lock-register instead of a function matcher in 
font-lock-keywords?  A matcher function can be as complex as you want: 
you can have it always return nil to skip font-lock's default logic and 
instead apply patterns using font-lock-apply-highlight.  The nice thing 
about this scheme is that it's portable, it's worked for years, and 
it'll continue to work no matter what fontification scheme we choose. 
It's not any less powerful than a function registered with 
jit-lock-register (which, granted, I haven't actually used) and it 
composes better with other users of font-lock with respect to prepending 
and appending faces.

The neat thing about font-lock is that it allows creating rules using 
many different levels of sophistication.  It makes hard things possible 
and simple things easy. Plain regular expression defaulting to 
font-lock-keyword-face can coexist with recursive descent parsers in the 
same buffer, and mode authors can gradually increase the sophistication 
of their font-locking rules without having to leave the framework.  It's 
not perfect, particularly with respect to multiline matches, but it 
generally works pretty well and I don't see a good reason to bypass it.

Re: In praise of font-lock (Was: Re: [PATCH] Unconditional quit on SIGUSR2)

[Lennart Borgman]

On Mon, Mar 28, 2011 at 11:13 PM, Daniel Colascione
<dan.colascione@gmail.com> wrote:
> On 3/28/2011 1:45 PM, Lennart Borgman wrote:
>>>
>>> Also, I strongly discourage anyone using fontification-functions
>>> directly.
>>>  Using this function bypass font-lock entirely, breaking font-lock-mode,
>>> add-font-lock-keywords, highlight-phrase, and other Emacs features.  It
>>> also
>>> adds no expressive power over font-lock-keywords, and IMHO,
>>> fontification-functions should be documented as being for internal use
>>> only.
>>
>> Thanks, I misremembered. I am using jit-lock-register.
>
> Well, thanks for not blowing away font-lock.  Still, why did you decide to
> use jit-lock-register instead of a function matcher in font-lock-keywords?

I am using this in wrap-to-fill.el in nXhtml to add 'wrap-prefix to
lines wrapped by visual-line-mode.

I got the advice from Stefan to use jit-lock-register directly and it
is simple and works very good. The net effect is that it works even if
font-lock is not on.

>  A matcher function can be as complex as you want: you can have it always
> return nil to skip font-lock's default logic and instead apply patterns
> using font-lock-apply-highlight.  The nice thing about this scheme is that
> it's portable, it's worked for years, and it'll continue to work no matter
> what fontification scheme we choose. It's not any less powerful than a
> function registered with jit-lock-register (which, granted, I haven't
> actually used) and it composes better with other users of font-lock with
> respect to prepending and appending faces.
>
> The neat thing about font-lock is that it allows creating rules using many
> different levels of sophistication.  It makes hard things possible and
> simple things easy. Plain regular expression defaulting to
> font-lock-keyword-face can coexist with recursive descent parsers in the
> same buffer, and mode authors can gradually increase the sophistication of
> their font-locking rules without having to leave the framework.  It's not
> perfect, particularly with respect to multiline matches, but it generally
> works pretty well and I don't see a good reason to bypass it.
>

Re: [PATCH] Unconditional quit on SIGUSR2

[chad]

On Mar 28, 2011, at 12:56 PM, Daniel Colascione wrote:

> On 3/28/2011 12:52 PM, Lennart Borgman wrote:
>> But can't that be taken care of by letting font lock run again?
> 
> It depends on how fontification is being run.  If we're using jit-lock, then I believe we'll try again automatically.  If we're using some other approach, then fontification will likely just be canceled until the next time that portion of the buffer is modified. 

It also doesn't help the problem where emacs needs to read the user's mind to figure out if the user wanted to cancel font-lock or whatever other process was running. Experience with predecessors to font-lock that were interrupted by C-g (so, admittedly quite old) suggest that emacs will guess wrong often enough that users will (did) find the colorizing not worth the headache. We might be able to do better now, but it isn't obvious to me how, and C-g handling is already pretty quirky when you consider all of the platforms, IIRC.

*Chad

Re: [PATCH] Unconditional quit on SIGUSR2

[Eli Zaretskii]

> Date: Mon, 28 Mar 2011 12:49:53 -0700
> From: Daniel Colascione <dan.colascione@gmail.com>
> CC: emacs-devel@gnu.org
> 
> On 3/28/2011 12:41 PM, Eli Zaretskii wrote:
> > We are miscommunicating: I meant that delivering a SIGTERM will end up
> > in fatal_error_signal, which will save all that's worth saving, before
> > Emacs commits suicide.  Your patch achieves the same goal, as far as
> > saving unsaved work is concerned, except it uses SIGUSR2.
> 
> My patch does not do the same thing.  Instead, my patch returns the user 
> to an *interactive* context where he can continue using Emacs normally 
> after resolving whatever was causing the infinite loop (perhaps by 
> closing a buffer, or setting some configuration variables).  It is not 
> an unconditional quit the same way SIGTERM is.

I would not recommend anyone to continue using Emacs normally after
interrupting uninterruptible ops.

> It would solve that problem but introduce another: users have no way of 
> knowing went font-lock happens.  Innocently typing C-g at the wrong time 
> can terminate font lock and leave parts of the buffer unfontified.

If the user doesn't know when to type C-g, she won't know when to type
"kill PID -USR2", either.

Re: [PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

On 3/28/2011 3:00 PM, Eli Zaretskii wrote:
> I would not recommend anyone to continue using Emacs normally after
> interrupting uninterruptible ops.

You're not understanding me.

I'm not talking about interrupting arbitrary code.  Instead, I'm talking 
about interrupting in safe places in situations where quitting is 
disabled poor UI would otherwise result, not because the code being run 
is inherently delicate.  If interrupting in these places is dangerous, 
so is inserting an explicit (debug) in the code.

>> It would solve that problem but introduce another: users have no way of
>> knowing went font-lock happens.  Innocently typing C-g at the wrong time
>> can terminate font lock and leave parts of the buffer unfontified.
>
> If the user doesn't know when to type C-g, she won't know when to type
> "kill PID -USR2", either.

As covered in another branch of this thread, C-g *is* often ambiguous. 
It's inherently racy when we allow normal quitting of background work. 
Users would send this signal to Emacs only after noticing an 
uninterruptable hang.  Is that not clear?

Re: [PATCH] Unconditional quit on SIGUSR2

[chad]

[-- Attachment #1: Type: text/plain, Size: 449 bytes --]


On Mar 28, 2011, at 3:08 PM, Daniel Colascione wrote:
> 
> As covered in another branch of this thread, C-g *is* often ambiguous. It's inherently racy when we allow normal quitting of background work. Users would send this signal to Emacs only after noticing an uninterruptable hang.  Is that not clear?

If the method works out in practice, how would people feel about a patch to emacsclient adding a flag to send such signals?

*Chad



[-- Attachment #2: Type: text/html, Size: 762 bytes --]

Re: [PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

[-- Attachment #1.1: Type: text/plain, Size: 766 bytes --]

On 3/28/11 3:20 PM, chad wrote:
> If the method works out in practice, how would people feel about a
> patch to emacsclient adding a flag to send such signals?

That would be fine. For NT Emacs, the best approach would be for
emacsclient to suspend all threads in the target Emacs process, then use
CreateRemoteThread to inject code into a running Emacs.  This code would
trip the debugger the same way the signal handler in my patch does, then
resume the rest of Emacs.

Below is another version of the patch.  I've changed the code so that we
instead inspect a new debug-on-special-event variable, which by default
is the symbol sigusr2.  This way, we can extend the mechanism and use
another kind of special event for platforms that lack signals.


[-- Attachment #1.2: debugsig3.patch --]
[-- Type: text/plain, Size: 4865 bytes --]

=== modified file 'doc/emacs/trouble.texi'
--- doc/emacs/trouble.texi	2011-01-25 04:08:28 +0000
+++ doc/emacs/trouble.texi	2011-03-29 17:47:26 +0000
@@ -812,6 +812,14 @@
 This backtrace is useful for debugging such long loops, so if you can
 produce it, copy it into the bug report.
 
+@vindex debug-on-special-event
+If the normal quit process is ineffective, you can try sending Emacs
+the special event given in @code{debug-on-special-event}, which by
+default corresponds to SIGUSR2.  When Emacs receives this event, it
+stops what it's doing, sets @code{debug-on-quit} to @code{t}, and
+tries to break into the debugger in a variety of ways.  This process
+happens even in places where quitting is normally not allowed.
+
 @item
 Check whether any programs you have loaded into the Lisp world,
 including your @file{.emacs} file, set any variables that may affect the

=== modified file 'doc/lispref/debugging.texi'
--- doc/lispref/debugging.texi	2011-01-25 04:08:28 +0000
+++ doc/lispref/debugging.texi	2011-03-29 17:46:44 +0000
@@ -185,6 +185,20 @@
 when you quit.  @xref{Quitting}.
 @end defopt
 
+@defopt debug-on-special-event
+When this variable contains a symbol matching one of the special
+events in @code{special-event-map} (@pxref{Active Keymaps}) and Emacs
+receives the corresponding event, Emacs will break into the debugger
+instead of processing the event normally, setting @code{debug-on-quit}
+to @code{t} as a side effect.  This feature is useful for terminating
+infinite loops in places where quits would normally be ignored, such
+as in code that runs during redisplay.
+
+Currently, the only events Emacs recognizes as valid values for
+@code{debug-on-special-event} are @code{sigusr1} and @code{sigusr2}.
+
+@end defopt
+
 @node Function Debugging
 @subsection Entering the Debugger on a Function Call
 @cindex function call debugging

=== modified file 'lisp/cus-start.el'
--- lisp/cus-start.el	2011-03-27 10:55:07 +0000
+++ lisp/cus-start.el	2011-03-29 17:36:51 +0000
@@ -259,6 +259,11 @@
 	     (suggest-key-bindings keyboard (choice (const :tag "off" nil)
 						    (integer :tag "time" 2)
 						    (other :tag "on")))
+             (debug-on-special-event debug
+			    (choice (const :tag "None" nil)
+                                    (const :tag "When sent SIGUSR1" sigusr1)
+                                    (const :tag "When sent SIGUSR2" sigusr2))
+                            "24.1")
 
 ;; This is not good news because it will use the wrong
 ;; version-specific directories when you upgrade.  We need

=== modified file 'src/keyboard.c'
--- src/keyboard.c	2011-03-27 02:27:11 +0000
+++ src/keyboard.c	2011-03-29 17:29:47 +0000
@@ -7165,12 +7165,33 @@
 {
   int old_errno = errno;
   struct user_signal_info *p;
+  const char* special_event_name = NULL;
 
   SIGNAL_THREAD_CHECK (sig);
-
+  
+  if (SYMBOLP (Vdebug_on_special_event) &&
+      STRINGP (XSYMBOL (Vdebug_on_special_event)->xname))
+    {
+      special_event_name =
+        SSDATA (XSYMBOL (Vdebug_on_special_event)->xname);
+    }
+  
   for (p = user_signals; p; p = p->next)
     if (p->sig == sig)
       {
+        if (special_event_name &&
+            strcmp (special_event_name, p->name) == 0)
+          {
+            /* Enter the debugger in many ways.  */
+            debug_on_next_call = 1;
+            debug_on_quit = 1;
+            Vquit_flag = Qt;
+            Vinhibit_quit = Qnil;
+
+            /* Eat the event.  */
+            break;
+          }
+        
 	p->npending++;
 #ifdef SIGIO
 	if (interrupt_input)
@@ -12178,6 +12199,17 @@
 `deactivate-mark' call uses this to set the window selection.  */);
   Vsaved_region_selection = Qnil;
 
+  DEFVAR_LISP ("debug-on-special-event",
+               Vdebug_on_special_event,
+               doc: /* Enter debugger on this special event.
+When Emacs receives the special event specifed by this variable,
+it will try to break into the debugger as soon as possible instead of
+processing the event normally through `special-event-map'.
+
+Currently, the only meaningful values for this
+variable are `sigusr1' and `sigusr2'.  */);
+  Vdebug_on_special_event = intern_c_string ("sigusr2");
+
   /* Create the initial keyboard. */
   initial_kboard = (KBOARD *) xmalloc (sizeof (KBOARD));
   init_kboard (initial_kboard);

=== modified file 'src/lisp.h'
--- src/lisp.h	2011-03-27 02:27:11 +0000
+++ src/lisp.h	2011-03-28 13:06:41 +0000
@@ -2814,7 +2814,7 @@
 
 /* Defined in eval.c */
 extern Lisp_Object Qautoload, Qexit, Qinteractive, Qcommandp, Qdefun, Qmacro;
-extern Lisp_Object Qinhibit_quit;
+extern Lisp_Object Qinhibit_quit, Qdebug;
 extern Lisp_Object Vautoload_queue;
 extern Lisp_Object Vsignaling_function;
 extern int handling_signal;


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 685 bytes --]

On 3/29/11 10:55 AM, Daniel Colascione wrote:
> On 3/28/11 3:20 PM, chad wrote:
>> If the method works out in practice, how would people feel about a
>> patch to emacsclient adding a flag to send such signals?
> 
> That would be fine. For NT Emacs, the best approach would be for
> emacsclient to suspend all threads in the target Emacs process, then use
> CreateRemoteThread to inject code into a running Emacs.  This code would
> trip the debugger the same way the signal handler in my patch does, then
> resume the rest of Emacs.

Err, after actually *reading* w32term.c and w32fns.c, it looks like a
normal window message would suffice.  No need for code injection.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

[-- Attachment #1.1: Type: text/plain, Size: 409 bytes --]

On 3/29/11 10:55 AM, Daniel Colascione wrote:
>
> Below is another version of the patch.  I've changed the code so that we
> instead inspect a new debug-on-special-event variable, which by default
> is the symbol sigusr2.  This way, we can extend the mechanism and use
> another kind of special event for platforms that lack signals.
> 

Does anyone have any strong objections to merging this patch?

[-- Attachment #1.2: debugsig3.patch --]
[-- Type: text/plain, Size: 4865 bytes --]

=== modified file 'doc/emacs/trouble.texi'
--- doc/emacs/trouble.texi	2011-01-25 04:08:28 +0000
+++ doc/emacs/trouble.texi	2011-03-29 17:47:26 +0000
@@ -812,6 +812,14 @@
 This backtrace is useful for debugging such long loops, so if you can
 produce it, copy it into the bug report.
 
+@vindex debug-on-special-event
+If the normal quit process is ineffective, you can try sending Emacs
+the special event given in @code{debug-on-special-event}, which by
+default corresponds to SIGUSR2.  When Emacs receives this event, it
+stops what it's doing, sets @code{debug-on-quit} to @code{t}, and
+tries to break into the debugger in a variety of ways.  This process
+happens even in places where quitting is normally not allowed.
+
 @item
 Check whether any programs you have loaded into the Lisp world,
 including your @file{.emacs} file, set any variables that may affect the

=== modified file 'doc/lispref/debugging.texi'
--- doc/lispref/debugging.texi	2011-01-25 04:08:28 +0000
+++ doc/lispref/debugging.texi	2011-03-29 17:46:44 +0000
@@ -185,6 +185,20 @@
 when you quit.  @xref{Quitting}.
 @end defopt
 
+@defopt debug-on-special-event
+When this variable contains a symbol matching one of the special
+events in @code{special-event-map} (@pxref{Active Keymaps}) and Emacs
+receives the corresponding event, Emacs will break into the debugger
+instead of processing the event normally, setting @code{debug-on-quit}
+to @code{t} as a side effect.  This feature is useful for terminating
+infinite loops in places where quits would normally be ignored, such
+as in code that runs during redisplay.
+
+Currently, the only events Emacs recognizes as valid values for
+@code{debug-on-special-event} are @code{sigusr1} and @code{sigusr2}.
+
+@end defopt
+
 @node Function Debugging
 @subsection Entering the Debugger on a Function Call
 @cindex function call debugging

=== modified file 'lisp/cus-start.el'
--- lisp/cus-start.el	2011-03-27 10:55:07 +0000
+++ lisp/cus-start.el	2011-03-29 17:36:51 +0000
@@ -259,6 +259,11 @@
 	     (suggest-key-bindings keyboard (choice (const :tag "off" nil)
 						    (integer :tag "time" 2)
 						    (other :tag "on")))
+             (debug-on-special-event debug
+			    (choice (const :tag "None" nil)
+                                    (const :tag "When sent SIGUSR1" sigusr1)
+                                    (const :tag "When sent SIGUSR2" sigusr2))
+                            "24.1")
 
 ;; This is not good news because it will use the wrong
 ;; version-specific directories when you upgrade.  We need

=== modified file 'src/keyboard.c'
--- src/keyboard.c	2011-03-27 02:27:11 +0000
+++ src/keyboard.c	2011-03-29 17:29:47 +0000
@@ -7165,12 +7165,33 @@
 {
   int old_errno = errno;
   struct user_signal_info *p;
+  const char* special_event_name = NULL;
 
   SIGNAL_THREAD_CHECK (sig);
-
+  
+  if (SYMBOLP (Vdebug_on_special_event) &&
+      STRINGP (XSYMBOL (Vdebug_on_special_event)->xname))
+    {
+      special_event_name =
+        SSDATA (XSYMBOL (Vdebug_on_special_event)->xname);
+    }
+  
   for (p = user_signals; p; p = p->next)
     if (p->sig == sig)
       {
+        if (special_event_name &&
+            strcmp (special_event_name, p->name) == 0)
+          {
+            /* Enter the debugger in many ways.  */
+            debug_on_next_call = 1;
+            debug_on_quit = 1;
+            Vquit_flag = Qt;
+            Vinhibit_quit = Qnil;
+
+            /* Eat the event.  */
+            break;
+          }
+        
 	p->npending++;
 #ifdef SIGIO
 	if (interrupt_input)
@@ -12178,6 +12199,17 @@
 `deactivate-mark' call uses this to set the window selection.  */);
   Vsaved_region_selection = Qnil;
 
+  DEFVAR_LISP ("debug-on-special-event",
+               Vdebug_on_special_event,
+               doc: /* Enter debugger on this special event.
+When Emacs receives the special event specifed by this variable,
+it will try to break into the debugger as soon as possible instead of
+processing the event normally through `special-event-map'.
+
+Currently, the only meaningful values for this
+variable are `sigusr1' and `sigusr2'.  */);
+  Vdebug_on_special_event = intern_c_string ("sigusr2");
+
   /* Create the initial keyboard. */
   initial_kboard = (KBOARD *) xmalloc (sizeof (KBOARD));
   init_kboard (initial_kboard);

=== modified file 'src/lisp.h'
--- src/lisp.h	2011-03-27 02:27:11 +0000
+++ src/lisp.h	2011-03-28 13:06:41 +0000
@@ -2814,7 +2814,7 @@
 
 /* Defined in eval.c */
 extern Lisp_Object Qautoload, Qexit, Qinteractive, Qcommandp, Qdefun, Qmacro;
-extern Lisp_Object Qinhibit_quit;
+extern Lisp_Object Qinhibit_quit, Qdebug;
 extern Lisp_Object Vautoload_queue;
 extern Lisp_Object Vsignaling_function;
 extern int handling_signal;


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH] Unconditional quit on SIGUSR2

[joakim]

Daniel Colascione <dan.colascione@gmail.com> writes:

> On 3/29/11 10:55 AM, Daniel Colascione wrote:
>>
>> Below is another version of the patch.  I've changed the code so that we
>> instead inspect a new debug-on-special-event variable, which by default
>> is the symbol sigusr2.  This way, we can extend the mechanism and use
>> another kind of special event for platforms that lack signals.
>> 
>
> Does anyone have any strong objections to merging this patch?

I haven't used your patch but I often feel the need to be able to
interrupt an Emacs that does not respond for some reason. If your patch
gives me another tool to try in those cases, I would be very interested
in having it applied to trunk.

Would it be possible also to have some "run level 1" functionality? That
is, when the signal is sent, Emacs disables timers, font-lock etc...

>
> === modified file 'doc/emacs/trouble.texi'
> --- doc/emacs/trouble.texi	2011-01-25 04:08:28 +0000
> +++ doc/emacs/trouble.texi	2011-03-29 17:47:26 +0000
> @@ -812,6 +812,14 @@
>  This backtrace is useful for debugging such long loops, so if you can
>  produce it, copy it into the bug report.
>  
> +@vindex debug-on-special-event
> +If the normal quit process is ineffective, you can try sending Emacs
> +the special event given in @code{debug-on-special-event}, which by
> +default corresponds to SIGUSR2.  When Emacs receives this event, it
> +stops what it's doing, sets @code{debug-on-quit} to @code{t}, and
> +tries to break into the debugger in a variety of ways.  This process
> +happens even in places where quitting is normally not allowed.
> +
>  @item
>  Check whether any programs you have loaded into the Lisp world,
>  including your @file{.emacs} file, set any variables that may affect the
>
> === modified file 'doc/lispref/debugging.texi'
> --- doc/lispref/debugging.texi	2011-01-25 04:08:28 +0000
> +++ doc/lispref/debugging.texi	2011-03-29 17:46:44 +0000
> @@ -185,6 +185,20 @@
>  when you quit.  @xref{Quitting}.
>  @end defopt
>  
> +@defopt debug-on-special-event
> +When this variable contains a symbol matching one of the special
> +events in @code{special-event-map} (@pxref{Active Keymaps}) and Emacs
> +receives the corresponding event, Emacs will break into the debugger
> +instead of processing the event normally, setting @code{debug-on-quit}
> +to @code{t} as a side effect.  This feature is useful for terminating
> +infinite loops in places where quits would normally be ignored, such
> +as in code that runs during redisplay.
> +
> +Currently, the only events Emacs recognizes as valid values for
> +@code{debug-on-special-event} are @code{sigusr1} and @code{sigusr2}.
> +
> +@end defopt
> +
>  @node Function Debugging
>  @subsection Entering the Debugger on a Function Call
>  @cindex function call debugging
>
> === modified file 'lisp/cus-start.el'
> --- lisp/cus-start.el	2011-03-27 10:55:07 +0000
> +++ lisp/cus-start.el	2011-03-29 17:36:51 +0000
> @@ -259,6 +259,11 @@
>  	     (suggest-key-bindings keyboard (choice (const :tag "off" nil)
>  						    (integer :tag "time" 2)
>  						    (other :tag "on")))
> +             (debug-on-special-event debug
> +			    (choice (const :tag "None" nil)
> +                                    (const :tag "When sent SIGUSR1" sigusr1)
> +                                    (const :tag "When sent SIGUSR2" sigusr2))
> +                            "24.1")
>  
>  ;; This is not good news because it will use the wrong
>  ;; version-specific directories when you upgrade.  We need
>
> === modified file 'src/keyboard.c'
> --- src/keyboard.c	2011-03-27 02:27:11 +0000
> +++ src/keyboard.c	2011-03-29 17:29:47 +0000
> @@ -7165,12 +7165,33 @@
>  {
>    int old_errno = errno;
>    struct user_signal_info *p;
> +  const char* special_event_name = NULL;
>  
>    SIGNAL_THREAD_CHECK (sig);
> -
> +  
> +  if (SYMBOLP (Vdebug_on_special_event) &&
> +      STRINGP (XSYMBOL (Vdebug_on_special_event)->xname))
> +    {
> +      special_event_name =
> +        SSDATA (XSYMBOL (Vdebug_on_special_event)->xname);
> +    }
> +  
>    for (p = user_signals; p; p = p->next)
>      if (p->sig == sig)
>        {
> +        if (special_event_name &&
> +            strcmp (special_event_name, p->name) == 0)
> +          {
> +            /* Enter the debugger in many ways.  */
> +            debug_on_next_call = 1;
> +            debug_on_quit = 1;
> +            Vquit_flag = Qt;
> +            Vinhibit_quit = Qnil;
> +
> +            /* Eat the event.  */
> +            break;
> +          }
> +        
>  	p->npending++;
>  #ifdef SIGIO
>  	if (interrupt_input)
> @@ -12178,6 +12199,17 @@
>  `deactivate-mark' call uses this to set the window selection.  */);
>    Vsaved_region_selection = Qnil;
>  
> +  DEFVAR_LISP ("debug-on-special-event",
> +               Vdebug_on_special_event,
> +               doc: /* Enter debugger on this special event.
> +When Emacs receives the special event specifed by this variable,
> +it will try to break into the debugger as soon as possible instead of
> +processing the event normally through `special-event-map'.
> +
> +Currently, the only meaningful values for this
> +variable are `sigusr1' and `sigusr2'.  */);
> +  Vdebug_on_special_event = intern_c_string ("sigusr2");
> +
>    /* Create the initial keyboard. */
>    initial_kboard = (KBOARD *) xmalloc (sizeof (KBOARD));
>    init_kboard (initial_kboard);
>
> === modified file 'src/lisp.h'
> --- src/lisp.h	2011-03-27 02:27:11 +0000
> +++ src/lisp.h	2011-03-28 13:06:41 +0000
> @@ -2814,7 +2814,7 @@
>  
>  /* Defined in eval.c */
>  extern Lisp_Object Qautoload, Qexit, Qinteractive, Qcommandp, Qdefun, Qmacro;
> -extern Lisp_Object Qinhibit_quit;
> +extern Lisp_Object Qinhibit_quit, Qdebug;
>  extern Lisp_Object Vautoload_queue;
>  extern Lisp_Object Vsignaling_function;
>  extern int handling_signal;
>

-- 
Joakim Verona

Re: [PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1272 bytes --]

On 4/25/11 2:23 AM, joakim@verona.se wrote:
> Daniel Colascione <dan.colascione@gmail.com> writes:
> 
>> On 3/29/11 10:55 AM, Daniel Colascione wrote:
>>>
>>> Below is another version of the patch.  I've changed the code so that we
>>> instead inspect a new debug-on-special-event variable, which by default
>>> is the symbol sigusr2.  This way, we can extend the mechanism and use
>>> another kind of special event for platforms that lack signals.
>>>
>>
>> Does anyone have any strong objections to merging this patch?
> 
> I haven't used your patch but I often feel the need to be able to
> interrupt an Emacs that does not respond for some reason. If your patch
> gives me another tool to try in those cases, I would be very interested
> in having it applied to trunk.

I'll apply it to the trunk then.

> Would it be possible also to have some "run level 1" functionality? That
> is, when the signal is sent, Emacs disables timers, font-lock etc...

I don't think these features can be safely disabled from within a signal
handler, but it should be possible to that work when Emacs pops up into
the debugger.  The "run level 1" functionality seems more general, and
I'd like to focus this change on simply entering the debugger itself.



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH] Unconditional quit on SIGUSR2

[Stefan Monnier]

That reminds me of the idea to turn C-g C-g C-g into a "really break, no
matter what" key sequence.  Someone posted a patch for it a while back
and I didn't find the time to review it back then.

> === modified file 'doc/emacs/trouble.texi'
> --- doc/emacs/trouble.texi	2011-01-25 04:08:28 +0000
> +++ doc/emacs/trouble.texi	2011-03-29 17:47:26 +0000
> @@ -812,6 +812,14 @@
>  This backtrace is useful for debugging such long loops, so if you can
>  produce it, copy it into the bug report.
 
> +@vindex debug-on-special-event
> +If the normal quit process is ineffective, you can try sending Emacs
> +the special event given in @code{debug-on-special-event}, which by
> +default corresponds to SIGUSR2.  When Emacs receives this event, it
> +stops what it's doing, sets @code{debug-on-quit} to @code{t}, and
> +tries to break into the debugger in a variety of ways.  This process
> +happens even in places where quitting is normally not allowed.
> +
>  @item
>  Check whether any programs you have loaded into the Lisp world,
>  including your @file{.emacs} file, set any variables that may affect the

> === modified file 'doc/lispref/debugging.texi'
> --- doc/lispref/debugging.texi	2011-01-25 04:08:28 +0000
> +++ doc/lispref/debugging.texi	2011-03-29 17:46:44 +0000
> @@ -185,6 +185,20 @@
>  when you quit.  @xref{Quitting}.
>  @end defopt
 
> +@defopt debug-on-special-event
> +When this variable contains a symbol matching one of the special
> +events in @code{special-event-map} (@pxref{Active Keymaps}) and Emacs
> +receives the corresponding event, Emacs will break into the debugger
> +instead of processing the event normally, setting @code{debug-on-quit}
> +to @code{t} as a side effect.  This feature is useful for terminating
> +infinite loops in places where quits would normally be ignored, such
> +as in code that runs during redisplay.
> +
> +Currently, the only events Emacs recognizes as valid values for
> +@code{debug-on-special-event} are @code{sigusr1} and @code{sigusr2}.
> +
> +@end defopt

I'm not sure I'd want to advertise this feature in all manuals at this
point.  I'd rather see it in etc/DEBUG.  And it should include some info
about what happens after you get the backtrace: e.g. can you just hit
`q' to abort the inf-loop and go on using this Emacs sessions as if
nothing happened?

> +  if (SYMBOLP (Vdebug_on_special_event) &&
> +      STRINGP (XSYMBOL (Vdebug_on_special_event)->xname))

Use SYMBOL_NAME.  And I don't think STRINGP is needed here.

Another minor problem is that you use a name with "special-event" and
the doc makes references to "special-event-map" yet the two are
fundamentally unrelated (except that they both can handle SIGUSRn
signals).


        Stefan

Re: [PATCH] Unconditional quit on SIGUSR2

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1326 bytes --]

On 4/25/11 6:32 AM, Stefan Monnier wrote:
> That reminds me of the idea to turn C-g C-g C-g into a "really break, no
> matter what" key sequence.  Someone posted a patch for it a while back
> and I didn't find the time to review it back then.

That's okay as it's optional.  I don't want to kill
fontification because I lean on C-g.

> I'm not sure I'd want to advertise this feature in all manuals at this
> point.  I'd rather see it in etc/DEBUG. 

Fair enough.

> And it should include some info
> about what happens after you get the backtrace: e.g. can you just hit
> `q' to abort the inf-loop and go on using this Emacs sessions as if
> nothing happened?

Yes, and I'll add that.

>> +  if (SYMBOLP (Vdebug_on_special_event) &&
>> +      STRINGP (XSYMBOL (Vdebug_on_special_event)->xname))
> 
> Use SYMBOL_NAME.  And I don't think STRINGP is needed here.

Will do.

> Another minor problem is that you use a name with "special-event" and
> the doc makes references to "special-event-map" yet the two are
> fundamentally unrelated (except that they both can handle SIGUSRn
> signals).

The idea is that the event named by the variable is in the same
namespace as the events in special-event-map, though the variable
supports only a subset of them.  Do you have a better name in mind?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH] Unconditional quit on SIGUSR2

[Stefan Monnier]

>> Another minor problem is that you use a name with "special-event" and
>> the doc makes references to "special-event-map" yet the two are
>> fundamentally unrelated (except that they both can handle SIGUSRn
>> signals).
> The idea is that the event named by the variable is in the same
> namespace as the events in special-event-map, though the variable
> supports only a subset of them.

AFAIK this namespace is the same as used for all events (whether they
get caught by special-event-map or any other of the umpteen maps we
have), and indeed we have moved some bindings between global-map and
special-event-map after realizing that they were not handled at the
right level.

> Do you have a better name in mind?

So you could just name it debug-on-event.


        Stefan

Re: [PATCH] Unconditional quit on SIGUSR2

[Richard Stallman]

    That reminds me of the idea to turn C-g C-g C-g into a "really break, no
    matter what" key sequence.

Do you mean that it would quit even from code that does not allow
quitting?  Nowadays, why would you want that?  Isn't it always
possible to switch to or create another terminal?

The patch sent a few days ago handles SIGUSR2, which presumably you
send from another terminal.  That patch is useful as a way to get
into the debugger.

C-g C-g is already emergency escape; it gives you a way to get out of
Emacs when you can't quit.  It asks questions about what to do.  If
some other option is desirable advisable, we could make emergency
escape offer some other option.  I believe you can already syspend at
those questions by typing C-z.  That would enable you to attach with
GDB and debug the unquittable loop.

If it is a matter of a loop in Lisp code with quitting inhibited,
in such cases emergency stop could offer the option to quit out of it
and return to the C code that called that Lisp code.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.

Re: [PATCH] Unconditional quit on SIGUSR2

[Tom Tromey]

>>>>> "Daniel" == Daniel Colascione <dan.colascione@gmail.com> writes:

Daniel> (Speaking of debugging bytecode, by the way: would it be feasible to
Daniel> someday provide the ability to step through compiled functions
Daniel> bytecode-by-bytecode?)

I don't see why not.  You'd just have to annotate each opcode with a
call to some Lisp debugger function, and then of course write the
bytecode debugger in Lisp...

Trickier would be making this work without slowing down Elisp.  Even
that could be done if you were willing to have 2 copies of the
interpreter.

Alternatively, just compile bytecode to C and then use gdb.  :-)

Tom

Re: [PATCH] Unconditional quit on SIGUSR2

[Stefan Monnier]

> Trickier would be making this work without slowing down Elisp.  Even
> that could be done if you were willing to have 2 copies of the
> interpreter.

If the byte-code interpreter is generated from a description (think
vmgen, here), that's easy.  `vmgen' would also generate more efficient
code than what we have now.


        Stefan

Re: [PATCH] Unconditional quit on SIGUSR2

[Tom Tromey]

Tom> Trickier would be making this work without slowing down Elisp.  Even
Tom> that could be done if you were willing to have 2 copies of the
Tom> interpreter.

Stefan> If the byte-code interpreter is generated from a description (think
Stefan> vmgen, here), that's easy.  `vmgen' would also generate more efficient
Stefan> code than what we have now.

Speaking of ... a while back I wrote a patch to make the bytecode
interpreter be token-threaded.  I was going to commit it, but I thought
maybe I should wait for lexbind to be merged...?

Tom

Re: [PATCH] Unconditional quit on SIGUSR2

[Stefan Monnier]

> Speaking of ... a while back I wrote a patch to make the bytecode
> interpreter be token-threaded.  I was going to commit it, but I thought
> maybe I should wait for lexbind to be merged...?

It might be worth adapting it to the lexbind branch and testing its
performance there (i.e. test its performance impact on lexical-binding
code, such as the bytecomp/byte-opt/macroexp which have all been
converted to use lexical-binding).


        Stefan


As things stand on the lexbind branch, using lexical-binding does not
result in a speed improvement, although it could: basically, with
dynamic scoping a let is turned into:
- eval the inital value of the variable.
- call byte-varbind.
- evaluate the body of the let.
- when you need to read the variable, you call byte-varref.
- when you need to write to the variable, you call byte-varset.
- call byte-unbind.
whereas with lexical scoping a let is turned into:
- eval the inital value of the variable.
- do nothing
- evaluate the body of the let.
- when you need to read the variable, you call byte-stackref.
- when you need to write to the variable, you call byte-stackset.
- call byte-discard.
so we saved one call to byte-varbind and we replaced byte-varrefs by
byte-stackrefs, byte-varsets by byte-stackset, and byte-unbind by
byte-discard.  The replacements are very good because they are simpler
operations that can run faster.  But the overall performance is not
noticeably improved (and can be worsened) because the above benefits are
defeated by the fact that calls to mapcar often need to build a closure,
and worse: unwind-protect, condition-case, and catch need to build
closures as well.
Still, these last three could be improved to not necessitate closures (I
opted to not do that (yet) because it would require more substantial
changes and I want to focus on delivering the lexical semantics first,
and keep the performance for later).
OK, I got a bit lost, but what I intended to get at is that the simpler
byte-codes we now use (stackref/stackset/discard) spend a larger
proportion of their time in interpreter-overhead, so there's a chance
that token-threading will get a bit more benefits on such code that it
does on dynamically scoped code where more operations need to call out
to functions anyway.