why "in_sighandler"?

why "in_sighandler"?

[Stefan Monnier]

Could someone explain the difference between `handling_signal' and
`in_sighandler'?  Depending on the answer I'm likely to suggest a renaming,
since it seems that either they are the same and should be merged or they
are different and should say so directly in their name.


        Stefan

Re: why "in_sighandler"?

[Jan D.]

Stefan Monnier wrote:
> Could someone explain the difference between `handling_signal' and
> `in_sighandler'?  Depending on the answer I'm likely to suggest a renaming,
> since it seems that either they are the same and should be merged or they
> are different and should say so directly in their name.
>   

handling_signal may be non-zero even if no signal handler is invoked.  
For example, from UNBLOCK_INPUT (via reinvoke_input_signal => 
handle_async_input).  Also if SYNC_INPUT is used.  So handling_signal 
really means "handling X11/GUI event".  I needed a variable that was 
only set when the signal handler is running.  Maybe setting 
handling_signal is an error in the non-signalling cases, I don't really 
know.

    Jan D.

Re: why "in_sighandler"?

[Stefan Monnier]

>> Could someone explain the difference between `handling_signal' and
>> `in_sighandler'?  Depending on the answer I'm likely to suggest a renaming,
>> since it seems that either they are the same and should be merged or they
>> are different and should say so directly in their name.
>> 

> handling_signal may be non-zero even if no signal handler is invoked.
> For example, from UNBLOCK_INPUT (via reinvoke_input_signal =>
> handle_async_input).  Also if SYNC_INPUT is used.  So handling_signal really
> means "handling X11/GUI event".  I needed a variable that was only set when
> the signal handler is running.  Maybe setting handling_signal is an error in
> the non-signalling cases, I don't really know.

Oh, I see, thanks.

Yes, the distinction is subtle.  For what it's worth, I believe that
handling_signal shouldn't need to be set when handle_async_input is called
synchronously via reinvoke_input_signal.  I.e. it should behave the same as
in_sighanlder ;-).  But we shouldn't try such a change now, of course.


        Stefan

Re: why "in_sighandler"?

[YAMAMOTO Mitsuharu]

>>>>> On Sun, 20 Aug 2006 18:37:30 +0200, "Jan D." <jan.h.d@swipnet.se> said:

> I needed a variable that was only set when the signal handler is
> running.

I've got no response from you to my last message (*1) in the related
thread.  So I'm not sure if you are for or against my suggestion about
undoing `in_sighandler'-related changes and just changing the order of
lock/unlock and BLOCK_INPUT/UNBLOCK_INPUT in the previous version of
BLOCK_INPUT_ALLOC/UNBLOCK_INPUT_ALLOC as follows.

#define BLOCK_INPUT_ALLOC                       \
  do                                            \
    {                                           \
      if (pthread_self () == main_thread)       \
        BLOCK_INPUT;                            \
      pthread_mutex_lock (&alloc_mutex);        \
    }                                           \
  while (0)
#define UNBLOCK_INPUT_ALLOC                     \
  do                                            \
    {                                           \
      pthread_mutex_unlock (&alloc_mutex);      \
      if (pthread_self () == main_thread)       \
        UNBLOCK_INPUT;                          \
    }                                           \
  while (0)

(*1) http://lists.gnu.org/archive/html/emacs-pretest-bug/2006-08/msg00161.html

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: why "in_sighandler"?

[Jan Djärv]

YAMAMOTO Mitsuharu skrev:
>>>>>> On Sun, 20 Aug 2006 18:37:30 +0200, "Jan D." <jan.h.d@swipnet.se> said:
> 
>> I needed a variable that was only set when the signal handler is
>> running.
> 
> I've got no response from you to my last message (*1) in the related
> thread.  So I'm not sure if you are for or against my suggestion about
> undoing `in_sighandler'-related changes and just changing the order of
> lock/unlock and BLOCK_INPUT/UNBLOCK_INPUT in the previous version of
> BLOCK_INPUT_ALLOC/UNBLOCK_INPUT_ALLOC as follows.
> 
> #define BLOCK_INPUT_ALLOC                       \
>   do                                            \
>     {                                           \
>       if (pthread_self () == main_thread)       \
>         BLOCK_INPUT;                            \
>       pthread_mutex_lock (&alloc_mutex);        \
>     }                                           \
>   while (0)
> #define UNBLOCK_INPUT_ALLOC                     \
>   do                                            \
>     {                                           \
>       pthread_mutex_unlock (&alloc_mutex);      \
>       if (pthread_self () == main_thread)       \
>         UNBLOCK_INPUT;                          \
>     }                                           \
>   while (0)
> 


I don't like it, because it assumes things are "probably OK" to run in the 
signal handler (i.e. pthread_mutex_lock/unlock).  I still think it might hang 
if a Gnome thread is in mutex_lock and a signal arrives and then the signal 
handler also enters mutex_lock.  It is one assumtion against another, neither 
suggestion is without flaws (we need SYNC_INPUT for that).

	Jan D.

Re: why "in_sighandler"?

[YAMAMOTO Mitsuharu]

>>>>> On Mon, 21 Aug 2006 08:18:38 +0200, Jan Djärv <jan.h.d@swipnet.se> said:

> I don't like it, because it assumes things are "probably OK" to run
> in the signal handler (i.e. pthread_mutex_lock/unlock).  I still
> think it might hang if a Gnome thread is in mutex_lock and a signal
> arrives and then the signal handler also enters mutex_lock.

Does it mean the quote from IEEE Std 1003.1 did not convince you?

> It is one assumtion against another, neither suggestion is without
> flaws (we need SYNC_INPUT for that).

No assumption is needed to say that the current code leads to a
problem under a certain scenario.

Suppose that we abandon emacs_blocked_malloc and so on when
HAVE_GTK_AND_PTHREAD is defined.  I think it is as safe as other
non-GNU-malloc systems where emacs_blocked_malloc and so on are not
used, provided that malloc-related functions are thread-safe.  What do
you think about that?

				   YAMAMOTO Mitsuharu
				mituharu@math.chiba-u.ac.jp

Re: why "in_sighandler"?

[Jan Djärv]

YAMAMOTO Mitsuharu skrev:
>>>>>> On Mon, 21 Aug 2006 08:18:38 +0200, Jan Djärv <jan.h.d@swipnet.se> said:
> 
>> I don't like it, because it assumes things are "probably OK" to run
>> in the signal handler (i.e. pthread_mutex_lock/unlock).  I still
>> think it might hang if a Gnome thread is in mutex_lock and a signal
>> arrives and then the signal handler also enters mutex_lock.
> 
> Does it mean the quote from IEEE Std 1003.1 did not convince you?

In a way, but not the same way you interpret it :-).  The quoute is:

   "All functions not in the above table are considered to be unsafe
   with respect to signals. In the presence of signals, all functions
   defined by this volume of IEEE Std 1003.1-2001 shall behave as
   defined when called from or interrupted by a signal-catching
   function, with a single exception: when a signal interrupts an
   unsafe function and the signal-catching function calls an unsafe
   function, the behavior is undefined."

Read the single exception:  "when a signal interrupts an
   unsafe function and the signal-catching function calls an unsafe
   function, the behavior is undefined."

This is exactly what happens if a Gnome thread enters pthread_lock/unlock and 
a signal arrives and the signal handler later also calls pthread_lock/unlock. 
  This was the original problem.


> 
>> It is one assumtion against another, neither suggestion is without
>> flaws (we need SYNC_INPUT for that).
> 
> No assumption is needed to say that the current code leads to a
> problem under a certain scenario.
> 

No, but in the presense of signal handlers executing malloc, there is no 
scenario that is safe.  Your suggestion also has undefined behaviour accoding 
to the quote from the standard.  The one currently implemented has a race 
condition.

> Suppose that we abandon emacs_blocked_malloc and so on when
> HAVE_GTK_AND_PTHREAD is defined.  I think it is as safe as other
> non-GNU-malloc systems where emacs_blocked_malloc and so on are not
> used, provided that malloc-related functions are thread-safe.  What do
> you think about that?

This whole workaround with mutexes and blocks started because Emacs hanged on 
some system, I'm not sure if it was a GNU/Linux system or some kind of BSD 
variant.  So I don't think we should go back, the old problem happend every 
time the file selection box was opened on a Gnome system.

	Jan D.

Re: why "in_sighandler"?

[YAMAMOTO Mitsuharu]

>>>>> On Mon, 21 Aug 2006 09:19:13 +0200, Jan Djärv <jan.h.d@swipnet.se> said:

> Read the single exception: "when a signal interrupts an unsafe
> function and the signal-catching function calls an unsafe function,
> the behavior is undefined."

So, you interpret that ``what are interrupted by a signal'' are all
the functions in execution on the whole threads in the process, not
just a single function in execution on the thread that the signal is
delivered to?  I don't think that is a natural interpretation.

> No, but in the presense of signal handlers executing malloc, there
> is no scenario that is safe.  Your suggestion also has undefined
> behaviour accoding to the quote from the standard.

As I said earlier, the current Emacs implementation calls malloc
within a signal handler with the help of BLOCK_INPUT relying on the
assumption that it is safe unless the signal interrupted
malloc-related functions.  As malloc also internally uses some mutex
in order to make it thread-safe, I don't think allowing mutex
operations in a signal handler (with the help of BLOCK_INPUT, of
course) makes the situation worse.

>> Suppose that we abandon emacs_blocked_malloc and so on when
>> HAVE_GTK_AND_PTHREAD is defined.  I think it is as safe as other
>> non-GNU-malloc systems where emacs_blocked_malloc and so on are not
>> used, provided that malloc-related functions are thread-safe.  What
>> do you think about that?

> This whole workaround with mutexes and blocks started because Emacs
> hanged on some system, I'm not sure if it was a GNU/Linux system or
> some kind of BSD variant.  So I don't think we should go back, the
> old problem happend every time the file selection box was opened on
> a Gnome system.

Do you mean this one?

  http://lists.gnu.org/archive/html/emacs-pretest-bug/2004-11/msg00368.html

Then it's on a GNU/Linux system and emacs_blocked_malloc has already
existed and used, and we've never tried to abandon
emacs_blocked_malloc etc.  I think BSD variants don't use them by
default anyway.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: why "in_sighandler"?

[Jan Djärv]

YAMAMOTO Mitsuharu skrev:
>>>>>> On Mon, 21 Aug 2006 09:19:13 +0200, Jan Djärv <jan.h.d@swipnet.se> said:
> 
>> Read the single exception: "when a signal interrupts an unsafe
>> function and the signal-catching function calls an unsafe function,
>> the behavior is undefined."
> 
> So, you interpret that ``what are interrupted by a signal'' are all
> the functions in execution on the whole threads in the process, not
> just a single function in execution on the thread that the signal is
> delivered to?  I don't think that is a natural interpretation.

I don't see the line "what are interrupted by a signal" in the quote, so I'm 
not sure what you mean.

My interpretation is that it is unsafe to call lock/unlock from a thread and 
from a signal handler at the same time, if they operate on the same mutex. 
That is what the original hang was all about.

> 
>> No, but in the presense of signal handlers executing malloc, there
>> is no scenario that is safe.  Your suggestion also has undefined
>> behaviour accoding to the quote from the standard.
> 
> As I said earlier, the current Emacs implementation calls malloc
> within a signal handler with the help of BLOCK_INPUT relying on the
> assumption that it is safe unless the signal interrupted
> malloc-related functions.

But this only works for the Emacs main thread, not the Gnome threads.

	Jan D.

Re: why "in_sighandler"?

[YAMAMOTO Mitsuharu]

>>>>> On Mon, 21 Aug 2006 10:46:36 +0200, Jan Djärv <jan.h.d@swipnet.se> said:

>>> Read the single exception: "when a signal interrupts an unsafe
>>> function and the signal-catching function calls an unsafe
>>> function, the behavior is undefined."
>> 
>> So, you interpret that ``what are interrupted by a signal'' are all
>> the functions in execution on the whole threads in the process, not
>> just a single function in execution on the thread that the signal
>> is delivered to?  I don't think that is a natural interpretation.

> I don't see the line "what are interrupted by a signal" in the
> quote, so I'm not sure what you mean.

> My interpretation is that it is unsafe to call lock/unlock from a
> thread and from a signal handler at the same time, if they operate
> on the same mutex.  That is what the original hang was all about.

The standard says "when a signal interrupts an unsafe function ...".
My interpretation is that a signal interrupts *only* the function in
execution on the thread that the signal is delivered to, but the
signal does not interrupt the functions in execution on the other
threads.

>> As I said earlier, the current Emacs implementation calls malloc
>> within a signal handler with the help of BLOCK_INPUT relying on the
>> assumption that it is safe unless the signal interrupted
>> malloc-related functions.

> But this only works for the Emacs main thread, not the Gnome
> threads.

And according to the above interpretation, functions that are executed
on non-main threads are irrelevant.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: why "in_sighandler"?

[Richard Stallman]

    handling_signal may be non-zero even if no signal handler is invoked.  
    For example, from UNBLOCK_INPUT (via reinvoke_input_signal => 
    handle_async_input).  Also if SYNC_INPUT is used.  So handling_signal 
    really means "handling X11/GUI event".  I needed a variable that was 
    only set when the signal handler is running.

Could you please install comments which clearly explain this?

Re: why "in_sighandler"?

[Jan Djärv]

YAMAMOTO Mitsuharu skrev:

> 
> The standard says "when a signal interrupts an unsafe function ...".
> My interpretation is that a signal interrupts *only* the function in
> execution on the thread that the signal is delivered to, but the
> signal does not interrupt the functions in execution on the other
> threads.
> 

A signal interrupts all threads on a uni-processor machine.  The fact that the 
signal is delivered to just one thread is another matter.  Also, it has more 
to do with reentrance of the function in question rather than which threads 
are executing what.

	Jan D.

Re: why "in_sighandler"?

[Jan Djärv]

Richard Stallman skrev:
>     handling_signal may be non-zero even if no signal handler is invoked.  
>     For example, from UNBLOCK_INPUT (via reinvoke_input_signal => 
>     handle_async_input).  Also if SYNC_INPUT is used.  So handling_signal 
>     really means "handling X11/GUI event".  I needed a variable that was 
>     only set when the signal handler is running.
> 
> Could you please install comments which clearly explain this?

Done.

	Jan D.

Re: why "in_sighandler"?

[YAMAMOTO Mitsuharu]

>>>>> On Mon, 21 Aug 2006 13:32:30 +0200, Jan Djärv <jan.h.d@swipnet.se> said:

> A signal interrupts all threads on a uni-processor machine.

I can hardly believe that.  Can you give some references saying so?

The same quote from IEEE Std 1003.1 again:

  All functions not in the above table are considered to be unsafe
  with respect to signals. In the presence of signals, all functions
  defined by this volume of IEEE Std 1003.1-2001 shall behave as
  defined when called from or interrupted by a signal-catching
  function, with a single exception: when a signal interrupts an
  unsafe function and the signal-catching function calls an unsafe
  function, the behavior is undefined.

  http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_04.html

It says "interrupted by a signal-catching function".  And the
signal-catching function is executed on the thread that the signal is
delivered to.  Given no other notes, it is natural to interpret that a
signal only interrupts the function in the thread that the
signal-catching function is executed.

"Multithreading in the Solaris Environment" says:

  An asynchronous signal interrupts the signalled thread at some
  arbitrary point in its execution

  http://www.sun.com/software/whitepapers/solaris9/multithread.pdf
  (P13)

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: why "in_sighandler"?

[Jan Djärv]

YAMAMOTO Mitsuharu skrev:
>>>>>> On Mon, 21 Aug 2006 13:32:30 +0200, Jan Djärv <jan.h.d@swipnet.se> said:
> 
>> A signal interrupts all threads on a uni-processor machine.
> 
> I can hardly believe that.  Can you give some references saying so?

How can it be otherwise?  If you only have one program counter, that program 
counter must be changed to the signal handler no matter how many threads you 
have, i.e. any previous execution (regardless of thread) is interrupted.

> 
> The same quote from IEEE Std 1003.1 again:
> 
>   All functions not in the above table are considered to be unsafe
>   with respect to signals. In the presence of signals, all functions
>   defined by this volume of IEEE Std 1003.1-2001 shall behave as
>   defined when called from or interrupted by a signal-catching
>   function, with a single exception: when a signal interrupts an
>   unsafe function and the signal-catching function calls an unsafe
>   function, the behavior is undefined.
> 
>   http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_04.html
> 
> It says "interrupted by a signal-catching function".  And the
> signal-catching function is executed on the thread that the signal is
> delivered to.  Given no other notes, it is natural to interpret that a
> signal only interrupts the function in the thread that the
> signal-catching function is executed.
> 
> "Multithreading in the Solaris Environment" says:
> 
>   An asynchronous signal interrupts the signalled thread at some
>   arbitrary point in its execution
> 

It is semantics.  When I say interrupted I mean that the signal handling 
function starts to run.  You obviously mean something else.  But this is a 
side issue, it has more to do with reentrance of the function interrupted 
rather than which thread is currently running.

	Jan D.

Re: why "in_sighandler"?

[YAMAMOTO Mitsuharu]

>>>>> On Tue, 22 Aug 2006 08:38:54 +0200, Jan Djärv <jan.h.d@swipnet.se> said:

> How can it be otherwise?  If you only have one program counter, that
> program counter must be changed to the signal handler no matter how
> many threads you have, i.e. any previous execution (regardless of
> thread) is interrupted.

Not-running threads have already been *interrupted* by context
switching.  Are they interrupted by a signal again?

> When I say interrupted I mean that the signal handling function
> starts to run.  You obviously mean something else.  But this is a
> side issue, it has more to do with reentrance of the function
> interrupted rather than which thread is currently running.

I don't understand why non-signalled threads are relevant as long as a
signal handler only executes thread-safe functions.  The problem of
async-signal-unsafe function is that a thread that took a lock in the
normal context may try to take the same lock in a signal handler
context.  The thread cannot go back to the normal context where the
lock will be released afterwards, but just waits for the lock in the
signal handler.  As a result, the thread gets stuck.  That's
irrelevant to the other threads.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: why "in_sighandler"?

[Jan Djärv]

YAMAMOTO Mitsuharu skrev:
>>>>>> On Tue, 22 Aug 2006 08:38:54 +0200, Jan Djärv <jan.h.d@swipnet.se> said:
> 
>> How can it be otherwise?  If you only have one program counter, that
>> program counter must be changed to the signal handler no matter how
>> many threads you have, i.e. any previous execution (regardless of
>> thread) is interrupted.
> 
> Not-running threads have already been *interrupted* by context
> switching.  Are they interrupted by a signal again?

What exactly do you mean by interrupted?  In the sense that they don't get to 
execute, yes they are interrupted.  When a signal handler is running, no 
threads can run (on a single CPU machine), hence they are all interrupted.


> I don't understand why non-signalled threads are relevant as long as a
> signal handler only executes thread-safe functions.  The problem of
> async-signal-unsafe function is that a thread that took a lock in the
> normal context may try to take the same lock in a signal handler
> context.  The thread cannot go back to the normal context where the
> lock will be released afterwards, but just waits for the lock in the
> signal handler.  As a result, the thread gets stuck.  That's
> irrelevant to the other threads.

That can't be it.  The mutex is recursive, so a thread is able to take it 
multiple times.

	Jan D.

Re: why "in_sighandler"?

[YAMAMOTO Mitsuharu]

>>>>> On Tue, 22 Aug 2006 10:23:40 +0200, Jan Djärv <jan.h.d@swipnet.se> said:

>>> How can it be otherwise?  If you only have one program counter,
>>> that program counter must be changed to the signal handler no
>>> matter how many threads you have, i.e. any previous execution
>>> (regardless of thread) is interrupted.
>> 
>> Not-running threads have already been *interrupted* by context
>> switching.  Are they interrupted by a signal again?

> What exactly do you mean by interrupted?  In the sense that they
> don't get to execute, yes they are interrupted.

I should have used "..." instead of *...*.  I tried to follow your
definition of "interrupt".

> When a signal handler is running, no threads can run (on a single
> CPU machine), hence they are all interrupted.

The signalled-thread is/becomes running just before the signal handler
is executed.  The thread is not "interrupt"ed by context switching,
but by a signal.

>> I don't understand why non-signalled threads are relevant as long
>> as a signal handler only executes thread-safe functions.  The
>> problem of async-signal-unsafe function is that a thread that took
>> a lock in the normal context may try to take the same lock in a
>> signal handler context.  The thread cannot go back to the normal
>> context where the lock will be released afterwards, but just waits
>> for the lock in the signal handler.  As a result, the thread gets
>> stuck.  That's irrelevant to the other threads.

> That can't be it.  The mutex is recursive, so a thread is able to
> take it multiple times.

Recursive mutex is implemented using a simpler lock mechanism.  That's
why I used the term "lock" instead of "mutex".

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: why "in_sighandler"?

[Andreas Schwab]

Jan Djärv <jan.h.d@swipnet.se> writes:

> What exactly do you mean by interrupted?  In the sense that they don't get
> to execute, yes they are interrupted.  When a signal handler is running,
> no threads can run (on a single CPU machine), hence they are all
> interrupted.

Signal handler execution can be preempted like any other code path in your
program.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: why "in_sighandler"?

[Jan Djärv]

YAMAMOTO Mitsuharu skrev:

> 
> Recursive mutex is implemented using a simpler lock mechanism.  That's
> why I used the term "lock" instead of "mutex".

All mutexes in glibc on GNU/Linux are implemented with a lower level lock.  As 
I said before, both fixes have some drawbacks.  If you want to install your 
fix, go right ahead, it will work equally well.

	Jan D.