[PATCH] At-exit hook

[PATCH] At-exit hook

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 1922 bytes --]

Hi,

I think it would be good to have an at-exit-hook which is run atexit(). The
motivation is that Guile can provide bindings for libraries which may want
to clean up resources at exit. If the at-exit-hook exists, this would then
be one way to make sure that the linked in library can do this at exit. You
could say that it is not needed since the bindings themselves could call
atexit(), but this presumes that the bindings are written in C. Nowadays
with (system foreign) and Nyacc, the bindings could very well be written
entirely in Scheme.

An alternative would then, of course, be to create a Guile binding for
atexit(), but as you can see from the included patch, providing this
functionality as a hook gives us better control with regards to running
cleanup code in the proper context.

I can apply this patch with proper additions to NEWS etc myself, but I
wanted to first give you a chance to protest.

So, what do you say?

Best regards,
Mikael

diff --git a/libguile/init.c b/libguile/init.c
index 3df8c5ae5..2bf69c9f9 100644
--- a/libguile/init.c
+++ b/libguile/init.c
@@ -327,9 +327,12 @@ invoke_main_func (void *body_data)
 scm_i_pthread_mutex_t scm_i_init_mutex = SCM_I_PTHREAD_MUTEX_INITIALIZER;
 int scm_initialized_p = 0;

+SCM scm_at_exit_hook;
+
 static void *
 really_cleanup_for_exit (void *unused)
 {
+  scm_c_run_hook (scm_at_exit_hook, SCM_EOL);
   scm_flush_all_ports ();
   return NULL;
 }
@@ -351,6 +354,13 @@ cleanup_for_exit ()
   scm_with_guile (really_cleanup_for_exit, NULL);
 }

+static void
+init_at_exit_hook ()
+{
+  scm_at_exit_hook = scm_make_hook (SCM_INUM0);
+  scm_c_define ("at-exit-hook", scm_at_exit_hook);
+}
+
 void
 scm_i_init_guile (void *base)
 {
@@ -504,6 +514,7 @@ scm_i_init_guile (void *base)
   scm_init_rw ();
   scm_init_extensions ();

+  init_at_exit_hook ();
   atexit (cleanup_for_exit);
   scm_load_startup_files ();
   scm_init_load_should_auto_compile ();

[-- Attachment #2: Type: text/html, Size: 2284 bytes --]

RE: [PATCH] At-exit hook

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 408 bytes --]

‘atexit’ functions are run at ‘exit’. ‘exit’ can be run from signal handlers (*). Since the hook runs Scheme code, it could do a lot of AC-unsafe things, resulting in problems.

(*) glibc documentation says ‘exit’ is AC-unsafe, but this is unsupported by POSIX AFAICT. OTOH the same applies to even ‘malloc’, so likely I’m looking in the wrong places.

Best regards,
Maxime Devos

[-- Attachment #2: Type: text/html, Size: 1726 bytes --]

Re: [PATCH] At-exit hook

[Nala Ginrut]

[-- Attachment #1: Type: text/plain, Size: 600 bytes --]

How about using sigprocmask to disable async signals temporarily?


On Thu, Nov 7, 2024 at 8:23 PM Maxime Devos <maximedevos@telenet.be> wrote:

> ‘atexit’ functions are run at ‘exit’. ‘exit’ can be run from signal
> handlers (*). Since the hook runs Scheme code, it could do a lot of
> AC-unsafe things, resulting in problems.
>
>
>
> (*) glibc documentation says ‘exit’ is AC-unsafe, but this is unsupported
> by POSIX AFAICT. OTOH the same applies to even ‘malloc’, so likely I’m
> looking in the wrong places.
>
>
>
> Best regards,
>
> Maxime Devos
>

[-- Attachment #2: Type: text/html, Size: 1385 bytes --]

Re: [PATCH] At-exit hook

[Mailer]

On Thu, 7 Nov 2024 12:23:08 +0100
Maxime Devos <maximedevos@telenet.be> wrote:
> ‘atexit’ functions are run at ‘exit’. ‘exit’ can be run from signal
> handlers (*). Since the hook runs Scheme code, it could do a lot of
> AC-unsafe things, resulting in problems.
> 
> (*) glibc documentation says ‘exit’ is AC-unsafe, but this is
> unsupported by POSIX AFAICT. OTOH the same applies to even ‘malloc’,
> so likely I’m looking in the wrong places.

I think you meant async-signal-safe (AS-safe).  'exit' is not a-s-s and
cannot be called in a signal handler (for example it can flush buffers)
whereas '_exit' is a-s-s.  Furthermore a registered handler cannot
itself safely call 'exit'.

I believe the main reason that use of 'atexit' or 'on_exit' is
discouraged is that it does not handle abnormal process termination.
(Registered handlers also don't run on termination by '_exit', but that
is usually what you want.)

Chris

Re: [PATCH] At-exit hook

[Mailer]

On Thu, 7 Nov 2024 12:09:25 +0000
Mailer <vine24683579@gmail.com> wrote:
> On Thu, 7 Nov 2024 12:23:08 +0100
> Maxime Devos <maximedevos@telenet.be> wrote:
> > ‘atexit’ functions are run at ‘exit’. ‘exit’ can be run from signal
> > handlers (*). Since the hook runs Scheme code, it could do a lot of
> > AC-unsafe things, resulting in problems.
> > 
> > (*) glibc documentation says ‘exit’ is AC-unsafe, but this is
> > unsupported by POSIX AFAICT. OTOH the same applies to even ‘malloc’,
> > so likely I’m looking in the wrong places.
> 
> I think you meant async-signal-safe (AS-safe).  'exit' is not a-s-s and
> cannot be called in a signal handler (for example it can flush buffers)
> whereas '_exit' is a-s-s.  Furthermore a registered handler cannot
> itself safely call 'exit'.
> 
> I believe the main reason that use of 'atexit' or 'on_exit' is
> discouraged is that it does not handle abnormal process termination.
> (Registered handlers also don't run on termination by '_exit', but that
> is usually what you want.)

I believe also that use of 'atexit' is discouraged in dynamically linked
libraries because of the uncertain timing of the unloading of the
library, but I think in fact glibc is OK with this, so I guess it may
depend on your libc.

Chris

Re: [PATCH] At-exit hook

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 2267 bytes --]

OK, so people have brought up two issues:

1. It is for various reasons not recommended to call atexit() from a
dynamically linked library (which Guile already does before my suggested
change, n.b.).

2. It is not async signal safe.

A suggested remedy would then be:

Instead of calling the at-exit-hook from really_cleanup_for_exit, we could
call it (still within an scm_with_guile) from the end of scm_boot_guile(),
just before exit(), with the disadvantage that it wouldn't be called if
main_func() calls exit on its own. It's kind of a pity that we didn't early
on introduce some kind of scm_finalize_guile() which the user would have to
call when done with the library...

And, well, perhaps we should block asyncs, but I don't know about signals
with this new setup.

Best regards,
Mikael

On Thu, Nov 7, 2024 at 1:26 PM Mailer <vine24683579@gmail.com> wrote:

> On Thu, 7 Nov 2024 12:09:25 +0000
> Mailer <vine24683579@gmail.com> wrote:
> > On Thu, 7 Nov 2024 12:23:08 +0100
> > Maxime Devos <maximedevos@telenet.be> wrote:
> > > ‘atexit’ functions are run at ‘exit’. ‘exit’ can be run from signal
> > > handlers (*). Since the hook runs Scheme code, it could do a lot of
> > > AC-unsafe things, resulting in problems.
> > >
> > > (*) glibc documentation says ‘exit’ is AC-unsafe, but this is
> > > unsupported by POSIX AFAICT. OTOH the same applies to even ‘malloc’,
> > > so likely I’m looking in the wrong places.
> >
> > I think you meant async-signal-safe (AS-safe).  'exit' is not a-s-s and
> > cannot be called in a signal handler (for example it can flush buffers)
> > whereas '_exit' is a-s-s.  Furthermore a registered handler cannot
> > itself safely call 'exit'.
> >
> > I believe the main reason that use of 'atexit' or 'on_exit' is
> > discouraged is that it does not handle abnormal process termination.
> > (Registered handlers also don't run on termination by '_exit', but that
> > is usually what you want.)
>
> I believe also that use of 'atexit' is discouraged in dynamically linked
> libraries because of the uncertain timing of the unloading of the
> library, but I think in fact glibc is OK with this, so I guess it may
> depend on your libc.
>
> Chris
>
>

[-- Attachment #2: Type: text/html, Size: 3042 bytes --]

RE: [PATCH] At-exit hook

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1486 bytes --]

On Thu, 7 Nov 2024 12:23:08 +0100
>Maxime Devos <maximedevos@telenet.be> wrote:
>> ‘atexit’ functions are run at ‘exit’. ‘exit’ can be run from signal
>> handlers (*). Since the hook runs Scheme code, it could do a lot of
>> AC-unsafe things, resulting in problems.
>> 
>> (*) glibc documentation says ‘exit’ is AC-unsafe, but this is
>> unsupported by POSIX AFAICT. OTOH the same applies to even ‘malloc’,
>> so likely I’m looking in the wrong places.

>I think you meant async-signal-safe (AS-safe).  'exit' is not a-s-s and
>cannot be called in a signal handler (for example it can flush buffers)
>whereas '_exit' is a-s-s.  Furthermore a registered handler cannot
>itself safely call 'exit'. […]

No, I did mean exactly what I wrote. Read the glibc documentation of ‘exit’ and you’ll see. (Likewise for the POSIX page for ‘exit’ – POSIX does not seem to restrict things to _outside_ signal handlers.)

Also, when two authorative sources (POSIX and glibc in this case) have contrary claims, then simply repeating one of those claim does not help at all, you would need to explain the cause of the discrepancy instead.

That ‘exit’ flushes buffers does not imply that ‘exit’ is async-unsafe, alternatives include buffer flushing being safe, ‘exit’ having its own implementation of flushing that is AC-safe, or ‘you may call ‘exit’ but only if no files (as in FILE*) are open’.

Best regards,
Maxime Devos

[-- Attachment #2: Type: text/html, Size: 3846 bytes --]

Re: [PATCH] At-exit hook

[Mailer]

On Thu, 7 Nov 2024 17:10:45 +0100
Maxime Devos <maximedevos@telenet.be> wrote:

> On Thu, 7 Nov 2024 12:23:08 +0100
> >Maxime Devos <maximedevos@telenet.be> wrote:
> >> ‘atexit’ functions are run at ‘exit’. ‘exit’ can be run from signal
> >> handlers (*). Since the hook runs Scheme code, it could do a lot of
> >> AC-unsafe things, resulting in problems.
> >> 
> >> (*) glibc documentation says ‘exit’ is AC-unsafe, but this is
> >> unsupported by POSIX AFAICT. OTOH the same applies to even ‘malloc’,
> >> so likely I’m looking in the wrong places.
> 
> >I think you meant async-signal-safe (AS-safe).  'exit' is not a-s-s and
> >cannot be called in a signal handler (for example it can flush buffers)
> >whereas '_exit' is a-s-s.  Furthermore a registered handler cannot
> >itself safely call 'exit'. […]
> 
> No, I did mean exactly what I wrote. Read the glibc documentation of ‘exit’ and you’ll see. (Likewise for the POSIX page for ‘exit’ – POSIX does not seem to restrict things to _outside_ signal handlers.)
> 
> Also, when two authorative sources (POSIX and glibc in this case) have contrary claims, then simply repeating one of those claim does not help at all, you would need to explain the cause of the discrepancy instead.
> 
> That ‘exit’ flushes buffers does not imply that ‘exit’ is async-unsafe, alternatives include buffer flushing being safe, ‘exit’ having its own implementation of flushing that is AC-safe, or ‘you may call ‘exit’ but only if no files (as in FILE*) are open’.
> 
> Best regards,
> Maxime Devos

You have lost me.  "AC-safe" means async-cancel-safe.  It is irrelevant:

1. Only three POSIX functions are async cancelation safe, namely
pthread_cancel, pthread_setcancelstate, and pthread_setcanceltype.  See
the POSIX standard of 2017 (the only one I have to hand), General
Information, paragraph 2.9.5, Async-Cancel Safety: "No other functions
in this volume of POSIX.1-2017 are required to be async-cancel-safe."
The GNU documentation says the same.

2. No one ever uses asynchronous cancelation anyway, partly because of
that.  Deferred thread cancelation at safe cancelation points is the
only cancelation used in practice.

3. AC-Safety has no bearing on the current discussion in any case.

On Async-Signal Safety, whatever you may say, 'exit' is not on the
POSIX list of async-signal-safe functions.  See the POSIX standard of
2017, General Information, paragraph 2.4.3, Signal Actions: "Any
function not in the above table may be unsafe with respect to
signals."   Do 'man 7 signal-safety', also at
https://man7.org/linux/man-pages/man7/signal-safety.7.html, to see your
implementation's list, which includes '_exit' but not 'exit' (on my
distribution), thus conforming with POSIX.

AS-Safety is probably also irrelevant because as I understand it guile
implements its own deferred signal delivery with asyncs, which may or
not permit guile's exit to be invoked in an async handler (I have never
examined it to find out).  POSIX and glibc documentation is not
authoritative on that.

Chris

RE: [PATCH] At-exit hook

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 3309 bytes --]

On Thu, 7 Nov 2024 17:10:45 +0100
Maxime Devos <maximedevos@telenet.be> wrote:

> On Thu, 7 Nov 2024 12:23:08 +0100
> >Maxime Devos <maximedevos@telenet.be> wrote:
> >> ‘atexit’ functions are run at ‘exit’. ‘exit’ can be run from signal
> >> handlers (*). Since the hook runs Scheme code, it could do a lot of
> >> AC-unsafe things, resulting in problems.
> >> 
> >> (*) glibc documentation says ‘exit’ is AC-unsafe, but this is
> >> unsupported by POSIX AFAICT. OTOH the same applies to even ‘malloc’,
> >> so likely I’m looking in the wrong places.
> 
> >I think you meant async-signal-safe (AS-safe).  'exit' is not a-s-s and
> >cannot be called in a signal handler (for example it can flush buffers)
> >whereas '_exit' is a-s-s.  Furthermore a registered handler cannot
> >itself safely call 'exit'. […]
> 
> No, I did mean exactly what I wrote. Read the glibc documentation of ‘exit’ and you’ll see. (Likewise for the POSIX page for ‘exit’ – POSIX does not seem to restrict things to _outside_ signal handlers.)
> 
> Also, when two authorative sources (POSIX and glibc in this case) have contrary claims, then simply repeating one of those claim does not help at all, you would need to explain the cause of the discrepancy instead.
> 
> That ‘exit’ flushes buffers does not imply that ‘exit’ is async-unsafe, alternatives include buffer flushing being safe, ‘exit’ having its own implementation of flushing that is AC-safe, or ‘you may call ‘exit’ but only if no files (as in FILE*) are open’.
> 
> Best regards,
> Maxime Devos

>>You have lost me.  "AC-safe" means async-cancel-safe.  It is irrelevant: […]

Right, I meant AS-unsafe(=async signal safe). (but not AS-safe).

>On Async-Signal Safety, whatever you may say, 'exit' is not on the
POSIX list of async-signal-safe functions.  See the POSIX standard of
2017, General Information, paragraph 2.4.3, Signal Actions: "Any
function not in the above table may be unsafe with respect to
signals."   Do 'man 7 signal-safety', also at
https://man7.org/linux/man-pages/man7/signal-safety.7.html, to see your
implementation's list, which includes '_exit' but not 'exit' (on my
distribution), thus conforming with POSIX.

This explains the discrepancy: POSIX confusingly separates the AS-safety information from the other documentation of the function.

> AS-Safety is probably also irrelevant because as I understand it guile
implements its own deferred signal delivery with asyncs, which may or
not permit guile's exit to be invoked in an async handler (I have never
examined it to find out).  POSIX and glibc documentation is not
authoritative on that.

I wasn’t considering guile’s exit, I was considering C’s exit, and under the assumption that it is AS-safe. Guile’s ‘exit’ procedure is irrelevant here (and IIRC doesn’t actually exist).

Guile does not block all signals or install signal handlers for everything. Hence, one of these could have a signal handler that is run as a POSIX signal handler, and (under ‘exit’ is AS-safe assumption) runs ‘exit’. Then the proposed Scheme exit hooks would be invoked, and could get in trouble (no malloc from within a signal handler, etc.).

Best regards,
Maxime Devos.



[-- Attachment #2: Type: text/html, Size: 7659 bytes --]

Re: [PATCH] At-exit hook

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]

Hi all,

I've come to the conclusion that this problem is unsolvable and I will
abandon the idea of an at-exit-hook. The most serious potential consequence
of running it in really_cleanup_for_exit(), which is indirectly invoked
through atexit(), is that the resources to clean up or the Guile library
itself has already been deallocated.

Also, calling it from scm_boot_guile is a half measure since the
application might exit from main_func(). I thank Chris and Maxime for
pointing out the problems.

I actually think that I tried to do this (introduce an at-exit-hook) 25
years ago, with the same eventual realizations, but didn't recall that at
first. Brace for 2050! :-))

Best regards,
Mikael

[-- Attachment #2: Type: text/html, Size: 860 bytes --]