bug#17036: Continuation for Emacs: invoking a process on exit?

bug#17036: Continuation for Emacs: invoking a process on exit?

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 649 bytes --]

Is there a way to give Emacs itself a continuation, i.e. a command to exec
when it exits? Copious searching and cursory examination of the source code
(grepping for atexit, exit, and looking at emacs.c in some more detail)
suggest not.

This would be useful for restarting having updated my configuration (some
of which is non-idempotent), as it would save having manually to issue a
new "emacs" command having waited for it to shut down; overall, up to
several brain-seconds if I don't just sit and watch the process.

It also seems appropriately Lispy to allow a Lisp system's final action to
be to call a continuation...

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 822 bytes --]

bug#17036: Continuation for Emacs: invoking a process on exit?

[Andreas Schwab]

Reuben Thomas <rrt@sc3d.org> writes:

> Is there a way to give Emacs itself a continuation, i.e. a command to exec
> when it exits?

kill-emacs-hook?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#17036: Continuation for Emacs: invoking a process on exit?

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 446 bytes --]

On 18 March 2014 22:52, Andreas Schwab <schwab@linux-m68k.org> wrote:

> Reuben Thomas <rrt@sc3d.org> writes:
>
> > Is there a way to give Emacs itself a continuation, i.e. a command to
> exec
> > when it exits?
>
> kill-emacs-hook?
>

That's not a tail-call: to reexec Emacs, it needs to be a proper exec. It
might work from kill-emacs-hook, but it's surely not safe? Emacs hasn't
finished shutting down when it runs...

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 897 bytes --]

bug#17036: Continuation for Emacs: invoking a process on exit?

[Glenn Morris]

I could imagine a `restart-emacs' command having some small utility.

bug#17036: Continuation for Emacs: invoking a process on exit?

[Stefan]

> I could imagine a `restart-emacs' command having some small utility.

Could the w32 build support something like POSIX's `exec'?


        Stefan

bug#17036: Continuation for Emacs: invoking a process on exit?

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 305 bytes --]

On 19 March 2014 13:10, Stefan <monnier@iro.umontreal.ca> wrote:

> > I could imagine a `restart-emacs' command having some small utility.
>
> Could the w32 build support something like POSIX's `exec'?


Windows has execvp... http://msdn.microsoft.com/en-us/library/3xw6zy53.aspx

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 801 bytes --]

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> Date: Wed, 19 Mar 2014 13:19:28 +0000
> From: Reuben Thomas <rrt@sc3d.org>
> Cc: Andreas Schwab <schwab@linux-m68k.org>, 17036@debbugs.gnu.org
> 
> On 19 March 2014 13:10, Stefan <monnier@iro.umontreal.ca> wrote:
> 
> > > I could imagine a `restart-emacs' command having some small utility.
> >
> > Could the w32 build support something like POSIX's `exec'?
> 
> 
> Windows has execvp... http://msdn.microsoft.com/en-us/library/3xw6zy53.aspx

Don't believe the sales people.  MS's execvp is buggy, and even if we
forget about those bugs, it won't do what is expected here: it won't
keep the file descriptors open in the original process still open in
the overlaid process.  That's because there's no 'exec' system call on
Windows, so execvp is _emulated_: the original process simply invokes
the new one as its child process, and then immediately exits.

So the answer to Stefan is: no, this cannot be done on Windows, not
without some custom code to let the re-executed Emacs inherit all of
the file descriptors which were open in the original Emacs process.

bug#17036: Continuation for Emacs: invoking a process on exit?

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1406 bytes --]

On 19 March 2014 16:51, Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Wed, 19 Mar 2014 13:19:28 +0000
> > From: Reuben Thomas <rrt@sc3d.org>
> > Cc: Andreas Schwab <schwab@linux-m68k.org>, 17036@debbugs.gnu.org
> >
> > On 19 March 2014 13:10, Stefan <monnier@iro.umontreal.ca> wrote:
> >
> > > > I could imagine a `restart-emacs' command having some small utility.
> > >
> > > Could the w32 build support something like POSIX's `exec'?
> >
> >
> > Windows has execvp...
> http://msdn.microsoft.com/en-us/library/3xw6zy53.aspx
>
> Don't believe the sales people.  MS's execvp is buggy, and even if we
> forget about those bugs, it won't do what is expected here: it won't
> keep the file descriptors open in the original process still open in
> the overlaid process.  That's because there's no 'exec' system call on
> Windows, so execvp is _emulated_: the original process simply invokes
> the new one as its child process, and then immediately exits.
>

That's good enough for restart-emacs.


> So the answer to Stefan is: no, this cannot be done on Windows, not
> without some custom code to let the re-executed Emacs inherit all of
> the file descriptors which were open in the original Emacs process.
>

It's fine for what i had in mind, namely Emacs simply launching another
command with arguments, much as a Lisp callcc. This could be documented as
a limitation on Windows.

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 2418 bytes --]

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> Date: Wed, 19 Mar 2014 21:14:22 +0000
> From: Reuben Thomas <rrt@sc3d.org>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Andreas Schwab <schwab@linux-m68k.org>, 
> 	17036@debbugs.gnu.org
> 
> > Don't believe the sales people.  MS's execvp is buggy, and even if we
> > forget about those bugs, it won't do what is expected here: it won't
> > keep the file descriptors open in the original process still open in
> > the overlaid process.  That's because there's no 'exec' system call on
> > Windows, so execvp is _emulated_: the original process simply invokes
> > the new one as its child process, and then immediately exits.
> >
> 
> That's good enough for restart-emacs.

Maybe so, it's hard to say, since you never described what that should
do.

> > So the answer to Stefan is: no, this cannot be done on Windows, not
> > without some custom code to let the re-executed Emacs inherit all of
> > the file descriptors which were open in the original Emacs process.
> >
> 
> It's fine for what i had in mind, namely Emacs simply launching another
> command with arguments, much as a Lisp callcc. This could be documented as
> a limitation on Windows.

I very much doubt that this limitation would not render the whole
issue moot on Windows.  E.g., how will restart-emacs then be different
from a simple call-process?  But again, since you didn't say what the
feature is supposed to do, ...

bug#17036: Continuation for Emacs: invoking a process on exit?

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1979 bytes --]

On 20 March 2014 03:45, Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Wed, 19 Mar 2014 21:14:22 +0000
> > From: Reuben Thomas <rrt@sc3d.org>
> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Andreas Schwab <
> schwab@linux-m68k.org>,
> >       17036@debbugs.gnu.org
> >
> > > Don't believe the sales people.  MS's execvp is buggy, and even if we
> > > forget about those bugs, it won't do what is expected here: it won't
> > > keep the file descriptors open in the original process still open in
> > > the overlaid process.  That's because there's no 'exec' system call on
> > > Windows, so execvp is _emulated_: the original process simply invokes
> > > the new one as its child process, and then immediately exits.
> > >
> >
> > That's good enough for restart-emacs.
>
> Maybe so, it's hard to say, since you never described what that should
> do.
>

I didn't discuss the command (it was Glenn Morris who suggested the name),
but in my original bug report I said: "This would be useful for restarting
having updated my configuration...as it would save having manually to issue a
new 'emacs' command..." For this, a simple "exec emacs" is enough, but why
not throw in command-line arguments too.


>  I very much doubt that this limitation would not render the whole
> issue moot on Windows.  E.g., how will restart-emacs then be different
> from a simple call-process?


Because Emacs does not continue running after it exits. As I said in my
second email to this bug: "...to reexec Emacs, it needs to be a proper exec
[so that] Emacs has[...] finished shutting down when it runs."

If you simply use CallProcess (or fork/exec on POSIX systems), then the
newly-started emacs will be in contention with the old one, even if the old
one has nearly finished exiting.


>  But again, since you didn't say what the
> feature is supposed to do, ...
>

A tail-call, but for processes. (BTW, sorry to have mentioned call/cc
earlier, that was a bad analogy.)

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 3263 bytes --]

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> Date: Thu, 20 Mar 2014 12:02:49 +0000
> From: Reuben Thomas <rrt@sc3d.org>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Andreas Schwab <schwab@linux-m68k.org>, 
> 	17036@debbugs.gnu.org
> 
> I didn't discuss the command (it was Glenn Morris who suggested the name),
> but in my original bug report I said: "This would be useful for restarting
> having updated my configuration...as it would save having manually to issue a
> new 'emacs' command..." For this, a simple "exec emacs" is enough, but why
> not throw in command-line arguments too.

I'm probably missing something, because I don't see how "exec emacs"
will do what you want.  See below.

> >  I very much doubt that this limitation would not render the whole
> > issue moot on Windows.  E.g., how will restart-emacs then be different
> > from a simple call-process?
> 
> Because Emacs does not continue running after it exits. As I said in my
> second email to this bug: "...to reexec Emacs, it needs to be a proper exec
> [so that] Emacs has[...] finished shutting down when it runs."
> 
> If you simply use CallProcess (or fork/exec on POSIX systems), then the
> newly-started emacs will be in contention with the old one, even if the old
> one has nearly finished exiting.

What do you mean by "in contention"?  What contention do you envision?

> A tail-call, but for processes.

But AFAIU, 'exec' is not a tail-call.  It doesn't shut down the
invoking process; in particular, the atexit and on_exit handlers are
not run.  Depending on where you invoke it in Emacs, even the
kill-emacs-hook might not run.  Therefore, you cannot control whether
everything you get in an orderly shutdown, which you will then need
for the restart, will be in order.

There are also all kinds of small details, like the lock files left
behind by the original process -- the PID remains the same after
'exec', AFAIK.  Etc.

IOW, I'm not entirely sure 'exec' will do what you want.

bug#17036: Continuation for Emacs: invoking a process on exit?

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1117 bytes --]

On 20 March 2014 17:43, Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Thu, 20 Mar 2014 12:02:49 +0000
> > From: Reuben Thomas <rrt@sc3d.org>
> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Andreas Schwab <
> schwab@linux-m68k.org>,
> >       17036@debbugs.gnu.org
> >
> > I didn't discuss the command (it was Glenn Morris who suggested the
> name),
> > but in my original bug report I said: "This would be useful for
> restarting
> > having updated my configuration...as it would save having manually to
> issue a
> > new 'emacs' command..." For this, a simple "exec emacs" is enough, but
> why
> > not throw in command-line arguments too.
>
> I'm probably missing something, because I don't see how "exec emacs"
> will do what you want.


I'm sorry, I seem to have made a total hash of explaining something really
simple. In effect, I want kill-emacs-and-exec, which takes a list of
arguments, runs kill-emacs, and then execs the argument list.

In fact, since kill-emacs can't currently take a list, it could be extended
to do so.

save-buffers-kill-emacs could be likewise extended.

Does that make sense now?

[-- Attachment #2: Type: text/html, Size: 1826 bytes --]

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> Date: Thu, 20 Mar 2014 23:10:19 +0000
> From: Reuben Thomas <rrt@sc3d.org>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Andreas Schwab <schwab@linux-m68k.org>, 
> 	17036@debbugs.gnu.org
> 
> I'm sorry, I seem to have made a total hash of explaining something really
> simple. In effect, I want kill-emacs-and-exec, which takes a list of
> arguments, runs kill-emacs, and then execs the argument list.

As long as it does what you want, fine.  But please note that this is
not the same as exiting Emacs and starting a new session, because the
original Emacs didn't shut down all the way, and the PID is the same.

bug#17036: Continuation for Emacs: invoking a process on exit?

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1260 bytes --]

On 21 March 2014 07:53, Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Thu, 20 Mar 2014 23:10:19 +0000
> > From: Reuben Thomas <rrt@sc3d.org>
> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Andreas Schwab <
> schwab@linux-m68k.org>,
> >       17036@debbugs.gnu.org
> >
> > I'm sorry, I seem to have made a total hash of explaining something
> really
> > simple. In effect, I want kill-emacs-and-exec, which takes a list of
> > arguments, runs kill-emacs, and then execs the argument list.
>
> As long as it does what you want, fine.  But please note that this is
> not the same as exiting Emacs and starting a new session, because the
> original Emacs didn't shut down all the way, and the PID is the same.
>

I think I'm still being unclear, sorry.

I am assuming that all the regular atexit handlers have already been
called, and that Emacs is really about to exit. (That is what I mean by
saying that kill-emacs has been run.) So, this could be implemented by
ensuring that the first atexit handler to be registered on startup checks a
"kill-emacs-and-exec" flag, and if it is set, does the exec.
Correspondingly, kill-emacs-and-exec would set the flag, store the
arguments in a suitable place, and then tail-call to kill-emacs.

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 2003 bytes --]

bug#17036: Continuation for Emacs: invoking a process on exit?

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1588 bytes --]

On 21 March 2014 10:09, Reuben Thomas <rrt@sc3d.org> wrote:

>
> On 21 March 2014 07:53, Eli Zaretskii <eliz@gnu.org> wrote:
>
>> > Date: Thu, 20 Mar 2014 23:10:19 +0000
>> > From: Reuben Thomas <rrt@sc3d.org>
>> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Andreas Schwab <
>> schwab@linux-m68k.org>,
>> >       17036@debbugs.gnu.org
>> >
>> > I'm sorry, I seem to have made a total hash of explaining something
>> really
>> > simple. In effect, I want kill-emacs-and-exec, which takes a list of
>> > arguments, runs kill-emacs, and then execs the argument list.
>>
>> As long as it does what you want, fine.  But please note that this is
>> not the same as exiting Emacs and starting a new session, because the
>> original Emacs didn't shut down all the way, and the PID is the same.
>>
>
> I think I'm still being unclear, sorry.
>
> I am assuming that all the regular atexit handlers have already been
> called, and that Emacs is really about to exit. (That is what I mean by
> saying that kill-emacs has been run.) So, this could be implemented by
> ensuring that the first atexit handler to be registered on startup checks a
> "kill-emacs-and-exec" flag, and if it is set, does the exec.
> Correspondingly, kill-emacs-and-exec would set the flag, store the
> arguments in a suitable place, and then tail-call to kill-emacs.
>

And if there is a problem with keeping the same PID, or with file
descriptors still being open, or any other kind of resource that is
released on process exit, then by all means have the atexit handler
mentioned above fork().

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 2729 bytes --]

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> Date: Fri, 21 Mar 2014 10:09:10 +0000
> From: Reuben Thomas <rrt@sc3d.org>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Andreas Schwab <schwab@linux-m68k.org>, 
> 	17036@debbugs.gnu.org
> 
> > As long as it does what you want, fine.  But please note that this is
> > not the same as exiting Emacs and starting a new session, because the
> > original Emacs didn't shut down all the way, and the PID is the same.
> >
> 
> I think I'm still being unclear, sorry.
> 
> I am assuming that all the regular atexit handlers have already been
> called, and that Emacs is really about to exit. (That is what I mean by
> saying that kill-emacs has been run.) So, this could be implemented by
> ensuring that the first atexit handler to be registered on startup checks a
> "kill-emacs-and-exec" flag, and if it is set, does the exec.

Assuming a call to 'exec' is allowed in an atexit handler.  (I don't
know if it is.)

bug#17036: Continuation for Emacs: invoking a process on exit?

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1412 bytes --]

On 21 March 2014 10:18, Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Fri, 21 Mar 2014 10:09:10 +0000
> > From: Reuben Thomas <rrt@sc3d.org>
> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Andreas Schwab <
> schwab@linux-m68k.org>,
> >       17036@debbugs.gnu.org
> >
> > > As long as it does what you want, fine.  But please note that this is
> > > not the same as exiting Emacs and starting a new session, because the
> > > original Emacs didn't shut down all the way, and the PID is the same.
> > >
> >
> > I think I'm still being unclear, sorry.
> >
> > I am assuming that all the regular atexit handlers have already been
> > called, and that Emacs is really about to exit. (That is what I mean by
> > saying that kill-emacs has been run.) So, this could be implemented by
> > ensuring that the first atexit handler to be registered on startup
> checks a
> > "kill-emacs-and-exec" flag, and if it is set, does the exec.
>
> Assuming a call to 'exec' is allowed in an atexit handler.  (I don't
> know if it is.)
>

The only restrictions I can find are that a) if an atexit handler calls
_exit, the remaining handlers are not called; b) if the process is
terminated by a signal, the handlers are not called. The only thing you
can't do is call exit() or longjmp(). You can even call "atexit" from an
atexit handler (the new handler is pushed on the front of the remaining
queue).

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 2203 bytes --]

bug#17036: Continuation for Emacs: invoking a process on exit?

[Lars Ingebrigtsen]

Glenn Morris <rgm@gnu.org> writes:

> I could imagine a `restart-emacs' command having some small utility.

I've now added this command to Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#17036: Continuation for Emacs: invoking a process on exit?

[Reuben Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 405 bytes --]

Thanks for this! I look forward to it.

--
https://rrt.sc3d.org

On Sun, 17 Apr 2022, 12:38 Lars Ingebrigtsen, <larsi@gnus.org> wrote:

> Glenn Morris <rgm@gnu.org> writes:
>
> > I could imagine a `restart-emacs' command having some small utility.
>
> I've now added this command to Emacs 29.
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no
>

[-- Attachment #2: Type: text/html, Size: 951 bytes --]

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> Resent-From: Lars Ingebrigtsen <larsi@gnus.org>
> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
> Resent-CC: bug-gnu-emacs@gnu.org
> Resent-Sender: help-debbugs@gnu.org
> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sun, 17 Apr 2022 13:38:10 +0200
> Cc: 17036@debbugs.gnu.org, Andreas Schwab <schwab@linux-m68k.org>,
>  Reuben Thomas <rrt@sc3d.org>
> 
> Glenn Morris <rgm@gnu.org> writes:
> 
> > I could imagine a `restart-emacs' command having some small utility.
> 
> I've now added this command to Emacs 29.

As implemented, it won't work reliably on MS-Windows, because execvp
there doesn't do what you think it should.  I think we should use
sys_spawnve instead.

bug#17036: Continuation for Emacs: invoking a process on exit?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> As implemented, it won't work reliably on MS-Windows, because execvp
> there doesn't do what you think it should.  I think we should use
> sys_spawnve instead.

Ah, I grepped for execvp to see whether we already used it, but didn't
notice that the hits were from Gnulib.

I'm not familiar with sys_spawnve -- can you do the adjustments?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rgm@gnu.org,  17036@debbugs.gnu.org,  schwab@linux-m68k.org,  rrt@sc3d.org
> Date: Sun, 17 Apr 2022 14:08:49 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > As implemented, it won't work reliably on MS-Windows, because execvp
> > there doesn't do what you think it should.  I think we should use
> > sys_spawnve instead.
> 
> Ah, I grepped for execvp to see whether we already used it, but didn't
> notice that the hits were from Gnulib.
> 
> I'm not familiar with sys_spawnve -- can you do the adjustments?

I don't think I will have the time.  It isn't a simple job, because
just calling sys_spawnve will not do -- that function currently
supports only Emacs sub-processes.

FTR, I will document below the potential issues with the current
implementation of kill-emacs/restart-emacs:

  . when kill-emacs is called with RESTART non-nil, the value of ARG
    is ignored; this should at least be documented;
  . the exit status of the restarted Emacs is discarded, so it will
    not be available to the parent program, at least on MS-Windows,
    and also if execvp fails for some reason;
  . the semantics of the file descriptors open in the original Emacs
    process is not clear to me: will they remain open in the restarted
    Emacs, if the original Emacs opened them without CLOEXEC?
  . does the restarted Emacs belong to the same process group? should
    it?
  . on MS-Windows, if any of the argv[] command-line arguments have
    embedded whitespace, the restarted Emacs will not get the same
    elements in its argv[] array, because the Windows API for starting
    processes accepts the command-line arguments as a single string

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> Resent-From: Eli Zaretskii <eliz@gnu.org>
> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
> Resent-CC: bug-gnu-emacs@gnu.org
> Resent-Sender: help-debbugs@gnu.org
> Date: Sun, 17 Apr 2022 15:34:55 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: rgm@gnu.org, 17036@debbugs.gnu.org, schwab@linux-m68k.org, rrt@sc3d.org
> 
> > From: Lars Ingebrigtsen <larsi@gnus.org>
> > Cc: rgm@gnu.org,  17036@debbugs.gnu.org,  schwab@linux-m68k.org,  rrt@sc3d.org
> > Date: Sun, 17 Apr 2022 14:08:49 +0200
> > 
> > Eli Zaretskii <eliz@gnu.org> writes:
> > 
> > > As implemented, it won't work reliably on MS-Windows, because execvp
> > > there doesn't do what you think it should.  I think we should use
> > > sys_spawnve instead.
> > 
> > Ah, I grepped for execvp to see whether we already used it, but didn't
> > notice that the hits were from Gnulib.
> > 
> > I'm not familiar with sys_spawnve -- can you do the adjustments?
> 
> I don't think I will have the time.  It isn't a simple job, because
> just calling sys_spawnve will not do -- that function currently
> supports only Emacs sub-processes.
> 
> FTR, I will document below the potential issues with the current
> implementation of kill-emacs/restart-emacs:
> 
>   . when kill-emacs is called with RESTART non-nil, the value of ARG
>     is ignored; this should at least be documented;
>   . the exit status of the restarted Emacs is discarded, so it will
>     not be available to the parent program, at least on MS-Windows,
>     and also if execvp fails for some reason;
>   . the semantics of the file descriptors open in the original Emacs
>     process is not clear to me: will they remain open in the restarted
>     Emacs, if the original Emacs opened them without CLOEXEC?
>   . does the restarted Emacs belong to the same process group? should
>     it?
>   . on MS-Windows, if any of the argv[] command-line arguments have
>     embedded whitespace, the restarted Emacs will not get the same
>     elements in its argv[] array, because the Windows API for starting
>     processes accepts the command-line arguments as a single string

And one more:

  . my reading of the code in 'main' is that the argv[] array is
    sorted as it is processed, so the restarted Emacs will get the
    arguments in a different order (not that it should matter too
    much, I think, but still: it isn't exactly the same invocation)

bug#17036: Continuation for Emacs: invoking a process on exit?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>   . when kill-emacs is called with RESTART non-nil, the value of ARG
>     is ignored; this should at least be documented;

Emacs doesn't exit, so I thought it would be self-evident that ARG
(which is all about the exit code) is ignored.

>   . the exit status of the restarted Emacs is discarded, so it will
>     not be available to the parent program, at least on MS-Windows,
>     and also if execvp fails for some reason;

Again, Emacs doesn't exit, so...

>   . the semantics of the file descriptors open in the original Emacs
>     process is not clear to me: will they remain open in the restarted
>     Emacs, if the original Emacs opened them without CLOEXEC?

I thought we opened all file descriptors with CLOEXEC?  If not, that's a
bug, since we'd be leaking file descriptors to programs we start with
`call-process', for instance.

>   . does the restarted Emacs belong to the same process group? should
>     it?

I think so, and I guess so?

>   . on MS-Windows, if any of the argv[] command-line arguments have
>     embedded whitespace, the restarted Emacs will not get the same
>     elements in its argv[] array, because the Windows API for starting
>     processes accepts the command-line arguments as a single string

Sounds like we should just document that this doesn't work on Windows,
then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#17036: Continuation for Emacs: invoking a process on exit?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> And one more:
>
>   . my reading of the code in 'main' is that the argv[] array is
>     sorted as it is processed, so the restarted Emacs will get the
>     arguments in a different order (not that it should matter too
>     much, I think, but still: it isn't exactly the same invocation)

Hm, so we do (in sort_args), so I guess we should store the real argv
somewhere.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> Date: Sun, 17 Apr 2022 15:34:55 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: rgm@gnu.org, 17036@debbugs.gnu.org, schwab@linux-m68k.org, rrt@sc3d.org
> 
> > From: Lars Ingebrigtsen <larsi@gnus.org>
> > Cc: rgm@gnu.org,  17036@debbugs.gnu.org,  schwab@linux-m68k.org,  rrt@sc3d.org
> > Date: Sun, 17 Apr 2022 14:08:49 +0200
> > 
> > Eli Zaretskii <eliz@gnu.org> writes:
> > 
> > > As implemented, it won't work reliably on MS-Windows, because execvp
> > > there doesn't do what you think it should.  I think we should use
> > > sys_spawnve instead.
> > 
> > Ah, I grepped for execvp to see whether we already used it, but didn't
> > notice that the hits were from Gnulib.
> > 
> > I'm not familiar with sys_spawnve -- can you do the adjustments?
> 
> I don't think I will have the time.  It isn't a simple job, because
> just calling sys_spawnve will not do -- that function currently
> supports only Emacs sub-processes.

I eventually had an implementation idea that is much simpler than the
above, so I went ahead and implemented it.  It currently doesn't work
in a -nw session, I hope to debug that later.

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rgm@gnu.org,  17036@debbugs.gnu.org,  schwab@linux-m68k.org,  rrt@sc3d.org
> Date: Sun, 17 Apr 2022 14:49:23 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >   . when kill-emacs is called with RESTART non-nil, the value of ARG
> >     is ignored; this should at least be documented;
> 
> Emacs doesn't exit, so I thought it would be self-evident that ARG
> (which is all about the exit code) is ignored.

Let's see how evident is it ;-)

> >   . the exit status of the restarted Emacs is discarded, so it will
> >     not be available to the parent program, at least on MS-Windows,
> >     and also if execvp fails for some reason;
> 
> Again, Emacs doesn't exit, so...

I meant when the restarted Emacs exits.  It _can_ exit, can't it?

> >   . the semantics of the file descriptors open in the original Emacs
> >     process is not clear to me: will they remain open in the restarted
> >     Emacs, if the original Emacs opened them without CLOEXEC?
> 
> I thought we opened all file descriptors with CLOEXEC?  If not, that's a
> bug, since we'd be leaking file descriptors to programs we start with
> `call-process', for instance.

What about stadin/stdout/stderr etc.?

> >   . does the restarted Emacs belong to the same process group? should
> >     it?
> 
> I think so, and I guess so?

Is that guaranteed? should we make sure it's so?  Or is it
unimportant?

> >   . on MS-Windows, if any of the argv[] command-line arguments have
> >     embedded whitespace, the restarted Emacs will not get the same
> >     elements in its argv[] array, because the Windows API for starting
> >     processes accepts the command-line arguments as a single string
> 
> Sounds like we should just document that this doesn't work on Windows,
> then.

It works now, at least in the GUI sessions.

Some other questions related to this:

  . AFAIU, the execvp'ed process inherits the environment of the
    calling process, so any changes to the environment will be
    propagated to the child, right?  Do we want that?

  . What about the working directory?  If the original Emacs was
    invoked with --chdir, the restart happens in another directory;
    moreover, relative program name in argv[0] and relative directory
    name in --chdir may not work.  Is that a problem?

  . What should happen to client frames when Emacs is restarted?  What
    does happen with the current implementation, .e.g if some of the
    client frames are on other displays?

A lot of this stuff has to do with the semantics of "restarting"
Emacs: what exactly does that mean, and what should users expect/not
expect when they restart Emacs?

bug#17036: Continuation for Emacs: invoking a process on exit?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> >   . the exit status of the restarted Emacs is discarded, so it will
>> >     not be available to the parent program, at least on MS-Windows,
>> >     and also if execvp fails for some reason;
>> 
>> Again, Emacs doesn't exit, so...
>
> I meant when the restarted Emacs exits.  It _can_ exit, can't it?

If you exit from the new process, then you'll get the exit code from
that next call to `kill-emacs'?

>> I thought we opened all file descriptors with CLOEXEC?  If not, that's a
>> bug, since we'd be leaking file descriptors to programs we start with
>> `call-process', for instance.
>
> What about stadin/stdout/stderr etc.?

Those are inherited, I think.  (Which is good.)

>> >   . does the restarted Emacs belong to the same process group? should
>> >     it?
>> 
>> I think so, and I guess so?
>
> Is that guaranteed? should we make sure it's so?  Or is it
> unimportant?

I think that whatever the OS does here by default is what's correct, so
it's not something we need to worry about.

>> Sounds like we should just document that this doesn't work on Windows,
>> then.
>
> It works now, at least in the GUI sessions.

Great!

> Some other questions related to this:
>
>   . AFAIU, the execvp'ed process inherits the environment of the
>     calling process, so any changes to the environment will be
>     propagated to the child, right?  Do we want that?

I'm not quite sure.  We could save envp and call execve instead with
that saved envp (I think?), but I think using the environment is what we
want here.  Probably?  But perhaps that should be an option...

>   . What about the working directory?  If the original Emacs was
>     invoked with --chdir, the restart happens in another directory;
>     moreover, relative program name in argv[0] and relative directory
>     name in --chdir may not work.  Is that a problem?

Ah, yes it is.  --chdir and relative names stops this from working, so
that needs fixing, I think. 

>   . What should happen to client frames when Emacs is restarted?  What
>     does happen with the current implementation, .e.g if some of the
>     client frames are on other displays?

I expect those to disappear.  We're restarting Emacs, after all.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rgm@gnu.org,  17036@debbugs.gnu.org,  schwab@linux-m68k.org,  rrt@sc3d.org
> Date: Sun, 17 Apr 2022 16:49:42 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> >   . the exit status of the restarted Emacs is discarded, so it will
> >> >     not be available to the parent program, at least on MS-Windows,
> >> >     and also if execvp fails for some reason;
> >> 
> >> Again, Emacs doesn't exit, so...
> >
> > I meant when the restarted Emacs exits.  It _can_ exit, can't it?
> 
> If you exit from the new process, then you'll get the exit code from
> that next call to `kill-emacs'?

Do you see that returned to the shell?

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> Date: Sun, 17 Apr 2022 17:29:20 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: rgm@gnu.org, 17036@debbugs.gnu.org, schwab@linux-m68k.org, rrt@sc3d.org
> 
> I eventually had an implementation idea that is much simpler than the
> above, so I went ahead and implemented it.  It currently doesn't work
> in a -nw session, I hope to debug that later.

The problems in the -nw session have something top do with the parent
releasing the console device and the restarted child attaching to it;
I'm unable to get that to work (not really surprising, as I know very
little about the subtleties of the Windows console sharing).  So for
now I just made restart-emacs fail in the -nw case on MS-Windows.

Btw, calling 'error' in this place, which we also do on Posix
platforms, is not very useful: this is past the call to
shut_down_emacs, so all 'error' does at this point is exit with an
error status, and the error message is lost.  Something to keep in
mind, I guess.

bug#17036: Continuation for Emacs: invoking a process on exit?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Btw, calling 'error' in this place, which we also do on Posix
> platforms, is not very useful: this is past the call to
> shut_down_emacs, so all 'error' does at this point is exit with an
> error status, and the error message is lost.  Something to keep in
> mind, I guess.

It prints the error message OK on Debian, at least, but I guess
emacs_perror is the function to use here?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rgm@gnu.org,  17036@debbugs.gnu.org,  schwab@linux-m68k.org,  rrt@sc3d.org
> Date: Sun, 17 Apr 2022 18:02:15 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Btw, calling 'error' in this place, which we also do on Posix
> > platforms, is not very useful: this is past the call to
> > shut_down_emacs, so all 'error' does at this point is exit with an
> > error status, and the error message is lost.  Something to keep in
> > mind, I guess.
> 
> It prints the error message OK on Debian, at least, but I guess
> emacs_perror is the function to use here?

I guess so.  Although that will be lost as well if Emacs is invoked as
a GUI program from some desktop shortcut or in some other fancy way
which redirects the standard handles to the great void...

bug#17036: Continuation for Emacs: invoking a process on exit?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> If you exit from the new process, then you'll get the exit code from
>> that next call to `kill-emacs'?
>
> Do you see that returned to the shell?

When you finally exit (without restarting), the function returns the
same value as always, so I think I must be misunderstanding what you
mean here.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#17036: Continuation for Emacs: invoking a process on exit?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> I guess so.  Although that will be lost as well if Emacs is invoked as
> a GUI program from some desktop shortcut or in some other fancy way
> which redirects the standard handles to the great void...

Yup, but we've already shut down Emacs at this point, so I don't think
we can do much more.

But there is one check we could do at a more meaningful point: We could
check whether argv[0] points to a binary that exists at the start of
`kill-emacs' and then signal an error.  That will probably be the (by
far) most common problem in practice, so it's worth doing that, I think.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#17036: Continuation for Emacs: invoking a process on exit?

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rgm@gnu.org,  17036@debbugs.gnu.org,  schwab@linux-m68k.org,  rrt@sc3d.org
> Date: Mon, 18 Apr 2022 10:48:17 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> If you exit from the new process, then you'll get the exit code from
> >> that next call to `kill-emacs'?
> >
> > Do you see that returned to the shell?
> 
> When you finally exit (without restarting), the function returns the
> same value as always, so I think I must be misunderstanding what you
> mean here.

I was just asking whether this was checked and worked correctly.