bug#54667: 29.0.50; posix_spawn breaks emacs-gdb

bug#54667: 29.0.50; posix_spawn breaks emacs-gdb

[Herman, Géza]

There's a gdb frontend: https://github.com/weirdNox/emacs-gdb

With the emacs commit "a60053f836 Use posix_spawn if possible.", 
emacs-gdb doesn't work properly: when starting an executable, gdb says 
that it's running, but in reality, it doesn't start. The process itself 
is started, gdb attaches to it, but the process still not run for some 
reason.

You can reproduce this:
1. install emacs-gdb
2. M-x gdb-executable, enter any executable, like "/bin/ls". emacs-gdb 
should open a new frame.
3. Then press f5 (this executes gdb-run-or-continue), this should start 
the process in gdb.

Before the mentioned commit, this worked, and "ls" was run properly. But 
with this commit, "ls" isn't started.

I checked this with a recent master 
(bd5d136777ef30f36807c7e690413846ed38fce1), and still happens. Adding

#undef USABLE_POSIX_SPAWN
#define USABLE_POSIX_SPAWN 0

to callproc.c at line 49 fixes the issue.

In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 
3.24.20, cairo version 1.16.0)
  of 2022-04-01 built on zetor
Repository revision: bd5d136777ef30f36807c7e690413846ed38fce1
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description: Debian GNU/Linux bullseye/sid

Configured using:
  'configure --with-native-compilation --without-compress-install
  --with-xinput2'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3
THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
   value of $LC_ALL: C.UTF-8
   value of $LANG: en_US.UTF-8
   value of $XMODIFIERS: @im=none
   locale-coding-system: utf-8-unix

Major mode: C++//l

Minor modes in effect:
   gdb-keys-mode: t
   tooltip-mode: t
   global-eldoc-mode: t
   show-paren-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   line-number-mode: t
   indent-tabs-mode: t
   transient-mark-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   abbrev-mode: t

Load-path shadows:
/home/geza/.emacs.d/elpa/transient-20220216.2303/transient hides 
/usr/local/share/emacs/29.0.50/lisp/transient
~/.emacs.d/lisp/emacs-gdb/gdb-mi hides 
/usr/local/share/emacs/29.0.50/lisp/progmodes/gdb-mi

Features:
(shadow sort mail-extr emacsbug message yank-media rmc puny dired
dired-loaddefs rfc822 mml mml-sec epa derived epg rfc6068 epg-config
gnus-util text-property-search time-date mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils gdb-mi comp comp-cstr
warnings rx cl-extra gdb-module hydra lv comint ansi-color ring help-fns
radix-tree cl-print debug backtrace help-mode find-func finder-inf info
helm-easymenu advice package browse-url url url-proxy url-privacy
url-expand url-methods url-history url-cookie url-domsuf url-util
mailcap url-handlers url-parse auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache json map url-vars seq gv subr-x
byte-opt bytecomp byte-compile cconv cc-mode cc-fonts cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs cl-loaddefs cl-lib
iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice
simple cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian
slovak czech european ethiopic indian cyrillic chinese composite
emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help
abbrev obarray cl-preloaded button loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 186444 9283)
  (symbols 48 14946 0)
  (strings 32 49762 3465)
  (string-bytes 1 1931765)
  (vectors 16 27008)
  (vector-slots 8 517036 11837)
  (floats 8 65 232)
  (intervals 56 890 0)
  (buffers 992 20))

bug#54667: 29.0.50; posix_spawn breaks emacs-gdb

[Robert Pluim]

>>>>> On Fri, 1 Apr 2022 12:51:43 +0200, Herman@debbugs.gnu.org, Géza <geza.herman@gmail.com> said:

    Herman> There's a gdb frontend: https://github.com/weirdNox/emacs-gdb
    Herman> With the emacs commit "a60053f836 Use posix_spawn if possible.",
    Herman> emacs-gdb doesn't work properly: when starting an executable, gdb says
    Herman> that it's running, but in reality, it doesn't start. The process
    Herman> itself is started, gdb attaches to it, but the process still not run
    Herman> for some reason.

    Herman> You can reproduce this:
    Herman> 1. install emacs-gdb
    Herman> 2. M-x gdb-executable, enter any executable, like "/bin/ls". emacs-gdb
    Herman> should open a new frame.
    Herman> 3. Then press f5 (this executes gdb-run-or-continue), this should
    Herman> start the process in gdb.

    Herman> Before the mentioned commit, this worked, and "ls" was run
    Herman> properly. But with this commit, "ls" isn't started.

    Herman> I checked this with a recent master
    Herman> (bd5d136777ef30f36807c7e690413846ed38fce1), and still happens. Adding

    Herman> #undef USABLE_POSIX_SPAWN
    Herman> #define USABLE_POSIX_SPAWN 0

    Herman> to callproc.c at line 49 fixes the issue.

Thereʼs a patch from Jürgen Hötzel in <86o82mvybj.fsf@hoetzel.info> on
emacs-devel that should fix it (I haven't had a chance to fully test
it).

Robert
-- 

bug#54667: 29.0.50; posix_spawn breaks emacs-gdb

[Herman, Géza]

I can confirm that it fixes the problem.

For reference, here's the patch: 
https://lists.gnu.org/archive/html/emacs-devel/2022-03/msg00067.html

(Btw., according to this email thread, vfork is faster than posix_spawn 
on Linux, so maybe it'd make sense to use posix_spawn only on non-Linux 
platforms?)

On 2022-04-01 14:16, Robert Pluim wrote:
>>>>>> On Fri, 1 Apr 2022 12:51:43 +0200, Herman@debbugs.gnu.org, Géza <geza.herman@gmail.com> said:
>      Herman> There's a gdb frontend: https://github.com/weirdNox/emacs-gdb
>      Herman> With the emacs commit "a60053f836 Use posix_spawn if possible.",
>      Herman> emacs-gdb doesn't work properly: when starting an executable, gdb says
>      Herman> that it's running, but in reality, it doesn't start. The process
>      Herman> itself is started, gdb attaches to it, but the process still not run
>      Herman> for some reason.
>
>      Herman> You can reproduce this:
>      Herman> 1. install emacs-gdb
>      Herman> 2. M-x gdb-executable, enter any executable, like "/bin/ls". emacs-gdb
>      Herman> should open a new frame.
>      Herman> 3. Then press f5 (this executes gdb-run-or-continue), this should
>      Herman> start the process in gdb.
>
>      Herman> Before the mentioned commit, this worked, and "ls" was run
>      Herman> properly. But with this commit, "ls" isn't started.
>
>      Herman> I checked this with a recent master
>      Herman> (bd5d136777ef30f36807c7e690413846ed38fce1), and still happens. Adding
>
>      Herman> #undef USABLE_POSIX_SPAWN
>      Herman> #define USABLE_POSIX_SPAWN 0
>
>      Herman> to callproc.c at line 49 fixes the issue.
>
> Thereʼs a patch from Jürgen Hötzel in <86o82mvybj.fsf@hoetzel.info> on
> emacs-devel that should fix it (I haven't had a chance to fully test
> it).
>
> Robert

bug#54667: 29.0.50; posix_spawn breaks emacs-gdb

[Robert Pluim]

>>>>> On Fri, 1 Apr 2022 15:29:50 +0200, "Herman, Géza" <geza.herman@gmail.com> said:

    Herman> I can confirm that it fixes the problem.

Thanks, Iʼll see if I can get around to committing it this weekend.

    Herman> For reference, here's the patch:
    Herman> https://lists.gnu.org/archive/html/emacs-devel/2022-03/msg00067.html

    Herman> (Btw., according to this email thread, vfork is faster than
    Herman> posix_spawn on Linux, so maybe it'd make sense to use posix_spawn only
    Herman> on non-Linux platforms?)

In emacs-28 we only use posix_spawn on macOS because its vork is
sub-optimal. I donʼt remember the rationale for switching to using it
everywhere, itʼs undoubtedly in the archives somewhere.

Thanks

Robert
-- 

bug#54667: 29.0.50; posix_spawn breaks emacs-gdb

[Robert Pluim]

tags 54667 fixed
close 54667 29.1
quit

>>>>> On Fri, 01 Apr 2022 16:44:13 +0200, Robert Pluim <rpluim@gmail.com> said:

>>>>> On Fri, 1 Apr 2022 15:29:50 +0200, "Herman, Géza" <geza.herman@gmail.com> said:
    Herman> I can confirm that it fixes the problem.

    Robert> Thanks, Iʼll see if I can get around to committing it this weekend.

    Herman> For reference, here's the patch:
    Herman> https://lists.gnu.org/archive/html/emacs-devel/2022-03/msg00067.html

    Herman> (Btw., according to this email thread, vfork is faster than
    Herman> posix_spawn on Linux, so maybe it'd make sense to use posix_spawn only
    Herman> on non-Linux platforms?)

    Robert> In emacs-28 we only use posix_spawn on macOS because its vork is
    Robert> sub-optimal. I donʼt remember the rationale for switching to using it
    Robert> everywhere, itʼs undoubtedly in the archives somewhere.

Closing.
Committed as 8103b060d8

bug#54667: 29.0.50; posix_spawn breaks emacs-gdb

[Philipp Stephani]

> Am 01.04.2022 um 16:44 schrieb Robert Pluim <rpluim@gmail.com>:
> 
>>>>>> On Fri, 1 Apr 2022 15:29:50 +0200, "Herman, Géza" <geza.herman@gmail.com> said:
> 
>    Herman> I can confirm that it fixes the problem.
> 
> Thanks, Iʼll see if I can get around to committing it this weekend.
> 
>    Herman> For reference, here's the patch:
>    Herman> https://lists.gnu.org/archive/html/emacs-devel/2022-03/msg00067.html
> 
>    Herman> (Btw., according to this email thread, vfork is faster than
>    Herman> posix_spawn on Linux, so maybe it'd make sense to use posix_spawn only
>    Herman> on non-Linux platforms?)
> 
> In emacs-28 we only use posix_spawn on macOS because its vork is
> sub-optimal. I donʼt remember the rationale for switching to using it
> everywhere, itʼs undoubtedly in the archives somewhere.

My reasoning back then was:

1. Using fork/vfork has a few pitfalls (can't call async-signal-unsafe functions in the child process) that posix_spawn avoids (by not allowing arbitrary code between fork and exec).  Therefore, using posix_spawn is simpler and more obviously correct.
2. Because posix_spawn offers less functionality than fork+exec, it could be faster than the latter, e.g. by having it implemented in the kernel and avoiding page table copies.
3. Since we only run CI on GNU/Linux, it's useful to have only one codepath on all Unix-like systems so that issues that only appear on macOS are less likely to go unnoticed.

These arguments are still mostly correct, but there are some counterpoints:
1. We need to keep the fork/vfork code path anyway since posix_spawn doesn't allow us to correctly set up a pseudoterminal, so we still need to deal with the pitfalls.
2. posix_spawn is indeed much faster on macOS, but not so on GNU/Linux.  (I think it's unnecessarily slowed down on GNU/Linux by what I consider a bug in the POSIX standard.)
3. This is still true, but I'm not sure how much it matters given that the implementations of posix_spawn are completely different on GNU/Linux and macOS.