bug#47439: 27.2; In daemon mode, if after-init-hook errors out, the server does not start

bug#47439: 27.2; In daemon mode, if after-init-hook errors out, the server does not start

[Gilles]

1. Make an init file with the following line: (add-hook 'after-init-hook 'barf)
2. Start Emacs in daemon mode: HOME=$PWD emacs --no-site-file --daemon=foo

Expected behavior: Emacs starts normally (including starting the
server), with the error logged in *Messages* (and optionally on Emacs's
stderr), as happens for errors in the init file or in
emacs-startup-hook.

Actual behavior: the server does not start, so there is no way to
open a frame other than the initial daemon frame.

I did most of the investigation with a debug build of Emacs 27.2 on
Linux, but I observed the same symptoms with Emacs 27.1 on macOS,
with the official Ubuntu 20.04 build of Emacs 26.3, and with the
official Ubuntu 16.04 build of Emacs 24.5.1.

In GNU Emacs 27.2 (build 2, x86_64-pc-linux-gnu, X toolkit, Xaw scroll bars)
 of 2021-03-27 built on darkstar
Repository revision: deef5efafb70f4b171265b896505b92b6eef24e6
Repository branch: HEAD
Windowing system distributor 'The X.Org Foundation', version 11.0.12009000
System Description: Ubuntu 20.04.2 LTS

Configured using:
 'configure --with-x-toolkit=lucid --enable-checking=yes,glyphs
 --enable-check-lisp-object-type
 --prefix=/home/gilles/Packages/emacs-27.2-dbg 'CFLAGS=-g3 -O0'
 LDFLAGS=-g3'

Configured features:
XPM JPEG TIFF GIF PNG SOUND GSETTINGS GLIB NOTIFY INOTIFY LIBSELINUX
GNUTLS LIBXML2 FREETYPE HARFBUZZ XFT ZLIB TOOLKIT_SCROLL_BARS LUCID X11
XDBE XIM MODULES THREADS PDUMPER GMP

Important settings:
  value of $LC_CTYPE: en_US.UTF-8
  locale-coding-system: utf-8-unix

Backtrace of the daemon process:
[New LWP 1377142]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f214d60f246 in __pselect (nfds=8, readfds=0x7ffeb73ecd70,
writefds=0x7ffeb73ecdf0, exceptfds=0x0, timeout=<optimized out>,
sigmask=<optimized out>) at ../sysdeps/unix/sysv/linux/pselect.c:48
48      ../sysdeps/unix/sysv/linux/pselect.c: No such file or directory.
SIGINT is used by the debugger.
Are you sure you want to change it? (y or n) [answered Y; input not
from terminal]
DISPLAY = :0
TERM = xterm
Breakpoint 1 at 0x562be7588dbd: file emacs.c, line 379.
Breakpoint 2 at 0x562be753a6f6: file xterm.c, line 10145.
#0  0x00007f214d60f246 in __pselect (nfds=8, readfds=0x7ffeb73ecd70,
writefds=0x7ffeb73ecdf0, exceptfds=0x0, timeout=<optimized out>,
sigmask=<optimized out>) at ../sysdeps/unix/sysv/linux/pselect.c:48
#1  0x0000562be7845fbe in really_call_select (arg=0x7ffeb73ecc60) at
thread.c:586
#2  0x0000562be76ac0ff in flush_stack_call_func (func=0x562be7845ef3
<really_call_select>, arg=0x7ffeb73ecc60) at alloc.c:4951
#3  0x0000562be78460ba in thread_select (func=0x7f214d60f180
<__pselect>, max_fds=8, rfds=0x7ffeb73ecd70, wfds=0x7ffeb73ecdf0,
efds=0x0, timeout=0x7ffeb73ed3a0, sigmask=0x0) at thread.c:616
#4  0x0000562be78ae770 in xg_select (fds_lim=8, rfds=0x7ffeb73ed410,
wfds=0x7ffeb73ed490, efds=0x0, timeout=0x7ffeb73ed3a0, sigmask=0x0) at
xgselect.c:117
#5  0x0000562be77b7c7c in wait_reading_process_output (time_limit=0,
nsecs=0, read_kbd=-1, do_display=true, wait_for_cell=XIL(0),
wait_proc=0x0, just_wait_proc=0) at process.c:5572
#6  0x0000562be759f260 in kbd_buffer_get_event (kbp=0x7ffeb73ed770,
used_mouse_menu=0x7ffeb73edd75, end_time=0x0) at keyboard.c:3866
#7  0x0000562be759989d in read_event_from_main_queue (end_time=0x0,
local_getcjmp=0x7ffeb73edb80, used_mouse_menu=0x7ffeb73edd75) at
keyboard.c:2156
#8  0x0000562be7599cb3 in read_decoded_event_from_main_queue
(end_time=0x0, local_getcjmp=0x7ffeb73edb80, prev_event=XIL(0),
used_mouse_menu=0x7ffeb73edd75) at keyboard.c:2220
#9  0x0000562be759c030 in read_char (commandflag=1,
map=XIL(0x562be8a95aa3), prev_event=XIL(0),
used_mouse_menu=0x7ffeb73edd75, end_time=0x0) at keyboard.c:2830
#10 0x0000562be75ae73b in read_key_sequence (keybuf=0x7ffeb73edf60,
prompt=XIL(0), dont_downcase_last=false, can_return_switch_frame=true,
fix_current_buffer=true, prevent_redisplay=false) at keyboard.c:9554
#11 0x0000562be7596f70 in command_loop_1 () at keyboard.c:1350
#12 0x0000562be770e339 in internal_condition_case (bfun=0x562be7596ad2
<command_loop_1>, handlers=XIL(0x90), hfun=0x562be7596082 <cmd_error>)
at eval.c:1356
#13 0x0000562be7596693 in command_loop_2 (ignore=XIL(0)) at keyboard.c:1091
#14 0x0000562be770d71c in internal_catch (tag=XIL(0xcc60),
func=0x562be7596662 <command_loop_2>, arg=XIL(0)) at eval.c:1117
#15 0x0000562be759662d in command_loop () at keyboard.c:1070
#16 0x0000562be7595b49 in recursive_edit_1 () at keyboard.c:714
#17 0x0000562be7595d49 in Frecursive_edit () at keyboard.c:786
#18 0x0000562be758b75d in main (argc=3, argv=0x7ffeb73ee3e8) at emacs.c:2067
[Inferior 1 (process 1377141) detached]

bug#47439: 27.2; In daemon mode, if after-init-hook errors out, the server does not start

[Lars Ingebrigtsen]

Gilles <gilles.usenet@gmail.com> writes:

> 1. Make an init file with the following line: (add-hook 'after-init-hook 'barf)
> 2. Start Emacs in daemon mode: HOME=$PWD emacs --no-site-file --daemon=foo
>
> Expected behavior: Emacs starts normally (including starting the
> server), with the error logged in *Messages* (and optionally on Emacs's
> stderr), as happens for errors in the init file or in
> emacs-startup-hook.
>
> Actual behavior: the server does not start, so there is no way to
> open a frame other than the initial daemon frame.

The doc string for this variable is pretty explicit about errors:

---
There is no `condition-case' around the running of this hook;
therefore, if `debug-on-error' is non-nil, an error in one of
these functions will invoke the debugger.
---

So I think this is expected behaviour?  Anybody got an opinion?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47439: 27.2; In daemon mode, if after-init-hook errors out, the server does not start

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sun, 28 Mar 2021 15:56:01 +0200
> Cc: 47439@debbugs.gnu.org
> 
> There is no `condition-case' around the running of this hook;
> therefore, if `debug-on-error' is non-nil, an error in one of
> these functions will invoke the debugger.
> ---
> 
> So I think this is expected behaviour?  Anybody got an opinion?

The problem here is that if the daemon signals an error any place
during startup before it starts the server, there's no way at all to
communicate with the daemon, so you cannot know what happened and why.

So I think we should provide one of the following, in case of such
errors:

 . exit the daemon and leave the description of the problem on some
   disk file, or display it on the screen
 . start the server and allow clients to connect and see the error
   message, with or without the backtrace

bug#47439: 27.2; In daemon mode, if after-init-hook errors out, the server does not start

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> The problem here is that if the daemon signals an error any place
> during startup before it starts the server, there's no way at all to
> communicate with the daemon, so you cannot know what happened and why.
>
> So I think we should provide one of the following, in case of such
> errors:
>
>  . exit the daemon and leave the description of the problem on some
>    disk file, or display it on the screen
>  . start the server and allow clients to connect and see the error
>    message, with or without the backtrace

Ah, right -- in this error mode, the Emacs starts, but there's no
feedback on the error -- it just hangs, and if you `C-c' it on the
command line, the Emacs is running in the background, but the daemon
hasn't started.

So that's, indeed, a very bad way to handle errors here.

I don't know why the doc string here specifies that there's no error
handling of this hook in particular -- that's the default for hooks,
right?

It seems that it first appeared in 1992:

+  "Functions to call after loading the init file (~/.emacs).
+The call is not protected by a condition-case, so you can set `debug-on-error'
+in .emacs, and put all the actual code on `after-init-hook'.")

Which is even more confusing -- it seems like the point here is that
this is a way to run init code, but get debugging?  We now have a
separate facility for that, so that bit doesn't seem very important any
more.

Anyway -- I think perhaps continuing on here and starting the daemon
might be the most useful solution here, perhaps?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47439: 27.2; In daemon mode, if after-init-hook errors out, the server does not start

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: gilles.usenet@gmail.com,  47439@debbugs.gnu.org
> Date: Sun, 28 Mar 2021 16:43:48 +0200
> 
> I don't know why the doc string here specifies that there's no error
> handling of this hook in particular -- that's the default for hooks,
> right?
> 
> It seems that it first appeared in 1992:
> 
> +  "Functions to call after loading the init file (~/.emacs).
> +The call is not protected by a condition-case, so you can set `debug-on-error'
> +in .emacs, and put all the actual code on `after-init-hook'.")
> 
> Which is even more confusing -- it seems like the point here is that
> this is a way to run init code, but get debugging?  We now have a
> separate facility for that, so that bit doesn't seem very important any
> more.

It definitely predates the daemon. and I think it relates to what
"--debug-init" does.

> Anyway -- I think perhaps continuing on here and starting the daemon
> might be the most useful solution here, perhaps?

You mean, catching the error if we are starting the daemon? that's one
possibility, assuming that the error message will then be visible when
the first client connects.

In non-daemon mode I think we should keep not catching errors.

bug#47439: 27.2; In daemon mode, if after-init-hook errors out, the server does not start

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> You mean, catching the error if we are starting the daemon? that's one
> possibility, assuming that the error message will then be visible when
> the first client connects.
>
> In non-daemon mode I think we should keep not catching errors.

Yup, and yup.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47439: 27.2; In daemon mode, if after-init-hook errors out, the server does not start

[Gilles]

Looking at the code in startup.el, I think it would be ok to let
errors go uncaught (it would avoid any complication to debug those
errors), but the daemon should start anyway. How about putting most of
command-line in an unwind-protect? Or moving the daemon start outside
the unwind-protect in normal-top-level?

-- Gilles

On Sun, 28 Mar 2021 at 16:56, Lars Ingebrigtsen <larsi@gnus.org> wrote:
>
> Eli Zaretskii <eliz@gnu.org> writes:
>
> > You mean, catching the error if we are starting the daemon? that's one
> > possibility, assuming that the error message will then be visible when
> > the first client connects.
> >
> > In non-daemon mode I think we should keep not catching errors.
>
> Yup, and yup.
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no

bug#47439: 27.2; In daemon mode, if after-init-hook errors out, the server does not start

[Eli Zaretskii]

> From: Gilles <gilles.usenet@gmail.com>
> Date: Sun, 28 Mar 2021 19:04:44 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>, 47439@debbugs.gnu.org
> 
> Looking at the code in startup.el, I think it would be ok to let
> errors go uncaught (it would avoid any complication to debug those
> errors), but the daemon should start anyway. How about putting most of
> command-line in an unwind-protect? Or moving the daemon start outside
> the unwind-protect in normal-top-level?

The error should still be shown when you connect.