bug#64163: 29.0.92; Syntactic errors in early init when --debug-init is on

bug#64163: 29.0.92; Syntactic errors in early init when --debug-init is on

[Sebastian Miele]

Create a $DIR with an early-init.el containing just '(' and an init.el
with just '(error "Later")'.

Then running

  src/emacs --init-directory=$DIR

produces a warning buffer that lists both errors in order.  However,
when running

  src/emacs --init-directory=$DIR --debug-init

the debugger triggers on the error "Later", without any mention of the
error in the early init file.  This is unexpected.

It is as expected when early-init.el instead of a syntactic error
contains, e.g., just '(error "Early")'.  Then the debugger triggers on
the "Early" error.

The unexpected behavior also happens, when early-init.el does not have a
syntactic error per se, but (indirectly) loads a file that does.

In GNU Emacs 29.0.92 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.38, cairo version 1.17.8) of 2023-06-18 built on huette
Repository revision: a24e9e3fee59435422af0473b7ec585de2c13b4e
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12101008
System Description: Arch Linux

Configured using:
 'configure --with-x-toolkit=gtk --with-native-compilation=yes
 --with-tree-sitter --with-json --with-mailutils --with-imagemagick'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ
IMAGEMAGICK JPEG JSON LCMS2 LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3
THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XDBE XIM XINPUT2
XPM GTK3 ZLIB

Important settings:
  value of $LC_COLLATE: C
  value of $LANG: C.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
help-fns radix-tree cl-print byte-opt gv bytecomp byte-compile debug
backtrace help-mode find-func cl-loaddefs cl-lib rmc iso-transl tooltip
cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit xinput2 x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 54932 8781)
 (symbols 48 6191 0)
 (strings 32 19661 2752)
 (string-bytes 1 615525)
 (vectors 16 12469)
 (vector-slots 8 255805 14317)
 (floats 8 23 27)
 (intervals 56 279 0)
 (buffers 984 12))

bug#64163: 29.0.92; Syntactic errors in early init when --debug-init is on

[Eli Zaretskii]

> From: Sebastian Miele <iota@whxvd.name>
> Date: Mon, 19 Jun 2023 12:27:09 +0200
> 
> Create a $DIR with an early-init.el containing just '(' and an init.el
> with just '(error "Later")'.
> 
> Then running
> 
>   src/emacs --init-directory=$DIR
> 
> produces a warning buffer that lists both errors in order.  However,
> when running
> 
>   src/emacs --init-directory=$DIR --debug-init
> 
> the debugger triggers on the error "Later", without any mention of the
> error in the early init file.  This is unexpected.

First, the --init-directory=$DIR part is not relevant to the issue;
the same happens if you place such init files in your "normal" home
directory.

More to the point, that error in early-init.el does not trigger a
backtrace because 'end-of-file' error is in the list that is the value
of debug-ignored-errors.  So this problem is specific to that
particular kind of error in early-init.el.  If you trigger some other
error, which is not in the list of ignored ones, you will see a
backtrace.

IOW, this is the expected behavior.

I wonder whether the patch below could be the solution to this.
Stefan, WDYT?  Should we perhaps condition this binding by more
conditions, like only do this in interactive invocations or something?

diff --git a/lisp/startup.el b/lisp/startup.el
index b0f019d..bf16f14 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -1030,7 +1030,8 @@ startup--load-user-init-file
     (let ((debug-on-error debug-on-error-initial))
       (condition-case-unless-debug error
           (when init-file-user
-            (let ((init-file-name (funcall filename-function)))
+            (let ((init-file-name (funcall filename-function))
+                  (debug-ignored-errors nil))
 
               ;; If `user-init-file' is t, then `load' will store
               ;; the name of the file that it loads into

bug#64163: 29.0.92; Syntactic errors in early init when --debug-init is on

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> First, the --init-directory=$DIR part is not relevant to the issue;
> the same happens if you place such init files in your "normal" home
> directory.

[ IIUC he used it so we can reproduce his recipe without having to
  mess with our config.  Thanks Sebastien for that effort :-)  ]

> More to the point, that error in early-init.el does not trigger
> a backtrace because 'end-of-file' error is in the list that is the
> value of debug-ignored-errors.  So this problem is specific to that
> particular kind of error in early-init.el.  If you trigger some other
> error, which is not in the list of ignored ones, you will see
> a backtrace.
>
> IOW, this is the expected behavior.
>
> I wonder whether the patch below could be the solution to this.

I think `debug-ignored-errors` should keep its default value when
`--debug-init` is not used.

IOW `--debug-init` should be the one that sets `debug-ignored-errors` to
nil (only while loading the init files, tho).

> Stefan, WDYT?  Should we perhaps condition this binding by more
> conditions, like only do this in interactive invocations or something?

That would also be worth a try, yes.


        Stefan

bug#64163: 29.0.92; Syntactic errors in early init when --debug-init is on

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Sebastian Miele <iota@whxvd.name>,  64163@debbugs.gnu.org
> Date: Tue, 20 Jun 2023 10:24:31 -0400
> 
> > First, the --init-directory=$DIR part is not relevant to the issue;
> > the same happens if you place such init files in your "normal" home
> > directory.
> 
> [ IIUC he used it so we can reproduce his recipe without having to
>   mess with our config.  Thanks Sebastien for that effort :-)  ]

I usually start a new shell and set HOME there to point to some
scratch directory.

The problem with using --init-directory= in this particular case is
that this is a new feature in Emacs 29, so I was initially afraid we
have some subtle bug in it.

> I think `debug-ignored-errors` should keep its default value when
> `--debug-init` is not used.
> 
> IOW `--debug-init` should be the one that sets `debug-ignored-errors` to
> nil

So you are saying it should be condition on init-file-debug being
non-nil?

Incidentally, do you happen to know the difference between that
variable having the value t and 'startup'?  I don't see anywhere that
we assign any different non-nil value to it.

> (only while loading the init files, tho).

OK.

> > Stefan, WDYT?  Should we perhaps condition this binding by more
> > conditions, like only do this in interactive invocations or something?
> 
> That would also be worth a try, yes.

OK, thanks.

bug#64163: 29.0.92; Syntactic errors in early init when --debug-init is on

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> IOW `--debug-init` should be the one that sets `debug-ignored-errors` to
>> nil
>
> So you are saying it should be condition on init-file-debug being
> non-nil?

Yup.

> Incidentally, do you happen to know the difference between that
> variable having the value t and 'startup'?

I'm not familiar with this code, no.

AFAICT, `startup` is a special value for `debug-on-error-initial` rather
than for `init-file-debug`.

I expect this is intended to be a value to which noone would set
`debug-on-error`, tho it would make sense to use a more verbose/unlikely
value, both to better detect user settings of `debug-on-error` and to
help document the code.

Maybe something like the patch below (I also renamed the local
variables so they don't look like global variables).


        Stefan


diff --git a/lisp/startup.el b/lisp/startup.el
index 835ad785af1..5b842055c58 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -1021,17 +1021,17 @@ startup--load-user-init-file
 
 This function sets `user-init-file' to the name of the loaded
 init-file, or to a default value if loading is not possible."
-  (let ((debug-on-error-from-init-file nil)
-        (debug-on-error-should-be-set nil)
-        (debug-on-error-initial
-         (if (eq init-file-debug t)
-             'startup
-           init-file-debug))
+  (let ((d-o-e-from-init-file nil)
+        (d-o-e-initial
+         (if init-file-debug 'startup--witness debug-on-error))
+        (d-i-e-from-init-file nil)
+        (d-i-e-initial
+         (if init-file-debug '(startup--witness) debug-ignored-errors))
         ;; The init file might contain byte-code with embedded NULs,
         ;; which can cause problems when read back, so disable nul
         ;; byte detection.  (Bug#52554)
         (inhibit-null-byte-detection t))
-    (let ((debug-on-error debug-on-error-initial))
+    (let ((debug-on-error d-o-e-initial))
       (condition-case-unless-debug error
           (when init-file-user
             (let ((init-file-name (funcall filename-function)))
@@ -1112,12 +1112,15 @@ startup--load-user-init-file
 
       ;; If we can tell that the init file altered debug-on-error,
       ;; arrange to preserve the value that it set up.
-      (or (eq debug-on-error debug-on-error-initial)
-          (setq debug-on-error-should-be-set t
-                debug-on-error-from-init-file debug-on-error)))
-
-    (when debug-on-error-should-be-set
-      (setq debug-on-error debug-on-error-from-init-file))))
+      (or (eq debug-on-error d-o-e-initial)
+          (setq d-o-e-from-init-file (list debug-on-error)))
+      (or (eq debug-ignored-errors d-i-e-initial)
+          (setq d-i-e-from-init-file (list debug-ignored-errors))))
+
+    (when d-o-e-from-init-file
+      (setq debug-on-error (car d-o-e-from-init-file)))
+    (when d-i-e-from-init-file
+      (setq debug-ignored-errors (car d-i-e-from-init-file)))))
 
 (defvar lisp-directory nil
   "Directory where Emacs's own *.el and *.elc Lisp files are installed.")

bug#64163: 29.0.92; Syntactic errors in early init when --debug-init is on

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: iota@whxvd.name,  64163@debbugs.gnu.org
> Date: Tue, 20 Jun 2023 13:07:46 -0400
> 
> >> IOW `--debug-init` should be the one that sets `debug-ignored-errors` to
> >> nil
> >
> > So you are saying it should be condition on init-file-debug being
> > non-nil?
> 
> Yup.

Thanks, now done on the master branch.

> > Incidentally, do you happen to know the difference between that
> > variable having the value t and 'startup'?
> 
> I'm not familiar with this code, no.
> 
> AFAICT, `startup` is a special value for `debug-on-error-initial` rather
> than for `init-file-debug`.
> 
> I expect this is intended to be a value to which noone would set
> `debug-on-error`, tho it would make sense to use a more verbose/unlikely
> value, both to better detect user settings of `debug-on-error` and to
> help document the code.
> 
> Maybe something like the patch below (I also renamed the local
> variables so they don't look like global variables).

You know me: I'm not a big fan of renaming, unless really necessary...