obscure compilation error with cl-defstruct &aux boa constructor

obscure compilation error with cl-defstruct &aux boa constructor

[Madhu]

This is a self-inflicted injury,, but nevertheless here goes:
lisp/emacs-lisp/lisp-mode.el has a struct

#+begin_src emacs lisp
(cl-defstruct (lisp-indent-state
               (:constructor nil)
               (:constructor lisp-indent-initial-state
                             (&aux (ppss (lisp-ppss))
                                   (ppss-point (point))
                                   (stack (make-list (1+ (car ppss)) nil)))))
  stack ;; Cached indentation, per depth.
  ppss
  ppss-point)
#+end_src

I had a local patch (suggested by Noam Postavsky) which retained the old
open-paren-in-column-0-is-defun-start behaviour with an overriding
advice on lisp-ppss.  This patch modified the args to the
lisp-indent-initial-state constructor to look like this:

#+begin_src emacs_lisp
                             (&optional (ppss (lisp-ppss))
                              &aux (ppss-point (point))
                                   (stack (make-list (1+ (car ppss)) nil)))))
#+end_src

Recently this leads to an obscure compilation/bootstrap error.

#+begin_example
    Loading textmodes/paragraphs...
    Loading progmodes/prog-mode...
    Loading /dev/shm/emacs/lisp/emacs-lisp/lisp-mode.el (source)...
    Eager macro-expansion failure: (void-variable emacs-lisp-mode-syntax-table)
    Eager macro-expansion failure: (void-variable emacs-lisp-mode-syntax-table)

    Error: void-variable (emacs-lisp-mode-syntax-table)
      mapbacktrace(#[1028 "!!!!B5@!A5!!!" [princ "  " prin1 "(" "  (" " " ")\n"] 7 "\n\n(fn EVALD FUNC ARGS FLAGS)"])
      debug-early-backtrace()
      debug-early(error (void-variable emacs-lisp-mode-syntax-table))
      byte-code(!##" [emacs-lisp-mode-syntax-table make-syntax-table modify-syntax-entry 8216 "(’  " 8217 ")‘  "] 5)
      (defvar help-mode-syntax-table (byte-code !##" [emacs-lisp-mode-syntax-table make-syntax-table modify-syntax-entry 8216 "(’  " 8217 ")‘  "] 5) ("/dev/shm/emacs/lisp/help-mode.elc" . 4653))
      require(help-mode)
      byte-code("#!E\"" [set-advertised-calling-convention cl-prettyexpand (form) "27.1" require help-mode add-to-list describe-symbol-backends nil cl-find-class #[771 "!" [cl-describe-type] 5 "\n\n(fn S B F)"]] 5)
      cl-subseq((&cl-defs (nil (cl-tag-slot) (stack) (ppss) (ppss-point)) &optional (ppss (lisp-ppss)) &aux (ppss-point (point)) (stack (make-list (1+ (car ppss)) nil))) 0 4)
      cl--make-usage-args((&cl-defs (nil (cl-tag-slot) (stack) (ppss) (ppss-point)) &optional (ppss (lisp-ppss)) &aux (ppss-point (point)) (stack (make-list (1+ (car ppss)) nil))))
      cl--transform-lambda(((&cl-defs (nil (cl-tag-slot) (stack) (ppss) (ppss-point)) &optional (ppss (lisp-ppss)) &aux (ppss-point (point)) (stack (make-list (1+ (car ppss)) nil))) "Constructor for objects of type `lisp-indent-state'." (declare (side-effect-free t)) (record 'lisp-indent-state stack ppss ppss-point)) lisp-indent-initial-state)
      #[642 "B\"BB" [defun cl--transform-lambda] 8 ("/dev/shm/emacs/lisp/emacs-lisp/cl-macs.elc" . 6880)](lisp-indent-initial-state (&cl-defs (nil (cl-tag-slot) (stack) (ppss) (ppss-point)) &optional (ppss (lisp-ppss)) &aux (ppss-point (point)) (stack (make-list (1+ (car ppss)) nil))) "Constructor for objects of type `lisp-indent-state'." (declare (side-effect-free t)) (record 'lisp-indent-state stack ppss ppss-point))
      (cl-defun lisp-indent-initial-state (&cl-defs (nil (cl-tag-slot) (stack) (ppss) (ppss-point)) &optional (ppss (lisp-ppss)) &aux (ppss-point (point)) (stack (make-list (1+ (car ppss)) nil))) "Constructor for objects of type `lisp-indent-state'." (declare (side-effect-free t)) (record 'lisp-indent-state stack ppss ppss-point))
      eval-buffer(#<buffer  *load*> nil "/dev/shm/emacs/lisp/emacs-lisp/lisp-mode.el" nil t)
      load-with-code-conversion("/dev/shm/emacs/lisp/emacs-lisp/lisp-mode.el" "/dev/shm/emacs/lisp/emacs-lisp/lisp-mode.el" nil nil)
      load("emacs-lisp/lisp-mode")
      load("loadup.el")
    Symbol’s value as variable is void: emacs-lisp-mode-syntax-table
    make[1]: *** [Makefile:927: bootstrap-emacs.pdmp] Error 255
    make[1]: Leaving directory '/dev/shm/emacs/src'
    make: *** [Makefile:469: src] Error 2
#+end_example

The error goes away if I remove the &aux (and thus make all the
arguments optional)

Might this be a bug from a recent change? (I've not looked at the
changes yet, so this is a note to self)

Re: obscure compilation error with cl-defstruct &aux boa constructor

[Stefan Monnier]

Madhu [2022-04-16 22:16:18] wrote:
>       byte-code("#!E\"" [set-advertised-calling-convention cl-prettyexpand (form) "27.1" require help-mode add-to-list describe-symbol-backends nil cl-find-class #[771 "!" [cl-describe-type] 5 "\n\n(fn S B F)"]] 5)
>       cl-subseq((&cl-defs (nil (cl-tag-slot) (stack) (ppss) (ppss-point)) &optional (ppss (lisp-ppss)) &aux (ppss-point (point)) (stack (make-list (1+ (car ppss)) nil))) 0 4)

It looks like some frames are missing between these two.

> The error goes away if I remove the &aux (and thus make all the
> arguments optional)

I must say that seeing your backtrace I can't understand why your code
doesn't work or rather why our code works (the call to `cl-subseq`
depends on the presence of `&aux` which is already present in our code).

> Might this be a bug from a recent change? (I've not looked at the
> changes yet, so this is a note to self)

The bootstrap is delicate and ad-hoc.
Sometimes innocuous changes can break it.


        Stefan

Re: obscure compilation error with cl-defstruct &aux boa constructor

[Madhu]

* Stefan Monnier <jwvtuarzrem.fsf-monnier+emacs @gnu.org> :
Wrote on Sun, 17 Apr 2022 11:16:30 -0400:
> Madhu [2022-04-16 22:16:18] wrote:
[...]
>> (stack (make-list (1+ (car ppss)) nil))) 0 4)
>
> It looks like some frames are missing between these two.

I think I posted the full backtrace, but there was other cruft on HEAD,
so it is not the same error you'd get on master with the single
indicated change. which would be

#+begin_example
Loading /dev/shm/emacs/lisp/progmodes/prog-mode.el (source)...
Loading /dev/shm/emacs/lisp/emacs-lisp/lisp-mode.el (source)...
Eager macro-expansion failure: (void-function cl-subseq)
Eager macro-expansion failure: (void-function cl-subseq)
Symbol’s function definition is void: cl-subseq
make[1]: *** [Makefile:912: bootstrap-emacs.pdmp] Error 255
#+end_example

>> The error goes away if I remove the &aux (and thus make all the
>> arguments optional)
>
> I must say that seeing your backtrace I can't understand why your code
> doesn't work or rather why our code works (the call to `cl-subseq`
> depends on the presence of `&aux` which is already present in our code).
>
>> Might this be a bug from a recent change? (I've not looked at the
>> changes yet, so this is a note to self)
>
> The bootstrap is delicate and ad-hoc.
> Sometimes innocuous changes can break it.

I see it isn't a recent error. to compile the indicated form I had a
workaround in loadup.el that defined cl-subseq (after loading "simple"),
since around 2019-03-14. (somehow I'd lost that workaround this time
around)