bug#55323: 29.0.50; Session-compiled interactive form gives (invalid-function #<symbol list at 476>)

bug#55323: 29.0.50; Session-compiled interactive form gives (invalid-function #<symbol list at 476>)

[Bob Rogers]

In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, cairo version 1.16.0)
 of 2022-05-08 built on orion
Repository revision: 278b18a460caf34e422847d10ac3f0b62bef4996
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description: openSUSE Leap 15.3

   The problem occurs with an evaluated "interactive" form in a defun
that is compiled with compile-defun -- and it may have escaped notice
until now because it doesn't seem to happen if the source is part of
Emacs (i.e. is compiled in a file that git knows about).

   1.  emacs -Q

   2.  Find an elisp file with an interactive form that does not belong
to an Emacs working copy.  I used the align-highlight-rule snippet
below, copied from lisp/align.el.

   3.  Evaluate the first two forms, and do "M-x compile-defun" to
compile the third.

   4.  Attempt to invoke the command via "M-x align-highlight-rule RET".

What you should see then is a backtrace that starts something like this:

	Debugger entered--Lisp error: (invalid-function #<symbol list at 476>)
	  (#<symbol list at 476> (#<symbol region-beginning at 482>) ...)
	  call-interactively(align-highlight-rule record nil)
	  command-execute(align-highlight-rule record)

Disassembly shows that the interactive form is not compiled, and the
arglist is full of #<symbol X at Y>:

	byte code for align-highlight-rule:
	  doc:  Highlight the whitespace which a given rule would have modified. ...
	  args: (#<symbol beg at 49> #<symbol end at 53> #<symbol title at 57> ...)
	 interactive: (#<symbol list at 476> (#<symbol region-beginning at 482>) ...)
	0	constant  intern
	1	varref	  title
	2	call	  1
	3	varref	  align-mode-exclude-rules-list

Disassembling the in-tree version, whether from "M-x compile-defun" or
file compilation, shows neither of these problems (go figure).

   And of course "M-x eval-defun" and "M-x byte-compile-file" continue
to DTRT, so I am not in any hurry for a fix.  TIA,

					-- Bob Rogers
					   http://www.rgrjr.com/

------------------------------------------------------------------------

(require 'align)

(setq debug-on-error t)

(defun align-highlight-rule (beg end title &optional rules exclude-rules)
  "Highlight the whitespace which a given rule would have modified.
BEG and END mark the extent of the region.  TITLE identifies the rule
that should be highlighted.  If RULES or EXCLUDE-RULES is set to a
list of rules (see `align-rules-list'), it can be used to override the
default alignment rules that would have been used to identify the text
to be colored."
  (interactive
   (list (region-beginning) (region-end)
	 (completing-read
	  "Title of rule to highlight: "
	  (mapcar
           (lambda (rule)
             (list (symbol-name (car rule))))
	   (append (or align-mode-rules-list align-rules-list)
		   (or align-mode-exclude-rules-list
		       align-exclude-rules-list))) nil t)))
  (let ((ex-rule (assq (intern title)
		       (or align-mode-exclude-rules-list
			   align-exclude-rules-list)))
	face)
    (align-unhighlight-rule)
    (align-region
     beg end 'entire
     (or rules (if ex-rule
		   (or exclude-rules align-mode-exclude-rules-list
		       align-exclude-rules-list)
		 (or align-mode-rules-list align-rules-list)))
     (unless ex-rule (or exclude-rules align-mode-exclude-rules-list
			 align-exclude-rules-list))
     (lambda (b e mode)
       (if (and mode (listp mode))
           (if (equal (symbol-name (car mode)) title)
               (setq face (cons align-highlight-change-face
                                align-highlight-nochange-face))
             (setq face nil))
         (when face
           (let ((overlay (make-overlay b e)))
             (setq align-highlight-overlays
                   (cons overlay align-highlight-overlays))
             (overlay-put overlay 'face
                          (if mode
                              (car face)
                            (cdr face))))))))))

------------------------------------------------------------------------
Configured using:
 'configure --with-dbus=no --with-gsettings=no --with-gif=ifavailable
 --with-tiff=no --with-gnutls=yes --with-gconf=no'

Configured features:
ACL CAIRO FREETYPE GIF GLIB GMP GNUTLS HARFBUZZ JPEG LIBSELINUX LIBXML2
MODULES NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS
TOOLKIT_SCROLL_BARS X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Debugger

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  buffer-read-only: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media rmc puny
dired dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg
rfc6068 epg-config gnus-util time-date mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils help-fns radix-tree
cl-print debug backtrace help-mode find-func compile
text-property-search comint ansi-color ring align vc-git diff-mode
easy-mmode vc-dispatcher cl-loaddefs cl-lib seq gv subr-x byte-opt
bytecomp byte-compile cconv iso-transl tooltip eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice simple cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop
case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads inotify
dynamic-setting font-render-setting cairo move-toolbar gtk x-toolkit
xinput2 x multi-tty make-network-process emacs)

Memory information:
((conses 16 63054 8598)
 (symbols 48 7628 1)
 (strings 32 21891 2878)
 (string-bytes 1 716772)
 (vectors 16 15096)
 (vector-slots 8 202859 11567)
 (floats 8 28 36)
 (intervals 56 345 0)
 (buffers 992 13))

bug#55323: 29.0.50; Session-compiled interactive form gives (invalid-function #<symbol list at 476>)

[Alan Mackenzie]

Hello, Bob.

On Sun, May 08, 2022 at 15:29:12 -0400, Bob Rogers wrote:
> In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, cairo version 1.16.0)
>  of 2022-05-08 built on orion
> Repository revision: 278b18a460caf34e422847d10ac3f0b62bef4996
> Repository branch: master
> Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
> System Description: openSUSE Leap 15.3

>    The problem occurs with an evaluated "interactive" form in a defun
> that is compiled with compile-defun -- and it may have escaped notice
> until now because it doesn't seem to happen if the source is part of
> Emacs (i.e. is compiled in a file that git knows about).

>    1.  emacs -Q

>    2.  Find an elisp file with an interactive form that does not belong
> to an Emacs working copy.  I used the align-highlight-rule snippet
> below, copied from lisp/align.el.

>    3.  Evaluate the first two forms, and do "M-x compile-defun" to
> compile the third.

>    4.  Attempt to invoke the command via "M-x align-highlight-rule RET".

> What you should see then is a backtrace that starts something like this:

> 	Debugger entered--Lisp error: (invalid-function #<symbol list at 476>)
> 	  (#<symbol list at 476> (#<symbol region-beginning at 482>) ...)
> 	  call-interactively(align-highlight-rule record nil)
> 	  command-execute(align-highlight-rule record)

> Disassembly shows that the interactive form is not compiled, and the
> arglist is full of #<symbol X at Y>:

> 	byte code for align-highlight-rule:
> 	  doc:  Highlight the whitespace which a given rule would have modified. ...
> 	  args: (#<symbol beg at 49> #<symbol end at 53> #<symbol title at 57> ...)
> 	 interactive: (#<symbol list at 476> (#<symbol region-beginning at 482>) ...)
> 	0	constant  intern
> 	1	varref	  title
> 	2	call	  1
> 	3	varref	  align-mode-exclude-rules-list

> Disassembling the in-tree version, whether from "M-x compile-defun" or
> file compilation, shows neither of these problems (go figure).

Thanks for taking the trouble to report this bug.  I can reproduce it
here.  At a guess, the critical detail is having align-highlight-rule
in a different file's buffer on doing M-x compile-defun.

>    And of course "M-x eval-defun" and "M-x byte-compile-file" continue
> to DTRT, so I am not in any hurry for a fix.  TIA,

I hope to be able to fix this within a few days.

> 					-- Bob Rogers
> 					   http://www.rgrjr.com/

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#55323: 29.0.50; Session-compiled interactive form gives (invalid-function #<symbol list at 476>)

[Alan Mackenzie]

Hello again, Bob.

On Sun, May 08, 2022 at 15:29:12 -0400, Bob Rogers wrote:
> In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, cairo version 1.16.0)
>  of 2022-05-08 built on orion
> Repository revision: 278b18a460caf34e422847d10ac3f0b62bef4996
> Repository branch: master
> Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
> System Description: openSUSE Leap 15.3

>    The problem occurs with an evaluated "interactive" form in a defun
> that is compiled with compile-defun -- and it may have escaped notice
> until now because it doesn't seem to happen if the source is part of
> Emacs (i.e. is compiled in a file that git knows about).

I think what's different is that the source has been copied into a
different file, and that file doesn't have a

    ;; -*- lexical-binding:t -*-

at the top.  :-)

>    1.  emacs -Q

>    2.  Find an elisp file with an interactive form that does not belong
> to an Emacs working copy.  I used the align-highlight-rule snippet
> below, copied from lisp/align.el.

>    3.  Evaluate the first two forms, and do "M-x compile-defun" to
> compile the third.

>    4.  Attempt to invoke the command via "M-x align-highlight-rule RET".

> What you should see then is a backtrace that starts something like this:

> 	Debugger entered--Lisp error: (invalid-function #<symbol list at 476>)
> 	  (#<symbol list at 476> (#<symbol region-beginning at 482>) ...)
> 	  call-interactively(align-highlight-rule record nil)
> 	  command-execute(align-highlight-rule record)

> Disassembly shows that the interactive form is not compiled, and the
> arglist is full of #<symbol X at Y>:

> 	byte code for align-highlight-rule:
> 	  doc:  Highlight the whitespace which a given rule would have modified. ...
> 	  args: (#<symbol beg at 49> #<symbol end at 53> #<symbol title at 57> ...)
> 	 interactive: (#<symbol list at 476> (#<symbol region-beginning at 482>) ...)
> 	0	constant  intern
> 	1	varref	  title
> 	2	call	  1
> 	3	varref	  align-mode-exclude-rules-list

Yes.  For some reason, the interactive form is not compiled when both
lexical-binding is nil AND the form looks like (list .....).  I don't
know why this is, and suspect it's a remnant of a very old bug fix which
is no longer relevant.

> Disassembling the in-tree version, whether from "M-x compile-defun" or
> file compilation, shows neither of these problems (go figure).

See above.

>    And of course "M-x eval-defun" and "M-x byte-compile-file" continue
> to DTRT, so I am not in any hurry for a fix.  TIA,

Would you please try out the following patch, which removes the
positions from the symbols with positions in the arglist and the
interactive form.  Then please let me know how it goes.  Thanks!



diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el
index 28237d67d2..c282d79446 100644
--- a/lisp/emacs-lisp/bytecomp.el
+++ b/lisp/emacs-lisp/bytecomp.el
@@ -3084,7 +3084,8 @@ byte-compile-lambda
                        ;; which may include "calls" to
                        ;; internal-make-closure (Bug#29988).
                        lexical-binding)
-                   (setq int `(,(car int) ,newform)))))
+                   (setq int `(,(car int) ,newform))
+                 (setq int (byte-run-strip-symbol-positions int))))) ; for compile-defun.
             ((cdr int)                  ; Invalid (interactive . something).
 	     (byte-compile-warn-x int "malformed interactive spec: %s"
 				  int))))
@@ -3099,7 +3100,7 @@ byte-compile-lambda
                                         (byte-compile-make-lambda-lexenv
                                          arglistvars))
                                    reserved-csts))
-          (bare-arglist arglist))
+          (bare-arglist (byte-run-strip-symbol-positions arglist))) ; for compile-defun.
       ;; Build the actual byte-coded function.
       (cl-assert (eq 'byte-code (car-safe compiled)))
       (let ((out



> 					-- Bob Rogers
> 					   http://www.rgrjr.com/

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#55323: 29.0.50; Session-compiled interactive form gives (invalid-function #<symbol list at 476>)

[Bob Rogers]

   From: Alan Mackenzie <acm@muc.de>
   Date: Wed, 11 May 2022 15:20:04 +0000

   Hello again, Bob.

   On Sun, May 08, 2022 at 15:29:12 -0400, Bob Rogers wrote:

   . . .

   >    The problem occurs with an evaluated "interactive" form in a defun
   > that is compiled with compile-defun -- and it may have escaped notice
   > until now because it doesn't seem to happen if the source is part of
   > Emacs (i.e. is compiled in a file that git knows about).

   I think what's different is that the source has been copied into a
   different file, and that file doesn't have a

       ;; -*- lexical-binding:t -*-

   at the top.  :-)

Aha!  That makes much more sense.  I first encountered this in some of
my own code which I've been lazy about switching to "lexical-binding:t".

   . . .

   > Disassembly shows that the interactive form is not compiled, and the
   > arglist is full of #<symbol X at Y>:

   . . .

   Yes.  For some reason, the interactive form is not compiled when both
   lexical-binding is nil AND the form looks like (list .....).  I don't
   know why this is, and suspect it's a remnant of a very old bug fix which
   is no longer relevant.

   >    And of course "M-x eval-defun" and "M-x byte-compile-file" continue
   > to DTRT, so I am not in any hurry for a fix.  TIA,

   Would you please try out the following patch, which removes the
   positions from the symbols with positions in the arglist and the
   interactive form.  Then please let me know how it goes.  Thanks!

   . . .

   -- 
   Alan Mackenzie (Nuremberg, Germany).

The patch does work when I run my test case (though I get a different
error because I'm not invoking align-highlight-rule appropriately).  And
(as you say) the "interactive" form is compiled when lexical-binding
is nil.

					-- Bob

bug#55323: 29.0.50; Session-compiled interactive form gives (invalid-function #<symbol list at 476>)

[Alan Mackenzie]

Hello, Bob.

On Thu, May 12, 2022 at 18:45:00 -0400, Bob Rogers wrote:

[ .... ]

> The patch does work when I run my test case (though I get a different
> error because I'm not invoking align-highlight-rule appropriately).  And
> (as you say) the "interactive" form is compiled when lexical-binding
> is nil.

Thanks for the testing.  I've now committed the patch (slightly
enhanced), and I'm closing the bug with this post.

> 					-- Bob

-- 
Alan Mackenzie (Nuremberg, Germany).