bug#41618: 28.0.50; Can't byte-compile an edebugged macro

bug#41618: 28.0.50; Can't byte-compile an edebugged macro

[Philipp Stephani]

1. Define some arbitrary macro:

  (defmacro foo ())

2. Edebug it using C-u C-M-x.

3. Attempt to byte-compile it using M-: (byte-compile 'foo).

This produces an error:

  Wrong type argument: listp, #[0 "\300\207" [nil] 1]

The stack trace is

Debugger entered--Lisp error: (wrong-type-argument listp #f(compiled-function () #<bytecode 0x1e0000171e91>))
  eval((macro . #f(compiled-function () #<bytecode 0x1e0000171e91>)) t)
  #f(compiled-function (form) #<bytecode -0x16fada817203f185>)(foo)
  byte-compile(foo)
  eval((byte-compile 'foo) t)
  eval-expression((byte-compile 'foo) nil nil 127)
  funcall-interactively(eval-expression (byte-compile 'foo) nil nil 127)
  call-interactively(eval-expression nil nil)
  command-execute(eval-expression)



In GNU Emacs 28.0.50 (build 19, x86_64-pc-linux-gnu, GTK+ Version 3.24.14, cairo version 1.16.0)
 of 2020-05-30
Repository revision: 3dbe6530b124436550dae4db6cd4b7b380e95377
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12007000
System Description: Debian GNU/Linux rodete

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --enable-gcc-warnings=warn-only
 --enable-gtk-deprecation-warnings --without-pop --with-mailutils
 --enable-checking --enable-check-lisp-object-type --with-modules
 'CFLAGS=-O0 -ggdb3''

Configured features:
XPM JPEG TIFF GIF PNG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY
LIBSELINUX GNUTLS FREETYPE HARFBUZZ ZLIB TOOLKIT_SCROLL_BARS GTK3 X11
XDBE XIM MODULES THREADS PDUMPER GMP

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec epa epg epg-config gnus-util
rmail rmail-loaddefs text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils phst skeleton
derived edmacro kmacro pcase ffap thingatpt url url-proxy url-privacy
url-expand url-methods url-history url-cookie url-domsuf url-util
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json map url-vars mailcap subr-x rx gnutls puny seq
byte-opt gv bytecomp byte-compile cconv dbus xml compile comint
ansi-color ring cl-loaddefs cl-lib tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 69265 7079)
 (symbols 48 8959 1)
 (strings 32 23829 1591)
 (string-bytes 1 769536)
 (vectors 16 13265)
 (vector-slots 8 177353 6149)
 (floats 8 25 35)
 (intervals 56 212 0)
 (buffers 992 11))

-- 
Google Germany GmbH
Erika-Mann-Straße 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich.  Falls Sie diese fälschlicherweise erhalten haben
sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie
alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail
an die falsche Person gesendet wurde.

This e-mail is confidential.  If you received this communication by mistake,
please don’t forward it to anyone else, please erase all copies and
attachments, and please let me know that it has gone to the wrong person.

bug#41618: 28.0.50; Can't byte-compile an edebugged macro

[Alan Mackenzie]

Hello, Philipp.

In article <mailman.730.1590857164.2541.bug-gnu-emacs@gnu.org> you wrote:

> 1. Define some arbitrary macro:

>   (defmacro foo ())

> 2. Edebug it using C-u C-M-x.

This was not actually relevant.  A simple evaluation with C-M-x produces
the same error.

> 3. Attempt to byte-compile it using M-: (byte-compile 'foo).

> This produces an error:

>   Wrong type argument: listp, #[0 "\300\207" [nil] 1]

> The stack trace is

> Debugger entered--Lisp error: (wrong-type-argument listp #f(compiled-function () #<bytecode 0x1e0000171e91>))
>   eval((macro . #f(compiled-function () #<bytecode 0x1e0000171e91>)) t)
>   #f(compiled-function (form) #<bytecode -0x16fada817203f185>)(foo)
>   byte-compile(foo)
>   eval((byte-compile 'foo) t)
>   eval-expression((byte-compile 'foo) nil nil 127)
>   funcall-interactively(eval-expression (byte-compile 'foo) nil nil 127)
>   call-interactively(eval-expression nil nil)
>   command-execute(eval-expression)

This was quite a simple bug.  At the end of byte-compile, the code does
two things:
(i) If the argument to byte-compile is a symbol, the result is eval'd.
(ii) If a macro is being compiled, 'macro is pushed onto the result.

When both of these things were necessary, they were being done in the
wrong order, throwing the error.

I've committed a fix to the emacs-27 branch, and it should reach master
the next time "somebody" copies the commits over.  In the mean time,
here's that patch, should you want to apply it to your system now:



diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el
index 72dbfd74b1..22e648e44b 100644
--- a/lisp/emacs-lisp/bytecomp.el
+++ b/lisp/emacs-lisp/bytecomp.el
@@ -2755,14 +2755,15 @@ byte-compile
         ;; Expand macros.
         (setq fun (byte-compile-preprocess fun))
         (setq fun (byte-compile-top-level fun nil 'eval))
-        (if macro (push 'macro fun))
         (if (symbolp form)
             ;; byte-compile-top-level returns an *expression* equivalent to the
             ;; `fun' expression, so we need to evaluate it, tho normally
             ;; this is not needed because the expression is just a constant
             ;; byte-code object, which is self-evaluating.
-            (fset form (eval fun t))
-          fun)))))))
+            (setq fun (eval fun t)))
+        (if macro (push 'macro fun))
+        (if (symbolp form) (fset form fun))
+        fun))))))
 
 (defun byte-compile-sexp (sexp)
   "Compile and return SEXP."


> In GNU Emacs 28.0.50 (build 19, x86_64-pc-linux-gnu, GTK+ Version 3.24.14, cairo version 1.16.0)
>  of 2020-05-30
> Repository revision: 3dbe6530b124436550dae4db6cd4b7b380e95377
> Repository branch: master
> Windowing system distributor 'The X.Org Foundation', version 11.0.12007000
> System Description: Debian GNU/Linux rodete

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#41618: 28.0.50; Can't byte-compile an edebugged macro

[Eli Zaretskii]

> Date: 31 May 2020 17:01:19 -0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: 41618-done@debbugs.gnu.org
> 
> I've committed a fix to the emacs-27 branch

Thanks, but why to emacs-27?  Is this bug a regression from Emacs 26?
If not, it doesn't sound to me like a problem that should be fixed
urgently, the use case is not really that frequent, and there's an
easy workaround.

I'd prefer not to push anything to emacs-27 that doesn't absolutely
have to be there.

bug#41618: 28.0.50; Can't byte-compile an edebugged macro

[Alan Mackenzie]

Hello, Eli.

On Sun, May 31, 2020 at 20:08:17 +0300, Eli Zaretskii wrote:
> > Date: 31 May 2020 17:01:19 -0000
> > From: Alan Mackenzie <acm@muc.de>
> > Cc: 41618-done@debbugs.gnu.org

> > I've committed a fix to the emacs-27 branch

> Thanks, but why to emacs-27?  Is this bug a regression from Emacs 26?

It was difficult to decide, but emacs-27 seemed right; the bug was
almost certainly a regression from Emacs 26, the erroneous commit having
been made on 2019-07-27.  Also the fix was utterly safe, having been
exhaustively tested (i.e. make bootstrap worked).

> If not, it doesn't sound to me like a problem that should be fixed
> urgently, the use case is not really that frequent, and there's an
> easy workaround.

It produced an obscure error message, for which only somebody in the know
would be able to apply a workaround.

> I'd prefer not to push anything to emacs-27 that doesn't absolutely
> have to be there.

OK.  I've felt a bit unclear about this over the past few weeks.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#41618: 28.0.50; Can't byte-compile an edebugged macro

[Eli Zaretskii]

> Date: Sun, 31 May 2020 17:41:53 +0000
> Cc: 41618@debbugs.gnu.org, p.stephani2@gmail.com
> From: Alan Mackenzie <acm@muc.de>
> 
> > Thanks, but why to emacs-27?  Is this bug a regression from Emacs 26?
> 
> It was difficult to decide, but emacs-27 seemed right; the bug was
> almost certainly a regression from Emacs 26, the erroneous commit having
> been made on 2019-07-27.

Which commit was that?  I don't think you mentioned that.

bug#41618: 28.0.50; Can't byte-compile an edebugged macro

[Philipp Stephani]

Am So., 31. Mai 2020 um 19:01 Uhr schrieb Alan Mackenzie <acm@muc.de>:
>
> Hello, Philipp.
>
> In article <mailman.730.1590857164.2541.bug-gnu-emacs@gnu.org> you wrote:
>
> > 1. Define some arbitrary macro:
>
> >   (defmacro foo ())
>
> > 2. Edebug it using C-u C-M-x.
>
> This was not actually relevant.  A simple evaluation with C-M-x produces
> the same error.
>
> > 3. Attempt to byte-compile it using M-: (byte-compile 'foo).
>
> > This produces an error:
>
> >   Wrong type argument: listp, #[0 "\300\207" [nil] 1]
>
> > The stack trace is
>
> > Debugger entered--Lisp error: (wrong-type-argument listp #f(compiled-function () #<bytecode 0x1e0000171e91>))
> >   eval((macro . #f(compiled-function () #<bytecode 0x1e0000171e91>)) t)
> >   #f(compiled-function (form) #<bytecode -0x16fada817203f185>)(foo)
> >   byte-compile(foo)
> >   eval((byte-compile 'foo) t)
> >   eval-expression((byte-compile 'foo) nil nil 127)
> >   funcall-interactively(eval-expression (byte-compile 'foo) nil nil 127)
> >   call-interactively(eval-expression nil nil)
> >   command-execute(eval-expression)
>
> This was quite a simple bug.  At the end of byte-compile, the code does
> two things:
> (i) If the argument to byte-compile is a symbol, the result is eval'd.
> (ii) If a macro is being compiled, 'macro is pushed onto the result.
>
> When both of these things were necessary, they were being done in the
> wrong order, throwing the error.
>
> I've committed a fix to the emacs-27 branch, and it should reach master
> the next time "somebody" copies the commits over.  In the mean time,
> here's that patch, should you want to apply it to your system now:
>


Thanks for the quick fix. Confirmed that it's fixed on the emacs-27 branch.

bug#41618: 28.0.50; Can't byte-compile an edebugged macro

[Philipp Stephani]

Am So., 31. Mai 2020 um 19:56 Uhr schrieb Eli Zaretskii <eliz@gnu.org>:
>
> > Date: Sun, 31 May 2020 17:41:53 +0000
> > Cc: 41618@debbugs.gnu.org, p.stephani2@gmail.com
> > From: Alan Mackenzie <acm@muc.de>
> >
> > > Thanks, but why to emacs-27?  Is this bug a regression from Emacs 26?
> >
> > It was difficult to decide, but emacs-27 seemed right; the bug was
> > almost certainly a regression from Emacs 26, the erroneous commit having
> > been made on 2019-07-27.
>
> Which commit was that?  I don't think you mentioned that.

My guess is on 1c8405e33e814a372fa349313521b015c3601605, but I haven't
formally bisected.

bug#41618: 28.0.50; Can't byte-compile an edebugged macro

[Alan Mackenzie]

Hello, Eli.

On Sun, May 31, 2020 at 20:56:30 +0300, Eli Zaretskii wrote:
> > Date: Sun, 31 May 2020 17:41:53 +0000
> > Cc: 41618@debbugs.gnu.org, p.stephani2@gmail.com
> > From: Alan Mackenzie <acm@muc.de>

> > > Thanks, but why to emacs-27?  Is this bug a regression from Emacs 26?

> > It was difficult to decide, but emacs-27 seemed right; the bug was
> > almost certainly a regression from Emacs 26, the erroneous commit having
> > been made on 2019-07-27.

> Which commit was that?  I don't think you mentioned that.

I didn't, sorry.  It was:

commit 1c8405e33e814a372fa349313521b015c3601605
Author: Stefan Monnier <monnier@iro.umontreal.ca>
Date:   Sat Jul 27 17:28:10 2019 -0400

    * lisp/emacs-lisp/bytecomp.el (byte-compile-out-toplevel): Fix
    * bug#34757

    This fix was provided by Pip Cet <pipcet@gmail.com>.  It tightens the
    code that tries to recognize a bytecode sequence as being a simple
    function call (to then decompile it), which occasionally misfired.

    I added some minor changes found while investigating this issue.

    (byte-compile): Handle corner case where byte-compile-top-level returns
    a non-self-evaluating expression.
    (byte-compile-out-toplevel): Remove support for `progn` and `t` values
    of output-type which aren't used anywhere.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#41618: 28.0.50; Can't byte-compile an edebugged macro

[Eli Zaretskii]

> Date: Sun, 31 May 2020 18:04:33 +0000
> Cc: 41618@debbugs.gnu.org, p.stephani2@gmail.com
> From: Alan Mackenzie <acm@muc.de>
> 
> > > It was difficult to decide, but emacs-27 seemed right; the bug was
> > > almost certainly a regression from Emacs 26, the erroneous commit having
> > > been made on 2019-07-27.
> 
> > Which commit was that?  I don't think you mentioned that.
> 
> I didn't, sorry.  It was:
> 
> commit 1c8405e33e814a372fa349313521b015c3601605
> Author: Stefan Monnier <monnier@iro.umontreal.ca>
> Date:   Sat Jul 27 17:28:10 2019 -0400

Then I guess it's okay to fix this on emacs-27.  Thanks.