bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 3166 bytes --]

Hi!

The construct 'eval-when-compile' seems to throw debug off. When pressing
C-u C-M-x on the following, an error is issued. This worked correctly in
24.3. This makes it hard to use edebug on functions in cc-mode
(like c-font-lock-declarations), as it makes heavy use of eval-when-compile.

(defun test (limit)
  (eval-when-compile
    (boundp 'parse-sexp-lookup-properties)))

Sincerely,
    Anders Lidgren


In GNU Emacs 24.3.50.2 (x86_64-apple-darwin13.0.0, NS apple-appkit-1265.00)
 of 2013-12-17 on macpro.lan
Bzr revision: 115421
monnier@iro.umontreal.ca-20131208081150-4bq2y1cgta14nfmj
Windowing system distributor `Apple', version 10.3.1265
Configured using:
 `configure --with-ns'

Important settings:
  value of $LC_CTYPE: UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-u C-M-x C-SPC <down> <down> <down> <escape> w <up>
<up> <up> <up> s-f s-f <escape> f <escape> f <escape>
d <escape> d <escape> d <escape> d <down> C-a C-x C-s
<up> <up> s-f <right> <right> <right> <right> <right>
<right> <right> <right> <right> <right> <backspace>
<backspace> <backspace> t e s t <down> <down> <up>
C-u C-M-x C-x C-s <up> C-a C-SPC C-g C-SPC <down> <down>
<down> <escape> w <menu-bar> <help-menu> <send-ema
cs-bug-report>

Recent messages:
Saving file /Users/anders/emacs/src/misc/bug-edebug-eval-when-compile.el...
Wrote /Users/anders/emacs/src/misc/bug-edebug-eval-when-compile.el
Beginning of buffer
Edebug: test
edebug-enter: Symbol's value as variable is void: limit
Saving file /Users/anders/emacs/src/misc/bug-edebug-eval-when-compile.el...
Wrote /Users/anders/emacs/src/misc/bug-edebug-eval-when-compile.el
Mark activated
Quit
Mark activated

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils misearch multi-isearch edebug easymenu cl-loaddefs
cl-lib vc-dispatcher vc-svn time-date tooltip electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel ns-win tool-bar dnd fontset
image regexp-opt fringe tabulated-list newcomment lisp-mode prog-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
cocoa ns multi-tty emacs)

[-- Attachment #2: Type: text/html, Size: 4452 bytes --]

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 577 bytes --]

Anders Lindgren wrote:
> The construct 'eval-when-compile' seems to throw debug off. When
> pressing C-u C-M-x on the following, an error is issued. This worked
> correctly in 24.3. This makes it hard to use edebug on functions in
> cc-mode (like c-font-lock-declarations), as it makes heavy use of
> eval-when-compile.

> (defun test (limit)
>   (eval-when-compile
>     (boundp 'parse-sexp-lookup-properties)))

I'm not sure of it but the patch below seems to solve it.
This works for the bug#14646 case[1], too.  WDYT?

[1] <http://thread.gmane.org/gmane.emacs.bugs/80778>


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1009 bytes --]

--- lisp/emacs-lisp/byte-run.el~	2013-11-21 22:12:14 +0000
+++ lisp/emacs-lisp/byte-run.el	2013-12-19 23:45:41 +0000
@@ -391,7 +391,7 @@
   "Like `progn', but evaluates the body at compile time if you're compiling.
 Thus, the result of the body appears to the compiler as a quoted constant.
 In interpreted code, this is entirely equivalent to `progn'."
-  (declare (debug (def-body)) (indent 0))
+  (declare (debug (&rest def-form)) (indent 0))
   (list 'quote (eval (cons 'progn body) lexical-binding)))
 
 (defmacro eval-and-compile (&rest body)
--- lisp/progmodes/cc-defs.el~	2013-06-02 22:06:34 +0000
+++ lisp/progmodes/cc-defs.el	2013-12-19 23:45:40 +0000
@@ -1137,7 +1137,7 @@
 ;; Make edebug understand the macros.
 ;(eval-after-load "edebug" ; 2006-07-09: def-edebug-spec is now in subr.el.
 ;  '(progn
-(def-edebug-spec cc-eval-when-compile t)
+(def-edebug-spec cc-eval-when-compile (&rest def-form))
 (def-edebug-spec c-point t)
 (def-edebug-spec c-set-region-active t)
 (def-edebug-spec c-safe t)

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 906 bytes --]

Hi!

Unfortunately, the problem still seems to be there. I tried this on latest
trunk (as of yesterday) with the patch applied and I still get an error
when I do C-u C-M-x on the code I originally supplied.

    -- Anders


On Fri, Dec 20, 2013 at 1:01 AM, Katsumi Yamaoka <yamaoka@jpl.org> wrote:

> Anders Lindgren wrote:
> > The construct 'eval-when-compile' seems to throw debug off. When
> > pressing C-u C-M-x on the following, an error is issued. This worked
> > correctly in 24.3. This makes it hard to use edebug on functions in
> > cc-mode (like c-font-lock-declarations), as it makes heavy use of
> > eval-when-compile.
>
> > (defun test (limit)
> >   (eval-when-compile
> >     (boundp 'parse-sexp-lookup-properties)))
>
> I'm not sure of it but the patch below seems to solve it.
> This works for the bug#14646 case[1], too.  WDYT?
>
> [1] <http://thread.gmane.org/gmane.emacs.bugs/80778>
>
>

[-- Attachment #2: Type: text/html, Size: 1424 bytes --]

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Katsumi Yamaoka]

Anders Lindgren wrote:
> Unfortunately, the problem still seems to be there. I tried this on
> latest trunk (as of yesterday) with the patch applied and I still get
> an error when I do C-u C-M-x on the code I originally supplied.

Could you make sure that the edebug spec of the `eval-when-compile'
macro definition (i.e. in byte-run.el) has been changed into
`(&rest def-form)'?
Otherwise please try `C-M-x' on the patched `eval-when-compile' macro.

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 985 bytes --]

Hi!

I just tried this and the error is still there.

On an emacs with bzr revision 115609, I started Emacs with -Q, and ran C-u
C-M-x on the code, and I got the error. I inspected the eval-when-compile
macro and it did have the form suggested by the patch in the bug report you
reported, i.e. "(declare (debug (def-body)) (indent 0))". Just to make sure
I ran C-M-x on it and tried again, but the error is still there...

    -- Anders


On Fri, Dec 20, 2013 at 9:28 AM, Katsumi Yamaoka <yamaoka@jpl.org> wrote:

> Anders Lindgren wrote:
> > Unfortunately, the problem still seems to be there. I tried this on
> > latest trunk (as of yesterday) with the patch applied and I still get
> > an error when I do C-u C-M-x on the code I originally supplied.
>
> Could you make sure that the edebug spec of the `eval-when-compile'
> macro definition (i.e. in byte-run.el) has been changed into
> `(&rest def-form)'?
> Otherwise please try `C-M-x' on the patched `eval-when-compile' macro.
>

[-- Attachment #2: Type: text/html, Size: 1500 bytes --]

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Katsumi Yamaoka]

Anders Lindgren wrote:
> I just tried this and the error is still there.

Well, if you tried `C-u C-M-x' on `c-font-lock-declarations' and
the like, you needed to have patched cc-defs.el in addition to
byte-run.el.

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 773 bytes --]

Sorry, I don't follow you here...

I get the problem when I do C-u C-M-x on both the small cut-down example of
my original bug report as well as on c-font-lock-declarations.

I haven't seen any patch to cc-defs.el in any of the mails I have received
on this matter.

I get the problem when I start Emacs with the "-Q" option, based on plain
build from a clean bzr checkout. I don't have any local patches applied, if
that is what you are suggesting.

    -- Anders


On Fri, Dec 20, 2013 at 10:53 AM, Katsumi Yamaoka <yamaoka@jpl.org> wrote:

> Anders Lindgren wrote:
> > I just tried this and the error is still there.
>
> Well, if you tried `C-u C-M-x' on `c-font-lock-declarations' and
> the like, you needed to have patched cc-defs.el in addition to
> byte-run.el.
>
>

[-- Attachment #2: Type: text/html, Size: 1248 bytes --]

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Katsumi Yamaoka]

Anders Lindgren <andlind@gmail.com> wrote:
> I haven't seen any patch to cc-defs.el in any of the mails I have received on
> this matter.

<http://article.gmane.org/gmane.emacs.bugs/82252>
-> <http://cache.gmane.org//gmane/emacs/bugs/82252-001.bin>

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 829 bytes --]

Hi!

OK, now I found the patch, and I now understand what you mean by the
"(&rest def-form)".

It seems to have solved one problem, but created another. When using C-u
C-M-x, everything works ok. However, the normal C-M-x command issues the
error "Too deep - perhaps infinite loop in spec?" When enabling
debug-on-error, it looks like a very deep call stack (754 entries).
Curiously, it works if I mark the function with the region and do
"eval-region", it works properly.

    -- Anders


On Fri, Dec 20, 2013 at 1:44 PM, Katsumi Yamaoka <yamaoka@jpl.org> wrote:

> Anders Lindgren <andlind@gmail.com> wrote:
> > I haven't seen any patch to cc-defs.el in any of the mails I have
> received on
> > this matter.
>
> <http://article.gmane.org/gmane.emacs.bugs/82252>
> -> <http://cache.gmane.org//gmane/emacs/bugs/82252-001.bin>
>

[-- Attachment #2: Type: text/html, Size: 1502 bytes --]

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Katsumi Yamaoka]

Anders Lindgren wrote:
> It seems to have solved one problem, but created another. When using
> C-u C-M-x, everything works ok. However, the normal C-M-x command
> issues the error "Too deep - perhaps infinite loop in spec?" When
> enabling debug-on-error, it looks like a very deep call stack (754
> entries). Curiously, it works if I mark the function with the region
> and do "eval-region", it works properly.

Sorry for the delay.  But
I'm unable to reproduce such an infloop-like error so far.
Could you show the name or the definition of the function
causing it?

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Katsumi Yamaoka]

Katsumi Yamaoka wrote:
> Anders Lindgren wrote:
>> It seems to have solved one problem, but created another. When using
>> C-u C-M-x, everything works ok. However, the normal C-M-x command
>> issues the error "Too deep - perhaps infinite loop in spec?" When
>> enabling debug-on-error, it looks like a very deep call stack (754
>> entries). Curiously, it works if I mark the function with the region
>> and do "eval-region", it works properly.

> Sorry for the delay.  But
> I'm unable to reproduce such an infloop-like error so far.
> Could you show the name or the definition of the function
> causing it?

I've committed my patch since I think it's better at least than
not working.  Please reopen the bug if you get a reproducible error.

bug#16184: incorrect Edebug spec for eval-and-compile

[Gemini Lasswell]

This bug also exists in eval-and-compile, which needs the same fix as
eval-when-compile.

(defun test (limit)
  (eval-and-compile
    (boundp 'parse-sexp-lookup-properties)))

To reproduce:

1. emacs -Q
2. enter the above code into *scratch*
3. C-u C-M-x

Result: edebug-after: Wrong type argument: consp, nil

bug#16184: confirmed in 25.1

[David Bremner]

[-- Attachment #1: Type: text/plain, Size: 135 bytes --]


I tried the example from message #44 and got the same failure.
it does say "Edebug: test" on the previous line of *Messages*, fwiw

d

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 629 bytes --]

bug#16184: confirmed on 25.1.1

[Benaiah Mischenko]

The eval-and-compile test case from message #44 fails for me in emacs -Q as well, with the following *Messages* output:

Edebug: test
edebug-after: Wrong type argument: consp, nil

The output of M-x emacs-version is as follows:

GNU Emacs 25.1.1 (x86_64-apple-darwin15.6.0, NS appkit-1404.47 Version 10.11.6 (Build 15G31)) of 2016-09-21

Turning on toggle-debug-on-error and running it again gave me the following backtrace:

Debugger entered--Lisp error: (wrong-type-argument consp nil)
  edebug-before(1)
  (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties)))
  (progn (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties))))
  eval((progn (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties)))) nil)
  #[128 "\301\302\303\x03B\b\"D\207" [lexical-binding quote eval progn] 5 1783882]((edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties))))
  macroexpand((eval-and-compile (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties)))) nil)
  macroexp-macroexpand((eval-and-compile (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties)))) nil)
  macroexp--expand-all((eval-and-compile (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties)))))
  macroexp--all-forms((edebug-after (edebug-before 0) 3 (eval-and-compile (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties))))) 1)
  #[514 "\300\x01\301\"\211\204\x10

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Alan Mackenzie]

Hello, Anders.

On Wed, Dec 18, 2013 at 11:58:23AM +0100, Anders Lindgren wrote:
> Hi!

> The construct 'eval-when-compile' seems to throw debug off. When pressing
> C-u C-M-x on the following, an error is issued. This worked correctly in
> 24.3. This makes it hard to use edebug on functions in cc-mode
> (like c-font-lock-declarations), as it makes heavy use of eval-when-compile.

> (defun test (limit)
>   (eval-when-compile
>     (boundp 'parse-sexp-lookup-properties)))

> Sincerely,
>     Anders Lidgren

[ .... ].

This has been bugging me off and on for years, too.  If I'd noticed your
bug report back in 2013, I might have got round to looking at it then,
but I thought it was just me that was suffering.  :-(

I think I now understand what's happening, and I have a tentative fix,
though this might be more of a workaround than fix, I'm not sure.

Anyhow, this is the form I've been testing with:

    (defun test (limit)
      (let ((foo
             (cc-eval-when-compile
               (boundp 'parse-sexp-lookup-properties))))))

, which clearly needs (require 'cc-defs) before it if you don't have CC
Mode loaded already.

#########################################################################

Essential background information
================================

When edebug instruments a function, it inserts a call to edebug-enter at
the start of the function.  Regularly throughout the function, it inserts
calls to edebug-before and edebug-after, in forms which look something
like:

    (edebug-before 2)
    (edebug-after (edebug-before 2) 3 ...)

.  edebug maintains a stack of numerical "offsets" (whatever they are)
called edebug-offset-indices, consing a 0 onto it in edebug-enter, each
time an instrumented function is entered.  At each edebug-before or
edebug-after, the top element of this stack is modified with:

    (setcar edebug-offset-indices before-index) ; where before-index is,
say 2.

The offset at the top of edebug-offset-indices is used to position point
each time edebug displays the buffer being debugged before waiting for
user input.

#########################################################################

The bug mechanism is this: The macro-expander working on
cc-eval-when-compile, and edebug's instrumentation produce a form
something like this:

    (edebug-enter 'test (limit)
                  (function (lambda ......
                   (cc-eval-when-compile
                    (edebug-after (edebug-before 2) 3
                                  (boundp 'parse-sexp-lookup-properties)
                   .....)))))

.  During this instrumentation, the inside of the cc-eval-when-compile
form gets run, in particular, the (edebug-before 2).  At this stage,
since edebug-enter hasn't been run at all, edebug-offset-indices is still
nil.  The (edebug-before 2) thus attempts (setcar nil 2), which throws
the error.

#########################################################################

My proposed solution/workaround is to initialise edebug-offset-indices to
'(0) rather than nil, giving edebug-before/after a cons to work on, even
when they're called "at the top level".

This seems to work, sort of.  In my test form above, when I do C-u C-M-x,
the function `test' gets successfully instrumented.  However, if I
replace cc-eval-when-compile with eval-when-compile and do C-u C-M-x, the
behaviour is different: during the instrumentation, edebug steps through
the `boundp' form.  I don't yet understand where this difference comes
from.

So, here's the patch, which I'd be grateful if you would try out.  It's
based on the savannah master:



diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
index 66117b8..5dc6f89 100644
--- a/lisp/emacs-lisp/edebug.el
+++ b/lisp/emacs-lisp/edebug.el
@@ -2052,9 +2052,9 @@ edebug-stack
 (defvar edebug-stack-depth -1)
 ;; Index of last edebug-stack item.
 
-(defvar edebug-offset-indices nil)
+(defvar edebug-offset-indices (list 0))
 ;; Stack of offset indices of visited edebug sexps.
-;; Should be nil at the top level.
+;; Should be nil at the top level.  No longer!  Extensive comment needed here.
 ;; Each function adds one cons.  Top is modified with setcar.
 
 
This patch clearly needs commenting added to it, but I'm in much too
prolix a frame of mind to attempt that now.  ;-(

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Alan Mackenzie]

Bug fixed.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 5516 bytes --]

Hi!

I just got around to test this. Now, `eval-when-comple' seems to be
working. Unfortunately, `eval-and-comple' is still broken.

If I do C-u M-C-x eval the following with `debug-on-error' set to t, I get
the backtrace below:

(defun test (limit)
  (eval-and-compile
    (boundp 'parse-sexp-lookup-properties)))


Debugger entered--Lisp error: (void-variable edebug-freq-count)
  edebug-before(1)
  (edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties)))
  (progn (edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties))))
  eval((progn (edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties)))) nil)
  #[128 "\301\302\303 B \"D\207" [lexical-binding quote eval progn] 5
1802730]((edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties))))
  macroexpand((eval-and-compile (edebug-after (edebug-before 1) 2 (boundp
(quote parse-sexp-lookup-properties)))) nil)
  macroexp-macroexpand((eval-and-compile (edebug-after (edebug-before 1) 2
(boundp (quote parse-sexp-lookup-properties)))) nil)
  macroexp--expand-all((eval-and-compile (edebug-after (edebug-before 1) 2
(boundp (quote parse-sexp-lookup-properties)))))
  macroexp--all-forms((edebug-after (edebug-before 0) 3 (eval-and-compile
(edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties))))) 1)
  #[514 "\300 \301\"\211\204 \302 \242\303\"\202g\304 !\204+\3051$\306\307
! \"0\202*\310\311 \"\262 \210\312 \242\" \242 =\203b \242\302
\242\303\"\211\262 \232\203J \242\202e \312 \"\240\210\211
\242=\203\\\211\202e\313 !\202e\313 !\262 \207" [function-get
compiler-macro macroexp--all-forms 1 functionp (debug error)
autoload-do-load indirect-function message "macroexp--expand-all: %S"
macroexp--compiler-macro macroexp--expand-all] 8 "\n\n(fn FORM
FUNC)"](((edebug-after (edebug-before 0) 3 (eval-and-compile (edebug-after
(edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties))))))
edebug-after)
  macroexp--expand-all((edebug-after (edebug-before 0) 3 (eval-and-compile
(edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties))))))
  macroexp--all-forms((lambda nil (edebug-after (edebug-before 0) 3
(eval-and-compile (edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties)))))) 2)
  macroexp--expand-all((function (lambda nil (edebug-after (edebug-before
0) 3 (eval-and-compile (edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties))))))))
  macroexp--all-forms((edebug-enter (quote test) (list limit) (function
(lambda nil (edebug-after (edebug-before 0) 3 (eval-and-compile
(edebug-after (edebug-before 1) 2 (boundp ...))))))) 1)
  #[514 "\300 \301\"\211\204 \302 \242\303\"\202g\304 !\204+\3051$\306\307
! \"0\202*\310\311 \"\262 \210\312 \242\" \242 =\203b \242\302
\242\303\"\211\262 \232\203J \242\202e \312 \"\240\210\211
\242=\203\\\211\202e\313 !\202e\313 !\262 \207" [function-get
compiler-macro macroexp--all-forms 1 functionp (debug error)
autoload-do-load indirect-function message "macroexp--expand-all: %S"
macroexp--compiler-macro macroexp--expand-all] 8 "\n\n(fn FORM
FUNC)"](((edebug-enter (quote test) (list limit) (function (lambda nil
(edebug-after (edebug-before 0) 3 (eval-and-compile (edebug-after ... 2
...))))))) edebug-enter)
  macroexp--expand-all((edebug-enter (quote test) (list limit) (function
(lambda nil (edebug-after (edebug-before 0) 3 (eval-and-compile
(edebug-after (edebug-before 1) 2 (boundp ...))))))))
  macroexp--all-forms((lambda (limit) (edebug-enter (quote test) (list
limit) (function (lambda nil (edebug-after (edebug-before 0) 3
(eval-and-compile (edebug-after ... 2 ...))))))) 2)
  macroexp--expand-all((function (lambda (limit) (edebug-enter (quote test)
(list limit) (function (lambda nil (edebug-after (edebug-before 0) 3
(eval-and-compile ...))))))))
  macroexp--all-forms((defalias (quote test) (function (lambda (limit)
(edebug-enter (quote test) (list limit) (function (lambda nil (edebug-after
... 3 ...))))))) 1)
  #[514 "\300 \301\"\211\204 \302 \242\303\"\202g\304 !\204+\3051$\306\307
! \"0\202*\310\311 \"\262 \210\312 \242\" \242 =\203b \242\302
\242\303\"\211\262 \232\203J \242\202e \312 \"\240\210\211
\242=\203\\\211\202e\313 !\202e\313 !\262 \207" [function-get
compiler-macro macroexp--all-forms 1 functionp (debug error)
autoload-do-load indirect-function message "macroexp--expand-all: %S"
macroexp--compiler-macro macroexp--expand-all] 8 "\n\n(fn FORM
FUNC)"](((defalias (quote test) (function (lambda (limit) (edebug-enter
(quote test) (list limit) (function (lambda nil ...))))))) defalias)
  macroexp--expand-all((defun test (limit) (edebug-enter (quote test) (list
limit) (function (lambda nil (edebug-after (edebug-before 0) 3
(eval-and-compile (edebug-after ... 2 ...))))))))
  macroexpand-all((defun test (limit) (edebug-enter (quote test) (list
limit) (function (lambda nil (edebug-after (edebug-before 0) 3
(eval-and-compile (edebug-after ... 2 ...))))))))
  eval-sexp-add-defvars((defun test (limit) (edebug-enter (quote test)
(list limit) (function (lambda nil (edebug-after (edebug-before 0) 3
(eval-and-compile (edebug-after ... 2 ...))))))))
  edebug-eval-defun((4))
  apply(edebug-eval-defun (4))
  eval-defun((4))
  funcall-interactively(eval-defun (4))
  call-interactively(eval-defun nil nil)
  command-execute(eval-defun)

    -- Anders


On Thu, Dec 29, 2016 at 10:29 AM, Alan Mackenzie <acm@muc.de> wrote:

> Bug fixed.
>
> --
> Alan Mackenzie (Nuremberg, Germany).
>

[-- Attachment #2: Type: text/html, Size: 6606 bytes --]

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Gemini Lasswell]

Alan Mackenzie <acm@muc.de> writes:

>
> .  During this instrumentation, the inside of the cc-eval-when-compile
> form gets run, in particular, the (edebug-before 2).  At this stage,
> since edebug-enter hasn't been run at all, edebug-offset-indices is still
> nil.  The (edebug-before 2) thus attempts (setcar nil 2), which throws
> the error.
>

I haven't able to reproduce the bug with cc-eval-when-compile,
just eval-and-compile. But the thing that is supposed to make Edebug
wrap a form in edebug-enter is the use of def-form or def-body in the
Edebug spec. It works for eval-when-compile which has the Edebug spec
(&rest def-form). The body of eval-and-compile doesn't get wrapped
because its Edebug spec is t, so the bug happens there.

cc-eval-when-compile has the same Edebug spec as eval-when-compile, so
its body should get wrapped by edebug-enter. If that's not happening in
your Emacs, it's a bug in Edebug which is different from the
eval-and-compile Edebug spec bug.

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Alan Mackenzie]

Hello, Gemini.

On Fri, Dec 30, 2016 at 03:27:37PM -0800, Gemini Lasswell wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > .  During this instrumentation, the inside of the cc-eval-when-compile
> > form gets run, in particular, the (edebug-before 2).  At this stage,
> > since edebug-enter hasn't been run at all, edebug-offset-indices is still
> > nil.  The (edebug-before 2) thus attempts (setcar nil 2), which throws
> > the error.

> I haven't able to reproduce the bug with cc-eval-when-compile,
> just eval-and-compile. But the thing that is supposed to make Edebug
> wrap a form in edebug-enter is the use of def-form or def-body in the
> Edebug spec.

This is extremely poorly documented, even in the source code, if it is
documented at all.  I don't really understand the documentation of the
edebug spec in general.

> It works for eval-when-compile which has the Edebug spec (&rest
> def-form). The body of eval-and-compile doesn't get wrapped because
> its Edebug spec is t, so the bug happens there.

Ah!  I think I'm beginning to understand all this.  Looking at the
source code, it seems a def-form or def-body causes the form to be
wrapping by edebug-enter, but not much else does.

> cc-eval-when-compile has the same Edebug spec as eval-when-compile, so
> its body should get wrapped by edebug-enter. If that's not happening in
> your Emacs, it's a bug in Edebug which is different from the
> eval-and-compile Edebug spec bug.

I think you've persuaded me that my fix to this bug was wrong.  Rather
than adapting initialisations to cope with edebug-enter not being
called, I should have diagnosed _why_ an edebug-enter is not being
generated during instrumentation of a cc-eval-when-compile form.

I'm working on this now.  Thank you indeed for drawing my attention to
these edebug specs.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Alan Mackenzie]

Hello again, Anders.

On Fri, Dec 30, 2016 at 09:02:31PM +0100, Anders Lindgren wrote:
> Hi!

> I just got around to test this. Now, `eval-when-comple' seems to be
> working. Unfortunately, `eval-and-comple' is still broken.

> If I do C-u M-C-x eval the following with `debug-on-error' set to t, I get
> the backtrace below:

> (defun test (limit)
>   (eval-and-compile
>     (boundp 'parse-sexp-lookup-properties)))

[ snip backtrace ]

I got a tip from Gemini Lasswell last night.  What is broken is not
edebug.el itself, but the edebug specs applied to various macros.  For
macros like eval-\(when\|and\)-compile, there needs to be an
edebug-enter form compile around the macro.  Most edebug specs don't
compile this - only (I think) def-form and def-body do.  This is not
clearly documented.

Anyhow, I've changed the edebug spec of eval-and-compile to what it
should be (\(cc-\)?eval-when-compile already had it), and I think these
problems are all solved.

Would you please try out the following patch, and let me know sometime
that it's OK, then I can commit it.

Happy New Year again!



diff --git a/lisp/emacs-lisp/byte-run.el b/lisp/emacs-lisp/byte-run.el
index 9d2a048..e680ebb 100644
--- a/lisp/emacs-lisp/byte-run.el
+++ b/lisp/emacs-lisp/byte-run.el
@@ -474,7 +474,7 @@ eval-and-compile
 `progn', except that the value of the expression may be (but is
 not necessarily) computed at load time if eager macro expansion
 is enabled."
-  (declare (debug t) (indent 0))
+  (declare (debug (&rest def-form)) (indent 0))
   ;; When the byte-compiler expands code, this macro is not used, so we're
   ;; either about to run `body' (plain interpretation) or we're doing eager
   ;; macroexpansion.
diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
index 15b1389..fd3e48d 100644
--- a/lisp/emacs-lisp/edebug.el
+++ b/lisp/emacs-lisp/edebug.el
@@ -2052,15 +2052,10 @@ edebug-stack
 (defvar edebug-stack-depth -1)
 ;; Index of last edebug-stack item.
 
-(defvar edebug-offset-indices (list 0))
+(defvar edebug-offset-indices nil)
 ;; Stack of offset indices of visited edebug sexps.
 ;; Each function adds one cons.  Top is modified with setcar.
-;; Should be nil at the top level.  No longer!  There are occasions
-;; when this variable is modified by setcar before a cons has been
-;; pushed by `edebug-enter', e.g. when instrumenting
-;; `c-font-lock-declarations' in .../lisp/progmodes/cc-fonts.el.  So
-;; this variable must be initialized to a cons.
-
+;; Should be nil at the top level.
 (defvar edebug-entered nil
   ;; Non-nil if edebug has already been entered at this recursive edit level.
   ;; This should stay nil at the top level.

>     -- Anders

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 3086 bytes --]

Hi Alan!

This time it seems to be working properly with both `eval-when-compile' and
`eval-and-compile'!

Happy new year to you too!

    -- Anders

On Sat, Dec 31, 2016 at 5:22 PM, Alan Mackenzie <acm@muc.de> wrote:

> Hello again, Anders.
>
> On Fri, Dec 30, 2016 at 09:02:31PM +0100, Anders Lindgren wrote:
> > Hi!
>
> > I just got around to test this. Now, `eval-when-comple' seems to be
> > working. Unfortunately, `eval-and-comple' is still broken.
>
> > If I do C-u M-C-x eval the following with `debug-on-error' set to t, I
> get
> > the backtrace below:
>
> > (defun test (limit)
> >   (eval-and-compile
> >     (boundp 'parse-sexp-lookup-properties)))
>
> [ snip backtrace ]
>
> I got a tip from Gemini Lasswell last night.  What is broken is not
> edebug.el itself, but the edebug specs applied to various macros.  For
> macros like eval-\(when\|and\)-compile, there needs to be an
> edebug-enter form compile around the macro.  Most edebug specs don't
> compile this - only (I think) def-form and def-body do.  This is not
> clearly documented.
>
> Anyhow, I've changed the edebug spec of eval-and-compile to what it
> should be (\(cc-\)?eval-when-compile already had it), and I think these
> problems are all solved.
>
> Would you please try out the following patch, and let me know sometime
> that it's OK, then I can commit it.
>
> Happy New Year again!
>
>
>
> diff --git a/lisp/emacs-lisp/byte-run.el b/lisp/emacs-lisp/byte-run.el
> index 9d2a048..e680ebb 100644
> --- a/lisp/emacs-lisp/byte-run.el
> +++ b/lisp/emacs-lisp/byte-run.el
> @@ -474,7 +474,7 @@ eval-and-compile
>  `progn', except that the value of the expression may be (but is
>  not necessarily) computed at load time if eager macro expansion
>  is enabled."
> -  (declare (debug t) (indent 0))
> +  (declare (debug (&rest def-form)) (indent 0))
>    ;; When the byte-compiler expands code, this macro is not used, so we're
>    ;; either about to run `body' (plain interpretation) or we're doing
> eager
>    ;; macroexpansion.
> diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
> index 15b1389..fd3e48d 100644
> --- a/lisp/emacs-lisp/edebug.el
> +++ b/lisp/emacs-lisp/edebug.el
> @@ -2052,15 +2052,10 @@ edebug-stack
>  (defvar edebug-stack-depth -1)
>  ;; Index of last edebug-stack item.
>
> -(defvar edebug-offset-indices (list 0))
> +(defvar edebug-offset-indices nil)
>  ;; Stack of offset indices of visited edebug sexps.
>  ;; Each function adds one cons.  Top is modified with setcar.
> -;; Should be nil at the top level.  No longer!  There are occasions
> -;; when this variable is modified by setcar before a cons has been
> -;; pushed by `edebug-enter', e.g. when instrumenting
> -;; `c-font-lock-declarations' in .../lisp/progmodes/cc-fonts.el.  So
> -;; this variable must be initialized to a cons.
> -
> +;; Should be nil at the top level.
>  (defvar edebug-entered nil
>    ;; Non-nil if edebug has already been entered at this recursive edit
> level.
>    ;; This should stay nil at the top level.
>
> >     -- Anders
>
> --
> Alan Mackenzie (Nuremberg, Germany).
>

[-- Attachment #2: Type: text/html, Size: 3999 bytes --]

bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Alan Mackenzie]

Hello, Anders.

On Sun, Jan 01, 2017 at 01:30:44AM +0100, Anders Lindgren wrote:
> Hi Alan!

> This time it seems to be working properly with both `eval-when-compile' and
> `eval-and-compile'!

> Happy new year to you too!

Thanks!  I've just committed the fix.

>     -- Anders

-- 
Alan Mackenzie (Nuremberg, Germany).