bug#40766: 27.0.91; Cannot edebug cl-lambda-list edebug spec

[Alan Mackenzie <acm@muc.de>]

Hello again, Philipp.

On Thu, Apr 23, 2020 at 06:41:28 -0000, Alan Mackenzie wrote:
> In article <mailman.944.1587560406.3066.bug-gnu-emacs@gnu.org> you wrote:

> > emacs -Q
> > M-x find-library RET cl-macs RET
> > Navigate to the form `(def-edebug-spec cl-lambda-list ...)'
> > Hit C-u C-M-x to edebug it.

> > This results in an error with backtrace:

Yes.  There is a bug in the edebug specs for edebug-spec-list and
edebug-spec.  (Sorry about this sounding confusing.  Believe me, it's
nothing compared with the head banging involved in keeping various
levels of the debug specs mentally separate.  ;-)  These two specs are
the main components of the debug spec for def-edebug-spec.  (It's
probably still confusing at this stage.)  These two things look like
this:

    (def-edebug-spec edebug-spec-list
      ;; A list must have something in it, or it is nil, a symbolp
      ((edebug-spec . [&or nil edebug-spec])))

    (def-edebug-spec edebug-spec
      (&or
    ; edebug-spec-list   <==============================================
       (vector &rest edebug-spec)           ; matches a vector
       ("vector" &rest edebug-spec)         ; matches a vector spec
       ("quote" symbolp)
       edebug-spec-list ;; <============================================
       stringp
       [edebug-lambda-list-keywordp &rest edebug-spec]
       [keywordp gate edebug-spec]
       edebug-spec-p  ;; Including all the special ones e.g. form.
       symbolp;; a predicate
       ))

The target spec, let's continue calling it spec 2, reduced to it's
minimum required to trigger the bug, looks like:

    (def-edebug-spec acm-lambda-list
      (([&rest cl-lambda-arg]
        ["&optional" cl-&optional-arg &rest cl-&optional-arg]
        . [&or arg nil])))

.  The essential attributes are it has enclosing parens (at least)
containing two vectors followed by a dot followed by a third vector.

When spec 2 gets parsed by edebug-spec (see above), it eventually
reaches a dotted list containing the three vectors.  It parses this with
edebug-spec-list containing the first two vectors.  But BECAUSE
edebug-spec-list IS MISPLACED INSIDE edebug-spec (see the two arrowed
lines), the second vector triggers recognition by (vector ....) rather
than by edebug-spec-list.  This "dedottifies" the dotted list in spec 2,
so that when the dot in spec 2 is reached, edebug-spec-list doesn't know
how to handle it.  For some reason, the backtracking doesn't backtrack
here.

The solution to the problem is to move the edebug-spec-list from after
the (vector ...) and ("quote" ....) entries to before them, as indicated
above.  This causes the second vector to get handled as the first
element of the cdr of the list it's contained within.


> Yes, I get this too.  The debug-spec definition looks like this:

>     (def-edebug-spec cl-lambda-list
>       (([&rest cl-lambda-arg]
>         [&optional ["&optional" cl-&optional-arg &rest cl-&optional-arg]]
>         [&optional ["&rest" cl-lambda-arg]]
>         [&optional ["&key" [cl-&key-arg &rest cl-&key-arg]
>                     &optional "&allow-other-keys"]]
>         [&optional ["&aux" &rest
>                     &or (symbolp &optional def-form) symbolp]]
>         . [&or arg nil])))

> After playing around with it for a few hours, it is clear that the
> bug is triggered by a combination of (i) the dot on the last line; and
> (ii) the presence of at least one [&optional ...] before it, at the same
> level of nesting.  Possibly the nesting of [ ... ] structures might have
> something to do with it.

So, I propose installing the following patch to fix the bug:


diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
index bb7817f242..7995a9e5a2 100644
--- a/lisp/emacs-lisp/edebug.el
+++ b/lisp/emacs-lisp/edebug.el
@@ -2115,10 +2115,10 @@ edebug-spec-list
 
 (def-edebug-spec edebug-spec
   (&or
+   edebug-spec-list
    (vector &rest edebug-spec)		; matches a vector
    ("vector" &rest edebug-spec)		; matches a vector spec
    ("quote" symbolp)
-   edebug-spec-list
    stringp
    [edebug-lambda-list-keywordp &rest edebug-spec]
    [keywordp gate edebug-spec]


> > -- 
> > Google Germany GmbH
> > Erika-Mann-Straße 33
> > 80636 München

-- 
Alan Mackenzie (Nuremberg, Germany).