bug#24773: 26.0.50; Error when using Edebug on a function that uses :documentation

bug#24773: 26.0.50; Error when using Edebug on a function that uses :documentation

[Gemini Lasswell]

Edebug gives an error when you try to debug any function which uses
:documentation to create its docstring. To reproduce:

1. Start Emacs (with -Q if you like) and enter the following code into *scratch*:

;;;  -*- lexical-binding: t -*-

(defun make-animal (species noise)
  (lambda ()
    (:documentation (format "I'm a %s." species))
    (message "%s" noise)))

(defalias 'kermit (make-animal "frog" "ribbit"))
(message "Kermit says %s" (documentation 'kermit t))
(kermit)

2. M-x edebug-all-defs RET
3. M-x eval-buffer RET
4. g

Result: "Kermit says nil" is written to Messages, and execution stops
with the error "Symbol’s function definition is void: :documentation".

This bug happens not only with lambda but also defun, defmacro,
defmethod, pcase-lambda, cl-defun, etc.


In GNU Emacs 26.0.50.10 (x86_64-apple-darwin15.6.0, NS appkit-1404.47 Version 10.11.6 (Build 15G1004))
 of 2016-10-22 built on rainbow.local
Repository revision: f577b8efec324afc80aa4308efbfa40dcce30bbf
Windowing system distributor 'Apple', version 10.3.1404
Recent messages:
Edebugging all definitions is on.
You can run the command ‘edebug-all-defs’ with M-x -defs RET
Edebugging all definitions is on.
Edebug: edebug-anon1358
Edebug: make-animal
Kermit says nil
Go...
Symbol's function definition is void: :documentation
Go...
edebug-signal: Symbol’s function definition is void: :documentation
Configured using:
 'configure --with-ns --disable-ns-self-contained'

Configured features:
JPEG RSVG IMAGEMAGICK NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS
NS

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Emacs-Lisp

Minor modes in effect:
  highlight-symbol-nav-mode: t
  magit-auto-revert-mode: t
  global-git-commit-mode: t
  async-bytecomp-package-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  diff-auto-refine-mode: t
  ivy-mode: t
  buffer-face-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  pyvenv-mode: t
  shell-dirtrack-mode: t
  nameless-mode: t
  beacon-mode: t
  rainbow-mode: t
  column-enforce-mode: t
  volatile-highlights-mode: t
  region-state-mode: t
  ws-butler-global-mode: t
  ws-butler-mode: t
  show-smartparens-global-mode: t
  show-smartparens-mode: t
  smartparens-global-mode: t
  smartparens-global-strict-mode: t
  smartparens-strict-mode: t
  smartparens-mode: t
  which-key-mode: t
  modalka-mode: t
  recentf-mode: t
  global-auto-revert-mode: t
  winner-mode: t
  display-time-mode: t
  savehist-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug highlight-symbol locate magit-blame
magit-stash magit-bisect magit-remote magit-commit magit-sequence magit
magit-apply magit-wip magit-log magit-diff smerge-mode magit-core
magit-autorevert magit-process magit-popup magit-mode magit-git crm
magit-section magit-utils git-commit log-edit message rfc822 mml mml-sec
epa derived epg gnus-util rmail rmail-loaddefs mailabbrev gmm-utils
mailheader pcvs-util with-editor async-bytecomp async server tabify
noutline outline dash-at-point undo-tree diff bug-reference add-log
expand-region text-mode-expansions python-el-fgallina-expansions
er-basic-expansions expand-region-core expand-region-custom ielm
eieio-tests eieio-opt speedbar sb-image ezimage dframe eieio-base ert pp
find-func ewoc debug eieio-compat edebug vc-git diff-mode colir flx
dired dired-loaddefs counsel esh-util swiper ivy delsel ffap face-remap
guess-style smtpmail sendmail mail-utils yasnippet highlight-indentation
flymake company help-fns radix-tree elpy pyvenv elpy-refactor
smartparens-python python tramp-sh tramp tramp-compat tramp-loaddefs
trampver shell pcomplete format-spec json map grep compile files-x
cus-edit virtualenvwrapper gud comint nameless lisp-mnt ace-window avy
beacon smex ido deft debbugs soap-client mm-decode mm-bodies mm-encode
warnings rng-xsd rng-dt rng-util xsd-regexp xml rainbow-mode ansi-color
color s hydra lv column-enforce-mode etags xref project
volatile-highlights region-state ws-butler smartparens-config
smartparens thingatpt dash which-key modalka quail
smart-mode-line-dark-theme smart-mode-line advice rich-minority
whiteboard-theme classic-theme recentf tree-widget wid-edit autorevert
filenotify winner ring time cus-start cus-load savehist cap-words
superword subword use-package diminish bind-key easy-mmode finder-inf
edmacro kmacro info network-stream starttls url-http tls gnutls
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr url-gw
nsm subr-x puny url-cache url-auth url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util mailcap cl
package epg-config url-handlers url-parse auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs password-cache url-vars seq byte-opt
gv bytecomp byte-compile cl-extra help-mode easymenu cconv cl-loaddefs
pcase cl-lib time-date mule-util tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/ns-win ns-win
ucs-normalize term/common-win tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev obarray minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote kqueue cocoa ns multi-tty
make-network-process emacs)

Memory information:
((conses 16 785197 160038)
 (symbols 48 45770 0)
 (miscs 40 2961 2551)
 (strings 32 116734 150424)
 (string-bytes 1 3378243)
 (vectors 16 71418)
 (vector-slots 8 1235646 55579)
 (floats 8 1146 1541)
 (intervals 56 44259 4364)
 (buffers 976 38))

bug#24773: 26.0.50; Error when using Edebug on a function that uses :documentation

[Noam Postavsky]

On Sun, Oct 23, 2016 at 11:28 AM, Gemini Lasswell <gazally@runbox.com> wrote:
> Edebug gives an error when you try to debug any function which uses
> :documentation to create its docstring.

Is that a legitimate way to create a docstring?

I tried

(defun foo ()
  (:documentation "this is docmentation")
  t)
(foo)

And it didn't work (<f1> f foo RET says foo is not documented, and
running foo, throws Lisp error: (void-function :documentation)).

bug#24773: 26.0.50; Error when using Edebug on a function that uses :documentation

[Gemini Lasswell]

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> Is that a legitimate way to create a docstring?

Yes, but lexical-binding needs to be on for it to work. It's mentioned
in the NEWS for Emacs 25.

bug#24773: 26.0.50; Error when using Edebug on a function that uses :documentation

[npostavs]

tags 24773 confirmed
found 24773 25.1
quit

Gemini Lasswell <gazally@runbox.com> writes:

> Noam Postavsky <npostavs@users.sourceforge.net> writes:
>
>> Is that a legitimate way to create a docstring?
>
> Yes, but lexical-binding needs to be on for it to work. It's mentioned
> in the NEWS for Emacs 25.

Ah, I see it now, yes.

bug#24773: 26.0.50; Error when using Edebug on a function that uses :documentation

[Gemini Lasswell]

[-- Attachment #1: Type: text/plain, Size: 602 bytes --]

tags 24773 patch
quit

Here is a patch to add support for :documentation to the Edebug specs
of the macros where it can be used (see bug#28557 for a list of those,
and some related bugs).

cl-defgeneric was missing an Edebug spec and I reported that as
bug#27747. Since it uses :documentation, I wrote one for it and
included it here, so that bug will also be fixed by this patch.

Before this patch there weren't any Edebug specs in
lisp/emacs-lisp/generator.el, and lots of macros. Consequently,
while iterators can be instrumented, they don't work under Edebug.
I'll report that as a separate bug.


[-- Attachment #2: 0001-Create-new-Edebug-spec-for-docstrings-and-use-it-in-.patch --]
[-- Type: text/plain, Size: 8353 bytes --]

From d048fe3f0372011f4aef5cae0a89ccdbe3d59bf8 Mon Sep 17 00:00:00 2001
From: Gemini Lasswell <gazally@runbox.com>
Date: Fri, 29 Sep 2017 14:58:20 -0700
Subject: [PATCH] Create new Edebug spec for docstrings and use it in closures

Since (:documentation FORM) can be used to create a docstring
when lexical-binding is on, allow for that possibility in Edebug
specs (bug#24773).
* lisp/emacs-lisp/edebug.el: Define an Edebug spec for docstrings
called lambda-doc and modify the Edebug specs for defun and
defmacro to use it.
(edebug-instrument-function): Check for generic functions first,
to fix bug where edebug-step-in didn't work on methods now that
cl-defgeneric has an Edebug spec.
* lisp/subr.el (lambda): Modify Edebug spec to use lambda-doc.
* lisp/emacs-lisp/cl-generic.el (cl-defgeneric): Add Edebug spec
(bug#27747).
(cl-defmethod): Use lambda-doc in Edebug spec.
* lisp/emacs-lisp/cl-macs.el: Modify Edebug spec for
cl-declarations-or-string to use lambda-doc, and modify Edebug
spec for cl-lambda-expr to use cl-declarations-or-string.
* lisp/emacs-lisp/pcase.el (pcase-lambda): Modify Edebug spec to
use lambda-doc, as well as &define and def-body which are
necessary for using Edebug on code wrapped by lambda.
* lisp/emacs-lisp/generator.el (iter-defun, iter-lambda): Add
Edebug specs.
---
 lisp/emacs-lisp/cl-generic.el | 13 +++++++++++--
 lisp/emacs-lisp/cl-macs.el    |  6 +++---
 lisp/emacs-lisp/edebug.el     | 27 +++++++++++++++------------
 lisp/emacs-lisp/generator.el  |  6 ++++--
 lisp/emacs-lisp/pcase.el      |  2 +-
 lisp/subr.el                  |  3 +--
 6 files changed, 35 insertions(+), 22 deletions(-)

diff --git a/lisp/emacs-lisp/cl-generic.el b/lisp/emacs-lisp/cl-generic.el
index b2f76abd88..62befd4742 100644
--- a/lisp/emacs-lisp/cl-generic.el
+++ b/lisp/emacs-lisp/cl-generic.el
@@ -204,7 +204,16 @@ cl-defgeneric
 DEFAULT-BODY, if present, is used as the body of a default method.
 
 \(fn NAME ARGS [DOC-STRING] [OPTIONS-AND-METHODS...] &rest DEFAULT-BODY)"
-  (declare (indent 2) (doc-string 3))
+  (declare (indent 2) (doc-string 3)
+           (debug
+            (&define name cl-lambda-list lambda-doc
+                     [&rest [&or
+                             ("declare" &rest sexp)
+                             (":argument-precedence-order" &rest sexp)
+                             (&define ":method" [&rest atom]
+                                      cl-generic-method-args lambda-doc
+                                      def-body)]]
+                     def-body)))
   (let* ((doc (if (stringp (car-safe options-and-methods))
                   (pop options-and-methods)))
          (declarations nil)
@@ -422,7 +431,7 @@ cl-defmethod
                                     ; Like in CLOS spec, we support
                                     ; any non-list values.
              cl-generic-method-args     ; arguments
-             [ &optional stringp ]      ; documentation string
+             lambda-doc                 ; documentation string
              def-body)))                ; part to be debugged
   (let ((qualifiers nil))
     (while (not (listp args))
diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el
index 32ba0ac309..40eda1e0d6 100644
--- a/lisp/emacs-lisp/cl-macs.el
+++ b/lisp/emacs-lisp/cl-macs.el
@@ -190,7 +190,7 @@ cl-declarations
   (&rest ("cl-declare" &rest sexp)))
 
 (def-edebug-spec cl-declarations-or-string
-  (&or stringp cl-declarations))
+  (&or lambda-doc cl-declarations))
 
 (def-edebug-spec cl-lambda-list
   (([&rest arg]
@@ -447,8 +447,8 @@ cl-defmacro
 
 (def-edebug-spec cl-lambda-expr
   (&define ("lambda" cl-lambda-list
-	    ;;cl-declarations-or-string
-	    ;;[&optional ("interactive" interactive)]
+	    cl-declarations-or-string
+	    [&optional ("interactive" interactive)]
 	    def-body)))
 
 ;; Redefine function-form to also match cl-function
diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
index dbc56e272f..d00b14e803 100644
--- a/lisp/emacs-lisp/edebug.el
+++ b/lisp/emacs-lisp/edebug.el
@@ -1986,15 +1986,14 @@ defconst
 (def-edebug-spec defvar (symbolp &optional form stringp))
 
 (def-edebug-spec defun
-  (&define name lambda-list
-	   [&optional stringp]
+  (&define name lambda-list lambda-doc
            [&optional ("declare" &rest sexp)]
 	   [&optional ("interactive" interactive)]
 	   def-body))
 (def-edebug-spec defmacro
   ;; FIXME: Improve `declare' so we can Edebug gv-expander and
   ;; gv-setter declarations.
-  (&define name lambda-list [&optional stringp]
+  (&define name lambda-list lambda-doc
            [&optional ("declare" &rest sexp)] def-body))
 
 (def-edebug-spec arglist lambda-list)  ;; deprecated - use lambda-list.
@@ -2005,6 +2004,10 @@ lambda-list
     &optional ["&rest" arg]
     )))
 
+(def-edebug-spec lambda-doc
+  (&optional [&or stringp
+                  (&define ":documentation" def-form)]))
+
 (def-edebug-spec interactive
   (&optional &or stringp def-form))
 
@@ -3204,15 +3207,6 @@ edebug-instrument-function
 instrument cannot be found, signal an error."
   (let ((func-marker (get func 'edebug)))
     (cond
-     ((and (markerp func-marker) (marker-buffer func-marker))
-      ;; It is uninstrumented, so instrument it.
-      (with-current-buffer (marker-buffer func-marker)
-	(goto-char func-marker)
-	(edebug-eval-top-level-form)
-        (list func)))
-     ((consp func-marker)
-      (message "%s is already instrumented." func)
-      (list func))
      ((cl-generic-p func)
       (let ((method-defs (cl--generic-method-files func))
             symbols)
@@ -3227,6 +3221,15 @@ edebug-instrument-function
               (edebug-eval-top-level-form)
               (push (edebug-form-data-symbol) symbols))))
         symbols))
+     ((and (markerp func-marker) (marker-buffer func-marker))
+      ;; It is uninstrumented, so instrument it.
+      (with-current-buffer (marker-buffer func-marker)
+	(goto-char func-marker)
+	(edebug-eval-top-level-form)
+        (list func)))
+     ((consp func-marker)
+      (message "%s is already instrumented." func)
+      (list func))
      (t
       (let ((loc (find-function-noselect func t)))
 	(unless (cdr loc)
diff --git a/lisp/emacs-lisp/generator.el b/lisp/emacs-lisp/generator.el
index f3597cc387..23af6143a7 100644
--- a/lisp/emacs-lisp/generator.el
+++ b/lisp/emacs-lisp/generator.el
@@ -681,7 +681,8 @@ iter-defun
 When called as a function, NAME returns an iterator value that
 encapsulates the state of a computation that produces a sequence
 of values.  Callers can retrieve each value using `iter-next'."
-  (declare (indent defun))
+  (declare (indent defun)
+           (debug (&define name lambda-list lambda-doc def-body)))
   (cl-assert lexical-binding)
   (let* ((parsed-body (macroexp-parse-body body))
          (declarations (car parsed-body))
@@ -693,7 +694,8 @@ iter-defun
 (defmacro iter-lambda (arglist &rest body)
   "Return a lambda generator.
 `iter-lambda' is to `iter-defun' as `lambda' is to `defun'."
-  (declare (indent defun))
+  (declare (indent defun)
+           (debug (&define lambda-list lambda-doc def-body)))
   (cl-assert lexical-binding)
   `(lambda ,arglist
      ,(cps-generate-evaluator body)))
diff --git a/lisp/emacs-lisp/pcase.el b/lisp/emacs-lisp/pcase.el
index c703cae445..36af88423c 100644
--- a/lisp/emacs-lisp/pcase.el
+++ b/lisp/emacs-lisp/pcase.el
@@ -226,7 +226,7 @@ pcase-lambda
 formal argument can be any pattern accepted by `pcase' (a mere
 variable name being but a special case of it)."
   (declare (doc-string 2) (indent defun)
-           (debug ((&rest pcase-PAT) body)))
+           (debug (&define (&rest pcase-PAT) lambda-doc def-body)))
   (let* ((bindings ())
          (parsed-body (macroexp-parse-body body))
          (args (mapcar (lambda (pat)
diff --git a/lisp/subr.el b/lisp/subr.el
index cf15ec287f..e0ef3e5885 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -110,8 +110,7 @@ lambda
 
 \(fn ARGS [DOCSTRING] [INTERACTIVE] BODY)"
   (declare (doc-string 2) (indent defun)
-           (debug (&define lambda-list
-                           [&optional stringp]
+           (debug (&define lambda-list lambda-doc
                            [&optional ("interactive" interactive)]
                            def-body)))
   ;; Note that this definition should not use backquotes; subr.el should not
-- 
2.14.1

bug#24773: 26.0.50; Error when using Edebug on a function that uses :documentation

[Gemini Lasswell]

Gemini Lasswell <gazally@runbox.com> writes:
> Here is a patch to add support for :documentation to the Edebug specs
> of the macros where it can be used (see bug#28557 for a list of those,
> and some related bugs).

Patch pushed to emacs-26.