bug#25461: Missing doc strings for "," and ",@".

bug#25461: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello, Emacs.

With any existing Emacs, when I do

    C-h f , <CR>
    
or

    C-h f ,@ <CR>

, I get the response

    [No match]

.  This is bad.  From a user's point of view, these key sequences should
bring up doc strings for "," and ",@", especially seeing how important
these symbols are, and how difficult they are to use for a lisper with
limited experience.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: Missing doc strings for "," and ",@".

[Glenn Morris]

Alan Mackenzie wrote:

> With any existing Emacs, when I do
>
>     C-h f , <CR>
>     
> or
>
>     C-h f ,@ <CR>
>
> , I get the response
>
>     [No match]

But they aren't functions, so I don't see why you want describe-function
to report on them, or indeed how that would work.

With this and 25462, it seems to me that what you want is a
(context-sensitive?) type of help that does not exist yet, which would
report on special syntax used within macros and such.

(Both , and ,@ are indexed in the elisp manual, BTW.)

bug#25461: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello, Glenn.

On Tue, Jan 17, 2017 at 02:34:16PM -0500, Glenn Morris wrote:
> Alan Mackenzie wrote:

> > With any existing Emacs, when I do

> >     C-h f , <CR>

> > or

> >     C-h f ,@ <CR>

> > , I get the response

> >     [No match]

> But they aren't functions, so I don't see why you want describe-function
> to report on them, or indeed how that would work.

I think I want some means of displaying "their doc strings".  They're much
more like functions than variables, so for lack of any third option, C-h
f seems the best way of doing this display.

I've hacked out a solution for this, involving defining defuns to hold
the doc strings.  I hope to post this patch on bug-gnu-emacs later on
this evening (European time).

> With this and 25462, it seems to me that what you want is a
> (context-sensitive?) type of help that does not exist yet, which would
> report on special syntax used within macros and such.

No, nothing so general.  pcase has introduced a lot of confusion (for me,
at any rate, so probably also for a lot of hackers who aren't Emacs
contributors), so I would like the doc strings to reduce this confusion
as far is as reasonable.

> (Both , and ,@ are indexed in the elisp manual, BTW.)

That's as it should be.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: Missing doc strings for "," and ",@".

[Andreas Schwab]

On Jan 17 2017, Alan Mackenzie <acm@muc.de> wrote:

> I think I want some means of displaying "their doc strings".  They're much
> more like functions than variables,

They are neither, just syntactic elements.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#25461: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello, Andreas.

On Tue, Jan 17, 2017 at 09:36:14PM +0100, Andreas Schwab wrote:
> On Jan 17 2017, Alan Mackenzie <acm@muc.de> wrote:

> > I think I want some means of displaying "their doc strings".  They're much
> > more like functions than variables,

> They are neither, just syntactic elements.

I don't expect the typical novice Elisp hacker cares about such nice
distinctions.  She can get a doc string for `, so why not one for , or
,@?

Anyway, I'm going to be providing a patch to do this.  We'll see whether
people like it or not.

> Andreas.

> -- 
> Andreas Schwab, schwab@linux-m68k.org
> GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
> "And now for something completely different."

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: Missing doc strings for "," and ",@".

[Andreas Schwab]

On Jan 17 2017, Alan Mackenzie <acm@muc.de> wrote:

> I don't expect the typical novice Elisp hacker cares about such nice
> distinctions.

A doc string is not for introduction of basics, a novice is expected to
read the manual for that.

> She can get a doc string for `, so why not one for , or ,@?

The paren doesn't have a doc string either.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#25461: Missing doc strings for "," and ",@".

[Michael Heerdegen]

Alan Mackenzie <acm@muc.de> writes:

> I don't expect the typical novice Elisp hacker cares about such nice
> distinctions.  She can get a doc string for `, so why not one for , or
> ,@?

' and #' also "don't have a docstring".  That ` "has a docstring" is
just coincidence - because the reader transforms

  `expr

to

  (` expr)

and the symbol ` has an associated symbol-function.

FWIW when I first saw `, , and ,@, it looked strange enough to me to
open the manual, because the syntax seemed so unusual that I believed
that I missed an essential part of knowledge about Elisp.  At least this
is what I think happened...

But I agree that some people might try C-h f on any of these, and we
would not all die if we would show something useful in this case.  We
could just say that C-f explains symbol functions and "reader macros"
and the thing would still be consistent.

OTOH, I think that saying anything about `pcase' there would be a bad
idea.


Michael.

bug#25461: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello, Michael.

On Wed, Jan 18, 2017 at 02:44:28AM +0100, Michael Heerdegen wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > I don't expect the typical novice Elisp hacker cares about such nice
> > distinctions.  She can get a doc string for `, so why not one for , or
> > ,@?

> ' and #' also "don't have a docstring".  That ` "has a docstring" is
> just coincidence - because the reader transforms

>   `expr

> to

>   (` expr)

> and the symbol ` has an associated symbol-function.

I'm sure that's true.  But ' is not really difficult enough to need a
doc string (IMAO).  Well, it didn't used to be, before it became
(ab)used by pcase.  #' could certainly do with one, I think.

But in my patch (which I expect to post within a few minutes of this
post), I have created a mechanism by which other reader macros can have
doc strings added.

> FWIW when I first saw `, , and ,@, it looked strange enough to me to
> open the manual, because the syntax seemed so unusual that I believed
> that I missed an essential part of knowledge about Elisp.  At least this
> is what I think happened...

Yes, but unless you're a genius, you probably had to look at the manual
several times before you internalised the meaning of , and ,@, with a
fair bit of practice in between.  During that practice, you might well
have found opening the manual to have been a hassle which you could have
avoided, had there been doc strings at the time.

> But I agree that some people might try C-h f on any of these, and we
> would not all die if we would show something useful in this case.  We
> could just say that C-f explains symbol functions and "reader macros"
> and the thing would still be consistent.

Or, not say anything at all - just do it.  C-h f after all does explain
named functions.  That it also explains a couple of reader macros as a
bonus hardly needs going on about.

> OTOH, I think that saying anything about `pcase' there would be a bad
> idea.

I disagree.  I was confused fairly badly for a long time when pcase
appeared (without announcement, IIRC) and purloined the reader macros
which, up till then, had had single definite functionalities.  I assume
other people were and will be confused by pcase usages, so it will do no
harm to draw people's attention towards them.  Anyhow, I've included a
sentence about pcase at the end of the proposed doc string for ,.  I
don't think pcase uses ,@, or at least I haven't seen any such use of it.

> Michael.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello, Emacs.

On Mon, Jan 16, 2017 at 09:24:02PM +0000, GNU bug Tracking System wrote:
> Thank you for filing a new bug report with debbugs.gnu.org.

[ .... ]

Thanks for the comments, so far.  Here is a patch which gives , and ,@
doc strings.  It also creates a mechanism by which other reader macros
(such as ' and #') can be given doc strings later.

I'll just point out one thing I'm proposing changing.  That is, to use
princ rather than prin1 to print the name of the function.  This causes,
for example, backtick to be printed as

    `

rather than the

    \`

we get at the moment.  This might cause problems for function names
which include backslashes or unprintable characters such as \n.

Are there objections to me installing the patch in master?



diff --git a/lisp/emacs-lisp/backquote.el b/lisp/emacs-lisp/backquote.el
index 94c561c..86ca010 100644
--- a/lisp/emacs-lisp/backquote.el
+++ b/lisp/emacs-lisp/backquote.el
@@ -247,4 +247,33 @@ backquote-listify
 	tail))
      (t (cons 'list heads)))))
 
+\f
+;; Give `,' and `,@' documentation strings which can be examined by C-h f.
+(put '\, 'function-documentation
+     "`,' signals that the next form should be evaluated and inserted.
+It can occur only in `\\=`' constructs.
+
+For example:
+
+b              => (ba bb bc)		; assume b has this value
+\\=`(a ,b c)      => (a (ba bb bc) c)	; insert the value of b
+
+See also `\\=`' and `,@'.
+
+Note that `,' is also used by the macro `pcase' and the like,
+with radically different semantics.")
+(put '\, 'reader-macro t)
+
+(put '\,@ 'function-documentation
+     ",@ signals that the next form should be evaluated and spliced in.
+It can occur only in `\\=`' constructs.
+
+For example:
+
+b              => (ba bb bc)		; assume b has this value
+\\=`(a ,@b c)     => (a ba bb bc c)	; splice in the value of b
+
+See also `\\=`' and `,'.")
+(put '\,@ 'reader-macro t)
+
 ;;; backquote.el ends here
diff --git a/lisp/help-fns.el b/lisp/help-fns.el
index fa16fa0..069dc3c 100644
--- a/lisp/help-fns.el
+++ b/lisp/help-fns.el
@@ -115,13 +115,15 @@ describe-function
                 (if fn
                     (format "Describe function (default %s): " fn)
                   "Describe function: ")
-                #'help--symbol-completion-table #'fboundp t nil nil
+                #'help--symbol-completion-table
+                (lambda (f) (or (fboundp f) (get f 'function-documentation)))
+                t nil nil
                 (and fn (symbol-name fn)))))
      (unless (equal val "")
        (setq fn (intern val)))
      (unless (and fn (symbolp fn))
        (user-error "You didn't specify a function symbol"))
-     (unless (fboundp fn)
+     (unless (or (fboundp fn) (get fn 'function-documentation))
        (user-error "Symbol's function definition is void: %s" fn))
      (list fn)))
 
@@ -144,7 +146,7 @@ describe-function
 
     (save-excursion
       (with-help-window (help-buffer)
-        (prin1 function)
+        (princ function)
         ;; Use " is " instead of a colon so that
         ;; it is easier to get out the function name using forward-sexp.
         (princ " is ")
@@ -469,7 +471,8 @@ help-fns--signature
         (let ((fill-begin (point))
               (high-usage (car high))
               (high-doc (cdr high)))
-          (insert high-usage "\n")
+          (unless (get function 'reader-macro)
+            (insert high-usage "\n"))
           (fill-region fill-begin (point))
           high-doc)))))
 
@@ -565,18 +568,23 @@ describe-function-1
 	  (or (and advised
                    (advice--cd*r (advice--symbol-function function)))
 	      function))
-	 ;; Get the real definition.
-	 (def (if (symbolp real-function)
-		  (or (symbol-function real-function)
-		      (signal 'void-function (list real-function)))
-		real-function))
-	 (aliased (or (symbolp def)
-		      ;; Advised & aliased function.
-		      (and advised (symbolp real-function)
-			   (not (eq 'autoload (car-safe def))))
-                      (and (subrp def)
-                           (not (string= (subr-name def)
-                                         (symbol-name function))))))
+	 ;; Get the real definition, if any.
+	 (def (cond ((and (symbolp real-function)
+                          (symbol-function real-function)))
+                    ((and (symbolp real-function)
+                          (get real-function 'function-documentation))
+                     nil)
+                    ((symbolp real-function)
+                     (signal 'void-function (list real-function)))
+                    (t real-function)))
+	 (aliased (and def
+                       (or (symbolp def)
+                           ;; Advised & aliased function.
+                           (and advised (symbolp real-function)
+                                (not (eq 'autoload (car-safe def))))
+                           (and (subrp def)
+                                (not (string= (subr-name def)
+                                              (symbol-name function)))))))
 	 (real-def (cond
                     ((and aliased (not (subrp def)))
                      (let ((f real-function))
@@ -605,6 +613,8 @@ describe-function-1
     ;; Print what kind of function-like object FUNCTION is.
     (princ (cond ((or (stringp def) (vectorp def))
 		  "a keyboard macro")
+		 ((get function 'reader-macro)
+                  "a reader macro")
 		 ;; Aliases are Lisp functions, so we need to check
 		 ;; aliases before functions.
 		 (aliased


-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[npostavs]

Alan Mackenzie <acm@muc.de> writes:

>
> I'll just point out one thing I'm proposing changing.  That is, to use
> princ rather than prin1 to print the name of the function.  This causes,
> for example, backtick to be printed as
>
>     `
>
> rather than the
>
>     \`
>
> we get at the moment.  This might cause problems for function names
> which include backslashes or unprintable characters such as \n.

Perhaps only do this for symbols which have `reader-macro' set in their
plist?

> +;; Give `,' and `,@' documentation strings which can be examined by C-h f.
> +(put '\, 'function-documentation
> +     "`,' signals that the next form should be evaluated and inserted.

This conflates the reader macro with its particular use by the backquote
macro.  I think it's better to explain only the reader macro here, and
link to ` and pcase for more details.

> +It can occur only in `\\=`' constructs.

This isn't true.

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello, Noam.

On Wed, Jan 18, 2017 at 07:16:34PM -0500, npostavs@users.sourceforge.net wrote:
> Alan Mackenzie <acm@muc.de> writes:


> > I'll just point out one thing I'm proposing changing.  That is, to use
> > princ rather than prin1 to print the name of the function.  This causes,
> > for example, backtick to be printed as

> >     `

> > rather than the

> >     \`

> > we get at the moment.  This might cause problems for function names
> > which include backslashes or unprintable characters such as \n.

> Perhaps only do this for symbols which have `reader-macro' set in their
> plist?

That sounds like a good idea.

> > +;; Give `,' and `,@' documentation strings which can be examined by C-h f.
> > +(put '\, 'function-documentation
> > +     "`,' signals that the next form should be evaluated and inserted.

> This conflates the reader macro with its particular use by the backquote
> macro.  

The use by backquote isn't "a particular" use, as though there were
several equally valid uses.  This use is the prime one, and up until
fairly recently the only one.  Other uses are subsidiary (and I make
reference to the other one lower down).

> I think it's better to explain only the reader macro here, and
> link to ` and pcase for more details.

In other words, omit the semantics from the doc string.  I don't think
that's a good idea: "This macro transforms `foo' into ....., but you'll
have to look up ..... to see what it actually does.".  We don't do that
for macros like `when' and `unless', or any others I can think of, so
why do it for ,?

> > +It can occur only in `\\=`' constructs.

> This isn't true.

In what sense is it not true?  I've never seen a comma used otherwise in
a lisp program.  What misunderstanding could it cause in a (relatively
new) Lisp hacker reading it?  I realise that it is possible to write
(defun \, (...) ...), but that is surely of interest only to people
testing Emacs.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[Noam Postavsky]

On Thu, Jan 19, 2017 at 12:37 PM, Alan Mackenzie <acm@muc.de> wrote:
>
>> > +;; Give `,' and `,@' documentation strings which can be examined by C-h f.
>> > +(put '\, 'function-documentation
>> > +     "`,' signals that the next form should be evaluated and inserted.
>
>> This conflates the reader macro with its particular use by the backquote
>> macro.
>
> The use by backquote isn't "a particular" use, as though there were
> several equally valid uses.  This use is the prime one, and up until
> fairly recently the only one.  Other uses are subsidiary (and I make
> reference to the other one lower down).

I agree backquote is the primary use, it's reasonable to emphasize that.

>
>> I think it's better to explain only the reader macro here, and
>> link to ` and pcase for more details.
>
> In other words, omit the semantics from the doc string.  I don't think
> that's a good idea: "This macro transforms `foo' into ....., but you'll
> have to look up ..... to see what it actually does.".  We don't do that
> for macros like `when' and `unless', or any others I can think of, so
> why do it for ,?

Because "," isn't a macro, it doesn't have any semantics. It's
misleading to suggest that it does.

>
>> > +It can occur only in `\\=`' constructs.
>
>> This isn't true.
>
> In what sense is it not true?  I've never seen a comma used otherwise in
> a lisp program.  What misunderstanding could it cause in a (relatively
> new) Lisp hacker reading it?

In the sense that it's possible to write a macro that could treat the
"," differently. I don't know of any that currently do*, but that's
just convention and should be documented as such.

* `use-package' once handled unmatched "," as signifying eager
evaluation, but it no longer does that.

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello, Noam.

On Thu, Jan 19, 2017 at 01:10:24PM -0500, Noam Postavsky wrote:
> On Thu, Jan 19, 2017 at 12:37 PM, Alan Mackenzie <acm@muc.de> wrote:

> >> > +;; Give `,' and `,@' documentation strings which can be examined by C-h f.
> >> > +(put '\, 'function-documentation
> >> > +     "`,' signals that the next form should be evaluated and inserted.

[ .... ]

> I agree backquote is the primary use, it's reasonable to emphasize that.

> Because "," isn't a macro, it doesn't have any semantics. It's
                                        ^^^^
> misleading to suggest that it does.

We could degenerate into a long silly discussion about what "have" means.
, evaluates the form it precedes, and inserts it into the containing
list.  That's semantics enough for me.

> >> > +It can occur only in `\\=`' constructs.

> >> This isn't true.

> > In what sense is it not true?  I've never seen a comma used otherwise in
> > a lisp program.  What misunderstanding could it cause in a (relatively
> > new) Lisp hacker reading it?

> In the sense that it's possible to write a macro that could treat the
> "," differently. I don't know of any that currently do*, but that's
> just convention and should be documented as such.

OK, I've toned down that part of the doc string.  (See below.)

> * `use-package' once handled unmatched "," as signifying eager
> evaluation, but it no longer does that.

How about this, then:

        "`,' signals that the next form should be evaluated and inserted.
    It occurs in `\\=`' constructs.

    For example:

    b              => (ba bb bc)            ; assume b has this value
    \\=`(a ,b c)      => (a (ba bb bc) c)   ; insert the value of b

    See also `\\=`' and `,@'.

    (Note that ``' constructs (including `,'s) sometimes have different
    semantics.  This occurs, for example, with the macro `pcase' and other
    macros with similar names.)

?

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[Michael Heerdegen]

Alan Mackenzie <acm@muc.de> writes:

> > I agree backquote is the primary use, it's reasonable to emphasize
> > that.
>
> > Because "," isn't a macro, it doesn't have any semantics. It's
>                                         ^^^^
> > misleading to suggest that it does.
>
> We could degenerate into a long silly discussion about what "have" means.
> , evaluates the form it precedes, and inserts it into the containing
> list.  That's semantics enough for me.

It has a semantics - in the context of backquote.  It's the backquote
macro whose (expanded) code "inserts" something.  "," may have any
arbitrary semantics in another context, and per se, without context, it
has no fixed semantics.

>         "`,' signals that the next form should be evaluated and inserted.
>     It occurs in `\\=`' constructs.

I still find that wording misleading.


Regards,

Michael.

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[Michael Heerdegen]

Hi Alan,

> diff --git a/lisp/emacs-lisp/backquote.el b/lisp/emacs-lisp/backquote.el
> index 94c561c..86ca010 100644
> --- a/lisp/emacs-lisp/backquote.el
> +++ b/lisp/emacs-lisp/backquote.el
> @@ -247,4 +247,33 @@ backquote-listify
>  	tail))
>       (t (cons 'list heads)))))
>  
> +\f
> +;; Give `,' and `,@' documentation strings which can be examined by C-h f.
> +(put '\, 'function-documentation
> +     "`,' signals that the next form should be evaluated and inserted.
> +It can occur only in `\\=`' constructs.
> +
> +For example:
> +
> +b              => (ba bb bc)		; assume b has this value
> +\\=`(a ,b c)      => (a (ba bb bc) c)	; insert the value of b
> +
> +See also `\\=`' and `,@'.

I don't think this makes it easier for people to understand things.
This suggests that "," has some kind of meaning per se, that it is a
macro-like thing, etc, and that pcase somehow redefines it.

Of course there is a logic behind pcase's usage of ` and ,.  The
usage of these suggests a mental model for their "meaning".  But we
should not describe our mental models in docstrings.  That's only useful
for people sharing the same model.

The bindings of variables depend on context.  The semantics of functions
are fluent (advice, local functions).  And also the "meaning" of the
sexps created by ` and , depend on the context - in Lisp, the "meaning"
of all sexps depends on context.  (foo bar) can be a function call or a
part of a let variable binding or a list or a pcase pattern.  It's not
different for sexps involving `.

So I think we maximally should describe what the reader does with `
etc., so that people know what to search for in the manual or remember
what they already had learned.


+		 ((get function 'reader-macro)
+                  "a reader macro")

We don't have reader macros in Emacs.  If we had them, we could
implement ', ` etc. as reader macros.  But using this term in H-f is
confusing, because it is not used in the manual.



Regards,

Michael

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello, Michael.

On Thu, Jan 19, 2017 at 03:23:32AM +0100, Michael Heerdegen wrote:
> Hi Alan,

> > diff --git a/lisp/emacs-lisp/backquote.el b/lisp/emacs-lisp/backquote.el
> > index 94c561c..86ca010 100644
> > --- a/lisp/emacs-lisp/backquote.el
> > +++ b/lisp/emacs-lisp/backquote.el
> > @@ -247,4 +247,33 @@ backquote-listify
> >  	tail))
> >       (t (cons 'list heads)))))

> > +\f
> > +;; Give `,' and `,@' documentation strings which can be examined by C-h f.
> > +(put '\, 'function-documentation
> > +     "`,' signals that the next form should be evaluated and inserted.
> > +It can occur only in `\\=`' constructs.
> > +
> > +For example:
> > +
> > +b              => (ba bb bc)		; assume b has this value
> > +\\=`(a ,b c)      => (a (ba bb bc) c)	; insert the value of b
> > +
> > +See also `\\=`' and `,@'.

> I don't think this makes it easier for people to understand things.
> This suggests that "," has some kind of meaning per se, that it is a
> macro-like thing, etc, and that pcase somehow redefines it.

Yes.  All these things are true, aren't they?

> Of course there is a logic behind pcase's usage of ` and ,.  The
> usage of these suggests a mental model for their "meaning".  But we
> should not describe our mental models in docstrings.  That's only useful
> for people sharing the same model.

I'm not sure I'm following you here.  A high level description of a
function necessarily involves a mental model.  

> The bindings of variables depend on context.  The semantics of functions
> are fluent (advice, local functions).  And also the "meaning" of the
> sexps created by ` and , depend on the context - in Lisp, the "meaning"
> of all sexps depends on context.  (foo bar) can be a function call or a
> part of a let variable binding or a list or a pcase pattern.  It's not
> different for sexps involving `.

` has a specific meaning, and has had a high quality doc string for
ever.  , likewise has a specific meaning, but doesn't yet have a doc
string.

In Emacs Lisp, functions generally have well defined context-free
semantics, though there are exceptions.

> So I think we maximally should describe what the reader does with `
> etc., so that people know what to search for in the manual or remember
> what they already had learned.

We don't do that for other functions.  A function's doc string should be
a crisp summary of what a function _does_.  A doc string which directs
people to a manual, or is so confusing or unspecific that the reader is
forced to open a manual, is a failed doc string.

, has a definite precise function (disregarding its use in pcase for
now).  Have you any specific suggestions on how to improve my wording of
its doc string?

> +		 ((get function 'reader-macro)
> +                  "a reader macro")

> We don't have reader macros in Emacs.

The reader performs macro-like actions.  What is the correct alternative
term for what the reader does with ', `, ,, #', etc.?

> If we had them, we could implement ', ` etc. as reader macros.  But
> using this term in H-f is confusing, because it is not used in the
> manual.

What would be a less confusing alternative?

> Regards,

> Michael

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[Michael Heerdegen]

Alan Mackenzie <acm@muc.de> writes:

> > > +;; Give `,' and `,@' documentation strings which can be examined by C-h f.
> > > +(put '\, 'function-documentation
> > > +     "`,' signals that the next form should be evaluated and inserted.
> > > +It can occur only in `\\=`' constructs.
> > > +
> > > +For example:
> > > +
> > > +b              => (ba bb bc)		; assume b has this value
> > > +\\=`(a ,b c)      => (a (ba bb bc) c)	; insert the value of b
> > > +
> > > +See also `\\=`' and `,@'.
>
> > I don't think this makes it easier for people to understand things.
> > This suggests that "," has some kind of meaning per se, that it is a
> > macro-like thing, etc, and that pcase somehow redefines it.
>
> Yes.  All these things are true, aren't they?

That `,' "signals" something is too vague to be true or false.  As an
explanation of things, I don't find it good, and in the generality that
this sentence is speaking, I would say it's wrong.

That "," can only appear inside "`" is simply wrong.  That there are
currently no counterexamples in our sources doesn't make a wrong
sentence right.  And I don't think I'm exaggerating here.  Elisp is a
Lisp, and Lisp is a programmable programming language.  We want users to
extend Emacs, and we want developers to extend (E)Lisp, because that is
a big strength of the language.  Formulating rules that are not true but
restrict our usage of the language is bad.

It is also not the case that `pcase' redefines "`".  And yes, I've seen
uses of "," outside of "`" in Lisp.  And it's perfectly fine to use ","
for anything anyone wants - like any other symbol.

You doesn't seem to be confused by the fact that `rx' "(re-)uses" `and',
`or' etc.  What makes "`" different?  The fact that it is a "reader
macro"?

> > Of course there is a logic behind pcase's usage of ` and ,.  The
> > usage of these suggests a mental model for their "meaning".  But we
> > should not describe our mental models in docstrings.  That's only
> > useful for people sharing the same model.
>
> I'm not sure I'm following you here.  A high level description of a
> function necessarily involves a mental model.

Maybe.  But you are describing properties of your mental model instead
of properties of the language.  Your mental model led you to confusion
and wrong conclusions.  It's not optimal to describe these things.  It
will confuse others.

> ` has a specific meaning, and has had a high quality doc string for
> ever.

Because it has a `symbol-function'.

> , likewise has a specific meaning,

That's one part that led you to confusion.

> but doesn't yet have a doc string.

... because it is undefined.

> > So I think we maximally should describe what the reader does with `
> > etc., so that people know what to search for in the manual or remember
> > what they already had learned.
>
> We don't do that for other functions.  A function's doc string should be
> a crisp summary of what a function _does_.  A doc string which directs
> people to a manual, or is so confusing or unspecific that the reader is
> forced to open a manual, is a failed doc string.

But that's exactly the point: "," is not a function or a macro.  So how
it is used is arbitrary.  It's traditionally used by "`", but it's not
restricted to that.  There may be even more use cases in future Elisp.
And that is good.

> , has a definite precise function (disregarding its use in pcase for
> now).  Have you any specific suggestions on how to improve my wording of
> its doc string?
>
> > +		 ((get function 'reader-macro)
> > +                  "a reader macro")
>
> > We don't have reader macros in Emacs.
>
> The reader performs macro-like actions.  What is the correct alternative
> term for what the reader does with ', `, ,, #', etc.?

The manual seems to prefer the term "reader syntax".  I think we could
use this term.

You can say that the reader expands `X to (` X), and that "`" is a
defined macro.  You can say that the reader expands ,X to (\, X), and
that a frequent usage is in backquote forms.  We should not say that ","
"does" something, because this makes no sense and leads a person reading
this to false assumptions.


Regards,

Michael.

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello, Michael.

On Fri, Jan 20, 2017 at 00:12:18 UTC, Michael Heerdegen wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > > > +;; Give `,' and `,@' documentation strings which can be examined by C-h f.
> > > > +(put '\, 'function-documentation
> > > > +     "`,' signals that the next form should be evaluated and inserted.
> > > > +It can occur only in `\\=`' constructs.
> > > > +
> > > > +For example:
> > > > +
> > > > +b              => (ba bb bc)		; assume b has this value
> > > > +\\=`(a ,b c)      => (a (ba bb bc) c)	; insert the value of b
> > > > +
> > > > +See also `\\=`' and `,@'.

> > > I don't think this makes it easier for people to understand things.
> > > This suggests that "," has some kind of meaning per se, that it is a
> > > macro-like thing, etc, and that pcase somehow redefines it.

> > Yes.  All these things are true, aren't they?

> That `,' "signals" something is too vague to be true or false.  As an
> explanation of things, I don't find it good, and in the generality that
> this sentence is speaking, I would say it's wrong.

OK, I'll accept that.  I can't see where it's wrong, though.  I've
changed it to say "causes" rather than "signal".  (See below.)

> That "," can only appear inside "`" is simply wrong.

That's poor wording on my part.  What I really meant was that for , to
have the effect described, it must be in a ` construct.  I've amended
this bit too (see below).

[ .... ]

> It is also not the case that `pcase' redefines "`".

pcase causes ` to be expanded by the macro \`--pcase-macroexpander in
place of the macro \`.  What is that if it is not pcase redefining `?

> And yes, I've seen uses of "," outside of "`" in Lisp.  And it's
> perfectly fine to use "," for anything anyone wants - like any other
> symbol.

I disagree with you, here.  Using standard Elisp symbols to mean
something different is a recipe for confusion.  I don't think it is in
any way fine to use `if', `and', `car', .... for non standard purposes.
I also don't think it's OK to do the same with ` or ,.

> You don't seem to be confused by the fact that `rx' "(re-)uses" `and',
> `or' etc.  What makes "`" different?  The fact that it is a "reader
> macro"?

I wasn't aware of rx.  I don't think it was good to reuse `and' the way
it does, but (subjectively) I don't think it will have caused all that
much confusion.

> > > Of course there is a logic behind pcase's usage of ` and ,.  The
> > > usage of these suggests a mental model for their "meaning".  But we
> > > should not describe our mental models in docstrings.  That's only
> > > useful for people sharing the same model.

> > I'm not sure I'm following you here.  A high level description of a
> > function necessarily involves a mental model.

> Maybe.  But you are describing properties of your mental model instead
> of properties of the language.  Your mental model led you to confusion
> and wrong conclusions.

I disagree with you, here.  I'm describing the simple straighforward
normal effect of ,.  I don't understand why you're making such a big
deal out of it.

> It's not optimal to describe these things.  It will confuse others.

I disagree with you here, too.  , and ,@ have a simply stated meaning in
the typical case, far simpler than you're making it out to be.

> > ` has a specific meaning, and has had a high quality doc string for
> > ever.

> Because it has a `symbol-function'.

> > , likewise has a specific meaning,

> That's one part that led you to confusion.

> > but doesn't yet have a doc string.

> ... because it is undefined.

It is defined.  I intend that definition to be in a doc string.

> > > So I think we maximally should describe what the reader does with `
> > > etc., so that people know what to search for in the manual or remember
> > > what they already had learned.

> > We don't do that for other functions.  A function's doc string should be
> > a crisp summary of what a function _does_.  A doc string which directs
> > people to a manual, or is so confusing or unspecific that the reader is
> > forced to open a manual, is a failed doc string.

> But that's exactly the point: "," is not a function or a macro.

No, but it needs a doc string.

> So how it is used is arbitrary.

If that were the case, Emacs wouldn't even build.  ,'s use is no more
arbitrary than any other symbol in Lisp.

> It's traditionally used by "`", but it's not restricted to that.
> There may be even more use cases in future Elisp.  And that is good.

I disagree with that last sentiment.

> > , has a definite precise function (disregarding its use in pcase for
> > now).  Have you any specific suggestions on how to improve my wording of
> > its doc string?

> > > +		 ((get function 'reader-macro)
> > > +                  "a reader macro")

> > > We don't have reader macros in Emacs.

> > The reader performs macro-like actions.  What is the correct alternative
> > term for what the reader does with ', `, ,, #', etc.?

> The manual seems to prefer the term "reader syntax".  I think we could
> use this term.

No, that won't do.  , has semantics as well as syntax.

> You can say that the reader expands `X to (` X), and that "`" is a
> defined macro.  You can say that the reader expands ,X to (\, X), and
> that a frequent usage is in backquote forms.

That is describing the internal mechanism of , rather than the result.
It would be utterly useless and confusing for the people who need to
consult the meaning of , and ,@, namely new Elisp hackers.

> We should not say that "," "does" something, because this makes no
> sense and leads a person reading this to false assumptions.

It will enable them to understand code they are reading, and to start
writing ` constructs themselves.

Here is the latest proposed version of the doc string for ,,
incorporating some of the comments you've made.


        "`,' causes the next form to be evaluated and inserted.
    It occurs in `\\=`' constructs.

    For example:

    b              => (ba bb bc)            ; assume b has this value
    \\=`(a ,b c)      => (a (ba bb bc) c)   ; insert the value of b

    See also `\\=`' and `,@'.

    (Note that ``' constructs (including `,'s) sometimes have different
    semantics.  This occurs, for example, with the macro `pcase' and other
    macros with similar names.")
 

> Regards,

> Michael.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[Michael Heerdegen]

Hello Alan,

> pcase causes ` to be expanded by the macro \`--pcase-macroexpander in
> place of the macro \`.  What is that if it is not pcase redefining `?

pcase patterns are not evaluated normally.  When ` is used in patterns,
the definition of ` is not consulted.  Just like in

(let ((l '(\` 1 2)))
  (when (eq (car l) '\`)
    (apply #'+ (cdr l))))
==> 3

Is that code redefining "`"?

You mean something different with the term "definition": a complete
summarisation of the semantics of something in any context.  Something
like that doesn't exist in a programmable extensible language like Lisp.


> I disagree with you, here.  I'm describing the simple straighforward
> normal effect of ,.  I don't understand why you're making such a big
> deal out of it.

I don't want that we change the docs to the worse.


> > ... because it is undefined.
>
> It is defined.  I intend that definition to be in a doc string.

It is implicitly given a meaning by the implementation of "`".  That's
why it's documented in the documentation of "`".  For the same reason
that there is no separate documentation for :group just because it has a
meaning in defcustom, or there is no mentioning in the docstring that
the symbol `error' has a different meaning as car of a list that is an
error handler in condition-case, or that car has a different meaning in
(setf (car something) ...) or...

All these examples appear in symbolic expressions (i.e. lists) that are
not evaluated normally.  Like in the tiny example above.  If we try to
mention what a symbol potentially could mean in any sexp that is not
evaluated normally in this symbol's docstring, our documentation would
become very messy, because lists that are not evaluated normally are
very common in Lisp.  That's why we normally collect this information in
the documentation of the functions/macros that implement this meaning.
This is not a problem because it's easy to look at the context and
consult the documentation of the surrounding form.


> > So how it is used is arbitrary.
>
> If that were the case, Emacs wouldn't even build.

The introduction of pcase didn't cause Emacs not to build.  Simply
because it didn't change the definition of `.


> > We should not say that "," "does" something, because this makes no
> > sense and leads a person reading this to false assumptions.
>
> It will enable them to understand code they are reading, and to start
> writing ` constructs themselves.

But it won't really enable them to understand how it works.


> Here is the latest proposed version of the doc string for ,,
> incorporating some of the comments you've made.
>
>
>         "`,' causes the next form to be evaluated and inserted.
>     It occurs in `\\=`' constructs.
>
>     For example:
>
>     b              => (ba bb bc)            ; assume b has this value
>     \\=`(a ,b c)      => (a (ba bb bc) c)   ; insert the value of b
>
>     See also `\\=`' and `,@'.
>
>     (Note that ``' constructs (including `,'s) sometimes have different
>     semantics.  This occurs, for example, with the macro `pcase' and other
>     macros with similar names.")

I still think it's an error to describe a global semantics that as such
isn't existent.

I would rather prefer a wording (sorry if it's poor, my English is not
so super...) like

  ", before a symbolic expression X forms a read syntax that the Lisp
  reader expands like ,X -> (, X).  Such symbolic expressions have a
  special meaning in backquote forms (see ``') and in `pcase' patterns -
  see there."

I think it would be ok to append your backquote example above.


Regards,

Michael.

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello, Michael.

On Sat, Jan 21, 2017 at 03:26:06 +0100, Michael Heerdegen wrote:
> Hello Alan,

[ .... ]

> I don't want that we change the docs to the worse.

I somehow don't think you'll be happy with any modifications I make to
the proposed doc strings which leaves their basic structure unchanged.

How about starting on a different tack.  Would you accept , and ,@
having doc strings, with the entirety of these being:

[for ,]:
    See ``' (and also `pcase') for the details of `,'.

[for ,@]:
    See ``' for the details of `,@'.

?

[ .... ]

> It [ , ] is implicitly given a meaning by the implementation of "`".
> That's why it's documented in the documentation of "`".  For the same
> reason that there is no separate documentation for :group just because
> it has a meaning in defcustom, or there is no mentioning in the
> docstring that the symbol `error' has a different meaning as car of a
> list that is an error handler in condition-case, or that car has a
> different meaning in (setf (car something) ...) or...

> All these examples appear in symbolic expressions (i.e. lists) that are
> not evaluated normally.  Like in the tiny example above.  If we try to
> mention what a symbol potentially could mean in any sexp that is not
> evaluated normally in this symbol's docstring, our documentation would
> become very messy, because lists that are not evaluated normally are
> very common in Lisp.  That's why we normally collect this information in
> the documentation of the functions/macros that implement this meaning.
> This is not a problem because it's easy to look at the context and
> consult the documentation of the surrounding form.

, and ,@ are different: it is not obvious to the unexperienced what
"sexp" they belong to in something like

    `(if ,cond (progn ,@body))

.  How is a beginner supposed to connect up the ,@ in that example with
the ` which doesn't directly enclose or otherwise connect with it?

[ .... ]

(I'll address your points that I've snipped if they become relevant
again.)

> Regards,

> Michael.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: [Patch #2]: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello again, Michael.

On Sat, Jan 21, 2017 at 15:56:51 +0000, Alan Mackenzie wrote:
> Hello, Michael.

> On Sat, Jan 21, 2017 at 03:26:06 +0100, Michael Heerdegen wrote:

[ .... ]

> How about starting on a different tack.  Would you accept , and ,@
> having doc strings, with the entirety of these being:

> [for ,]:
>     See ``' (and also `pcase') for the details of `,'.

> [for ,@]:
>     See ``' for the details of `,@'.

> ?


I've hacked all this together.  What used to be called a "reader macro"
is now a "reader construct".  I've made the ``' a clickable link, too.

Is there anything else you think needs changing?



diff --git a/lisp/emacs-lisp/backquote.el b/lisp/emacs-lisp/backquote.el
index 94c561c..acc3fa4 100644
--- a/lisp/emacs-lisp/backquote.el
+++ b/lisp/emacs-lisp/backquote.el
@@ -247,4 +247,14 @@ backquote-listify
 	tail))
      (t (cons 'list heads)))))
 
+\f
+;; Give `,' and `,@' documentation strings which can be examined by C-h f.
+(put '\, 'function-documentation
+     "See `\\=`' (also `pcase') for the details of `,'.")
+(put '\, 'reader-construct t)
+
+(put '\,@ 'function-documentation
+     "See `\\=`' or `,' for the details of `,@'.")
+(put '\,@ 'reader-construct t)
+
 ;;; backquote.el ends here
diff --git a/lisp/help-fns.el b/lisp/help-fns.el
index fa16fa0..edbcd90 100644
--- a/lisp/help-fns.el
+++ b/lisp/help-fns.el
@@ -115,13 +115,15 @@ describe-function
                 (if fn
                     (format "Describe function (default %s): " fn)
                   "Describe function: ")
-                #'help--symbol-completion-table #'fboundp t nil nil
+                #'help--symbol-completion-table
+                (lambda (f) (or (fboundp f) (get f 'function-documentation)))
+                t nil nil
                 (and fn (symbol-name fn)))))
      (unless (equal val "")
        (setq fn (intern val)))
      (unless (and fn (symbolp fn))
        (user-error "You didn't specify a function symbol"))
-     (unless (fboundp fn)
+     (unless (or (fboundp fn) (get fn 'function-documentation))
        (user-error "Symbol's function definition is void: %s" fn))
      (list fn)))
 
@@ -144,7 +146,9 @@ describe-function
 
     (save-excursion
       (with-help-window (help-buffer)
-        (prin1 function)
+        (if (get function 'reader-construct)
+            (princ function)
+          (prin1 function))
         ;; Use " is " instead of a colon so that
         ;; it is easier to get out the function name using forward-sexp.
         (princ " is ")
@@ -469,7 +473,8 @@ help-fns--signature
         (let ((fill-begin (point))
               (high-usage (car high))
               (high-doc (cdr high)))
-          (insert high-usage "\n")
+          (unless (get function 'reader-construct)
+            (insert high-usage "\n"))
           (fill-region fill-begin (point))
           high-doc)))))
 
@@ -565,18 +570,21 @@ describe-function-1
 	  (or (and advised
                    (advice--cd*r (advice--symbol-function function)))
 	      function))
-	 ;; Get the real definition.
+	 ;; Get the real definition, if any.
 	 (def (if (symbolp real-function)
-		  (or (symbol-function real-function)
-		      (signal 'void-function (list real-function)))
+                  (cond ((symbol-function real-function))
+                        ((get real-function 'function-documentation)
+                         nil)
+                        (t (signal 'void-function (list real-function))))
 		real-function))
-	 (aliased (or (symbolp def)
-		      ;; Advised & aliased function.
-		      (and advised (symbolp real-function)
-			   (not (eq 'autoload (car-safe def))))
-                      (and (subrp def)
-                           (not (string= (subr-name def)
-                                         (symbol-name function))))))
+	 (aliased (and def
+                       (or (symbolp def)
+                           ;; Advised & aliased function.
+                           (and advised (symbolp real-function)
+                                (not (eq 'autoload (car-safe def))))
+                           (and (subrp def)
+                                (not (string= (subr-name def)
+                                              (symbol-name function)))))))
 	 (real-def (cond
                     ((and aliased (not (subrp def)))
                      (let ((f real-function))
@@ -605,6 +613,8 @@ describe-function-1
     ;; Print what kind of function-like object FUNCTION is.
     (princ (cond ((or (stringp def) (vectorp def))
 		  "a keyboard macro")
+		 ((get function 'reader-construct)
+                  "a reader construct")
 		 ;; Aliases are Lisp functions, so we need to check
 		 ;; aliases before functions.
 		 (aliased
diff --git a/lisp/help-mode.el b/lisp/help-mode.el
index a8d7294..3fb793e 100644
--- a/lisp/help-mode.el
+++ b/lisp/help-mode.el
@@ -328,7 +328,7 @@ help-xref-symbol-regexp
 		    "\\(source \\(?:code \\)?\\(?:of\\|for\\)\\)\\)"
 		    "[ \t\n]+\\)?"
 		    ;; Note starting with word-syntax character:
-		    "['`‘]\\(\\sw\\(\\sw\\|\\s_\\)+\\)['’]"))
+		    "['`‘]\\(\\sw\\(\\sw\\|\\s_\\)+\\|`\\)['’]"))
   "Regexp matching doc string references to symbols.
 
 The words preceding the quoted symbol can be used in doc strings to


-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: [Patch #2]: Missing doc strings for "," and ",@".

[Michael Heerdegen]

Alan Mackenzie <acm@muc.de> writes:

> I've hacked all this together.  What used to be called a "reader
> macro" is now a "reader construct".

Well, I liked my suggested wording more, but that's not a big surprise I
guess.  I wonder what others think?


> diff --git a/lisp/emacs-lisp/backquote.el b/lisp/emacs-lisp/backquote.el
> [...]

I see that you already installed you suggested patch.  After a short
test it seems to work as intended (FWIW, it doesn't work with Helm, I
guess because Helm doesn't consult a completion table but reimplements
the whole C-h f thing by itself).


Regards,

Michael.

bug#25461: [Patch #2]: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello, Michael.

On Tue, Jan 24, 2017 at 02:58:54 +0100, Michael Heerdegen wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > I've hacked all this together.  What used to be called a "reader
> > macro" is now a "reader construct".

> Well, I liked my suggested wording more, but that's not a big surprise I
> guess.  I wonder what others think?

As I said to Noam, I don't thing "read syntax" is quite right.

> > diff --git a/lisp/emacs-lisp/backquote.el b/lisp/emacs-lisp/backquote.el
> > [...]

> I see that you already installed you suggested patch.  After a short
> test it seems to work as intended (FWIW, it doesn't work with Helm, I
> guess because Helm doesn't consult a completion table but reimplements
> the whole C-h f thing by itself).

That sounds bad!  I wasn't actually aware of Helm, but at the moment I
don't see why it would need its own C-h f facility.

> Regards,

> Michael.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: [Patch #2]: Missing doc strings for "," and ",@".

[npostavs]

Alan Mackenzie <acm@muc.de> writes:

>
> I've hacked all this together.  What used to be called a "reader macro"
> is now a "reader construct".

I think "read syntax" is better just because the manual already uses
that term.

> +;; Give `,' and `,@' documentation strings which can be examined by C-h f.
> +(put '\, 'function-documentation
> +     "See `\\=`' (also `pcase') for the details of `,'.")
> +(put '\, 'reader-construct t)
> +
> +(put '\,@ 'function-documentation
> +     "See `\\=`' or `,' for the details of `,@'.")

Linking from ",@" to "," seems a bit unhelpful, since "," hardly says
anything and just links to "`".

bug#25461: [Patch #2]: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Hello, Noam.

On Mon, Jan 23, 2017 at 23:01:36 -0500, npostavs@users.sourceforge.net wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > I've hacked all this together.  What used to be called a "reader macro"
> > is now a "reader construct".

> I think "read syntax" is better just because the manual already uses
> that term.

I think this is a very fine point.  "Read syntax" tends to distract
attention from the semantics, I think.

> > +;; Give `,' and `,@' documentation strings which can be examined by C-h f.
> > +(put '\, 'function-documentation
> > +     "See `\\=`' (also `pcase') for the details of `,'.")
> > +(put '\, 'reader-construct t)
> > +
> > +(put '\,@ 'function-documentation
> > +     "See `\\=`' or `,' for the details of `,@'.")

> Linking from ",@" to "," seems a bit unhelpful, since "," hardly says
> anything and just links to "`".

Thanks, I actually spotted that and removed it before committing a patch
yesterday.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#25461: [Patch #2]: Missing doc strings for "," and ",@".

[Noam Postavsky]

On Tue, Jan 24, 2017 at 2:39 PM, Alan Mackenzie <acm@muc.de> wrote:
> "Read syntax" tends to distract attention from the semantics, I think.

I don't quite agree about that, but I think what you've committed is
perfectly fine, and there's not much to be gained by arguing the point
any further.

bug#25461: [Patch]: Missing doc strings for "," and ",@".

[Alan Mackenzie]

Bug fixed.

-- 
Alan Mackenzie (Nuremberg, Germany).