pcase docs

pcase docs

[Eli Zaretskii]

Would people who expressed  the desire and intent of contributing
better documentation for pcase please consider doing that in the next
few days?  The initial documentation effort for v21.5 is almost
complete, and we will have our first pretest shortly, so it would be
good to have this item covered when the pretest comes out.

If you are not comfortable with Texinfo, don't bother about the
markup: adding that is a simple job, and I can easily do it.  It's
coming up with good content that takes most of the documentation
effort.

While at that, please also recommend which part(s) of this NEWS entry
needs to be reflected in the ELisp manual:

  ** pcase
  *** New UPatterns `quote', `app', `cl-struct', `eieio', `seq', and `map'.
  *** New UPatterns can be defined with `pcase-defmacro'.

Thanks.

Re: pcase docs

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

>   ** pcase
>   *** New UPatterns `quote', `app', `cl-struct', `eieio', `seq', and `map'.
>   *** New UPatterns can be defined with `pcase-defmacro'.

All of them?  I guess that means I have more text to write for you.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: pcase docs

[Michael Heerdegen]

John Wiegley <jwiegley@gmail.com> writes:

> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
>
> >   ** pcase
> >   *** New UPatterns `quote', `app', `cl-struct', `eieio', `seq', and `map'.
> >   *** New UPatterns can be defined with `pcase-defmacro'.
>
> All of them?  I guess that means I have more text to write for you.

`cl-struct', `eieio', `seq', and `map': Not in the manual, as Eli had
said:

> I think they need not be in the manual, as they are specific to those
> packages.  I will probably move their references in NEWS to their
> package's sections.

Michael.

Re: pcase docs

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> > All of them?  I guess that means I have more text to write for you.

And pcase-defmacro is intended for developers of (internal) packages
like seq etc, so it must not have to be described broadly.  Just say
that it is basically like `defmacro': you define how to expand the
defined pattern to a new pattern used instead of the one defined.  This
is done recursively (like for defmacro), and when compiling (also
like for defmacro).  It's a simple thing.  A small example could be
helpful.

You could also make the definition of ` an example.  If you leave out
vectors, it's quite simple and very instructive:

--8<---------------cut here---------------start------------->8---
(pcase-defmacro \` (qpat)
  (cond
   ((eq (car-safe qpat) '\,) (cadr qpat))
   ((consp qpat)
    `(and (pred consp)
          (app car ,(list '\` (car qpat)))
          (app cdr ,(list '\` (cdr qpat)))))
   ((or (stringp qpat) (integerp qpat) (symbolp qpat)) `',qpat)
   (t (error "Unknown QPAT: %S" qpat))))
--8<---------------cut here---------------end--------------->8---



Michael.

Re: pcase docs

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Date: Fri, 22 Jan 2016 20:58:06 +0100
> Cc: emacs-devel@gnu.org
> 
> And pcase-defmacro is intended for developers of (internal) packages
> like seq etc, so it must not have to be described broadly.

Since we don't have a separate "Emacs Lisp for Developers of Internal
Packages" manual, I don't see why we should omit these or describe
them in less detail.  Otherwise, where would those developers learn
about the likes of pcase-defmacro?

Re: pcase docs

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> Since we don't have a separate "Emacs Lisp for Developers of Internal
> Packages" manual, I don't see why we should omit these or describe
> them in less detail.

I didn't mean it like that.  But I think we can assume that they will
already know how `defmacro' works - something I would not assume for
every user of Emacs.


Michael.

Re: pcase docs

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: jwiegley@gmail.com,  emacs-devel@gnu.org
> Date: Sat, 23 Jan 2016 11:16:01 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Since we don't have a separate "Emacs Lisp for Developers of Internal
> > Packages" manual, I don't see why we should omit these or describe
> > them in less detail.
> 
> I didn't mean it like that.  But I think we can assume that they will
> already know how `defmacro' works - something I would not assume for
> every user of Emacs.

Sorry, I don't understand: how can knowing what 'defmacro' does help
to know what 'pcase-defmacro' does?  At the very least, one needs to
know which patterns will be possible after using 'pcase-defmacro',
don't you agree?

Btw, this macro also has "issues".  The doc string is less than
helpful:

    "Define a new kind of pcase PATTERN, by macro expansion.
  Patterns of the form (NAME ...) will be expanded according
  to this macro."

It says nothing about ARGS, and the "by macro expansion" part sounds
strange -- what useful information does it supply here?

And if you turn to the users of this macro, in the hope of gaining
some understanding, you will see, for example, that map.el defines a
pattern whose NAME is "map":

  (pcase-defmacro map (&rest args)
    "Build a `pcase' pattern matching map elements.

  The `pcase' pattern will match each element of PATTERN against
  the corresponding elements of the map.

(btw: what PATTERN?), but then uses it like this:

  (pcase type
    (`list (map-pairs map))
    (`hash-table (map--into-hash-table map))
    (_ (error "Not a map type name: %S" type))))

IOW, it actually uses sub-classes of 'map' as patterns.  Hmm...

(You can now understand why the documentation of 'pcase-defmacro' I
added is rather laconic and in some sense unsatisfactory.)

As an aside, when I look at such doc strings, I always become
frustrated, because I must be extremely stupid to ask all these
questions, when no one else evidently cares, which must mean they all
understand this stuff, and only I alone don't, right?

Thanks.

Re: pcase docs

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> Sorry, I don't understand: how can knowing what 'defmacro' does help
> to know what 'pcase-defmacro' does?

It works extremely analogous.

>  At the very least, one needs to know which patterns will be possible
> after using 'pcase-defmacro', don't you agree?
>
> Btw, this macro also has "issues".  The doc string is less than
> helpful:
>
>     "Define a new kind of pcase PATTERN, by macro expansion.
>   Patterns of the form (NAME ...) will be expanded according
>   to this macro."
>
> It says nothing about ARGS, and the "by macro expansion" part sounds
> strange -- what useful information does it supply here?

I agree that it's quite vague.

> And if you turn to the users of this macro, in the hope of gaining
> some understanding, you will see, for example, that map.el defines a
> pattern whose NAME is "map":
>
>   (pcase-defmacro map (&rest args)
>     "Build a `pcase' pattern matching map elements.
>
>   The `pcase' pattern will match each element of PATTERN against
>   the corresponding elements of the map.

> (btw: what PATTERN?),

the pattern that is gotten by expanding.

> but then uses it like this:
>
>   (pcase type
>     (`list (map-pairs map))
>     (`hash-table (map--into-hash-table map))
>     (_ (error "Not a map type name: %S" type))))

(why does my definition look like

`(and (pred mapp)
        ,@(map--make-pcase-bindings args))

?)

> IOW, it actually uses sub-classes of 'map' as patterns.  Hmm...

> As an aside, when I look at such doc strings, I always become
> frustrated, because I must be extremely stupid to ask all these
> questions, when no one else evidently cares, which must mean they all
> understand this stuff, and only I alone don't, right?

Surely not!  I just had the time to read the code, carefully (I had to,
to be able to write my el-search.el thing).  Most others don't care
because they just don't care, I think.

Actually `pcase-defmacro' is easy to understand if you understood
`defmacro': You define a pattern by its name and by the arguments it
will accept.

When the pcase "compiler" finds a pattern that is defined as a macro, it
substitutes the "call" with the "code" returned by evaluating the macro
definition with the given arguments.  Exactly like Lisp macros are
expanded by the lisp compiler.  Just that we have pcase "code" (i.e.,
another pattern again) in this case.  This is done recursively, i.e. the
returned pattern can be a pcase macro, a include pcase macros, again,
etc.

Try to understand the definition of ` - it's a challenge, but if you
succeed, you will see that things are much easier than you had expected.


Michael.

Re: pcase docs

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: jwiegley@gmail.com,  emacs-devel@gnu.org
> Date: Sat, 23 Jan 2016 13:05:22 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Sorry, I don't understand: how can knowing what 'defmacro' does help
> > to know what 'pcase-defmacro' does?
> 
> It works extremely analogous.

Maybe so, but the purpose is very different, so I think the analogy
doesn't really help.

> >   (pcase-defmacro map (&rest args)
> >     "Build a `pcase' pattern matching map elements.
> >
> >   The `pcase' pattern will match each element of PATTERN against
> >   the corresponding elements of the map.
> 
> > (btw: what PATTERN?),
> 
> the pattern that is gotten by expanding.

Expanding what?

> > As an aside, when I look at such doc strings, I always become
> > frustrated, because I must be extremely stupid to ask all these
> > questions, when no one else evidently cares, which must mean they all
> > understand this stuff, and only I alone don't, right?
> 
> Surely not!  I just had the time to read the code, carefully (I had to,
> to be able to write my el-search.el thing).  Most others don't care
> because they just don't care, I think.

But this stuff is widely used, by several different individuals.  How
do they know what to put in the code, if they, like me, are confused
about the purpose and the usage?

> Actually `pcase-defmacro' is easy to understand if you understood
> `defmacro': You define a pattern by its name and by the arguments it
> will accept.

But 'pcase' patterns don't accept any arguments, they are matched
against a value of an expression.  So how to understand "arguments" in
this context?

> When the pcase "compiler" finds a pattern that is defined as a macro, it
> substitutes the "call" with the "code" returned by evaluating the macro
> definition with the given arguments.

What do you mean by "call" here?

And how to explain that a pattern that starts with "`(list" will
somehow end up using the pattern defined with "(pcase-defmacro map"?

> Try to understand the definition of ` - it's a challenge, but if you
> succeed, you will see that things are much easier than you had expected.

Sorry, I don't want to go that way.  Understanding too much of the
implementation will "contaminate" my mind, and I will no longer be
able to see this from the POV of a Lisp programmer who needs to study
this stuff from scratch.  I want to understand it without looking
under the hood too much, then I will be able to document it in a
useful way.

Thanks.

Re: pcase docs

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> > Surely not!  I just had the time to read the code, carefully (I had to,
> > to be able to write my el-search.el thing).  Most others don't care
> > because they just don't care, I think.
>
> But this stuff is widely used, by several different individuals.  How
> do they know what to put in the code, if they, like me, are confused
> about the purpose and the usage?

Dunno.  If it helps: I had spent really a lot of time into understanding
pcase.  More time than you can probably invest, doing all the work you
do.  And I made lots of errors over weeks after had finally internalized
everything.

> > Actually `pcase-defmacro' is easy to understand if you understood
> > `defmacro': You define a pattern by its name and by the arguments it
> > will accept.
>
> But 'pcase' patterns don't accept any arguments, they are matched
> against a value of an expression.  So how to understand "arguments" in
> this context?

pcase-defmacro can only define patterns that are used like function
calls, (pattern-name pattern-args), like

 
  (pred fun)
  (guard expr)
  (or pattern...)

In the sense of pcase-defmacro, FUN, EXPR or the PATTERNs play the role
of arguments of the pattern.

> > When the pcase "compiler" finds a pattern that is defined as a
> > macro, it substitutes the "call" with the "code" returned by
> > evaluating the macro definition with the given arguments.
>
> What do you mean by "call" here?

A usage, or appearance, of the pattern.


> And how to explain that a pattern that starts with "`(list" will
> somehow end up using the pattern defined with "(pcase-defmacro map"?

No, `map' doesn't define `, it defines a pattern named `map' (that
currently is rarely used, or nowhere).

Mmh, maybe you need just a simple example.  Let's define a pcase pattern
"greater-than" that matches any number greater than a specified one, so
that e.g.

(pcase 9
  ((greater-than 2) t))  ==> t

but

(pcase (+ 0 1)
  ((greater-than (* 2 5)) t))  ==> nil.

or

(pcase "Hallo"
  ((greater-than 2) t))  ==> nil.

You could define it like this:

(pcase-defmacro greater-than (number)
  `(and (pred numberp)
        (pred (lambda (x) (< ,number x)))))

Then any occurrence of (greater-than N) in any pcase pattern is simply
substituted with

  (and (pred numberp)
       (pred (lambda (x) (< N x))))

That's it.


Of course, the variable name "x" in the above definition could lead to
name clashes (as with "conventional" macros), so you would better use an
uninterned symbol.  But with the shortcut version of `pred', we instead
can just write

(pcase-defmacro greater-than (number)
  `(and (pred numberp) (pred (< ,number))))

and avoid that problem this way.


A different simple example: here is how you could define `guard' via
`pred':

(pcase-defmacro guard (expr)
  `(pred ,(lambda () expr)))


HTH,

Michael.

Re: pcase docs

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: jwiegley@gmail.com,  emacs-devel@gnu.org
> Date: Sat, 23 Jan 2016 15:22:08 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > > Surely not!  I just had the time to read the code, carefully (I had to,
> > > to be able to write my el-search.el thing).  Most others don't care
> > > because they just don't care, I think.
> >
> > But this stuff is widely used, by several different individuals.  How
> > do they know what to put in the code, if they, like me, are confused
> > about the purpose and the usage?
> 
> Dunno.  If it helps: I had spent really a lot of time into understanding
> pcase.  More time than you can probably invest, doing all the work you
> do.  And I made lots of errors over weeks after had finally internalized
> everything.

So maybe we should drop everything and make sure those doc strings are
corrected?

> > > Actually `pcase-defmacro' is easy to understand if you understood
> > > `defmacro': You define a pattern by its name and by the arguments it
> > > will accept.
> >
> > But 'pcase' patterns don't accept any arguments, they are matched
> > against a value of an expression.  So how to understand "arguments" in
> > this context?
> 
> pcase-defmacro can only define patterns that are used like function
> calls, (pattern-name pattern-args), like
> 
>  
>   (pred fun)
>   (guard expr)
>   (or pattern...)
> 
> In the sense of pcase-defmacro, FUN, EXPR or the PATTERNs play the role
> of arguments of the pattern.

I don't think I see any of this in the doc string...

> > And how to explain that a pattern that starts with "`(list" will
> > somehow end up using the pattern defined with "(pcase-defmacro map"?
> 
> No, `map' doesn't define `, it defines a pattern named `map' (that
> currently is rarely used, or nowhere).

That's the problem: that the pattern 'map' is not used.  Instead, I
see its sub-classes being used as pcase patterns.

> (pcase-defmacro greater-than (number)
>   `(and (pred numberp)
>         (pred (lambda (x) (< ,number x)))))
> 
> Then any occurrence of (greater-than N) in any pcase pattern is simply
> substituted with
> 
>   (and (pred numberp)
>        (pred (lambda (x) (< N x))))
> 
> That's it.

That's what I expected to see when I saw this:

  (pcase-defmacro map (&rest args)

I expected to see its use like this:

  (map SOMETHING)

But instead I see this:

  (pcase type
    (`list (map-pairs map))
    (`hash-table (map--into-hash-table map))
    (_ (error "Not a map type name: %S" type))))

IOW, there's no 'map', which was defined with pcase-defmacro, but
there are 'list' and 'hash-table', which weren't.  And btw, why "`list"
instead of "`(list"?

Re: pcase docs

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> So maybe we should drop everything and make sure those doc strings are
> corrected?

I think we have already made some good progress since then.  But where
things are unclear, things should be improved further.

> > In the sense of pcase-defmacro, FUN, EXPR or the PATTERNs play the role
> > of arguments of the pattern.
>
> I don't think I see any of this in the doc string...

... of pcase-defmacro.  You must read between the lines ;-)

Yes, it's missing.

> I expected to see its use like this:
>
>   (map SOMETHING)


Indeed.

> But instead I see this:
>
>   (pcase type
>     (`list (map-pairs map))
>     (`hash-table (map--into-hash-table map))
>     (_ (error "Not a map type name: %S" type))))

> IOW, there's no 'map', which was defined with pcase-defmacro, but
> there are 'list' and 'hash-table', which weren't.  And btw, why "`list"
> instead of "`(list"?

That's something completely unrelated to the pcase `map' pattern, just a
random usage of pcase.  It's equivalent to

(cl-case type
  ((list) (map-pairs map))
  ((hash-table) (map--into-hash-table map))
  (t (error "Not a map type name: %S" type)))

i.e., this just tests whether TYPE is bound to one of two constants.

A usage of the function `map-into' would look like:

  (map-into '((1 . a) (2 . b)) 'hash-table)

  ==> an equivalent hash table to this alist


A usage of the pcase map pattern would look like this:

(pcase '((1 . a) (2 . b) (3 . c))
  ((map (2 x)) x))

  ==> b

Semantic and doc of the `map' pcase pattern are still a bit strange (or
wrong), I must say.  I had hoped Nicolas would have time to improve the
doc a bit.


Michael.

Re: pcase docs

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 280 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Semantic and doc of the `map' pcase pattern are still a bit strange (or
> wrong), I must say.  I had hoped Nicolas would have time to improve the
> doc a bit.

You're right, it needs a bit of love.  I will work on it.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: pcase docs

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> But this stuff is widely used, by several different individuals.  How
> do they know what to put in the code, if they, like me, are confused
> about the purpose and the usage?

Buy the way, I think most of them had help or advice by Stefan.  I also
had to ask him several questions because I didn't understand everything.

Then I had a tea at night several times and meditated over the sources.
That helped.


Michael.

Re: pcase docs

[Alan Mackenzie]

Hello, Eli.

On Sat, Jan 23, 2016 at 01:43:07PM +0200, Eli Zaretskii wrote:

> (You can now understand why the documentation of 'pcase-defmacro' I
> added is rather laconic and in some sense unsatisfactory.)

> As an aside, when I look at such doc strings, I always become
> frustrated, because I must be extremely stupid to ask all these
> questions, when no one else evidently cares, which must mean they all
> understand this stuff, and only I alone don't, right?

No, you're not stupid, and you're not the only person who cares about
"economical" doc strings.  That was one of the reasons I raised a bug
about all the pcase stuff documentation a few weeks ago, and I think I
included things like the `pcase-defmacro' doc strings in that bug.

I admit my reaction to such things is usually to throw up my hands in
despair, then say to myself "I don't need this anyway" and pass on to
the next problem.  I suspect I'm not alone in this.  So a big thank-you
from me to you and Michael H., as a representative of all the people
similarly clueless about such things, that you're actually getting down
and sorting these things out.

> Thanks.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: pcase docs

[Eli Zaretskii]

> Date: Sat, 23 Jan 2016 12:28:00 +0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: Michael Heerdegen <michael_heerdegen@web.de>, jwiegley@gmail.com,
> 	emacs-devel@gnu.org
> 
> Hello, Eli.
> 
> On Sat, Jan 23, 2016 at 01:43:07PM +0200, Eli Zaretskii wrote:
> 
> > (You can now understand why the documentation of 'pcase-defmacro' I
> > added is rather laconic and in some sense unsatisfactory.)
> 
> > As an aside, when I look at such doc strings, I always become
> > frustrated, because I must be extremely stupid to ask all these
> > questions, when no one else evidently cares, which must mean they all
> > understand this stuff, and only I alone don't, right?
> 
> No, you're not stupid

That's a relief, thanks.

> So a big thank-you from me to you and Michael H., as a
> representative of all the people similarly clueless about such
> things, that you're actually getting down and sorting these things
> out.

Thanks.  Perhaps take a look at the new-and-improved pcase
documentation, and provide your subjective feedback.

RE: pcase docs

[Drew Adams]

> I admit my reaction to such things is usually to throw up my hands in
> despair, then say to myself "I don't need this anyway" and pass on to
> the next problem.  I suspect I'm not alone in this.  So a big thank-you
> from me to you and Michael H., as a representative of all the people
> similarly clueless about such things, that you're actually getting down
> and sorting these things out.

+1

Re: pcase docs

[Stefan Monnier]

> Sorry, I don't understand: how can knowing what 'defmacro' does help
> to know what 'pcase-defmacro' does?

(defmacro FOO (FARGS) BODY) takes an expression of the form (FOO AARGS),
and replaces it with the result of evaluating BODY in a context where
FARGS is bound to AARGS.

(pcase-defmacro FOO (FARGS) BODY) takes a pattern of the form (FOO AARGS),
and replaces it with the result of evaluating BODY in a context where
FARGS is bound to AARGS.


        Stefan "hoping this clarifies it.  Sorry for my poor docstrings"

Re: pcase docs

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Date: Fri, 22 Jan 2016 20:46:17 +0100
> 
> `cl-struct', `eieio', `seq', and `map': Not in the manual, as Eli had
> said:
> 
> > I think they need not be in the manual, as they are specific to those
> > packages.  I will probably move their references in NEWS to their
> > package's sections.

I did that now, thanks for a reminder.