Re: The poor state of documentation of pcase like things.

[Alan Mackenzie <acm@muc.de>]

Hello, Michael.

On Sat, Dec 19, 2015 at 09:42:01PM +0100, Michael Heerdegen wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > I wish you a happy one, though.

> You too!

Thanks!

> > > As far as I understand how Stefan used to work, most of the semantics of
> > > most of the pcase derivatives, like `pcase-let', are not yet 100% fixed,
> > > we are not yet sure how useful we are, or if they may later be better be
> > > replaced by other forms that are more general, etc.

> > Yet the said functions have been committed on the release branch

> I think Stefan used the term "don't advertise them yet too much" for
> such things.

> > and are already in widespread use throughout the Emacs sources.  As
> > Eli notes, there're around 90 uses of pcase-let and pcase-let*.

> Could be they have nearly all been placed there by Stefan ;-)

I think that's probable rather than possible.

> > They're essentially undocumented.  That makes the code that uses them
> > essentially unmaintainable, except by those who know, or are prepared to
> > guess, what these functions do.  I can't agree with you that this is
> > good.

> This can't be the final state, no doubt.  It must be fixed.

Glad you agree.

> But we have to do it very carefully, and maybe not completely now: if
> even Stefan was not sure everywhere what could be written in the
> documentation and if the code makes much sense, let's not engrave in
> stone details we are not sure about whether they are true or intended or
> useful.

I don't really know what pcase-let does.  I haven't a clue what
pcase-lambda, pcase-defmacro, ... do.

> This does not so much apply to `pcase-let' - I think it's quite clear
> what it does, though I didn't use it often myself.

It's not clear to me.  I resent being forced to guess, read the pcase.el
source, or give up on trying to understand significant portions of the
Emacs source code.  So, yes, I think `pcase-let' should be documented.

> OTOH, I'm not so sure about `pcase-lambda' for example.  How does/should
> a `pcase-lambda' behave when applied to something that it's "signature"
> doesn't match?  Is it maybe cleaner to move a pcase call just inside the
> `lambda'?  Is the semantics of pcase really the best thing one can use
> to hook pattern matching into a function, or is there a better one?  In
> practice, only a subset of pcase semantics makes sense for
> `pcase-lambda' most of the time (mostly, destructuring), you can have
> "only one clause" contrary to `pcase' - is the resulting code readable?;
> is `pcase-lambda' a good abstraction/idea?  If later we find it's a bad
> idea, it's not good if we advertise it too much in the manual, because
> we can't remove it so easily then.

> > The pcase docstring is getting better, yes.  It stil doesn't document
> > explicitly that the normal meanings of ` and , are suspended.

> Mmh, but it also doesn't say that the normal meanings of and, or, ' and
> let are suspended.

Good point.  Perhaps it should say that about these symbols, too.  And
perhaps the entries in the Elisp manual for `and', `or', ``', `,', etc.
should warn about the non-uniform meanings within pcase.  The notion of
kicking a wasp's nest springs to mind.

> I think it doesn't have to, because a pcase pattern is not an
> expression that is evaluated, so symbols with known names can't have
> the same semantics as a pattern than as a defined function.

That would be a good argument if only experts read the doc string.
Thoroughly confused people also read doc strings.  I don't think we
should presuppose such sophisticated discernment in the typical reader.

> > There is still no explicit statement of what pcase's ` and , mean.

> But do we agree that the current docstring completely explains the
> semantics, even if the definition is recursive?

I don't agree.  Completely missing is something like "pcase compares the
value with each pattern in turn until one matches, and it then evaluates
the corresponding BODY, returning the result of the last body form
evaluated.  If no pattern matches, nil is returned.".  The somewhat
offensive "perform ML-style pattern matching on that value" is no
substitute for this.

There is no mention of ``' and `,' in the section on patterns.  Or is
\\='VAL really meant to be \\=`VAL?

> > Maybe I'm expecting too much, and ` and , have no intrinsic meanings in
> > pcase.  But I don't believe that is the case.

> ``' is a pcase macro (at least now).  pcase macros are like Lisp macros,
> but while Lisp macros expand to Lisp code, pcase macros expand to pcase
> patterns.

> The backquote macro ``' in Lisp is a utility that is used to build
> lists.  pcase's ``' is used to build patterns that match lists.  The
> only difference for unquoting (`,'), so to say, is that in the first
> case it "inserts" an arbitrary "value" at that position, and in the
> second case, "inserts" an arbitrary pcase pattern (that is responsible to
> match the element at that place, instead of being a specified constant
> that matches only itself via `equal'.)

But what does ``' _do_?  What it normally does is well explained in its
own doc string (which will need modification for pcase).  But what does
``' _do_ within pcase?  Saying "it is used to build patterns ...." is a
cop out.  The difficulty in describing its (and `,''s) semantics seems
to be a weakness in pcase.

> If you only unquote symbols, like in

>    `(1 2 ,x)

> you can imagine that x is already bound to the "right" value, then the
> thing works much like the standard backquote.  That analogy is not so
> useful for things like

>    `(1 2 ,(pred stringp))

> I guess, though you still can imagine that the pattern matches any
> three-element list whose first two elements are 1, 2, and as third
> element, anything that fulfills the predicate "stringp" can be
> "inserted".

> You can verify by yourself that elements in a backquoted list in pcase
> "behave" as if they were quoted with a quote (see the 'VAL form) - thus
> the name "QPattern" - and that the rules for quoted atoms apply as
> expected, and such things.

Are you saying that "QPattern" has no more conceptual meaning than
"patterns which are backquoted"?

> [ While writing this, I see now that the doc of ``' in the pcase doc is
>   a typical mathematical definition: it is absolutely correct, but
>   doesn't say "what it's good for" and "how to use it".  To be honest I
>   also had much trouble to understand it when I learned `pcase'. ]

I think I've got some reasonable partial understanding of ``', but I'm
unhappy that there's no concise description of what it does.

> BTW, there is no `,@' unquote splicing in pcase's backquote because it
> can lead to ambiguity when matching (and would be slow).

OK.

> Regards,

> Michael.

-- 
Alan Mackenzie (Nuremberg, Germany).