Re: Make peg.el a built-in library?

[Eric Abrahamsen <eric@ericabrahamsen.net>]

Adam Porter <adam@alphapapa.net> writes:

> FWIW, I've been happy using peg.el in org-ql.  I use it to parse strings
> like:
>
>   "todo:WAITING scheduled:from=2021-08-01,to=2021-08-31"
>
> into a sexp like:
>
>   (and (todo "WAITING")
>        (scheduled :from "2021-08-01" :to "2021-08-31"))
>
> You can see the code I use here:
>
> https://github.com/alphapapa/org-ql/blob/master/org-ql.el#L854

Whoo, I've been trying to get enough of a handle on the parsing actions
to write a documentation patch for them -- now I'm seeing what Helmut
meant by "semantically unintuitive". The sum total of docs regarding
actions is:

A "stack action" takes VARs from the "value stack" and pushes the result
of evaluating FORMs to that stack.

So lower-level pexs need to explicitly push values onto the stack. They
can do that with either one of the built-in "operators" (substring,
region, replace, list), or by using the pattern:

(and <your pex> `(VARS... -- FORM...))

Which confused me mightily until I realized that the backquoted sexp was
essentially a lambda with funny syntax:

`(VARS... -- FORM...) ==> (lambda (vars...) form...)
`(-- FORM...) ==> (lambda () form...)

You don't actually need the leading `and' if you're writing a top-level
pex, it only seems necessary if you're lining up a series of them under
an `or'.

A built-in operator pushes a value onto the stack. No operator (or stack
action) means no push. An action lambda with no argument but a return
value simply pushes that value onto the stack. An action lambda that
accepts arguments consumes values from the stack, and then pushes a new
value (its return value) onto the stack.

So lower-level pexs can take values from the stack and push new ones
back onto the stack, and higher-level pexs can pick those up later. But
because higher-level pexs often simply "or" lower-level pexs, the
developer has to be consistent with the number and type of pushed
values: if a high-level pex looks like:

(foo (or baz bar) `(str -- (upcase str)))

Then the contract is that both the "baz" and "bar" pexs (or an even
lower-level pex referred to by them) will push a single string value
onto the stack (probably with the "substring" operator). Essentially we
need to be calling our upper-level lambda with the right number/type of
argument(s).

If this email makes no sense, it's because I'm halfway through trying to
understand this library. I guess I could wish that these action forms
were simply callables, since they're clearly modeled after function
calls. Anyhoo, I'm going to try to confirm all of the above, and then at
least add to the commentary section for the main package file.

Eric