unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
From: Eric Abrahamsen <eric@ericabrahamsen.net>
To: emacs-devel@gnu.org
Subject: Re: [PATCH] Re: Make peg.el a built-in library?
Date: Wed, 16 Nov 2022 10:15:11 -0800	[thread overview]
Message-ID: <875yfe7ols.fsf@ericabrahamsen.net> (raw)
In-Reply-To: 87bkp7ct7f.fsf@localhost

Ihor Radchenko <yantar92@posteo.net> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Okay, here's a first stab. I read the paper, and understood about half
>> of it, which seemed like enough. It was interesting to see that the
>> paper explicitly calls out the exact greedy-matching behavior I'd
>> encountered.
>
> Thanks!

And thanks for the review! I'll add in all your simpler notes; more
responses below.

>> +  Emacs Lisp provide several tools for parsing and matching text, from
>
> provides
>
>> +regular expressions (@pxref{Regular Expressions}) to full @acronym{LL}
>> +grammar parsers (@pxref{Top,, Bovine parser development, bovine}).
>> +@dfn{Parsing Expression Grammars} (@acronym{PEG}) are another approach
>> +to text parsing that offer more structure and composibility than
>> +regular expressions, but less complexity than context-free grammars.
>> +
>> +A @acronym{PEG} parser is defined as a list of named rules, each of
>> +which match text patterns, and/or contain references to other rules.
>> +Parsing is initiated with the function @code{peg-run} or the macro
>> +@code{peg-parse}, and parses text after point in the current buffer,
>> +using a given set of rules.
>> +
>> +The definition of each rule is referred to as a @dfn{parsing
>> +expression} (@acronym{PEX}), and can consist of a literal string, a
>> +regexp-like character range or set, a peg-specific construct
>> +resembling an elisp function call, a reference to another rule, or a
>> +combination of any of these.  A grammar is expressed as a set of rules
>> +in which one rule is typically treated as a ``top-level'' or
>> +``entry-point'' rule.  For instance:
>> +
>> +@example
>> +@group
>> +((number sign digit (* digit))
>> + (sign   (or "+" "-" ""))
>> + (digit  [0-9]))
>> +@end group
>> +@end example
>> +
>> +The above grammar could be used directly in a call to
>> +@code{peg-parse}, in which the first rule is considered the
>> +``entry-point'' rule:
>> +
>> +@example
>> +(peg-parse
>> +  ((number sign digit (* digit))
>> +   (sign   (or "+" "-" ""))
>> +   (digit  [0-9])))
>> +@end example
>> +
>> +Or set as the value of a variable, and the variable used in a
>> +combination of calls to @code{with-peg-rules} and @code{peg-run},
>> +where the ``entry-point'' rule is given explicitly:
>> +
>> +@example
>> +(defvar number-grammar
>> +        '((number sign digit (* digit))
>> +          (sign (or "+" "-" ""))
>> +          (digit [0-9])))
>> +
>> +(with-peg-rules number-grammar
>> +  (peg-run (peg number)))
>> +@end example
>> +
>> +By default, calls to @code{peg-run} or @code{peg-parse} produce no
>> +output: parsing simply moves point.  In order to return or otherwise
>> +act upon parsed strings, rules can include @dfn{actions}, see
>> +@xref{Parsing Actions} for more information.
>> +
>> +Individual rules can also be defined using a more @code{defun}-like
>> +syntax, using the macro @code{define-peg-rule}:
>> +
>> +@example
>> +(define-peg-rule digit ()
>> +  [0-9])
>> +@end example
>> +
>> +This allows the rule to be referred to by name within calls to
>> +@code{peg-run} or @code{peg-parse} elsewhere, and also allows the use
>> +of function arguments in the rule body.
>> +
>> +@node PEX Definitions
>> +@section PEX Definitions
>> +
>> +Parsing expressions can be defined using the following syntax:
>> +
>> +@table @code
>> +@item (and E1 E2 ...)
>> +A sequence of PEXs that must all be matched.  The @code{and} form is
>> +optional and implicit.
>> +
>> +@item (or E1 E2 ...)
>> +Prioritized choices, meaning that, as in Elisp, the choices are tried
>> +in order, and the first successful match is used.
>
> It is worth highlighting that it is different from CFGs.
>
>> +@item (* E)
>> +Zero or more of an expression, as the regexp ``*''.
>> +
>> +@item (+ E)
>> +One or more of an expression, as the regexp ``+''.
>
> It is worth highlighting the greedy part here and referring to &A and
> !A.

I don't believe there is separate syntax for &A and !A -- those are
written (if A) and (not A).

>> +@item SYMBOL
>> +A symbol representing a previously-define PEG rule.
>
> defined
>
>> +By default the process of parsing simply moves point in the current
>> +buffer, ultimately returning @code{t} if the parsing succeeds, and
>> +@code{nil} if it doesn't.  It's also possible to define ``actions''
>> +that can run arbitrary Elisp at certain points during parsing.  These
>> +actions can affect something called the @dfn{parsing stack}: a list of
>> +values built up during the course of parsing.  If the stack is
>> +non-@code{nil} at the end of parsing, it is returned as the final
>> +value of the parsing process.
>
> Actions are only run when the expression matches; with point moved after
> the match, right? What about &A and !A?

That's right, actions only run if the parsing succeeds, and they run all
at once at the end. Maybe I can move all discussons of parsing success
vs failure into one place.

>> +There must be values on the stack before they can be popped and
>> +returned.
>
> What if there is just one value in the stack while the action required two?
>
>> +@item (list E)
>> +Match E, collect all values produced by E (and its sub-expressions)
>> +into a list, and push that list to the stack.
>> +@end table
>
> This one is not very clear. Does it imply that E is recursively wrapped
> into substring?

It's not very clear because I don't fully understand it! It does not
implicitly create any value-returning calls (such as `substring'). I
think what it means is that, by default, values returned by actions are
all spliced into a single flat list. If you need some of those values to
be returned in a sub-list, you can use this form.

It's a bit tricky to use because the E in (list E) could potentially
descend many levels and branch out into any number of sub-expressions,
so you need to have a clear mental model of what values might ultimately
be coming out of E. I guess that's also true for the whole thing,
though.

>> +It is up to the grammar author to keep track of which rules and
>> +sub-rules push values to the stack, and the state of the stack at any
>> +given point in the parsing.  If an action pops values from an empty
>> +stack, the symbols will be bound to @code{nil}.
>
> The part about popping out of empty stack looks out of scope. Maybe move
> it to earlier discussion of variable bindings in actions?

Okay, I'll remove this, and just add a shorter note up above about empty
stacks.

Thanks again,
Eric




  reply	other threads:[~2022-11-16 18:15 UTC|newest]

Thread overview: 100+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-25 18:52 Make peg.el a built-in library? Eric Abrahamsen
2021-08-26  6:17 ` Eli Zaretskii
2021-08-26 15:34   ` Eric Abrahamsen
2021-09-09  4:36     ` Eric Abrahamsen
2021-09-19 15:25       ` Eric Abrahamsen
2021-09-30 19:44       ` Stefan Monnier
2021-09-30 20:34         ` Adam Porter
2021-10-01  8:14           ` Augusto Stoffel
2021-10-01 18:05           ` Stefan Monnier
2021-10-01 18:40             ` Eric Abrahamsen
2021-10-02  3:57               ` Stefan Monnier
2021-10-02  7:32             ` Adam Porter
2021-10-02 14:45               ` Stefan Monnier
2021-10-02 15:13                 ` Adam Porter
2021-08-26 17:02 ` Adam Porter
2021-08-26 17:25   ` Eric Abrahamsen
2021-08-27  3:17   ` Eric Abrahamsen
2021-08-27  6:41     ` Helmut Eller
2021-08-27 16:57       ` Eric Abrahamsen
2021-09-26 10:59       ` Augusto Stoffel
2021-09-26 15:06         ` Eric Abrahamsen
2021-09-26 18:36           ` Augusto Stoffel
2021-09-27 16:18             ` Eric Abrahamsen
2021-09-27 22:34         ` Richard Stallman
2021-09-28  3:52           ` Eric Abrahamsen
2021-09-28  8:09             ` tomas
2021-09-28  9:32               ` Helmut Eller
2021-09-28 10:45                 ` tomas
2021-09-28 15:24               ` Augusto Stoffel
2021-09-30  6:04             ` Richard Stallman
2021-10-01  3:27               ` Eric Abrahamsen
2021-10-09  1:31 ` Michael Heerdegen
2021-10-09  5:28   ` Michael Heerdegen
2021-10-09  8:12     ` Helmut Eller
2021-10-09 12:52       ` Stefan Monnier
2021-10-10  5:49         ` Helmut Eller
2021-10-14 10:25       ` Michael Heerdegen
2021-10-09 12:54   ` Stefan Monnier
2021-10-09 16:47     ` Eric Abrahamsen
2021-10-10  4:20       ` Michael Heerdegen
2021-10-10 21:40         ` Eric Abrahamsen
2021-10-13  2:58           ` Michael Heerdegen
2021-10-09 16:49   ` Eric Abrahamsen
2021-10-10  3:43     ` Stefan Monnier
2021-10-10  4:46       ` Michael Heerdegen
2021-10-10  5:58         ` Helmut Eller
2021-10-10 13:56           ` Stefan Monnier
2021-10-22 16:33           ` Michael Heerdegen
2021-10-31 23:43           ` Michael Heerdegen
2021-11-15 23:16             ` Michael Heerdegen
2022-11-07  3:33 ` Ihor Radchenko
2022-11-07 19:46   ` Eric Abrahamsen
2022-11-08  6:57     ` Helmut Eller
2022-11-08  8:51       ` Ihor Radchenko
2022-11-10  4:04       ` Richard Stallman
2022-11-10  5:25         ` tomas
2022-11-10  8:15           ` Eli Zaretskii
2022-11-10  8:29             ` tomas
2022-11-11  4:36           ` Richard Stallman
2022-11-08  8:47     ` Ihor Radchenko
2022-11-08 16:18       ` Eric Abrahamsen
2022-11-08 19:08         ` tomas
2022-11-08 19:42           ` Eric Abrahamsen
2022-11-16  4:27             ` [PATCH] " Eric Abrahamsen
2022-11-16  5:07               ` tomas
2022-11-16  5:39                 ` Eric Abrahamsen
2022-11-16 15:53                   ` tomas
2022-11-16  6:24               ` Ihor Radchenko
2022-11-16 18:15                 ` Eric Abrahamsen [this message]
2022-11-17 12:21                   ` Ihor Radchenko
2022-11-27  1:46                     ` Eric Abrahamsen
2022-11-27  8:57                       ` Eli Zaretskii
2022-11-28  1:09                         ` Eric Abrahamsen
2022-11-28 12:16                           ` Eli Zaretskii
2023-09-25  1:30                             ` Eric Abrahamsen
2023-09-25  2:27                               ` Adam Porter
2023-09-25 13:00                                 ` Alexander Adolf
2024-03-24 14:19                               ` Ihor Radchenko
2024-03-24 15:32                                 ` Eli Zaretskii
2024-03-25  1:45                                   ` Eric Abrahamsen
2023-01-11  7:39               ` Michael Heerdegen
2023-01-11  8:04                 ` Ihor Radchenko
2023-01-11 11:01                   ` Michael Heerdegen
2023-01-11 11:32                     ` tomas
2023-02-05 12:10                     ` Ihor Radchenko
2023-02-05 15:41                       ` Eduardo Ochs
2023-02-05 15:45                         ` Ihor Radchenko
2023-02-05 16:19                           ` Eduardo Ochs
2023-02-05 16:50                             ` Ihor Radchenko
2023-02-09  5:44                         ` Jean Louis
2023-02-06  0:33                       ` Michael Heerdegen
2022-11-08 14:01     ` Stefan Monnier
2022-11-08 14:42       ` tomas
2022-11-08 15:08       ` Visuwesh
2022-11-08 16:29         ` Juanma Barranquero
2022-12-02 20:20           ` Augusto Stoffel
2022-11-08 16:10       ` Eric Abrahamsen
2022-11-08 18:59         ` tomas
2022-11-08 19:42           ` Eric Abrahamsen
2022-11-08 22:03             ` Tim Cross

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.gnu.org/software/emacs/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=875yfe7ols.fsf@ericabrahamsen.net \
    --to=eric@ericabrahamsen.net \
    --cc=emacs-devel@gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).