From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eric Abrahamsen Newsgroups: gmane.emacs.devel Subject: Re: [PATCH] Re: Make peg.el a built-in library? Date: Wed, 16 Nov 2022 10:15:11 -0800 Message-ID: <875yfe7ols.fsf@ericabrahamsen.net> References: <875yvtbbn3.fsf@ericabrahamsen.net> <877d07a16u.fsf@localhost> <87tu3asg2r.fsf@ericabrahamsen.net> <87edud25ov.fsf@localhost> <87a6511ku0.fsf@ericabrahamsen.net> <87wn85z0zl.fsf@ericabrahamsen.net> <87leobplpv.fsf_-_@ericabrahamsen.net> <87bkp7ct7f.fsf@localhost> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="29351"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) To: emacs-devel@gnu.org Cancel-Lock: sha1:trDcG7jWVPV/9KRfkysI68xbNUw= Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Nov 16 20:38:19 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ovOEl-0007Na-8l for ged-emacs-devel@m.gmane-mx.org; Wed, 16 Nov 2022 20:38:19 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ovOEe-0003kv-8u; Wed, 16 Nov 2022 14:38:12 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ovMwU-0003JM-Nm for emacs-devel@gnu.org; Wed, 16 Nov 2022 13:15:23 -0500 Original-Received: from ciao.gmane.io ([116.202.254.214]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ovMwS-00087O-Ly for emacs-devel@gnu.org; Wed, 16 Nov 2022 13:15:22 -0500 Original-Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1ovMwQ-0001xY-UY for emacs-devel@gnu.org; Wed, 16 Nov 2022 19:15:18 +0100 X-Injected-Via-Gmane: http://gmane.org/ Received-SPF: pass client-ip=116.202.254.214; envelope-from=ged-emacs-devel@m.gmane-mx.org; helo=ciao.gmane.io X-Spam_score_int: -15 X-Spam_score: -1.6 X-Spam_bar: - X-Spam_report: (-1.6 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Wed, 16 Nov 2022 14:38:10 -0500 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:299979 Archived-At: Ihor Radchenko writes: > Eric Abrahamsen writes: > >> Okay, here's a first stab. I read the paper, and understood about half >> of it, which seemed like enough. It was interesting to see that the >> paper explicitly calls out the exact greedy-matching behavior I'd >> encountered. > > Thanks! And thanks for the review! I'll add in all your simpler notes; more responses below. >> + Emacs Lisp provide several tools for parsing and matching text, from > > provides > >> +regular expressions (@pxref{Regular Expressions}) to full @acronym{LL} >> +grammar parsers (@pxref{Top,, Bovine parser development, bovine}). >> +@dfn{Parsing Expression Grammars} (@acronym{PEG}) are another approach >> +to text parsing that offer more structure and composibility than >> +regular expressions, but less complexity than context-free grammars. >> + >> +A @acronym{PEG} parser is defined as a list of named rules, each of >> +which match text patterns, and/or contain references to other rules. >> +Parsing is initiated with the function @code{peg-run} or the macro >> +@code{peg-parse}, and parses text after point in the current buffer, >> +using a given set of rules. >> + >> +The definition of each rule is referred to as a @dfn{parsing >> +expression} (@acronym{PEX}), and can consist of a literal string, a >> +regexp-like character range or set, a peg-specific construct >> +resembling an elisp function call, a reference to another rule, or a >> +combination of any of these. A grammar is expressed as a set of rules >> +in which one rule is typically treated as a ``top-level'' or >> +``entry-point'' rule. For instance: >> + >> +@example >> +@group >> +((number sign digit (* digit)) >> + (sign (or "+" "-" "")) >> + (digit [0-9])) >> +@end group >> +@end example >> + >> +The above grammar could be used directly in a call to >> +@code{peg-parse}, in which the first rule is considered the >> +``entry-point'' rule: >> + >> +@example >> +(peg-parse >> + ((number sign digit (* digit)) >> + (sign (or "+" "-" "")) >> + (digit [0-9]))) >> +@end example >> + >> +Or set as the value of a variable, and the variable used in a >> +combination of calls to @code{with-peg-rules} and @code{peg-run}, >> +where the ``entry-point'' rule is given explicitly: >> + >> +@example >> +(defvar number-grammar >> + '((number sign digit (* digit)) >> + (sign (or "+" "-" "")) >> + (digit [0-9]))) >> + >> +(with-peg-rules number-grammar >> + (peg-run (peg number))) >> +@end example >> + >> +By default, calls to @code{peg-run} or @code{peg-parse} produce no >> +output: parsing simply moves point. In order to return or otherwise >> +act upon parsed strings, rules can include @dfn{actions}, see >> +@xref{Parsing Actions} for more information. >> + >> +Individual rules can also be defined using a more @code{defun}-like >> +syntax, using the macro @code{define-peg-rule}: >> + >> +@example >> +(define-peg-rule digit () >> + [0-9]) >> +@end example >> + >> +This allows the rule to be referred to by name within calls to >> +@code{peg-run} or @code{peg-parse} elsewhere, and also allows the use >> +of function arguments in the rule body. >> + >> +@node PEX Definitions >> +@section PEX Definitions >> + >> +Parsing expressions can be defined using the following syntax: >> + >> +@table @code >> +@item (and E1 E2 ...) >> +A sequence of PEXs that must all be matched. The @code{and} form is >> +optional and implicit. >> + >> +@item (or E1 E2 ...) >> +Prioritized choices, meaning that, as in Elisp, the choices are tried >> +in order, and the first successful match is used. > > It is worth highlighting that it is different from CFGs. > >> +@item (* E) >> +Zero or more of an expression, as the regexp ``*''. >> + >> +@item (+ E) >> +One or more of an expression, as the regexp ``+''. > > It is worth highlighting the greedy part here and referring to &A and > !A. I don't believe there is separate syntax for &A and !A -- those are written (if A) and (not A). >> +@item SYMBOL >> +A symbol representing a previously-define PEG rule. > > defined > >> +By default the process of parsing simply moves point in the current >> +buffer, ultimately returning @code{t} if the parsing succeeds, and >> +@code{nil} if it doesn't. It's also possible to define ``actions'' >> +that can run arbitrary Elisp at certain points during parsing. These >> +actions can affect something called the @dfn{parsing stack}: a list of >> +values built up during the course of parsing. If the stack is >> +non-@code{nil} at the end of parsing, it is returned as the final >> +value of the parsing process. > > Actions are only run when the expression matches; with point moved after > the match, right? What about &A and !A? That's right, actions only run if the parsing succeeds, and they run all at once at the end. Maybe I can move all discussons of parsing success vs failure into one place. >> +There must be values on the stack before they can be popped and >> +returned. > > What if there is just one value in the stack while the action required two? > >> +@item (list E) >> +Match E, collect all values produced by E (and its sub-expressions) >> +into a list, and push that list to the stack. >> +@end table > > This one is not very clear. Does it imply that E is recursively wrapped > into substring? It's not very clear because I don't fully understand it! It does not implicitly create any value-returning calls (such as `substring'). I think what it means is that, by default, values returned by actions are all spliced into a single flat list. If you need some of those values to be returned in a sub-list, you can use this form. It's a bit tricky to use because the E in (list E) could potentially descend many levels and branch out into any number of sub-expressions, so you need to have a clear mental model of what values might ultimately be coming out of E. I guess that's also true for the whole thing, though. >> +It is up to the grammar author to keep track of which rules and >> +sub-rules push values to the stack, and the state of the stack at any >> +given point in the parsing. If an action pops values from an empty >> +stack, the symbols will be bound to @code{nil}. > > The part about popping out of empty stack looks out of scope. Maybe move > it to earlier discussion of variable bindings in actions? Okay, I'll remove this, and just add a shorter note up above about empty stacks. Thanks again, Eric