From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Noah Lavine Newsgroups: gmane.lisp.guile.devel Subject: Re: PEG Patches Date: Sun, 6 Mar 2011 20:28:15 -0500 Message-ID: References: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary=20cf307cfcd8edfc14049dda685a X-Trace: dough.gmane.org 1299463893 13338 80.91.229.12 (7 Mar 2011 02:11:33 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Mon, 7 Mar 2011 02:11:33 +0000 (UTC) To: guile-devel Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Mon Mar 07 03:11:28 2011 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1PwPuw-0005QM-FD for guile-devel@m.gmane.org; Mon, 07 Mar 2011 03:11:26 +0100 Original-Received: from localhost ([127.0.0.1]:54041 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PwPk0-0007jX-KB for guile-devel@m.gmane.org; Sun, 06 Mar 2011 21:00:08 -0500 Original-Received: from [140.186.70.92] (port=52993 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PwPFL-00026g-Ml for guile-devel@gnu.org; Sun, 06 Mar 2011 20:29:49 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PwPFA-0001Wk-7p for guile-devel@gnu.org; Sun, 06 Mar 2011 20:28:17 -0500 Original-Received: from mail-vw0-f41.google.com ([209.85.212.41]:55731) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1PwPFA-0001Wa-3o for guile-devel@gnu.org; Sun, 06 Mar 2011 20:28:16 -0500 Original-Received: by vws13 with SMTP id 13so4189469vws.0 for ; Sun, 06 Mar 2011 17:28:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=5L71OMWDBlxYmAEO6rL02YeNMF1hHhqFZ2XGOcfKF7M=; b=Bgs6ZrcDbvTUkBoX+/dhQKngMJ085UJsAQVuVa3SwOYUT/xcUXUd1SYqcHsDWEpekk gPS1qs+ARuVrXbwMn8Lui2w2dLNdjduPgkZMsSqEqsbvjovWpUxpxjKVHrpro68Y72C5 xkFnoZwRzmUOhtg3+FcmrMlPAQikgng9EjuyE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=ey76yl8k7FKSOF11iW81R9PyISiqxm0b95odfECtJtGofWF483MsYHrXzRt4/Tg1V4 wL4qUJ1sY3TXZQsKPomNHFmHquRMmFb3knN1eMP4oVGx75sPWop5KGr8j9HigOcKLxhs uzEGb/EPOR7DrVAGEpWHGn5HCGNzBtHSykq0U= Original-Received: by 10.52.98.130 with SMTP id ei2mr4546843vdb.198.1299461295700; Sun, 06 Mar 2011 17:28:15 -0800 (PST) Original-Received: by 10.52.166.194 with HTTP; Sun, 6 Mar 2011 17:28:15 -0800 (PST) In-Reply-To: X-Google-Sender-Auth: 6OKTeVoXOUI5dZSTC3_bylnpbr4 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 209.85.212.41 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:11803 Archived-At: --20cf307cfcd8edfc14049dda685a Content-Type: text/plain; charset=ISO-8859-1 Here's another patch, which in retrospect may be the most useful of the series. It adds a section called "PEG Internals" to the manual, and begins documenting how PEG actually works. This should make hacking PEG a lot easier. Noah On Sun, Mar 6, 2011 at 12:25 AM, Noah Lavine wrote: > Hello all, > > Attached is a series of patches I've made for the wip-mlucy branch. It > splits the PEG code into several little modules which go in > module/ice-9/peg/. The original peg source file becomes very little. > At the end it finally loses its big eval-when wrapper. > > There's one part of this that I'm not satisfied with, which is that > define-nonterminal goes into module/ice-9/peg/string-peg.scm, which is > supposed to be solely for pegs-as-strings. The reason for this is that > I got compiler errors if I didn't do this, and I couldn't figure out > how to stop them. I would appreciate it if someone would take a look > and try to find what I missed. > > Also, a note about future ideas - the current PEG code can only parse > strings. However, there is almost nothing string-specific about the > parsing code - just a few calls to string-ref and substring in > codegen.scm. I'd like to see this extended to parse vectors filled > with arbitrary objects. This would let you use a tokenizer with it, > which is the easiest way to implement C correctly, and also probably > the easiest way to store line number information with tokens, which is > necessary for ultimately giving good error messages from PEG parsers. > > Thanks, > Noah > --20cf307cfcd8edfc14049dda685a Content-Type: application/octet-stream; name="0001-Document-PEG-Internals.patch" Content-Disposition: attachment; filename="0001-Document-PEG-Internals.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gkypnpbu7 RnJvbSBhYjk3MzlmMjg1NWNiNDA1MWZmMjQwMjlkODRjZjU5NjMxZjIwNWEzIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBOb2FoIExhdmluZSA8bmxhdmluZUBoYXZlcmZvcmQuZWR1PgpE YXRlOiBTdW4sIDYgTWFyIDIwMTEgMjA6MjQ6MTMgLTA1MDAKU3ViamVjdDogW1BBVENIXSBEb2N1 bWVudCBQRUcgSW50ZXJuYWxzCgogKiBkb2MvcmVmL2FwaS1wZWcudGV4aTogYWRkIGEgbWFudWFs IHNlY3Rpb24gYWJvdXQgdGhlIFBFRyBpbnRlcm5hbHMuCi0tLQogZG9jL3JlZi9hcGktcGVnLnRl eGkgfCAgIDcxICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr KysrCiAxIGZpbGVzIGNoYW5nZWQsIDcxIGluc2VydGlvbnMoKyksIDAgZGVsZXRpb25zKC0pCgpk aWZmIC0tZ2l0IGEvZG9jL3JlZi9hcGktcGVnLnRleGkgYi9kb2MvcmVmL2FwaS1wZWcudGV4aQpp bmRleCBiNTMxMzlkLi4wYzgzMzY1IDEwMDY0NAotLS0gYS9kb2MvcmVmL2FwaS1wZWcudGV4aQor KysgYi9kb2MvcmVmL2FwaS1wZWcudGV4aQpAQCAtMzUsNiArMzUsNyBAQCByZWZlcmVuY2UsIGFu ZCBhIHR1dG9yaWFsLgogKiBQRUcgU3ludGF4IFJlZmVyZW5jZTo6CiAqIFBFRyBBUEkgUmVmZXJl bmNlOjoKICogUEVHIFR1dG9yaWFsOjoKKyogUEVHIEludGVybmFsczo6CiBAZW5kIG1lbnUKIAog QG5vZGUgUEVHIFN5bnRheCBSZWZlcmVuY2UKQEAgLTkyMSwzICs5MjIsNzMgQEAgaW5zdGFudGlh dGlvbiBvZiBhIGNvbW1vbiBwYXR0ZXJuIGZvciBtYXRjaGluZyBzeW50YWN0aWNhbGx5IGlycmVs ZXZhbnQKIGluZm9ybWF0aW9uLiAgU2luY2UgaXQncyB0YWdnZWQgd2l0aCBAY29kZXs8fSBhbmQg ZW5kcyB3aXRoIEBjb2Rleyp9IGl0CiB3b24ndCBjbHV0dGVyIHVwIHRoZSBwYXJzZSB0cmVlcyAo YWxsIHRoZSBlbXB0eSBsaXN0cyB3aWxsIGJlIGRpc2NhcmRlZAogZHVyaW5nIHRoZSBjb21wcmVz c2lvbiBzdGVwKSBhbmQgaXQgd2lsbCBuZXZlciBjYXVzZSBwYXJzaW5nIHRvIGZhaWwuCisKK0Bu b2RlIFBFRyBJbnRlcm5hbHMKK0BzdWJzZWN0aW9uIFBFRyBJbnRlcm5hbHMKKworQSBQRUcgcGFy c2VyIHRha2VzIGEgc3RyaW5nIGFzIGlucHV0IGFuZCBhdHRlbXB0cyB0byBwYXJzZSBpdCBhcyBh IGdpdmVuCitub250ZXJtaW5hbC4gVGhlIGtleSBpZGVhIG9mIHRoZSBQRUcgaW1wbGVtZW50YXRp b24gaXMgdGhhdCBldmVyeQorbm9udGVybWluYWwgaXMganVzdCBhIGZ1bmN0aW9uIHRoYXQgdGFr ZXMgYSBzdHJpbmcgYXMgYW4gYXJndW1lbnQgYW5kCithdHRlbXB0cyB0byBwYXJzZSB0aGF0IHN0 cmluZyBhcyBpdHMgbm9udGVybWluYWwuIFRoZSBmdW5jdGlvbnMgYWx3YXlzCitzdGFydCBmcm9t IHRoZSBiZWdpbm5pbmcsIGJ1dCBhIHBhcnNlIGlzIGNvbnNpZGVyZWQgc3VjY2Vzc2Z1bCBpZiB0 aGVyZQoraXMgbWF0ZXJpYWwgbGVmdCBvdmVyIGF0IHRoZSBlbmQuCisKK1RoaXMgbWFrZXMgaXQg ZWFzeSB0byBtb2RlbCBkaWZmZXJlbnQgUEVHIHBhcnNpbmcgb3BlcmF0aW9ucy4gRm9yCitpbnN0 YW5jZSwgY29uc2lkZXIgdGhlIFBFRyBncmFtbWFyIEBjb2RleyJhYiJ9LCB3aGljaCBjb3VsZCBh bHNvIGJlCit3cml0dGVuIEBjb2RleyhhbmQgImEiICJiIil9LiBJdCBtYXRjaGVzIHRoZSBzdHJp bmcgYGBhYicnLiBIZXJlJ3MgaG93Cit0aGF0IG1pZ2h0IGJlIGltcGxlbWVudGVkIGluIHRoZSBQ RUcgc3R5bGU6CisKK0BsaXNwCisoZGVmaW5lIChtYXRjaC1hbmQtYS1iIHN0cikKKyAgKG1hdGNo LWEgc3RyKQorICAobWF0Y2gtYiBzdHIpKQorQGVuZCBsaXNwCisKK0FzIHlvdSBjYW4gc2VlLCB0 aGUgdXNlIG9mIGZ1bmN0aW9ucyBwcm92aWRlcyBhbiBlYXN5IHdheSB0byBtb2RlbAorc2VxdWVu Y2luZy4gSW4gYSBzaW1pbGFyIHdheSwgb25lIGNvdWxkIG1vZGVsIEBjb2RleyhvciBhIGIpfSB3 aXRoCitzb21ldGhpbmcgbGlrZSB0aGUgZm9sbG93aW5nOgorCitAbGlzcAorKGRlZmluZSAobWF0 Y2gtb3ItYS1iIHN0cikKKyAgKG9yIChtYXRjaC1hIHN0cikgKG1hdGNoLWIgc3RyKSkpCitAZW5k IGxpc3AKKworSGVyZSB0aGUgc2VtYW50aWNzIG9mIGEgUEVHIEBjb2Rle29yfSBleHByZXNzaW9u IG1hcCBuYXR1cmFsbHkgb250bworU2NoZW1lJ3MgQGNvZGV7b3J9IG9wZXJhdG9yLiBUaGlzIGZ1 bmN0aW9uIHdpbGwgYXR0ZW1wdCB0byBydW4KK0Bjb2RleyhtYXRjaC1hIHN0cil9LCBhbmQgcmV0 dXJuIGl0cyByZXN1bHQgaWYgaXQgc3VjY2VlZHMuIE90aGVyd2lzZSBpdAord2lsbCBydW4gQGNv ZGV7KG1hdGNoLWIgc3RyKX0uCisKK09mIGNvdXJzZSwgdGhlIGNvZGUgYWJvdmUgd291bGRuJ3Qg cXVpdGUgd29yay4gV2UgbmVlZCBzb21lIHdheSBmb3IgdGhlCitwYXJzaW5nIGZ1bmN0aW9ucyB0 byBjb21tdW5pY2F0ZS4gVGhlIGFjdHVhbCBpbnRlcmZhY2UgdXNlZCBpcyBiZWxvdy4KKworQHN1 YnN1YmhlYWRpbmcgUGFyc2luZyBGdW5jdGlvbiBJbnRlcmZhY2UKKworQSBwYXJzaW5nIGZ1bmN0 aW9uIHRha2VzIHRocmVlIGFyZ3VtZW50cyAtIGEgc3RyaW5nLCB0aGUgbGVuZ3RoIG9mIHRoYXQK K3N0cmluZywgYW5kIHRoZSBwb3NpdGlvbiBpbiB0aGF0IHN0cmluZyBpdCBzaG91bGQgc3RhcnQg cGFyc2luZyBhdC4gSW4KK2VmZmVjdCwgdGhlIHBhcnNpbmcgZnVuY3Rpb25zIHBhc3MgYXJvdW5k IHN1YnN0cmluZ3MgaW4gcGllY2VzIC0gdGhlCitmaXJzdCBhcmd1bWVudCBpcyBhIGJ1ZmZlciBv ZiBjaGFyYWN0ZXJzLCBhbmQgdGhlIHNlY29uZCB0d28gZ2l2ZSBhCityYW5nZSB3aXRoaW4gdGhh dCBidWZmZXIgdGhhdCB0aGUgcGFyc2luZyBmdW5jdGlvbiBzaG91bGQgbG9vayBhdC4KKworUGFy c2luZyBmdW5jdGlvbnMgcmV0dXJuIGVpdGhlciAjZiwgaWYgdGhleSBmYWlsZWQgdG8gbWF0Y2gg dGhlaXIKK25vbnRlcm1pbmFsLCBvciBhIGxpc3Qgd2hvc2UgZmlyc3QgZWxlbWVudCBtdXN0IGJl IGFuIGludGVnZXIKK3JlcHJlc2VudGluZyB0aGUgZmluYWwgcG9zaXRpb24gaW4gdGhlIHN0cmlu ZyB0aGV5IG1hdGNoZWQgYW5kIHdob3NlIGNkcgorY2FuIGJlIGFueSBvdGhlciBkYXRhIHRoZSBm dW5jdGlvbiB3aXNoZXMgdG8gcmV0dXJuLCBvciAnKCkgaWYgaXQKK2RvZXNuJ3QgaGF2ZSBhbnkg bW9yZSBkYXRhLgorCitUaGUgb25lIGNhdmVhdCBpcyB0aGF0IGlmIHRoZSBleHRyYSBkYXRhIGl0 IHJldHVybnMgaXMgYSBsaXN0LCBhbnkKK2FkamFjZW50IHN0cmluZ3MgaW4gdGhhdCBsaXN0IHdp bGwgYmUgYXBwZW5kZWQgYnkgQGNvZGV7cGVnLXBhcnNlfS4gRm9yCitpbnN0YW5jZSwgaWYgYSBw YXJzaW5nIGZ1bmN0aW9uIHJldHVybnMgQGNvZGV7KDEzICgiYSIgImIiICJjIikpfSwKK0Bjb2Rl e3BlZy1wYXJzZX0gd2lsbCB0YWtlIEBjb2RleygxMyAoImFiYyIpKX0gYXMgaXRzIHZhbHVlLgor CitGb3IgZXhhbXBsZSwgaGVyZSBpcyBhIGZ1bmN0aW9uIHRvIG1hdGNoIGBgYWInJyB1c2luZyB0 aGUgYWN0dWFsCitpbnRlcmZhY2UuCisKK0BsaXNwCisoZGVmaW5lIChtYXRjaC1hLWIgc3RyIGxl biBwb3MpCisgICAoYW5kICg8PSAoKyBwb3MgMikgbGVuKQorICAgICAgICAoc3RyaW5nPSBzdHIg ImFiIiBwb3MgKCsgcG9zIDIpKQorICAgICAgICAobGlzdCAoKyBwb3MgMikgJygpKSkpIDsgd2Ug cmV0dXJuIG5vIGV4dHJhIGluZm9ybWF0aW9uCitAZW5kIGxpc3AKKworVGhlIGFib3ZlIGZ1bmN0 aW9uIGNhbiBiZSB1c2VkIHRvIG1hdGNoIGEgc3RyaW5nIGJ5IHJ1bm5pbmcKK0Bjb2RleyhwZWct cGFyc2UgbWF0Y2gtYS1iICJhYiIpfS4KLS0gCjEuNy40LjEKCg== --20cf307cfcd8edfc14049dda685a--