From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: HiPhish Newsgroups: gmane.lisp.guile.user Subject: Re: A macro containing a mini-macro? Date: Fri, 02 Nov 2018 23:32:10 +0100 Message-ID: <3896308.xfKRN2Y66f@aleksandar-ixtreme-m5740> References: <2093628.1NtSHukUaa@aleksandar-ixtreme-m5740> <87tvm95eq2.fsf@netris.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1541197847 1849 195.159.176.226 (2 Nov 2018 22:30:47 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 2 Nov 2018 22:30:47 +0000 (UTC) Cc: guile-user@gnu.org To: Mark H Weaver Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Fri Nov 02 23:30:43 2018 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gIhxi-0000Mj-32 for guile-user@m.gmane.org; Fri, 02 Nov 2018 23:30:42 +0100 Original-Received: from localhost ([::1]:53589 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gIhzo-00053s-Jy for guile-user@m.gmane.org; Fri, 02 Nov 2018 18:32:52 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:59408) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gIhzQ-00052f-O9 for guile-user@gnu.org; Fri, 02 Nov 2018 18:32:30 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gIhzK-0002GT-GZ for guile-user@gnu.org; Fri, 02 Nov 2018 18:32:27 -0400 Original-Received: from mout02.posteo.de ([185.67.36.66]:34215) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gIhzH-0002CF-42 for guile-user@gnu.org; Fri, 02 Nov 2018 18:32:20 -0400 Original-Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id B10E12400E5 for ; Fri, 2 Nov 2018 23:32:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.de; s=2017; t=1541197932; bh=kdRQr7MSy1Jc8xo/UnIcUSvJagg0wAvfeIVRZCy5/qI=; h=From:To:Cc:Subject:Date:From; b=k2myOYTvWTbSQvd9mqbxWEkd/U5SAh3S9F/7tueC19QAiJYk/lM6IwgyF9QZrWigX Bk66QukvHn/1uEC2veVwGuHbYU5MyKexhD9Iq68xnIhkOJHhl/bjoRshLR2QTDb96H QAvQ5bqcaO+zgBlaguWUWOF8TYMtwEKjApX0xcKXVorTjljZDx/jul51rOdynM5vAW M9n+0jilVn7MNxDA56dqNZuXTtKoDrKySFk/N69xjb9gajN57A/ZptsD8YNG6JKuLJ agLegR6RRvQnbIaD22KrccBBRb8NpV0MTg0MRnhyfNsgZP2MkD2hoZQa/ZE4+NGkSq cF6IudVv1zhDg== Original-Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 42mxdl67Q2z9rxH; Fri, 2 Nov 2018 23:32:11 +0100 (CET) In-Reply-To: <87tvm95eq2.fsf@netris.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 185.67.36.66 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:14957 Archived-At: Thank you very much for the in-depth explanation, and sorry for answering o= nly=20 now. I haven't been able to get around to it until now, and this is exactly= =20 what I was looking for. One more question if you feel like answering: where= =20 can I learn properly about Scheme macros? Metaprogramming is one of the mos= t=20 powerful features in Lisp, but I haven't come across a really good explanat= ion=20 of all its facets. On Saturday, 29 September 2018 02:28:37 CET Mark H Weaver wrote: > Hi, >=20 > HiPhish writes: > > Hello Schemers, > >=20 > > I have written a small macro for writing test specifications: > > (define-syntax test-cases > > =20 > > (syntax-rules () > > =20 > > ((_ title > > =20 > > (given (byte byte* ...)) > > ...) > > =20 > > (begin > > =20 > > (test-begin title) > > (call-with-values (=CE=BB () (open-bytevector-output-port)) > > =20 > > (=CE=BB (out get-bv) > > =20 > > (pack given out) > > (let ((received (get-bv)) > > =20 > > (expected (u8-list->bytevector '(byte byte* ...)))) > > =20 > > (test-assert (bytevector=3D? received expected))))) > > =20 > > ... > > (test-end title))))) > >=20 > > The idea is that I can specify a series of test cases where each case > > consists>=20 > > of an object and a sequence of bytes which this object is to be seriali= zed=20 to: > > (test-cases "Single precision floating point numbers" > > =20 > > (+3.1415927410125732 (#xCA #b01000000 #b01001001 #b00001111 > > #b11011011)) > > (-3.1415927410125732 (#xCA #b11000000 #b01001001 #b00001111 > >=20 > > #b11011011))) > >=20 > > This works fine, but sometimes there is a sequence of the same bytes and > > it > >=20 > > would be more readable if I could write something like this: > > ((make-vector 16 0) (#xDC (16 #x00))) > >=20 > > instead of writing out 16 times `#x00`. This would require being able to > > make a distinction in the pattern whether `byte` is of the pattern > >=20 > > byte > >=20 > > or > >=20 > > (count byte) > >=20 > > and if it's the latter construct a list of `count` `byte`s (via > > `(make-list > > count byte)` for example) and splice it in. This distinction needs to be > > made for each byte specification because I want to mix actual bytes and > > these "RLE- encoded" byte specifications. > >=20 > > So I guess what I'm looking for is to have a `syntax-rules` inside a > > `syntax- rules` in a way. Can this be done? >=20 > It cannot be done with pure 'syntax-rules' macros, but it can certainly > be done with procedural macros, sometimes called 'syntax-case' macros. > Procedural macros are quite general, allowing you to write arbitrary > Scheme code that runs at compile time to inspect the macro operands and > generate arbitrary code. >=20 > I'll describe how to do this with macros further down, but let me begin > with the simple approach. >=20 > As rain1 suggested, this can be accomplished most easily by writing a > normal procedure to convert your compact bytevector notation into a > bytevector, and then having your macro expand into code that calls that > procedure. Here's working code to do that: >=20 > --8<---------------cut here---------------start------------->8--- > (use-modules (ice-9 match) > (srfi srfi-1) > (rnrs bytevectors)) >=20 > (define (compact-bytevector segments) > (u8-list->bytevector > (append-map (match-lambda > ((count byte) (make-list count byte)) > (byte (list byte))) > segments))) >=20 > (define-syntax test-cases > (syntax-rules () > ((_ title > (given (seg ...)) > ...) > (begin > (test-begin title) > (call-with-values (=CE=BB () (open-bytevector-output-port)) > (=CE=BB (out get-bv) > (pack given out) > (let ((received (get-bv)) > (expected (compact-bytevector '(seg ...)))) > (test-assert (bytevector=3D? received expected))))) > ... > (test-end title))))) >=20 >=20 > scheme@(guile-user)> (compact-bytevector '(#xDC (16 #x00))) > $2 =3D #vu8(220 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0) > scheme@(guile-user)> ,expand (test-cases "Single precision floating point > numbers" ((make-vector 16 0) (#xDC (16 #x00)))) $3 =3D (begin > (test-begin > "Single precision floating point numbers") > (call-with-values > (lambda () (open-bytevector-output-port)) > (lambda (out get-bv) > (pack (make-vector 16 0) out) > (let ((received (get-bv)) > (expected (compact-bytevector '(220 (16 0))))) > (test-assert (bytevector=3D? received expected))))) > (test-end > "Single precision floating point numbers")) > scheme@(guile-user)> > --8<---------------cut here---------------end--------------->8--- >=20 > Now, suppose it was important to do more of this work at macro expansion > time. For example, if efficiency was a concern, it might not be > acceptable to postpone the conversion of '(#xDC (16 #x00)) into > #vu8(220 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0) until run time. >=20 > It turns out that pure 'syntax-rules' macros are turing complete, but > they are limited in the ways that they can inspect the syntax objects > given to them as operands. In particular, they cannot inspect atomic > expressions, except to compare them with the finite set of literals in > the first operand to 'syntax-rules'. This is not sufficient to > interpret an arbitrary integer literal. It could only be done with > 'syntax-rules' macros if the 'count' field were represented using a > finite set of literals and/or list structure. E.g. it could be done if > the count were represented as a list of decimal digits like (1 4 2) for > 142. >=20 > In cases like this, we would normally turn to procedural macros, > e.g. 'syntax-case' macros. Here's a straightforward approach, reusing > the 'compact-bytevector' procedure given above, but calling it at > compile time instead of at run time: >=20 > --8<---------------cut here---------------start------------->8--- > (use-modules (ice-9 match) > (srfi srfi-1) > (rnrs bytevectors)) >=20 > (define (compact-bytevector segments) > (u8-list->bytevector > (append-map (match-lambda > ((count byte) (make-list count byte)) > (byte (list byte))) > segments))) >=20 > (define-syntax compact-bytevector-literal > (lambda (stx) > (syntax-case stx () > ((_ (seg ...)) > (compact-bytevector (syntax->datum #'(seg ...))))))) >=20 > (define-syntax test-cases > (syntax-rules () > ((_ title > (given (seg ...)) > ...) > (begin > (test-begin title) > (call-with-values (=CE=BB () (open-bytevector-output-port)) > (=CE=BB (out get-bv) > (pack given out) > (let ((received (get-bv)) > (expected (compact-bytevector-literal (seg ...)))) > (test-assert (bytevector=3D? received expected))))) > ... > (test-end title))))) > --8<---------------cut here---------------end--------------->8--- >=20 > Here, instead of having 'test-cases' expand into a procedure call to > 'compact-bytevector', it expands into a *macro* call to > 'compact-bytevector-literal'. The latter is a procedural macro, which > calls 'compact-bytevector' at compile time. >=20 > This approach is sufficient in this case, but I sense in your question a > desire to be able to perform more general inspection on the macro > operands and generation of the resulting code. This particular example > is not ideally suited for this task, but the following example code > comes a bit closer: >=20 > --8<---------------cut here---------------start------------->8--- > (define (segment-syntax->u8-list stx) > (syntax-case stx () > ((count byte) > (every number? (syntax->datum #'(count byte))) ;optional guard > (make-list (syntax->datum #'count) > (syntax->datum #'byte))) > (byte > (number? (syntax->datum #'byte)) ;optional guard > (list (syntax->datum #'byte))))) >=20 > (define (compact-bytevector-syntax->bytevector stx) > (syntax-case stx () > ((seg ...) > (u8-list->bytevector > (append-map segment-syntax->u8-list > #'(seg ...)))))) >=20 > (define-syntax compact-bytevector-literal > (lambda (stx) > (syntax-case stx () > ((_ (seg ...)) > (compact-bytevector-syntax->bytevector #'(seg ...)))))) > --8<---------------cut here---------------end--------------->8--- >=20 > I've omitted the 'test-cases' macro here because it's unchanged from the > previous example. Here we have two normal procedures that use > 'syntax-case', which might be a bit confusing. These are procedures > that accept syntax objects as arguments, and return normal data > structures. >=20 > In contrast to the previous example, which used 'syntax->datum' on the > entire compact-bytevector-literal, in this example we inspect and > destruct the syntax object itself using 'syntax-case'. This would be > needed in the more general case where identifiers (i.e. variable > references) might occur in the syntax objects. >=20 > Hopefully this gives you some idea of what can be done, but I don't > think this is the best example to explore these possibilities, since in > this case the normal procedural approach in my first code excerpt above > is simplest and most likely sufficient. >=20 > Regards, > Mark