Should psyntax pass through a compiled VM program?

Should psyntax pass through a compiled VM program?

[Neil Jerram]

I think that's the right eventual question, anyway.  The context is
running Alan Grover's mod_lisp-for-guile in 1.9.2.

The mod_lisp-for-guile code includes a use of read-hash-extend to
define a syntax for a compiled regular expression - so that you can
write things like

  (if (#m/^Error: +/ line)
      ...)

with a similar effect to Perl

  if (line ~= /^Error: +/)
    ...

So:

  (read-hash-extend #\m (lambda (c port)
                          ...
                          (lambda (string)
                            ...)))

In other words, the custom reader procedure returns another
procedure.  So the read code (for the above example) becomes

  (if (#<procedure (string)> line)
      ...)

In pre-VM Guile, (I assume) this worked because there wasn't a psyntax
pass, and because the evaluator regards procedures as self-evaluating.

But in VM Guile, the read code is

  (if (#<program ...> line)
      ...)

And psyntax errors when it hits the #<program>.

I've worked around this by rewriting the read-hash-extend code as

  (define-public (make-regexp-fn regexp-string)
    (lambda (string)
      ...))

  (read-hash-extend #\m (lambda (c port)
                          ...
                          `(make-regexp-fn ,regexp-string)))

but I wonder if there is a case for supporting the code as it was
before, by

- teaching psyntax to pass through a #<program> if it sees one

- making programs self-evaluating (and in fact they may already be so,
  I haven't checked).

Also, is it definitely correct to call read-hash-extend procedures
first, and do the psyntax pass later?  I guess it must be, as we need
to call the read-hash-extend procedure in order even to determine the
extent of the relevant lexeme.

Regards,
        Neil

Re: Should psyntax pass through a compiled VM program?

[Andy Wingo]

On Sun 13 Sep 2009 23:17, Neil Jerram <neil@ossau.uklinux.net> writes:

> I think that's the right eventual question, anyway.  The context is
> running Alan Grover's mod_lisp-for-guile in 1.9.2.
>
> The mod_lisp-for-guile code includes a use of read-hash-extend to
> define a syntax for a compiled regular expression - so that you can
> write things like
>
>   (if (#m/^Error: +/ line)
>       ...)
>
> with a similar effect to Perl
>
>   if (line ~= /^Error: +/)
>     ...
>
> So:
>
>   (read-hash-extend #\m (lambda (c port)
>                           ...
>                           (lambda (string)
>                             ...)))
>
> In other words, the custom reader procedure returns another
> procedure.

So what happens when you compile this file? You would have to serialize
that procedure somehow, a procedure that could be a closure, etc...

No, I think all macros have to return syntax and not values.

This particular case is a bit unfortunate, though. You really want to
compile the regex once, preferably at compile-time... ideally the regex
compiles to scheme, and thence to bytecode/native code, but that's not
how things are right now.

So yes, I think the best we can do is to do what you did:

>   (define-public (make-regexp-fn regexp-string)
>     (lambda (string)
>       ...))

And make-regexp-fn should memoize results.

> but I wonder if there is a case for supporting the code as it was
> before, by

I'm afraid not, because you need to be able to compile a file to
bytecode, write it out to disk, then load it later.

> Also, is it definitely correct to call read-hash-extend procedures
> first, and do the psyntax pass later?  I guess it must be, as we need
> to call the read-hash-extend procedure in order even to determine the
> extent of the relevant lexeme.

Right. More thought (and documentation!) is needed here, though.

Andy
-- 
http://wingolog.org/