Re: Precedence for reader extensions

[Mark H Weaver <mhw@netris.org>]

Mikael Djurfeldt <mikael@djurfeldt.com> writes:
> On Tue, Feb 19, 2013 at 12:33 AM, Mark H Weaver <mhw@netris.org> wrote:
>> Mikael Djurfeldt <mikael@djurfeldt.com> writes:
>>> I propose to simplify this to only two levels:
>>>
>>> 1. %read-hash-procedures
>>> 2. predefined syntax
>
> It turns out that the change I propose above was already implemented
> in read.c.  The effect just wasn't visible due to a bug in flush_ws
> which caused all #! to be erroneously removed if they exist as the
> outermost expression.

I'm not sure that I consider this a bug.  All of the tokens that
flush_ws removes can appear anywhere that whitespace is allowed, and are
considered whitespace to the caller (although reader directives may
modify the per-port reader options as a side-effect).

  #;<expr>     (sexp-comments)
  #!fold-case  (reader directives)
  #! ... !#    (shebang block comments)
  #| ... |#    (r6rs block comments)

> In the attached diff, I've fixed the flush_ws bug and cleaned up some
> garbage code in scm_read_sharp which was unreachable.
>
> Can I push this into the repository?

I'm uncomfortable with globally overriding standard read syntax.  In a
large scheme system such as Guile, there are many modules that use
'read' and expect it to act in accordance with standard lexical
conventions.

Therefore, I'd prefer to limit 'read-hash-extend' to adding new syntax
that would otherwise have been considered an error.  If you're going to
override standard read syntax, then I think it should only be done on a
per-port basis.

Therefore, I'd prefer a precedence closer to this:

  1. (possibly) per-port variant of %read-hash-procedures
  2. predefined syntax
  3. %read-hash-procedures

>> I don't think this would be sufficient.  The problem is that tokens of
>> the form "#!<symbol><delimiter>" have become standardized.  To name a
>> few examples, both R6RS and R7RS define the reader directives
>> #!fold-case and #!no-fold-case, R6RS has #!r6rs, and SRFI-105 has
>> #!curly-infix.  Guile also has #! ... !# block comments to help with the
>> handling of executable scripts.
>
> In what sense is it not sufficient?  In any case: The present diff
> doesn't remove any functionality or make performance worse.  It only
> removes some inconsistent behavior.  At the same time it allows
> support for mit-scheme #!optional and #!rest.

The problem with this approach is that it does not compose.  You want to
add #!optional and #!rest.  R6RS added #!r6rs, #!fold-case, and
#!no-fold-case.  SRFI-105 added #!curly-infix.  But there can be only
one read-hash-procedure for #!, and it's global to the entire system.

That's why I suggested a way to add new tokens of the form
"#!<symbol><delimiter>".  That way, you could add handlers for #!rest
and #!optional without interfering with the other #!<symbol><delimiter>
tokens.

What do you think?

    Regards,
      Mark