Re: [PATCH] add language/wisp to Guile?

["Marc Nieper-Wißkirchen" <marc@nieper-wisskirchen.de>]

Am Di., 28. Feb. 2023 um 05:27 Uhr schrieb Philip McGrath
<philip@philipmcgrath.com>:
>
> Hi,
>
> On Monday, February 27, 2023 2:26:47 AM EST Marc Nieper-Wißkirchen wrote:

[...]

> > Nevertheless, I am not sure whether it is relevant to the point I
> > tried to make.  The "#!r6rs" does not indicate a particular language
> > (so tools scanning for "#!r6rs" cannot assume that the file is indeed
> > an R6RS program/library).
>
> I think I had missed that some of your remarks are specifically  about the
> "#!r6rs" directive, not directives of the form "#!<identifier>" more generally.
> I agree that implementations have more responsibilities with respect to
> "#!r6rs", that the presence of "#!r6rs" in a file is not enough to conclude
> that the file is an R6RS program/library, and that a straightforward
> implementation of "#!r6rs" as reading like "#lang r6rs" in the manner of my
> previous examples would not conform to R6RS.

Yes, this summarizes it well.

> Also, on the broader question, my first preference would be for Guile to
> implement `#lang language/wisp`, not least to avoid the confusing subtleties
> here and the potential for humans to confuse `#!language/wisp` with a shebang
> line. I raise the possibility of `#!language/wisp` only as an alternative if
> people are more comfortable using a mechanism that R6RS specifically designed
> for implementation-defined extensions.

When wisp only changes the lexical syntax, `#!wisp` would be fine (and
it cannot be confused with a shebang line IMO because a shebang line
must begin with `#! ` or `#!/`.  However, the authors of the R6RS
clearly had minor changes of the lexical syntax in mind when they
introduced comments like `#!r6rs` or `#!chezscheme` (e.g. Chez Scheme
adds a syntax for gensyms).  As wisp radically changes how the text is
tokenized, something like `#!wisp` probably only follows the latter
but not the spirit of R6RS.

> Nonetheless, I'll try to explain why I think "#!r6rs" can be handled, and is
> handled by Racket, consistently with both "#lang r6rs" and the behavior
> specified in the report.
>
> >
> > Of course, R6RS gives implementations the freedom to modify the reader
> > in whatever way after, say, "#!foo-baz" was read.  Thus, "#!foo-baz"
> > could be defined to work like Racket's "#lang foo-baz," reading the
> > rest of the source as "(module ...)".  But as long as we stay within
> > the confines of R6RS, this will only raise an undefined exception
> > because, in general, "module" is not globally bound.
> >
>
> Before getting to the general point, specifically about "module" not being
> bound: in Racket, a root-level `module` form is handled quite similarly to the
> `library` form in R6RS, which says in 7.1 [1]:
>
> >>>> The names `library`, `export`, `import`, [...] appearing in the library
> syntax are part of the syntax and are not reserved, i.e., the same names can
> be used for other purposes within the library or even exported from or
> imported into a library with different meanings, without affecting their use in
> the `library` form.
>
> None of the libraries defined in R6RS export a binding for `library`: instead,
> the implementation must recognize it somehow, whether by handling it as a
> built-in or binding it in some environment not standardized by R6RS.
>
> (The `racket/base` library/language does in fact export a binding for `module`
> which can be used to create submodules with the same syntax as a root-level
> `module`, but that isn't relevant to the handling of a `root-level` module
> form itself.)

Sure, but not relevant.  I didn't say that module is bound at the
Racket top-level, only that an R6RS implementation wouldn't expect it
(and cannot interpret it because it is not bound).

> > I don't want to contradict you; I just mean that a plain "#!r6rs"
> > without a top-level language where "module" is bound is not equivalent
> > to "#lang" and that trying to switch to, say,  Elisp mode with
> > "#!elisp" would leave the boundaries of the Scheme reports (and when
> > this is done, this specific discussion is moot).
> >
> > [...]
> >
> > (It must be compatible with calling the procedures "read" and "eval"
> > directly, so "#!r6rs" must not wrap everything in some module form,
> > say.)
>
> Now I'll try to sketch Racket's handling of "#!r6rs" from an R6RS perspective.
> For the sake of a concrete example, lets consider this program:

It is obvious that one can do it; it is just outside the realm of R6RS
because the "non-conformant mode" can be any (even a C-interpreting
mode) that can listen to whatever magic numbers there may be in the
input.  That said, the use of "#!r6rs" as such a magic marker is not
in the spirit of the lexical syntax of R6RS (where it was introduced).
This has been my original point.  Doable, of course.

[...]

> > In an implementation that supports, say,
> > R6RS and R7RS, "#!r6rs" can only switch the lexical syntax but cannot
> > introduce forms that make the implementation change the semantics from
> > R7RS to R6RS, e.g., in the case of unquoted vector literals.
>
> I'm not very familiar with R7RS, and, quickly skimming R7RS Small, I didn't
> see a notion of directives other than `#!fold-case` and `#!no-fold-case`.
> (That's a bit ironic, given that the R6RS editors seem to have contemplated
> `#!r7rs` *before* they considered `#!r6rs`.) I think a similar technique could
> work in this case, though. From an R6RS perspective, at least, an
> implementation could implement a directive such that the `read` from `(rnrs)`
> would parse:
>
> >>>> #!r7rs #(1 2 3)
>
> as:
>
> >>>> (quote #(1 2 3))
>
> The other direction is a bit trickier, but the R7RS specification for `read`
> from `(scheme base)` does say that "implementations may support extended
> syntax to represent record types or other types that do not have datum
> representations." It seems an implementation could define a type "non-self-
> evaluating-vector" and have `read` from `(scheme base)` produce a value of
> that type when given:
>
> >>>> #!r6rs #(1 2 3)

This cannot work because the macros can detect the presence of quotes
before vector literals; the reader must not insert quotes that weren't
there.

The expander must be made aware of what the valid literal expressions
are.  In Racket, this can be done with #%datum, but this does not
easily translate to R6RS (because of the presence of unwrapped syntax
objects in R6RS).

> Presumably `eval` from `(scheme eval)` would raise an error if asked to
> evaluate such a datum, as it does if asked to evaluate an unquoted (), but
> `quote` from `(scheme base)` would arrange to replace such a datum with a
> vector.
>
> (I'm not at all sure that an implementation *should* do such a thing: I'm only
> trying to explain why I don't think the Scheme reports prohibit it.)

My point is that just switching the lexical syntax won't do it.  Of
course, you can interpret a leading `#!r7rs` as switching from
conformant to non-conformant R6RS mode, but this would again be a
misuse (I guess this is also the reason why Racket favors `#lang` over
`#!...`.

But even then, one has a problem because the change cannot be file
local.  If a library in R7RS mode exports a macro that is used in a
context with R6RS mode, the vector literals inserted by the macro
should still be interpreted in R7RS mode.  If we could have #%datum,
this wouldn't be a problem.

Thank you a lot for your insights and your detailed explanations,

Marc

> -Philip
>
> [1]: http://www.r6rs.org/final/html/r6rs/r6rs-Z-H-10.html#node_sec_7.1
> [2]: http://www.r6rs.org/final/html/r6rs-app/r6rs-app-Z-H-3.html#node_chap_A
> [3]: https://docs.racket-lang.org/r6rs/Running_Top-Level_Programs.html
> [4]: https://docs.racket-lang.org/r6rs/Installing_Libraries.html