Re: Proposal: Identifier properties for Guile

[Mikael Djurfeldt <mikael@djurfeldt.com>]

Dear Daphne,

What you propose sounds exciting! I also think that you have the right
background to pull this off in a nice way.

Obviously, Andy Wingo should give his view on this, but I thought that
your message should have some response. :-)

Best regards,
Mikael Djurfeldt

On Tue, Sep 10, 2024 at 11:08 PM Daphne Preston-Kendal
<dpk@nonceword.org> wrote:
>
> Hi!
>
> I want to implement identifier properties a la SRFI 213 in Guile’s
> psyntax.
> <https://srfi.schemers.org/srfi-213/srfi-213.html>
> These will be in R7RS Large (in the fascicle on macros coming out Real
> Soon Now), and are in fact the only major macro feature Guile is
> missing from R7 Large.
>
> I want to gather feedback on my proposed strategy to implement these
> into Guile before starting work. Along the way, I’ll explain some
> nuances of the semantics which the SRFI explains very badly (the
> fascicle on macros completely redid the text).
>
> First of all, why would you want identifier properties? As an
> admittedly pretty advanced macrologist, I’m convinced these are the
> best things since sliced bread. I’ve gathered some example use cases
> here:
> <https://codeberg.org/scheme/r7rs/wiki/Identifier-property-use-cases>
> and in fact the second use case is no longer a hypothetical as it was
> when I wrote that page, because I have actually implemented it:
> <https://codeberg.org/dpk/extensible-match>
> Getting that library working in Guile and Hoot is my main personal
> motivation for wanting this feature – apart from wanting to see an
> implementation support the R7-large spec which I wrote ;-)
>
> Identifier properties are fundamentally very similar to transformer
> bindings, and pose some of the same challenges. The main difference is
> that identifier properties are attached in addition to a binding,
> rather than changing the regular Scheme semantics of the binding.
> But like transformer bindings, it makes sense to divide them into
> top-level (module-level, in Guile) property definitions and local
> definitions.
>
> The latter are easier to deal with because they don’t need to touch
> the module system. The expander gets rid of them in the same way it
> gets rid of let-syntax; expanded code doesn’t need to care about them
> any more.
>
> Implementing this means changing the expander’s idea of a wrap so that
> it contains property bindings, much as has been done in Chez’s version
> of psyntax:
> <https://github.com/cisco/ChezScheme/blob/658e0b152abe3bc4ba889883d4dc218fef093aef/s/syntax.ss#L892>
> This is a non-obtrusive change entirely localized to the expander as
> it exists in a live Guile system. I don’t see any major issues here or
> have any questions.
>
> When we get to module-level property bindings, things get trickier.
> I’d like a bit of guidance here. First, to clarify the SRFI:
> identifier properties, as the name suggests, are attached to
> identifiers in the sense of define/import/export. If you import an
> identifier from module A which the unrelated module B has attached a
> property to, you won’t see that property – unless you *also* re-import
> the same identifier from module B. Identifiers are organized, though,
> by keys, which themselves look like identifiers but those identifiers
> are actually used only for their bindings in the sanse of
> ‘free-identifier=?’. (Keys have to be defined – the fallback to
> symbolic comparison doesn’t apply here.)
>
> To support this I will have to extend the definition of a module in
> boot-9.scm to add, effectively, a second obarray – this one mapping
> variables to mappings of bindings to property values, rather than
> variables to values directly. However, it looks to me like there is
> some duplicate definition between C and Scheme here, and I would also
> need to update libguile/modules.c to at least be aware of the new
> structure, and probably write the property mapping code in C there as
> well to make it work as I describe below. Is this a correct
> assumption?
>
> Looking at how syntax transformers work in the current code, the
> ‘define-property’ form will probably have to be considered primitive
> down to a fairly low level. As I understand it from looking at
> expansion and disassembly, (define-syntax x y) first expands into
> (define x (make-syntax-transformer something something y)), then on
> the way from Tree-IL to bytecode it becomes an intrinsic call which is
> effectively (define! (current-module) 'x) followed by a (set! 'x
> (make-syntax-transformer something something y)). It looks like there
> might be double definition here too: once in a direct route from
> Tree-IL to bytecode, another with the same source and target but via
> CPS.
>
> In any case, based on this, I will add a <toplevel-define-property>
> Tree-IL node which compiles to a new define-property! intrinsic the
> same way <toplevel-define> compiles to a define!. That primitive will
> add the var -> binding -> value mapping to the module. Since there is
> no set! operation on properties, I think everything can be done in one
> intrinsic. One minor question I will have to solve is how to represent
> the binding used for the key down at this level. I am sure there is a
> simple answer to this – possibly just a pair:
> (<original-module> . <var-name>)
>
> There is an alternative approach which looks easier, which would be to
> have define-property at the top level expand directly into a call to a
> new ‘module-define-property!’ procedure. But I assume there is a
> reason macro transformers and other definitions were done the way they
> are done …
>
> Also, because this reaches so far down into the internals of Guile, I
> assume it will be necessary to adapt some of this specially for Hoot
> as well. I haven’t looked at the Hoot sources yet – it might be an
> idea to wait until Andy Wingo has finished porting psyntax to Hoot
> before trying anything there.
>
> In any case, there is then one last step, which is to add code
> adding/merging properties when modules are imported into other
> modules. This should be a fairly simple hash table/alist/whatever
> union operation, but I haven’t looked into it yet.
>
> Since this does touch nearly every level of Guile’s compiler, I would
> appreciate some feedback and ideally the opportunity to consult
> occasionally while I’m working with someone who is more familiar with
> all of this code than I am.
>
> Many thanks in advance,
>
> --
> dpk (Daphne Preston-Kendal) ·· 12103 Berlin, Germany ·· http://dpk.io/
>   One Thing to name them all, One Thing to define them,
>   One Thing to place them in environments and bind them,
>   In the Lambda Order they are all first class.               — R2RS
>
>