Proposal: Identifier properties for Guile

[Daphne Preston-Kendal <dpk@nonceword.org>]

Hi!

I want to implement identifier properties a la SRFI 213 in Guile’s
psyntax.
<https://srfi.schemers.org/srfi-213/srfi-213.html>
These will be in R7RS Large (in the fascicle on macros coming out Real
Soon Now), and are in fact the only major macro feature Guile is
missing from R7 Large.

I want to gather feedback on my proposed strategy to implement these
into Guile before starting work. Along the way, I’ll explain some
nuances of the semantics which the SRFI explains very badly (the
fascicle on macros completely redid the text).

First of all, why would you want identifier properties? As an
admittedly pretty advanced macrologist, I’m convinced these are the
best things since sliced bread. I’ve gathered some example use cases
here:
<https://codeberg.org/scheme/r7rs/wiki/Identifier-property-use-cases>
and in fact the second use case is no longer a hypothetical as it was
when I wrote that page, because I have actually implemented it:
<https://codeberg.org/dpk/extensible-match>
Getting that library working in Guile and Hoot is my main personal
motivation for wanting this feature – apart from wanting to see an
implementation support the R7-large spec which I wrote ;-)

Identifier properties are fundamentally very similar to transformer
bindings, and pose some of the same challenges. The main difference is
that identifier properties are attached in addition to a binding,
rather than changing the regular Scheme semantics of the binding.
But like transformer bindings, it makes sense to divide them into
top-level (module-level, in Guile) property definitions and local
definitions.

The latter are easier to deal with because they don’t need to touch
the module system. The expander gets rid of them in the same way it
gets rid of let-syntax; expanded code doesn’t need to care about them
any more.

Implementing this means changing the expander’s idea of a wrap so that
it contains property bindings, much as has been done in Chez’s version
of psyntax:
<https://github.com/cisco/ChezScheme/blob/658e0b152abe3bc4ba889883d4dc218fef093aef/s/syntax.ss#L892>
This is a non-obtrusive change entirely localized to the expander as
it exists in a live Guile system. I don’t see any major issues here or
have any questions.

When we get to module-level property bindings, things get trickier.
I’d like a bit of guidance here. First, to clarify the SRFI:
identifier properties, as the name suggests, are attached to
identifiers in the sense of define/import/export. If you import an
identifier from module A which the unrelated module B has attached a
property to, you won’t see that property – unless you *also* re-import
the same identifier from module B. Identifiers are organized, though,
by keys, which themselves look like identifiers but those identifiers
are actually used only for their bindings in the sanse of
‘free-identifier=?’. (Keys have to be defined – the fallback to
symbolic comparison doesn’t apply here.)

To support this I will have to extend the definition of a module in
boot-9.scm to add, effectively, a second obarray – this one mapping
variables to mappings of bindings to property values, rather than
variables to values directly. However, it looks to me like there is
some duplicate definition between C and Scheme here, and I would also
need to update libguile/modules.c to at least be aware of the new
structure, and probably write the property mapping code in C there as
well to make it work as I describe below. Is this a correct
assumption?

Looking at how syntax transformers work in the current code, the
‘define-property’ form will probably have to be considered primitive
down to a fairly low level. As I understand it from looking at
expansion and disassembly, (define-syntax x y) first expands into
(define x (make-syntax-transformer something something y)), then on
the way from Tree-IL to bytecode it becomes an intrinsic call which is
effectively (define! (current-module) 'x) followed by a (set! 'x
(make-syntax-transformer something something y)). It looks like there
might be double definition here too: once in a direct route from
Tree-IL to bytecode, another with the same source and target but via
CPS.

In any case, based on this, I will add a <toplevel-define-property>
Tree-IL node which compiles to a new define-property! intrinsic the
same way <toplevel-define> compiles to a define!. That primitive will
add the var -> binding -> value mapping to the module. Since there is
no set! operation on properties, I think everything can be done in one
intrinsic. One minor question I will have to solve is how to represent
the binding used for the key down at this level. I am sure there is a
simple answer to this – possibly just a pair:
(<original-module> . <var-name>)

There is an alternative approach which looks easier, which would be to
have define-property at the top level expand directly into a call to a
new ‘module-define-property!’ procedure. But I assume there is a
reason macro transformers and other definitions were done the way they
are done …

Also, because this reaches so far down into the internals of Guile, I
assume it will be necessary to adapt some of this specially for Hoot
as well. I haven’t looked at the Hoot sources yet – it might be an
idea to wait until Andy Wingo has finished porting psyntax to Hoot
before trying anything there.

In any case, there is then one last step, which is to add code
adding/merging properties when modules are imported into other
modules. This should be a fairly simple hash table/alist/whatever
union operation, but I haven’t looked into it yet.

Since this does touch nearly every level of Guile’s compiler, I would
appreciate some feedback and ideally the opportunity to consult
occasionally while I’m working with someone who is more familiar with
all of this code than I am.

Many thanks in advance,

-- 
dpk (Daphne Preston-Kendal) ·· 12103 Berlin, Germany ·· http://dpk.io/
  One Thing to name them all, One Thing to define them,
  One Thing to place them in environments and bind them,
  In the Lambda Order they are all first class.               — R2RS