A lexical use-modules?

A lexical use-modules?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1539 bytes --]

Hi,

[ CC'ing guix-devel@ because this functionality could be useful in Guix
package definitions ]

Currently, use-modules cannot be meaningfully used inside procedures,
unless '(current-module)' is always the module in which the procedure
is defined.

I wondered if some kind of 'lexical use-modules' was possible, with
sufficient macroology and module reflection, and it looks like it is:

(use-modules (srfi srfi-1))
(define-syntax use-module/lexical
  ;; todo: integrate into (use-modules ...)?
  (lambda (s)
    (syntax-case s ()
      ((_ foo)
       (let* ((module-name (syntax->datum #'foo))
              (interface (resolve-interface module-name)))
         (define (binding->import name variable)
           (define name-syntax (datum->syntax s name))
           #`(define-syntax #,name-syntax (identifier-syntax (@ foo
#,(datum->syntax #'irrelevant name)))))
         #`(begin #,@(module-map binding->import interface)))))))

(define &exception 'top-level)
(let ()
  (use-module/lexical (ice-9 exceptions))
  (pk 'inner &exception raise-continuable))
;;; (inner #<record-type &exception> #<procedure raise-continuable (obj)>)

(pk 'outer &exception)
;;; (outer top-level)

(pk 'unbound-variable raise-continuable)
;; a backtrace!

Limitation: things like

(define (foo)
  (use-module/lexical (platform-specific-constants))
  (if on-linux?
      CONSTANT_ONLY_DEFINED_ON_LINUX
      CONSTANT_ONLY_DEFINED_ON_HURD))

won't work in cross-compilation contexts.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: A lexical use-modules?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1325 bytes --]

Stefan Israelsson Tampe schreef op zo 27-03-2022 om 11:53 [+0200]:
Hmm actually you only have lexical macros that reference module
variables. That is not lexical variables which have more optimisation
associated with it at least used to. But is better speedwise than the
hash lookups I suggested. Anyhow you can create a let with local
variables and set them from the module dynamically. Maybe wingos recent
work mean that your approach is best. Would be interesting to do some
tests to see. 

Optimisations are performed at expansion time:

(macroexpand
  #'(let ()
      (use-module/lexical (ice-9 exceptions))
      (do-stuff)))
;; the unused imports disappeared!
$6 = #<tree-il (call (toplevel do-stuff))> 

A variable is only dereferenced when it is used:

,optimize
  (let ()
    (use-module/lexical (ice-9 exceptions))
    (lambda () raise-continuable))
$7 = (lambda ()
  (@ (ice-9 exceptions) raise-continuable))

So lexical imports should be as fast as directly writing
(@ (foo) bar).  I don't see how creating new modules at runtime and
using 'module-ref' could make things faster (*) here, given that it just
seems like an extra layer of indirection and it probably prevents things
like inlining.

(*) I'm only counting runtime here, not compilation time.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: A lexical use-modules?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2114 bytes --]

[Re-added guile-devel# to CC]

Stefan Israelsson Tampe schreef op zo 27-03-2022 om 16:37 [+0200]:
> I mean that modul-ref is a hash-table lookup, and that's more
> expensive as you say. I suggested that lexical variables should be
> let bound
> because you can deduce more things with them e.g. in
> 
> (let ((a (@ x y)))    
>     (when (vector? a) code ...))
> 
> Then a is known to be a vector and the check can be avoided inside
> code ...
> You suggest to use (@ x y) everywhere which is not terrible,
> especially with wingo's recent work. But still not sure that all
> optimisations work for referencing (@ x y) all the time. 

I could do this, but:

  * this won't work for macros, so I would have to make a special case
    for them

  * the imported variables might not yet have been initialised.  E.g.:

    (lambda ()
      (use-modules/lexical (foobar))
      a-foo ; initially #false
      (initialise-foobar!)
      a-foo) ; now 'footastic'

    The problem with your proposal is that modifications to global
    variables would not be detected --- the idea of 'use-modules' is to
    import the _variables_, not their current _values_.

    Additionally, doing the latter would be backwards-incompatible.

  * not all imported variables might be used, so your proposal might
    make things worse.  E.g.:

    (define (foo)
      (use-modules (ice-9 exceptions))
      (if (do-stuff)
          'ok
          (raise (make-some-exception with variables from (ice-9
exceptions)))))

   In the normal case, the variables would not need to be dereferenced.

  * I just want to make lexical use-modules work and I'm not currently
    interested in (possibly futily) trying to squeezing out some extra
    % or ‰ of performance.

> Would be nice though to be able to select the vars to be bound and
> improve on compilation speed if there is a lot of baggage (large
> module and only few var used)

When making a patch for merging 'use-module/lexical' into 'use-
modules', I intend to preserve support for #:select.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: A lexical use-modules?

[Ludovic Courtès]

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

> I wondered if some kind of 'lexical use-modules' was possible, with
> sufficient macroology and module reflection, and it looks like it is:

I agree it would be useful.

Just yesterday wrote this (my goal here was to allow dynamic module
loading based on some condition);

--8<---------------cut here---------------start------------->8---
(define-syntax with-modules
  (syntax-rules ()
    "Dynamically load the given MODULEs at run time, making the chosen
bindings available within the lexical scope of BODY."
    ((_ ((module #:select (bindings ...)) rest ...) body ...)
     (let* ((iface (resolve-interface 'module))
            (bindings (module-ref iface 'bindings))
            ...)
       (with-modules (rest ...) body ...)))
    ((_ () body ...)
     (begin body ...))))
--8<---------------cut here---------------end--------------->8---

… which can be used like this:

--8<---------------cut here---------------start------------->8---
(with-modules (((fibers) #:select (spawn-fiber sleep))
               ((fibers channels)
                #:select (make-channel put-message get-message)))
  (let ((channel (make-channel)))
    (spawn-fiber
      …)))
--8<---------------cut here---------------end--------------->8---

Unlike ‘use-modules’, its clearly limited to the lexical scope of its
body.

IWBN to have a similar functionality in Guile proper, and it could
probably be made more efficient.

Ludo’.

Re: A lexical use-modules?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

Maxime Devos schreef op za 26-03-2022 om 20:21 [+0100]:
> (define-syntax use-module/lexical
>   ;; todo: integrate into (use-modules ...)?
>   (lambda (s)
>     (syntax-case s ()
>       ((_ foo)
>        (let* ((module-name (syntax->datum #'foo))
>               (interface (resolve-interface module-name)))
>          (define (binding->import name variable)
>            (define name-syntax (datum->syntax s name))
>            #`(define-syntax #,name-syntax (identifier-syntax (@ foo
> #,(datum->syntax #'irrelevant name)))))
>          #`(begin #,@(module-map binding->import interface)))))))

This probably does not interact perfectly with 'syntax-parameterize',
'bound-identifier=?'/'free-identifier=?' and 'syntax-local-binding',
though I guess it is good enough for most practical purposes.

Greetings,
MAxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]