module trickery

module trickery

[Thomas Bushnell, BSG]

I need to evaluate expressions in a defined module.  I do this with:

(eval-in-module EXPR special-module)

special-module is very carefully constructed.  It gets variable
definitions from certain other places, all of which work just fine.  I
also need it to inherit syntax definitions from a specified module.  I
use the following to grab the relevant syntax:

(module-use! special-module (resolve-module '(python syntax)))

The (python syntax) module creates syntax with define-syntax, and does
nothing out of the ordinary.

When I evaluate expressions in special-module which use the syntax
defined in (python syntax), things go into an infinite loop.  The
macro I'm testing is blindingly simple and works fine in a normal
environment.

I am not wedded to this particular arrangement: anything that imports
the macros from one module to another works, provided I get to use
define-syntax.

What might I be doing wrong?



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Marius Vollmer]

tb@becket.net (Thomas Bushnell, BSG) writes:

> (module-use! special-module (resolve-module '(python syntax)))
> 
> The (python syntax) module creates syntax with define-syntax, and does
> nothing out of the ordinary.
> 
> When I evaluate expressions in special-module which use the syntax
> defined in (python syntax), things go into an infinite loop.  The
> macro I'm testing is blindingly simple and works fine in a normal
> environment.

Please try what happens when you also do

  (use-modules (ice-9 syncase))
  (module-use! special-module (resolve-module '(ice-9 syncase)))
  (set-module-transformer! special-module syncase)

The 'module transformer' needs to be set up so that syntax-case
expansion can work.

> What might I be doing wrong?

Nothing, in fact.  The syntax-case and modules are not really done
yet, I'm afraid.

-- 
GPG: D5D4E405 - 2F9B BCCC 8527 692A 04E3  331E FAF8 226A D5D4 E405


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Thomas Bushnell, BSG]

Marius Vollmer <mvo@zagadka.ping.de> writes:

> tb@becket.net (Thomas Bushnell, BSG) writes:
> 
> > (module-use! special-module (resolve-module '(python syntax)))
> > 
> > The (python syntax) module creates syntax with define-syntax, and does
> > nothing out of the ordinary.
> > 
> > When I evaluate expressions in special-module which use the syntax
> > defined in (python syntax), things go into an infinite loop.  The
> > macro I'm testing is blindingly simple and works fine in a normal
> > environment.
> 
> Please try what happens when you also do
> 
>   (use-modules (ice-9 syncase))
>   (module-use! special-module (resolve-module '(ice-9 syncase)))
>   (set-module-transformer! special-module syncase)
> 
> The 'module transformer' needs to be set up so that syntax-case
> expansion can work.

No discernable effect.  Here is exactly what I have so far...

The macros are defined inside (define-module (python syntax)).

Then I want to incorporate those macros into my special module; which
is set up as follows:

(define python-toplevel (make-module))
(beautify-user-module! python-toplevel)
(module-use! python-toplevel (resolve-module '(python syntax)))
(module-use! python-toplevel (resolve-module '(ice-9 syncase)))
(set-module-transformer! python-toplevel syncase)

;;; Insert the definitions of the (python __builtin__) module
;;; into python-toplevel.  Note that this is not an import: we
;;; do *not* share bindings with the module.  That's because we
;;; shouldn't allow sets here to blow them away.
(module-for-each
 (lambda (name variable)
   (module-define! python-toplevel name (variable-ref variable)))
 (resolve-module '(python __builtin__)))


Then I want to evaluate expressions inside this module with the
following:

(eval-in-module SEXP python-toplevel)


I'm quite flexible here... anything that lets me set up the relevant
module and do eval-in-module will be fine by me.  Switching from Guile
1.4 to the latest 1.5 test release is not unacceptible, if it will
really work.

Thomas


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Rob Browning]

Marius, is eval-in-module the 1.4 equivalent of the current two arg
eval?  Also, how much (if you know off the top of your head) has
changed from 1.4 to 1.5 wrt to syntax-case?  Could your changes to
support/use export-syntax be relevant here?

Thomas, don't know if this is actually germane, but in case it helps,
I played around with syncase a bit a while back (when messing with
hobbit and when fixing syncase's gensym), and though much of what I
figured out has faded a bit.  One gross-level detail I recall is that
using (ice-9 syncase) redefines eval.  pssyntax.ss requires the scheme
implementation provide an eval that doesn't try to expand any further
(via calling sc-expand) whenever it's given a form whose car is
"noexpand".  See the bottom of syncase.scm for the relevant bit of
code.  Also note the hack used to make syncase work right in slib
which involves setting the '*sc-expander* property of 'define in that
module to '(define).

Hopefully we'll be unifying the various macro systems in the not too
distant future as part of our evaluation "clean up", and this will all
be a bit more straightforward.

> I'm quite flexible here... anything that lets me set up the relevant
> module and do eval-in-module will be fine by me.  Switching from
> Guile 1.4 to the latest 1.5 test release is not unacceptible, if it
> will really work.

Don't know offhand, but if you're up to it and it's not too much work,
the more people that can try out the tarfile, the better :>

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Marius Vollmer]

tb@becket.net (Thomas Bushnell, BSG) writes:

> I'm quite flexible here... anything that lets me set up the relevant
> module and do eval-in-module will be fine by me.  Switching from
> Guile 1.4 to the latest 1.5 test release is not unacceptible, if it
> will really work.

Here is something that works with 1.5.8 and CVS HEAD.  I don't know
whether it works with the 1.4 series, tho.  Is it similar enough to
what you are doing?

    (define-module (python syntax)
      :use-syntax (ice-9 syncase))

    (define-syntax foo
      (syntax-rules ()
        ((_ a)
         (display a))))

    (define-module (test))

    (define python-toplevel (make-module))
    (beautify-user-module! python-toplevel)
    (module-use! python-toplevel (resolve-module '(python syntax)))

    (eval '(foo "hi\n") python-toplevel)

Running it produces:

    $ guile -s x.scm
    hi

Thus, it seems that no additional tricks are needed, which is a good
thing.

-- 
GPG: D5D4E405 - 2F9B BCCC 8527 692A 04E3  331E FAF8 226A D5D4 E405


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Thomas Bushnell, BSG]

Marius Vollmer <mvo@zagadka.ping.de> writes:

> tb@becket.net (Thomas Bushnell, BSG) writes:
> 
> > I'm quite flexible here... anything that lets me set up the relevant
> > module and do eval-in-module will be fine by me.  Switching from
> > Guile 1.4 to the latest 1.5 test release is not unacceptible, if it
> > will really work.
> 
> Here is something that works with 1.5.8 and CVS HEAD.  I don't know
> whether it works with the 1.4 series, tho.  Is it similar enough to
> what you are doing?

Incidentally, the release date is last weekend, right?

:)


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Thomas Bushnell, BSG]

Marius Vollmer <mvo@zagadka.ping.de> writes:

> tb@becket.net (Thomas Bushnell, BSG) writes:
> 
> > I'm quite flexible here... anything that lets me set up the relevant
> > module and do eval-in-module will be fine by me.  Switching from
> > Guile 1.4 to the latest 1.5 test release is not unacceptible, if it
> > will really work.
> 
> Here is something that works with 1.5.8 and CVS HEAD.  I don't know
> whether it works with the 1.4 series, tho.  Is it similar enough to
> what you are doing?

Ok, I installed 1.5.8.  Here's a simple failure case with very little
trickery.  Loading the following file causes an apparent infinite
loop:

(define-module (foo)
  :use-syntax (ice-9 syncase))

(define-syntax whoo!
  (syntax-rules ()
    ((_ a b)
     (set! a b))))

(define-public (print-two)
  (let ((a 0))
    (whoo! a 2)
    (display a)))

(define-module (test))
(use-modules (foo))
(print-two)



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Marius Vollmer]

tb@becket.net (Thomas Bushnell, BSG) writes:

> Here's a simple failure case with very little trickery.

Yep, it's a real bug somewhere, and I'm afraid it is not easy to fix.

The syncase implementation that we are using assumes that there is a
single toplevel and doesn't really work well with our multiple ones.

What I think is happening is this: in order for syncase to do its job,
properties are placed on symbols.  Since we have multiple global
namespaces, a symbol does not uniquely name a singel macro.  So we put
that property on the variable object that is bound to the symbol in
the current module.  When the property is accessed, we again consult
the variable that is bound to the symbol in the current module.  This
will only work when we find the same variable both times.

However, at the time that print-two is called in your example, the
current module is (test), not (foo).  Since the code in print-two is
expanded lazily, the property lookup will look in the wrong module and
find the wrong (or no) variable.  This makes syncase go into a loop,
for reasons that I don't know.

A workaround is to export all macros along with the procedures that
use them.  This ensures that the same variable is found regardless
what module is current.  (The workaround sucks, of coure, but it
works.)

For your example:

    (define-module (foo)
      :use-syntax (ice-9 syncase)
      :export (whoo!)      ;; <- new

    (define-syntax whoo!
      (syntax-rules ()
        ((_ a b)
         (set! a b))))

    (define-public (print-two)
      (let ((a 0))
        (whoo! a 2)
        (display a)))

    (define-module (test))
    (use-modules (foo))
    (print-two)

If you can export your macros in this way, that would buy you time
until we have a real fix.

-- 
GPG: D5D4E405 - 2F9B BCCC 8527 692A 04E3  331E FAF8 226A D5D4 E405


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Tom Lord]

	> [modules vs. macros]

You might be able to bum the module-safe `syntax-rules' implementation
from systas.

-t



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Marius Vollmer]

Tom Lord <lord@regexps.com> writes:

> 	> [modules vs. macros]
> 
> You might be able to bum the module-safe `syntax-rules' implementation
> from systas.

Yes, it looks very nice and clean.

-- 
GPG: D5D4E405 - 2F9B BCCC 8527 692A 04E3  331E FAF8 226A D5D4 E405


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Marius Vollmer]

Marius Vollmer <mvo@zagadka.ping.de> writes:

> If you can export your macros in this way, that would buy you time
> until we have a real fix.

Or you can try the attached patch.  That should fix the bug for real.

Index: ice-9/syncase.scm
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/ice-9/syncase.scm,v
retrieving revision 1.18.2.7
diff -u -r1.18.2.7 syncase.scm
--- ice-9/syncase.scm	12 Mar 2002 21:21:15 -0000	1.18.2.7
+++ ice-9/syncase.scm	4 Sep 2002 20:10:11 -0000
@@ -65,10 +65,20 @@
 
 \f
 
+(define expansion-eval-closure (make-fluid))
+
+(define (env->eval-closure env)
+  (or (and env
+	   (car (last-pair env)))
+      (module-eval-closure the-root-module)))
+
 (define sc-macro
   (procedure->memoizing-macro
     (lambda (exp env)
-      (sc-expand exp))))
+      (with-fluids ((expansion-eval-closure (env->eval-closure env)))
+        (sc-expand exp)))))
+
+(fluid-set! expansion-eval-closure (env->eval-closure #f))
 
 ;;; Exported variables
 
@@ -135,13 +145,12 @@
 			  '())))
 
 (define the-syncase-module (current-module))
+(define the-syncase-eval-closure (module-eval-closure the-syncase-module))
 
 (define (putprop symbol key binding)
-  (let* ((m (current-module))
-	 (v (or (module-variable m symbol)
-		(module-make-local-var! m symbol))))
+  (let* ((v ((fluid-ref expansion-eval-closure) symbol #t)))
     (if (symbol-property symbol 'primitive-syntax)
-	(if (eq? (current-module) the-syncase-module)
+	(if (eq? (fluid-ref expansion-eval-closure) the-syncase-eval-closure)
 	    (set-object-property! (module-variable the-root-module symbol)
 				  key
 				  binding))
@@ -149,8 +158,7 @@
     (set-object-property! v key binding)))
 
 (define (getprop symbol key)
-  (let* ((m (current-module))
-	 (v (module-variable m symbol)))
+  (let* ((v ((fluid-ref expansion-eval-closure) symbol #f)))
     (and v (or (object-property v key)
 	       (let ((root-v (module-local-variable the-root-module symbol)))
 		 (and (equal? root-v v)


-- 
GPG: D5D4E405 - 2F9B BCCC 8527 692A 04E3  331E FAF8 226A D5D4 E405


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Thomas Bushnell, BSG]

Marius Vollmer <mvo@zagadka.ping.de> writes:

> A workaround is to export all macros along with the procedures that
> use them.  This ensures that the same variable is found regardless
> what module is current.  (The workaround sucks, of coure, but it
> works.)

As it happens, I had asked a while ago if there was a version of
define-syntax that did an export the way define-public does.  I
actually would *prefer* that these macros be public.  

So far from being a workaround, this is an all-round win.

You also posted a patch for syncase that should work for unexported
macros; will that be in 1.6?




_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Marius Vollmer]

tb@becket.net (Thomas Bushnell, BSG) writes:

> As it happens, I had asked a while ago if there was a version of
> define-syntax that did an export the way define-public does.

Hmm, should be a simple matter of

  (define-macro (define-syntax-public id . rest)
    `(begin 
      (define-syntax ,id ,@rest)
      (export ,id)))

Does that work?
 
> You also posted a patch for syncase that should work for unexported
> macros; will that be in 1.6?

No, I'm afraid not.

-- 
GPG: D5D4E405 - 2F9B BCCC 8527 692A 04E3  331E FAF8 226A D5D4 E405


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Thomas Bushnell, BSG]

Marius Vollmer <mvo@zagadka.ping.de> writes:

> Hmm, should be a simple matter of
> 
>   (define-macro (define-syntax-public id . rest)
>     `(begin 
>       (define-syntax ,id ,@rest)
>       (export ,id)))
> 
> Does that work?

Yeah, so does :export in the define-module form.  Neither of these is
mentioned in any documentation I could find... :(


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: module trickery

[Marius Vollmer]

tb@becket.net (Thomas Bushnell, BSG) writes:

> Yeah, so does :export in the define-module form.  Neither of these is
> mentioned in any documentation I could find... :(

Yes, much of the module system documentation is folklore...

(I will add define-syntax-public to (ice-9 synase).)

-- 
GPG: D5D4E405 - 2F9B BCCC 8527 692A 04E3  331E FAF8 226A D5D4 E405


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel