Re: macros, procedure->macro

Re: macros, procedure->macro

[Dirk Herrmann]

On 14 Jul 2002, Neil Jerram wrote:

> >>>>> "Dirk" == Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:
> 
>     Dirk> On 13 Jul 2002, Neil Jerram wrote:
>     >> Thanks, that's helpful.  So we won't support references to a macro
>     >> that is defined in a following top-level form, as in:
>     >> 
>     >> (define-macro (foo x) `(list ,(bar x) ,x))
>     >> (define-macro (bar x) `(* ,x ,x))
>     >> 
>     >> or is there a cunning plan that still allows us to support this?
> 
>     Dirk> I don't see why recursive macros shouldn't be possible.
[...]
>     Dirk> However, there are things that won't work any more:
>     Dirk>   (define (foo) (bar))
>     Dirk>   (define-macro (bar) #f)
>     Dirk>   (foo)
> 
> For the purposes of current discussion, I don't see the difference
> between this example and my one above.  (Note that my `(bar x)' above
> is unquoted, so equivalent to your `(bar)'.)  What do you intend to be
> the key difference?  (Perhaps you meant `(define foo (bar))'?)

To clarify I would like to re-state my answer to a previous mail to you:

On 10 Jul 2002, Neil Jerram wrote:

> I think the perhaps the point is not at what stage macro expansion
> happens, but how universal it is.  In other words, is macro expansion
> performed universally like reading, and so only blocked by quoting, or
> is it performed like evaluation, and so blocked/delayed by any number
> of special forms including lambda and if.

Macro expansion is neither performed like reading, nor like evaluation.
Macro expansion is blocked by quoting _and_ macro definitions.  It is not
blocked by lambda or other special forms.  Since macro expansion is
blocked by macro definitions, recursive macro definitions are possible.

Given this, it should be clear why
  (define (foo) (bar))
  (define-macro (bar) #f)
  (foo)
will not work:  The first line is expanded at once.  In contrast,
  (define-macro (foo x) `(list ,(bar x) ,x))
  (define-macro (bar x) `(* ,x ,x))
will work, because the macro definition will not be expanded.

However, it may be that different macro implementations with different
behaviours can co-exist.  For example, I don't see why you shouldn't be
able to mix calls to procedure->memoizing-macro with calls to
define-macro.  Then, for each of the different macro systems the behaviour
would have to be defined separately.  Not all possible systems will comply
with the demand for a system that supports compilation and efficient
execution, though.

Best regards,
Dirk Herrmann


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Marius Vollmer]

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

>   (define-macro (foo x) `(list ,(bar x) ,x))
>   (define-macro (bar x) `(* ,x ,x))
> will work, because the macro definition will not be expanded.

Careful, Dirk. :) The term 'macro definition' is not really helpful
here, I think.  On the lowest level, a macro can be thought of as an
ordinary function that is called during compilation.  In order to run
this function, the macros that it uses must be expanded, like in any
other function.  The literal data structures that are embedded in the
function are not expanded, of course (just as they aren't for any
other function).

However, a 'quasi-quote' form is not a literal data structure, while
'quote' is.  'foo' from above is equivalent to

    (define-macro (foo x) (list 'list (bar x) x))

Thus, the function of the 'foo' macro makes use of the 'bar' macro,
which therefore needs to be defined when the compiler macro-expands
'foo's function, which might be just after it reads it in.  Had it
been

    (define-macro (foo x) (list 'list '(bar x) x))

then '(bar x)' would be a literal data structure and the definition
for 'bar' would be needed when the 'foo' macro is expanded, which
happens probably after 'bar' has been defined.


(Dirk, I don't think you need this detailed lecturing, but I couldn't
stop myself.)

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Neil Jerram]

>>>>> "Dirk" == Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

    Dirk> To clarify I would like to re-state my answer to a previous mail to you:

Thanks - the following is exactly what I've been looking for.  (I'm
pretty sure that I have not previously received any such email,
though.)

    Dirk> Macro expansion is neither performed like reading, nor like evaluation.
    Dirk> Macro expansion is blocked by quoting _and_ macro definitions.  It is not
    Dirk> blocked by lambda or other special forms.  Since macro expansion is
    Dirk> blocked by macro definitions, recursive macro definitions are possible.

This sounds good.  But, given that the code for a macro definition
gets wrapped in a lambda, perhaps it would be simpler to say that all
lambdas block macroexpansion, and that macroexpansion of the code
inside the lambda happens when the lambda is first used.  This seems
well defined and easy to understand to me.

    Dirk> Given this, it should be clear why
    Dirk>   (define (foo) (bar))
    Dirk>   (define-macro (bar) #f)
    Dirk>   (foo)
    Dirk> will not work:  The first line is expanded at once.  In contrast,
    Dirk>   (define-macro (foo x) `(list ,(bar x) ,x))
    Dirk>   (define-macro (bar x) `(* ,x ,x))
    Dirk> will work, because the macro definition will not be expanded.

If we follow the all lambdas idea, then the `define' example would
work as well.  Is there any reason why this would be a _bad_ thing?

    Dirk> However, it may be that different macro implementations with different
    Dirk> behaviours can co-exist.  For example, I don't see why you shouldn't be
    Dirk> able to mix calls to procedure->memoizing-macro with calls to
    Dirk> define-macro.  Then, for each of the different macro systems the behaviour
    Dirk> would have to be defined separately.  Not all possible systems will comply
    Dirk> with the demand for a system that supports compilation and efficient
    Dirk> execution, though.

In general, yes.  I don't understand your specific example, though, as
define-macro is implemented using procedure->memoizing-macro, so they
are the same implementation.

        Neil


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On 15 Jul 2002, Neil Jerram wrote:

> >>>>> "Dirk" == Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:
> 
>     Dirk> Macro expansion is neither performed like reading, nor like evaluation.
>     Dirk> Macro expansion is blocked by quoting _and_ macro definitions.  It is not
>     Dirk> blocked by lambda or other special forms.  Since macro expansion is
>     Dirk> blocked by macro definitions, recursive macro definitions are possible.
> 
> This sounds good.  But, given that the code for a macro definition
> gets wrapped in a lambda, perhaps it would be simpler to say that all
> lambdas block macroexpansion, and that macroexpansion of the code
> inside the lambda happens when the lambda is first used.  This seems
> well defined and easy to understand to me.

Well, I have to correct myself (actually, Marius has already done so):  I
confused define-macro and its mechanisms with define-syntax. Certainly,
the following will not work:

>     Dirk>   (define (foo) (bar))
>     Dirk>   (define-macro (bar) #f)
>     Dirk>   (foo)

And _neither_ will

>     Dirk>   (define-macro (foo x) `(list ,(bar x) ,x))
>     Dirk>   (define-macro (bar x) `(* ,x ,x))

Sorry for the confusion (oh boy).  However, the following will work
(hopefully I am getting it right this time...):

      (use-syntax (ice-9 syncase))
      (define-syntax foo (syntax-rules () ((foo) (bar))))
      (define-syntax bar (syntax-rules () ((bar) (write "bar"))))
      (foo)

and here applys what I wanted to express, namely that the use of bar in
the line where foo is defined is not expanded at that moment.

> If we follow the all lambdas idea, then the `define' example would
> work as well.  Is there any reason why this would be a _bad_ thing?

First, it is confusing.  In the _current_ system, the following works,
since currently lambda blocks macro expansion:
   (define-macro (foo x) `(list ,(bar x) ,x))
   (define-macro (bar x) `(* ,x ,x))
   (foo)
Thus, we can define the following:
   (define-macro (foo) `'first)
   (define (bar) (foo))
   (define (yuck expand?) (if expand? (bar) #f))
   (yuck <some-input-data>)
   (define-macro (foo) `'second)
   (yuck #t) 
Now, what does the last expression deliver?  It depends on what the value
of <some-input-data> was.  This is not what I expect from a macro system.

Second, you can't optimize the execution phase by writing out expanded
code and read it in later:  Only code that has been executed once is
expanded.  How could you write out the expanded version of bar in the
above example? You can't since it depends on your value of
<some-input-data> what bar becomes.  Thus, you also can not provide a
pre-compiled version of bar.  A lot of compiler optimization techniques
are impossible to use, like function inlining etc.

Third, it complicates the execution phase, since you always have to be
aware of still unexpanded functions in the executor.

>     Dirk> However, it may be that different macro implementations with different
>     Dirk> behaviours can co-exist.  For example, I don't see why you shouldn't be
>     Dirk> able to mix calls to procedure->memoizing-macro with calls to
>     Dirk> define-macro.  Then, for each of the different macro systems the behaviour
>     Dirk> would have to be defined separately.  Not all possible systems will comply
>     Dirk> with the demand for a system that supports compilation and efficient
>     Dirk> execution, though.
> 
> In general, yes.  I don't understand your specific example, though, as
> define-macro is implemented using procedure->memoizing-macro, so they
> are the same implementation.

Well, in that paragraph above replace 'define-macro' with 'define-syntax'
and I hope things become clearer.

Best regards,
Dirk Herrmann


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On 7 Jul 2002, Marius Vollmer wrote:

> Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:
> 
> > So what is so special about the uses of procedure->macro at the places
> > above?  
> >
> > [checking for redefinitions.]
> 
> We need to restrict this redefining behavior of 'define-class' et all
> to the top-level.  Local class definitions should not redefine classes
> outside of their scope (that would lead to a funny version of dynamic
> scoping of classes, eew), and redefinitions directly in one scope
> should be an error, just like any other definitions.

OK, I have changed the definitions of define-generic and define-accessor
to only show their re-defining behaviour if executed on top-level.  I also
changed all three (define-class, define-generic and define-accessor) to
use mmacros, which then would not change their behaviour at all.  Finally,
I changed define-class in stklos.scm to also use a mmacro, which is also
safe if no other 'define-class' code uses macros any more.

That means, after I have checked these changes in, guile itself will be
clean of using the built-in so called "macros", which are not really
macros in the scheme sense.

> Redefinitions on the top-level do make sense and can be supported by a
> normal macro via explicit module manipulations, i.e.
[...]

I am not sure I like the way, goops is based on names instead of objects
at a lot of places.  For example, it is not possible to do the following:
  (use-modules (oop goops))
  (define-generic G)
  (define (get-generic) G)
  (define-class <foo> () (x #:accessor (get-generic)))
because instead of a value, define-class expects a symbol after the
#:accessor keyword.

Automatic redefinition of classes is IMO a critical issue:
Redefinition of a class means to change the structure of all objects
of the class that already exist.  This should be a well considered
step and should not happen automatically (I am not even sure whether
it is a good idea at all).  It is not too inconvenient to request the user
to call class-redefinition instead of define-class.  It will inhibit
accidential redefinitions.

Just my opinion...

Best regards
Dirk Herrmann


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Marius Vollmer]

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

> Automatic redefinition of classes is IMO a critical issue:
> Redefinition of a class means to change the structure of all objects
> of the class that already exist.  This should be a well considered
> step and should not happen automatically (I am not even sure whether
> it is a good idea at all).  It is not too inconvenient to request
> the user to call class-redefinition instead of define-class.  It
> will inhibit accidential redefinitions.

Class redefinitions are mainly useful during incremental development,
i.e. when you load a new version of a file that has already been
loaded previously.

I'd say it would be OK to reject (or warn about) redefinitions of any
kind when they occur in a single file, that is, when a name is defined
twice at two different locations in a file.

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On 7 Jul 2002, Marius Vollmer wrote:

> We need to restrict this redefining behavior of 'define-class' et all
> to the top-level.  Local class definitions should not redefine classes
> outside of their scope (that would lead to a funny version of dynamic
> scoping of classes, eew), and redefinitions directly in one scope
> should be an error, just like any other definitions.
> 
> For example,
> 
>     (let ((foo (find-class "<moo>")))
>       (define-class foo ...)
>       ...)
> 
> should not try to redefine the class that is the value of the location
> bound to foo by the let.  It should simply shadow the outer foo.

Just a side note:  define-class cannot be used other than on top-level.
The reason is, that define-class will define generic functions for
accessors that are not defined yet.  Getting this right in non-top-level
scopes seems quite difficult (if not impossible) to do cleanly.

> Redefinitions on the top-level do make sense and can be supported by a
> normal macro via explicit module manipulations, i.e.
> 
>     (define-class foo ...)
>     =>
>     (module-define-class (current-module) 'foo ...)
> 
> with
> 
>     (define (module-define-class mod sym ...)
>       (let* ((var (module-ensure-local-variable! mod 'foo))
>              (class (make-class ...)))
>         (if (variable-bound? var)
>             (redefine-class (variable-ref var) class))
>         (variable-set! var class)))

I am currently changing the definition of define-class accordingly.
However, I am not sure which of the following solutions to choose:

* use a helper function module-define-class.  This would have to be
  exported.  And, it could not be used like define-class, since you
  would not be able to pass an empty combination for an empty list
  of supers:  (module-define-class some-module some-name () ...)
  would not be allowed, since you would have to quote the ().

* use a helper macro module-define-class.  This would have to be
  exported as syntax.

* change the expanded code of define-class to hold all necessary
  instructions to do the module manipulations.  In this case, the
  mmacro would return the following:

  `(begin
     ;; define accessors
     ,@(pre-definitions (slots exp) env)
     ;; update the current-module
     (let ((class (class ,@(cddr exp) #:name ',name)))
       (let* ((var (module-ensure-local-variable!
                   (current-module) ',name))
              (old (and (variable-bound? var)
                   (variable-ref var))))
         (if (and old
                  (is-a? old <class>)
                  (memq <object> (class-precedence-list old)))
             (variable-set! var (class-redefinition old class))
             (variable-set! var class))))))))))))

Which solution should be used?  I will try to update define-generic and
define-accessor accordingly.

Best regards,
Dirk Herrmann


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On Wed, 10 Jul 2002, Dirk Herrmann wrote:

> I am currently changing the definition of define-class accordingly.
> However, I am not sure which of the following solutions to choose:
[...]
> * change the expanded code of define-class to hold all necessary
>   instructions to do the module manipulations.  In this case, the
>   mmacro would return the following:
[...]
> Which solution should be used?  I will try to update define-generic and
> define-accessor accordingly.

I have decided for the solution above to avoid having to export an
additional binding.  Now, define-class, define-generic and define-accessor
are now fixed such that they will work even if mmacros are expanded before
execution.

That means, scm_makmacro and procedure->macro are not used anymore within
guile.  If nobody objects, I will make them deprecated.  The next step is
to get rid of acros.

Best regards,
Dirk Herrmann


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Marius Vollmer]

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

> I have decided for the solution above to avoid having to export an
> additional binding.

I hope that we can avoid exporting bindings that are only used in
macro expansions.  I don't know exactly how (ice-9 syncase) deals with
this right now, but I'd say it is supposed to make the following work:

    (define-module (t1)
      :use-syntax (ice-9 syncase)
      :export-syntax foo)

    (define (bar) #t)

    (define-syntax foo
      (syntax-rules ()
        ((foo)
         (bar))))

    (define-module (t2)
      :use-syntax (ice-9 syncase)
      :use-module (t1))

    (foo)

Note that (t1) doesn't export bar, but the expansion of (foo) uses it.

I have some vague ideas of how to implement this... the basic one is
to add new syntax that allows one to refer to specific bindings from
specific modules, on a very low level.  For example, we could use

    (#:module-ref (t1) bar)

to get at the bar binding of the (t1) module, from anywhere.  Note
that we use a keyword in the operator position.

(ice-9 syncase) could then rewrite references to global variables to
use this form.

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On 1 Jul 2002, Gary Houston wrote:

> > From: Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de>
> > Date: Mon, 1 Jul 2002 21:56:23 +0200 (CEST)
> 
> > 1) Some macro expert should check that replacing the call to
> > procedure->macro in boot-9.scm by a call to procedure->memoizing-macro is
> > safe.
> 
> A few others worth checking:
> 
> ./guile-core/oop/goops/save.scm:  (procedure->macro
> ./guile-core/oop/goops/stklos.scm:    (procedure->macro
> ./guile-core/oop/goops.scm:    (procedure->macro
> ./guile-core/oop/goops.scm:  (procedure->macro
> ./guile-core/oop/goops.scm:  (procedure->macro

Thanks for pointing these out.  I will take a look at them.  My current
assumption is, that it should be generally safe to replace
procedure->macro by procedure->memoizing-macro if the following conditions
are met:

1) the macro procedure does not have a stateful behaviour and
2) the transformation of the macro procedure does not depend on the
content of the environment argument.

If I am not mistaken, these two conditions guarantee, that the transformer
will return the same transformed code with every execution.  Thus, it
could just memoize that code, since it would always be the same anyway.

Best regards,
Dirk


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On Wed, 3 Jul 2002, Dirk Herrmann wrote:

> On 1 Jul 2002, Gary Houston wrote:
> 
[...]
> > ./guile-core/oop/goops.scm:    (procedure->macro
> > ./guile-core/oop/goops.scm:  (procedure->macro
> > ./guile-core/oop/goops.scm:  (procedure->macro
> 
> Thanks for pointing these out.  I will take a look at them.
[...]

Except for the three places above, I think replacing procedure->macro with
procedure->memoizing-macro should be safe.

So what is so special about the uses of procedure->macro at the places
above?  They appear in the code for define-class, define-generic and
define-accessor.  In these macros, it is first checked whether the object
defined is already of the desired type.  That is, if you do a
(define-class foo ...), the define-class macro first checks if foo is
already a class.  If so, the class is re-defined, which means, already
existing objects of the former class will be re-defined together with
their class.  However, the macro define-class needs to check the
environment of the macro application in order to figure out, whether the
identifier to be defined was already bound to a class.

That is, these macros do actually make use of the dynamic nature of the
"macros".  It is not possible to simply replace them with mmacros.

Hmm?

Best regards,
Dirk


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Marius Vollmer]

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

> So what is so special about the uses of procedure->macro at the places
> above?  
>
> [checking for redefinitions.]

We need to restrict this redefining behavior of 'define-class' et all
to the top-level.  Local class definitions should not redefine classes
outside of their scope (that would lead to a funny version of dynamic
scoping of classes, eew), and redefinitions directly in one scope
should be an error, just like any other definitions.

For example,

    (let ((foo (find-class "<moo>")))
      (define-class foo ...)
      ...)

should not try to redefine the class that is the value of the location
bound to foo by the let.  It should simply shadow the outer foo.

Also,

    (let ()
      (define-class foo ...)
      (define-class foo ...)
      ...)

should be an error, just like

    (let ()
      (define foo ...)
      (define foo ...))

Redefinitions on the top-level do make sense and can be supported by a
normal macro via explicit module manipulations, i.e.

    (define-class foo ...)
    =>
    (module-define-class (current-module) 'foo ...)

with

    (define (module-define-class mod sym ...)
      (let* ((var (module-ensure-local-variable! mod 'foo))
             (class (make-class ...)))
        (if (variable-bound? var)
            (redefine-class (variable-ref var) class))
        (variable-set! var class)))

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

macros, procedure->macro

[Dirk Herrmann]

Hi,

Guile currently supports three types of builtin macros:  "acros", "macros"
and "mmacros".  Of these, only "mmacros" work like real syntax
transformers, i. e. only "mmacros" will lead to a changed code.  Instead,
"acros" and "macros" will not change the code, but instead may even treat
their syntactic expression differently every time the same code is
encountered.

That is, if we plan to split up syntax transformation,
compilation/memoization and execution, we could not remove "acros" and
"macros" from the execution, since they may behave differently every time
the same code is executed.

I therefore strongly suggest to get rid of "acros" and "macros".  As a
first step, I suggest to get rid of "macros" and their scheme-level
representative procedure->macro: As far as I see it, there is only one use
of "macros" in guile, namely in boot-9-scm as a call to procedure->macro.  
And, to me it seems this could be replaced without problems by a call to
procedure->memoizing-macro.

I ask you for the following:

1) Some macro expert should check that replacing the call to
procedure->macro in boot-9.scm by a call to procedure->memoizing-macro is
safe.

2) Every guile user should determine, whether it would be OK if we got rid
of "macros".

3) Decide officially, whether it is OK to remove "macros" from the head
branch.

If all results are positive, I will go ahead and remove the support
for "macros" from guile.  After that, I will take a close look at "acros"
and we will play a similar game with "acros" again...

Best regards, 
Dirk Herrmann


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Rob Browning]

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

> That is, if we plan to split up syntax transformation,
> compilation/memoization and execution, we could not remove "acros"
> and "macros" from the execution, since they may behave differently
> every time the same code is executed.
>
> I therefore strongly suggest to get rid of "acros" and "macros".  As
> a first step, I suggest to get rid of "macros" and their
> scheme-level representative procedure->macro

Presuming I'm remembering my most recent look in to this stuff
correctly (about 6 mos ago), I believe you're right, but Marius may
have further comments.

> If all results are positive, I will go ahead and remove the support
> for "macros" from guile.  After that, I will take a close look at "acros"
> and we will play a similar game with "acros" again...

It may also be important to consider pssyntax (i.e. syntax-case,
etc.).  In the end I'd like to have one unified macro system whose
behavior and interactions with the rest of guile are very clear.

Thanks

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On Mon, 1 Jul 2002, Rob Browning wrote:

> Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:
> 
> > If all results are positive, I will go ahead and remove the support
> > for "macros" from guile.  After that, I will take a close look at "acros"
> > and we will play a similar game with "acros" again...
> 
> It may also be important to consider pssyntax (i.e. syntax-case,
> etc.).  In the end I'd like to have one unified macro system whose
> behavior and interactions with the rest of guile are very clear.

True, but currently I am looking at the backend of evaluation, while
syntax-case is at the very front.  In the long term, I would like to see
the evaluation split into the following phases:

1) read
2) syntax-transformation
3) scheme-to-scheme-optimization
4) memoization
5) execution

Reading is obvious.  Syntax transformation is the "one unified macro
system" that you mention.  The result should be scheme code, which may
only hold a limited set of built-in syntactic forms.  The reason I'd like
to see a scheme to scheme optimization phase after this is, that these
optimizations would be target independent, and a lot of code from other
scheme implementations could be used here.  The memoization will be
dependent on the target code format (it could even compile to machine
code), and so will execution.

Best regards
Dirk


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Marius Vollmer]

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

> 1) Some macro expert should check that replacing the call to
> procedure->macro in boot-9.scm by a call to procedure->memoizing-macro is
> safe.

Err, I'm no macro expert, but I fully agree that "acros" and "macros"
should be removed.  So even if it would not be OK to just replace the
use of procedure->macro with procedure->memoizing-macro, we need to
find a solution that doesn't use the former.

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On 2 Jul 2002, Marius Vollmer wrote:

> Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:
> 
> > 1) Some macro expert should check that replacing the call to
> > procedure->macro in boot-9.scm by a call to procedure->memoizing-macro is
> > safe.
> 
> Err, I'm no macro expert, but I fully agree that "acros" and "macros"
> should be removed.  So even if it would not be OK to just replace the
> use of procedure->macro with procedure->memoizing-macro, we need to
> find a solution that doesn't use the former.

OK, lets assume we want to get rid of "acros" and "macros".  When should
that happen, and when should the corresponding functions be removed from
guile?  Removing these would change the interface.  According to our
standard procedure, this would mean going through a phase of deprecating
the corresponding functions.  However, this would mean, we could not
actually proceed with working on the evaluator, since as long as those
functions exist (even if deprecated) it is not possible to split up the
evaluator.

Hmm?

Best regards
Dirk


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Marius Vollmer]

Lest anyone gets confused and is worried that we are talking about
removing macros from Guile: this is not the case.  "macro" is the
internal name for a special kind of syntactic device that is much like
a normal macro but is expanded everytime it is evaluated.  Normal
macros are only expanded once.

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

> OK, lets assume we want to get rid of "acros" and "macros".  When should
> that happen, and when should the corresponding functions be removed from
> guile?

As soon as reasonable. :)

> Removing these would change the interface.  According to our
> standard procedure, this would mean going through a phase of
> deprecating the corresponding functions.  However, this would mean,
> we could not actually proceed with working on the evaluator, since
> as long as those functions exist (even if deprecated) it is not
> possible to split up the evaluator.

Is it really impossible?  I'd say it can be done, as long as we have
'local-eval'.

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On 7 Jul 2002, Marius Vollmer wrote:

> Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:
> 
> > OK, lets assume we want to get rid of "acros" and "macros".  When should
> > that happen, and when should the corresponding functions be removed from
> > guile?
> 
> As soon as reasonable. :)
> 
> > Removing these would change the interface.  According to our
> > standard procedure, this would mean going through a phase of
> > deprecating the corresponding functions.  However, this would mean,
> > we could not actually proceed with working on the evaluator, since
> > as long as those functions exist (even if deprecated) it is not
> > possible to split up the evaluator.
> 
> Is it really impossible?  I'd say it can be done, as long as we have
> 'local-eval'.

Could you explain how you think it should be done?  Especially how you
think one should obtain the local environment during execution.  Since you
seem to have thought about it, it could save me some time to make use of
your thoughts.

In any case, I think it should be a temporary solution to be able to go
through a phase of deprecation of acros and macros.

Best regards,
Dirk Herrmann


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Marius Vollmer]

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

> > > Removing these would change the interface.  According to our
> > > standard procedure, this would mean going through a phase of
> > > deprecating the corresponding functions.  However, this would mean,
> > > we could not actually proceed with working on the evaluator, since
> > > as long as those functions exist (even if deprecated) it is not
> > > possible to split up the evaluator.
> > 
> > Is it really impossible?  I'd say it can be done, as long as we have
> > 'local-eval'.
> 
> Could you explain how you think it should be done?  Especially how you
> think one should obtain the local environment during execution.

The local environment is just there during execution, isn't it?  We
need to keep using our current data structure for it that can be
searched for local variables.  We could not immediately switch to a
more streamlined data structure for environments that omits these
names.

Actually, we need only keep the old data structure for code that uses
"acros" or "macros".  This can be determined at memoization time.  So
if we are really want to switch to a different environment
representation, we can do that while still providing "acros" and
"macros", but it would probably too much work.

Then, when we have a 'traditional' environment, we can pass this
environment to the "acro" and "macro" transformers and use it to
memoize and execute the code returned by the "macro".

> In any case, I think it should be a temporary solution to be able to
> go through a phase of deprecation of acros and macros.

Yes.

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On 9 Jul 2002, Marius Vollmer wrote:

> The local environment is just there during execution, isn't it?  We
> need to keep using our current data structure for it that can be
> searched for local variables.  We could not immediately switch to a
> more streamlined data structure for environments that omits these
> names.

True, but with current guile, how do you _access_ the local environment
(when not making use of acros, macros and mmacros to obtain it)?  That is,
how would you actually use 'local-eval' to emulate acros and macros - if I
understand you correctly and that is what you want to do?

Best regards,
Dirk Herrmann


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Marius Vollmer]

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

> True, but with current guile, how do you _access_ the local environment
> (when not making use of acros, macros and mmacros to obtain it)?

I'm not sure, I understand the problem.  Currently, the evaluator
carries around the environment in the "env" variable.  It will
continue to do this, or?  From there, it can be passed to "acro" and
"mmacro" transformers.

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On 10 Jul 2002, Marius Vollmer wrote:

> Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:
> 
> > True, but with current guile, how do you _access_ the local environment
> > (when not making use of acros, macros and mmacros to obtain it)?
> 
> I'm not sure, I understand the problem.  Currently, the evaluator
> carries around the environment in the "env" variable.  It will
> continue to do this, or?  From there, it can be passed to "acro" and
> "mmacro" transformers.

Yes, but we want to separate macro expansion, memoization and execution,
right? That is, we first have to get rid of acros and macros and even
mmacros from within the execution phase.  And, we want to not remove them,
but deprecate them first.  And, we still want to proceed with the
separation of expansion, memoization and execution while still keeping
acros and macros around as deprecated features.  That means, we will have
to provide a means to emulate the behaviour of acros etc. _without_ having
them being treated specially by the executor.  I thought it was for this
that you suggested making use of 'local-eval'.

Now, you are right that the evaluator carries around the env variable, and
passes it to acros, macros and mmacros.  But, after the handling of acros
etc. has been removed from the execution phase, we must find a different
mechanism for passing the env variable to the code emulating acros and
macros.  I am referring to your response cited below:

On 7 Jul 2002, Marius Vollmer wrote:

> Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:
> 
> > OK, lets assume we want to get rid of "acros" and "macros".  When should
> > that happen, and when should the corresponding functions be removed from
> > guile?
> 
> As soon as reasonable. :)
> 
> > Removing these would change the interface.  According to our
> > standard procedure, this would mean going through a phase of
> > deprecating the corresponding functions.  However, this would mean,
> > we could not actually proceed with working on the evaluator, since
> > as long as those functions exist (even if deprecated) it is not
> > possible to split up the evaluator.
> 
> Is it really impossible?  I'd say it can be done, as long as we have
> 'local-eval'.

Now, what I am basically saying is, that I don't understand how the
solution that you seem to have in mind should look like.  That is, I'd
like to see a code example where you demonstrate how you would use
local-eval to emulate acros and macros even after the evaluator is split
up into memoization and execution.

Or am I completely misunderstanding you?

Best regards,
Dirk Herrmann


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Marius Vollmer]

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

> That means, we will have to provide a means to emulate the behaviour
> of acros etc. _without_ having them being treated specially by the
> executor.

I think we must treat the specially.  I don't see a way around that.
Actually, that's most of the reason why we want to get rid of them:
they can't be handled by a separate, off-line expansion pass.

So, while they are 'in deprecation', there needs to be code in the
executor for them.  When they have been removed, that wart will vanish
as well.

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Gary Houston]

> From: Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de>
> Date: Mon, 1 Jul 2002 21:56:23 +0200 (CEST)

> 1) Some macro expert should check that replacing the call to
> procedure->macro in boot-9.scm by a call to procedure->memoizing-macro is
> safe.

A few others worth checking:

./guile-core/oop/goops/save.scm:  (procedure->macro
./guile-core/oop/goops/stklos.scm:    (procedure->macro
./guile-core/oop/goops.scm:    (procedure->macro
./guile-core/oop/goops.scm:  (procedure->macro
./guile-core/oop/goops.scm:  (procedure->macro


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Neil Jerram]

>>>>> "Dirk" == Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

    Dirk> 1) Some macro expert should check that replacing the call to
    procedure-> macro in boot-9.scm by a call to
    procedure->memoizing-macro is safe.

This change is safe, since define-option-interface is only ever used
at top level.

    Dirk> 2) Every guile user should determine, whether it would be OK
    Dirk> if we got rid of "macros".

    Dirk> 3) Decide officially, whether it is OK to remove "macros" from the head
    Dirk> branch.

    Dirk> If all results are positive, I will go ahead and remove the support
    Dirk> for "macros" from guile.  After that, I will take a close look at "acros"
    Dirk> and we will play a similar game with "acros" again...

Before doing this, I think we need to state clearly when macro
expansion will happen in the future Guile.

There was a discussion on this just a few weeks ago which, to my mind,
was left unresolved.

(As far as I understand, the options are just after reading and just
before memoization, but just-after-reading makes recursive macro
definitions difficult, and just-before-memoization leads to rather
lame execution-dependent expansion.

Obviously, by definition, the exact timing only matters for
non-hygienic macros; but my assumption is that an awful lot of people
want non-hygienic macros.)

        Neil

PS. I think we should not mix up this discussion with one about class
redefinition!


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On 9 Jul 2002, Neil Jerram wrote:

>     Dirk> If all results are positive, I will go ahead and remove the support
>     Dirk> for "macros" from guile.  After that, I will take a close look at "acros"
>     Dirk> and we will play a similar game with "acros" again...
> 
> Before doing this, I think we need to state clearly when macro
> expansion will happen in the future Guile.
[...]
> (As far as I understand, the options are just after reading and just
> before memoization, but just-after-reading makes recursive macro
> definitions difficult, and just-before-memoization leads to rather
> lame execution-dependent expansion.

Hmmm, I don't quite understand you here - to me it seems that the big
difference is whether macro-expansion happens during execution, or before
execution.  If macro-expansion happens before execution, you could
basically have one big step "macro expansion + memoization/compilation"
which should cover both the just-after-reading and the
just-before-memoization options.  Sure, this big step is better separated
into different tasks, but that can be decided later.  Currently my focus
is on separating the big step from execution.

> Obviously, by definition, the exact timing only matters for
> non-hygienic macros; but my assumption is that an awful lot of people
> want non-hygienic macros.)

Could you give some examples for situations where the exact timing is
important?

Best regards,
Dirk Herrmann


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Neil Jerram]

>>>>> "Dirk" == Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

    Dirk> Could you give some examples for situations where the exact timing is
    Dirk> important?

Well, I was thinking of ...

http://mail.gnu.org/pipermail/guile-devel/2002-April/005085.html
http://mail.gnu.org/pipermail/guile-devel/2002-April/005102.html

but considering that the second of these was from yourself, perhaps
I'm misunderstanding something.

I think the perhaps the point is not at what stage macro expansion
happens, but how universal it is.  In other words, is macro expansion
performed universally like reading, and so only blocked by quoting, or
is it performed like evaluation, and so blocked/delayed by any number
of special forms including lambda and if.

Has this helped to explain my point at all?

        Neil


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On 10 Jul 2002, Neil Jerram wrote:

> I think the perhaps the point is not at what stage macro expansion
> happens, but how universal it is.  In other words, is macro expansion
> performed universally like reading, and so only blocked by quoting, or
> is it performed like evaluation, and so blocked/delayed by any number
> of special forms including lambda and if.

Ahh, I think I get your point.  Currently, macro expansion is done like
evaluation.  However, I think we agree that this is broken.  Therefore, I
am currently working towards a solution (in tiny little steps, I admit),
where macro expansion will be performed like reading (to use your words).

Best regards,
Dirk Herrmann


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Neil Jerram]

>>>>> "Dirk" == Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

    Dirk> On 10 Jul 2002, Neil Jerram wrote:
    >> I think the perhaps the point is not at what stage macro expansion
    >> happens, but how universal it is.  In other words, is macro expansion
    >> performed universally like reading, and so only blocked by quoting, or
    >> is it performed like evaluation, and so blocked/delayed by any number
    >> of special forms including lambda and if.

    Dirk> Ahh, I think I get your point.  Currently, macro expansion
    Dirk> is done like evaluation.  However, I think we agree that
    Dirk> this is broken.  Therefore, I am currently working towards a
    Dirk> solution (in tiny little steps, I admit), where macro
    Dirk> expansion will be performed like reading (to use your
    Dirk> words).

Thanks, that's helpful.  So we won't support references to a macro
that is defined in a following top-level form, as in:

(define-macro (foo x) `(list ,(bar x) ,x))
(define-macro (bar x) `(* ,x ,x))

or is there a cunning plan that still allows us to support this?

        Neil


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Clinton Ebadi]

On Friday 12 July 2002 20:09, Neil Jerram wrote:
> Thanks, that's helpful.  So we won't support references to a macro
> that is defined in a following top-level form, as in:
>
> (define-macro (foo x) `(list ,(bar x) ,x))
> (define-macro (bar x) `(* ,x ,x))
>
> or is there a cunning plan that still allows us to support this?

Just use r5rs macros:
(use-modules (ice-9 syncase))
(define-syntax foo (syntax-rules () ((foo x) (list (bar x) x))))
(define-syntax bar (syntax-rules () ((bar x) (* x x))))

guile> (foo 5)
(25 5)

-- 
http://unknownlamer.org
Facts do not cease to exist because they are ignored.
                -- Aldous Huxley
Flag Burner.

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Neil Jerram]

>>>>> "Clinton" == Clinton Ebadi <unknown_lamer@unknownlamer.org> writes:

    Clinton> On Friday 12 July 2002 20:09, Neil Jerram wrote:
    >> Thanks, that's helpful.  So we won't support references to a macro
    >> that is defined in a following top-level form, as in:
    >> 
    >> (define-macro (foo x) `(list ,(bar x) ,x))
    >> (define-macro (bar x) `(* ,x ,x))
    >> 
    >> or is there a cunning plan that still allows us to support this?

    Clinton> Just use r5rs macros:
    Clinton> (use-modules (ice-9 syncase))
    Clinton> (define-syntax foo (syntax-rules () ((foo x) (list (bar x) x))))
    Clinton> (define-syntax bar (syntax-rules () ((bar x) (* x x))))

    guile> (foo 5)
    Clinton> (25 5)

Thanks for pointing this out.  However, r5rs are by definition
hygienic, one of the corollaries of which (if I understand correctly)
is that it makes no difference when or how many times macros are
expanded.

With r5rs macros, therefore, we can easily delay expansion as late as
possible (e.g. immediately before evaluation) so as to support all
kinds of recursive references.

With non-hygienic macros, timing matters.  So, if Guile wants to
support non-hygienic macros at all (in addition to r5rs), it's
important to be precise about when they are expanded.

Regards,
        Neil


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Marius Vollmer]

Neil Jerram <neil@ossau.uklinux.net> writes:

> Thanks for pointing this out.  However, r5rs are by definition
> hygienic, one of the corollaries of which (if I understand correctly)
> is that it makes no difference when or how many times macros are
> expanded.

It does make a difference when we also consider redefinitions, and
definitions textually after the first use.  I.e., which definition of
bar is used in the following example?

    (define-syntax bar ...)
    (define (foo) (bar))
    (define-syntax bar ...)

> With r5rs macros, therefore, we can easily delay expansion as late as
> possible (e.g. immediately before evaluation) so as to support all
> kinds of recursive references.

I think it is more important to give us leeway to perform macro
expansion as early as possible, to make life easy for compilers.

> With non-hygienic macros, timing matters.  So, if Guile wants to
> support non-hygienic macros at all (in addition to r5rs), it's
> important to be precise about when they are expanded.

Another option would be to not be precise and just expect macro
writers to code their macros in such a way that they don't depend on a
the exact behavior of the implementation.  This might even be
desirable, to encourage people to write portable code.

The conservative assumption is that macros are expanded once, in a
environment that is totally different from the run-time environment.

But, of course, precisely specifying our implementation works is also
a good thing.

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Rob Browning]

Marius Vollmer <mvo@zagadka.ping.de> writes:

> But, of course, precisely specifying our implementation works is
> also a good thing.

Definitely, and if possible, having good warnings that you can enable
would be good too.

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Dirk Herrmann]

On 13 Jul 2002, Neil Jerram wrote:

> Thanks, that's helpful.  So we won't support references to a macro
> that is defined in a following top-level form, as in:
> 
> (define-macro (foo x) `(list ,(bar x) ,x))
> (define-macro (bar x) `(* ,x ,x))
> 
> or is there a cunning plan that still allows us to support this?

I don't see why recursive macros shouldn't be possible.  This, however,
depends on the implementation of the macro system used.  Clinton has
demonstrated that it does work with r5rs macros.  This will continue to
work, since even today the r5rs macros in guile are expanded prior to
execution.

However, there are things that won't work any more:
  (define (foo) (bar))
  (define-macro (bar) #f)
  (foo)
With today's guile, the expansion of foo is done at the first evalution in
contrast to the definition of foo.

Best regards,
Dirk


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: macros, procedure->macro

[Neil Jerram]

>>>>> "Dirk" == Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

    Dirk> On 13 Jul 2002, Neil Jerram wrote:
    >> Thanks, that's helpful.  So we won't support references to a macro
    >> that is defined in a following top-level form, as in:
    >> 
    >> (define-macro (foo x) `(list ,(bar x) ,x))
    >> (define-macro (bar x) `(* ,x ,x))
    >> 
    >> or is there a cunning plan that still allows us to support this?

    Dirk> I don't see why recursive macros shouldn't be possible.

Well, because when the first `define-macro' form is read
(universally), macroexpanded (universally) and evaluated (with the
quasiquoted part protected from evaluation by an implicit lambda),
`bar' is not yet defined.

    Dirk> This, however, depends on the implementation of the macro
    Dirk> system used.

Yes (in practice), but it should depend first on our specification of
how it will work.  AFAICT, if the specification is that macroexpansion
is `universal' in the way that `read' is, the above example will _not_
work.

    Dirk> However, there are things that won't work any more:
    Dirk>   (define (foo) (bar))
    Dirk>   (define-macro (bar) #f)
    Dirk>   (foo)

For the purposes of current discussion, I don't see the difference
between this example and my one above.  (Note that my `(bar x)' above
is unquoted, so equivalent to your `(bar)'.)  What do you intend to be
the key difference?  (Perhaps you meant `(define foo (bar))'?)

Regards,
        Neil


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel