(set! (@@ MOD NAME) EXP) considered harmful

(set! (@@ MOD NAME) EXP) considered harmful

[Mark H Weaver]

The ability to set! arbitrary module top-level variables from outside
the module, using the syntax (set! (@@ MOD NAME) EXP), destroys our
ability to several important optimizations.

As long as such ability exists, we must pessimistically assume that
any module top-level variable might change at any time, which means,
for example, that we cannot inline top-level procedure applications
from within the module itself.  This could be a HUGE efficiency win in
the common case where such top-level procedures are never set! from
within the module.

Now that we have proper hygienic macros, is this functionality still
required?  If we're going to remove it, pre-2.0 seems like a good time
to do so.  We won't get another opportunity for several years.

     Mark

Re: (set! (@@ MOD NAME) EXP) considered harmful

[Mark H Weaver]

Sorry, I meant to say that (set! (@ MOD NAME) EXP) should be
considered harmful as well.

     Mark


On Wed, Sep 02, 2009 at 12:17:11PM -0400, Mark H Weaver wrote:
> The ability to set! arbitrary module top-level variables from outside
> the module, using the syntax (set! (@@ MOD NAME) EXP), destroys our
> ability to several important optimizations.
> 
> As long as such ability exists, we must pessimistically assume that
> any module top-level variable might change at any time, which means,
> for example, that we cannot inline top-level procedure applications
> from within the module itself.  This could be a HUGE efficiency win in
> the common case where such top-level procedures are never set! from
> within the module.
> 
> Now that we have proper hygienic macros, is this functionality still
> required?  If we're going to remove it, pre-2.0 seems like a good time
> to do so.  We won't get another opportunity for several years.
> 
>      Mark
> 

Re: (set! (@@ MOD NAME) EXP) considered harmful

[Andy Wingo]

Hi Mark,

On Wed 02 Sep 2009 18:17, Mark H Weaver <mhw@netris.org> writes:

> The ability to set! arbitrary module top-level variables from outside
> the module, using the syntax (set! (@@ MOD NAME) EXP), destroys our
> ability to several important optimizations.
>
> As long as such ability exists, we must pessimistically assume that
> any module top-level variable might change at any time, which means,
> for example, that we cannot inline top-level procedure applications
> from within the module itself.  This could be a HUGE efficiency win in
> the common case where such top-level procedures are never set! from
> within the module.

I disagree, heartily.

Guile is a dynamic system, one where you can get into modules and
redefine functions. All you have to do is (set-current-module!
(resolve-module '(foo))) and (define bar #f) to do the equivalent of
(set! (@@ (foo) bar) #f). It's just like that.

And we should be proud of that, IMO! Your program might be a reified gem
once you're finished with it, but it normally doesn't arrive there
without much debugging.

Furthermore, (set! (@ ..) ..) is a /consequence/ of hygiene. A hygienic
macro that expands out to a set! of a variable in its context will
compile to a (set! (@@ ...) ...). You can always get at the
module-private vars, even with hygienic systems -- see the
eval-in-module hacks from ikarus if you don't believe me there.

Now, regarding optimizations. We can optimize whatever we like ;)

More seriously, there are many possibilities there. One is to define
your modules in lexical forms, as Chez does it -- so private functions
will always be inlined. See the end of section 2.4 on
http://www.scheme.com/csug7/use.html.

Another is to always present module-private bindings as candidates for
inlining.

Another is to allow the user to mark certain procedures as inlinable.
That will certainly be the case with `map', for example.

Another is to have an optimization knob you can crank up, understanding
that higher levels of optimization prevent some runtime modifiability.

In short, @ and @@ are lovely and useful, and we should be happy with
them. (set! (@@ ...) ...) is not an expected operation from user code,
and may reasonably be discounted even at low optimization levels. I
don't see the problem :)

Andy
-- 
http://wingolog.org/

Re: (set! (@@ MOD NAME) EXP) considered harmful

[Neil Jerram]

Andy Wingo <wingo@pobox.com> writes:

> Hi Mark,
>
> On Wed 02 Sep 2009 18:17, Mark H Weaver <mhw@netris.org> writes:
>
>> The ability to set! arbitrary module top-level variables from outside
>> the module, using the syntax (set! (@@ MOD NAME) EXP), destroys our
>> ability to several important optimizations.
>>
>> As long as such ability exists, we must pessimistically assume that
>> any module top-level variable might change at any time, which means,
>> for example, that we cannot inline top-level procedure applications
>> from within the module itself.  This could be a HUGE efficiency win in
>> the common case where such top-level procedures are never set! from
>> within the module.
>
> I disagree, heartily.
>
> Guile is a dynamic system, one where you can get into modules and
> redefine functions. All you have to do is (set-current-module!
> (resolve-module '(foo))) and (define bar #f) to do the equivalent of
> (set! (@@ (foo) bar) #f). It's just like that.
>
> And we should be proud of that, IMO! Your program might be a reified gem
> once you're finished with it, but it normally doesn't arrive there
> without much debugging.
>
> Furthermore, (set! (@ ..) ..) is a /consequence/ of hygiene. A hygienic
> macro that expands out to a set! of a variable in its context will
> compile to a (set! (@@ ...) ...). You can always get at the
> module-private vars, even with hygienic systems -- see the
> eval-in-module hacks from ikarus if you don't believe me there.
>
> Now, regarding optimizations. We can optimize whatever we like ;)
>
> More seriously, there are many possibilities there. One is to define
> your modules in lexical forms, as Chez does it -- so private functions
> will always be inlined. See the end of section 2.4 on
> http://www.scheme.com/csug7/use.html.
>
> Another is to always present module-private bindings as candidates for
> inlining.
>
> Another is to allow the user to mark certain procedures as inlinable.
> That will certainly be the case with `map', for example.
>
> Another is to have an optimization knob you can crank up, understanding
> that higher levels of optimization prevent some runtime modifiability.
>
> In short, @ and @@ are lovely and useful, and we should be happy with
> them. (set! (@@ ...) ...) is not an expected operation from user code,
> and may reasonably be discounted even at low optimization levels. I
> don't see the problem :)

I agree with all that.  Even as we do more static-feeling things like
compilation, we'd like to keep all the dynamic possibilities too.

Regarding optimization, maybe another possibility would be to have
detailed tracking of the underlying definitions that each piece of
optimized+compiled code depends on, and to recompile+reoptimize
whenever one of the underlying definitions changes.

Or is that just cloud cuckoo land?

(Ludovic recently suggested an optimization for constant additions, and
I assumed that would require this kind of rollback, if the definition of
`+' was later changed.)

      Neil

Re: (set! (@@ MOD NAME) EXP) considered harmful

[Ludovic Courtès]

Hello!

Neil Jerram <neil@ossau.uklinux.net> writes:

> (Ludovic recently suggested an optimization for constant additions, and
> I assumed that would require this kind of rollback, if the definition of
> `+' was later changed.)

I generally agree with the praise of dynamicity, but there’s a tension
between dynamicity and performance.

Mark’s example is a good illustration.  Likewise, many optimizations
sacrifice dynamicity: open coding (see ‘*primcall-ops*’ and
‘vm-i-scheme.c’), inlining, constant folding, etc.

One could event argue that the mixture of code affected by these
optimizations and unaffected code is fragile in the presence of global
redefinitions.

Perhaps the compiler should make it possible to choose a tradeoff
between dynamicity and performance, e.g., by actually honoring the ‘:O’
compile option?

Thanks,
Ludo’.

Re: (set! (@@ MOD NAME) EXP) considered harmful

[Andy Wingo]

On Fri 18 Sep 2009 00:16, ludo@gnu.org (Ludovic Courtès) writes:

> Perhaps the compiler should make it possible to choose a tradeoff
> between dynamicity and performance, e.g., by actually honoring the ‘:O’
> compile option?

Sure, sounds good to me. We should also have in-source directives, like
(declare (speed 3)) or such.

Andy
-- 
http://wingolog.org/

Re: (set! (@@ MOD NAME) EXP) considered harmful

[Neil Jerram]

Andy Wingo <wingo@pobox.com> writes:

> On Fri 18 Sep 2009 00:16, ludo@gnu.org (Ludovic Courtès) writes:
>
>> Perhaps the compiler should make it possible to choose a tradeoff
>> between dynamicity and performance, e.g., by actually honoring the ‘:O’
>> compile option?
>
> Sure, sounds good to me.

Yes, that would be great.  In practice I think it's clear that most
people will want a dynamicity-compatible optimization level while
they're developing or debugging a program, but the best possible
optimization, and needing no dynamicity, when that program is deployed.

      Neil