Elisp lexical-let

Elisp lexical-let

[Daniel Kraft]

Hi again,

I just had some doubts about the details of lexical-let for elisp, which 
seem to be not answered by the document I linked to...  Maybe you can 
clarify these to me.

Especially, the question is about "what happens" when a lexical variable 
is inside its scope again bound dynamically (say via let or a lambda 
expression).  Like in this example (I'm not sure if all my expected 
outputs are right, please correct me as needed):

(defvar x 1)
(defun foo () x)
(lexical-let ((x 2))
   x ; -> 2
   (foo) ; -> 1
   (setq x 3)
   x ; -> 3
   (foo) ; -> 1
   (let ((x 4))
     x ; -> 4?
     (foo) ; -> 4
     (setq x 5)
     x ; -> 5
     (foo) ; -> 5
     ) ; end the let
   x ; -> 3?
   (foo) ; -> 4
   )
x ; -> 4
(foo) ; -> 4

So in this case all references to x in the lexical scope of the let are 
to the dynamic value again, while the lexical is untouched and restored 
after leaving the let?

Have a nice evening!

Daniel

-- 
Done:  Arc-Bar-Cav-Ran-Rog-Sam-Tou-Val-Wiz
To go: Hea-Kni-Mon-Pri

Re: Elisp lexical-let

[Ken Raeburn]

On Jul 21, 2009, at 15:48, Daniel Kraft wrote:
> Especially, the question is about "what happens" when a lexical  
> variable is inside its scope again bound dynamically (say via let or  
> a lambda expression).

Oh, don't stop there... let's get some buffer-local or frame-local  
bindings into the mix too! :-)
There are probably corner cases where the only "specification" you're  
likely to find is "what Emacs does when you try it".  And, if the goal  
is to support the existing body of Lisp code (e.g., if Emacs 24 goes  
out with lexical binding support, and people start using it), there's  
a tradeoff between what seems logical or convenient to you, and what  
behavior of Emacs the existing code is going to expect.  Maybe not in  
weird corner cases, but cases like you describe above seem likely, and  
I think you'd want to mimic whatever behavior Emacs is going to do.

Ken

Re: Elisp lexical-let

[Daniel Kraft]

Ken Raeburn wrote:
> On Jul 21, 2009, at 15:48, Daniel Kraft wrote:
>> Especially, the question is about "what happens" when a lexical 
>> variable is inside its scope again bound dynamically (say via let or a 
>> lambda expression).
> 
> Oh, don't stop there... let's get some buffer-local or frame-local 
> bindings into the mix too! :-)
> There are probably corner cases where the only "specification" you're 
> likely to find is "what Emacs does when you try it".  And, if the goal 
> is to support the existing body of Lisp code (e.g., if Emacs 24 goes out 
> with lexical binding support, and people start using it), there's a 
> tradeoff between what seems logical or convenient to you, and what 
> behavior of Emacs the existing code is going to expect.  Maybe not in 
> weird corner cases, but cases like you describe above seem likely, and I 
> think you'd want to mimic whatever behavior Emacs is going to do.

It seemed really hard to me to find at least *basic* information about 
how the lexbind things works; I did build now an emacs with lexbind from 
trunk, but so far as I see this is not meant to implement "lexical-let" 
as the cl package does, but rather allows switching all bindings from 
dynamic to lexical within one source file.

While this is certainly something we could do, too (best via compiler 
options, I guess?), it is not what I had in mind as the "extension" -- 
this being implementing the lexical-let as additional construct that 
establishes lexical binding for certain variables just temporarily.

And checks with the cl package's implementation of lexical-let give the 
result, that an inner let does the same as if it was another 
lexical-let; that is, does not revert to dynamic binding but rather sets 
only the lexical value.

So, what are the opinions regarding lexical-let as an extension 
construct?  Regarding the behaviour, to me the one described above seems 
to be a consequence of the implementing with unwind-protect and not 
necessarily expected -- thus I suggest to implement the version I had in 
mind, namely that an inner let or argument binding inside a lambda 
reverts to dynamic binding for that inner scope.  This seems more 
consistent and reasonable to me.

Yours,
Daniel

-- 
Done:  Arc-Bar-Cav-Ran-Rog-Sam-Tou-Val-Wiz
To go: Hea-Kni-Mon-Pri

Re: Elisp lexical-let

[Marijn Schouten (hkBst)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Daniel Kraft wrote:
> Ken Raeburn wrote:
>> On Jul 21, 2009, at 15:48, Daniel Kraft wrote:
>>> Especially, the question is about "what happens" when a lexical
>>> variable is inside its scope again bound dynamically (say via let or
>>> a lambda expression).
>>
>> Oh, don't stop there... let's get some buffer-local or frame-local
>> bindings into the mix too! :-)
>> There are probably corner cases where the only "specification" you're
>> likely to find is "what Emacs does when you try it".  And, if the goal
>> is to support the existing body of Lisp code (e.g., if Emacs 24 goes
>> out with lexical binding support, and people start using it), there's
>> a tradeoff between what seems logical or convenient to you, and what
>> behavior of Emacs the existing code is going to expect.  Maybe not in
>> weird corner cases, but cases like you describe above seem likely, and
>> I think you'd want to mimic whatever behavior Emacs is going to do.
> 
> It seemed really hard to me to find at least *basic* information about
> how the lexbind things works; I did build now an emacs with lexbind from
> trunk, but so far as I see this is not meant to implement "lexical-let"
> as the cl package does, but rather allows switching all bindings from
> dynamic to lexical within one source file.
> 
> While this is certainly something we could do, too (best via compiler
> options, I guess?), it is not what I had in mind as the "extension" --
> this being implementing the lexical-let as additional construct that
> establishes lexical binding for certain variables just temporarily.
> 
> And checks with the cl package's implementation of lexical-let give the
> result, that an inner let does the same as if it was another
> lexical-let; that is, does not revert to dynamic binding but rather sets
> only the lexical value.
> 
> So, what are the opinions regarding lexical-let as an extension
> construct?  Regarding the behaviour, to me the one described above seems
> to be a consequence of the implementing with unwind-protect and not
> necessarily expected -- thus I suggest to implement the version I had in
> mind, namely that an inner let or argument binding inside a lambda
> reverts to dynamic binding for that inner scope.  This seems more
> consistent and reasonable to me.
> 
> Yours,
> Daniel
> 

Guile also has lexical and dynamic variables; the fluids[1]. Queinnec in his
book LiSP also describes a system that has (default) lexical and dynamic
variable, on page 44. In both cases to find the value of a non-default variable
a function is used. Translated to elisp where the situation is dynamic by
default you probably want something like `(lexical x)' to dereference the
lexical variable `x' and also lexical-set(q).

It seems to me that only the dereferencing of variables is dynamic or lexical,
not the binding. Thus you don't even need lexical-let and `(lexical x)' would be
`x' found in the lexical environment (if it isn't found you can generate an
error) and `x' would be searched for in the dynamic environment. Does that make
sense?

[1]:http://www.gnu.org/software/guile/manual/guile.html#Fluids-and-Dynamic-States

Marijn

- --
If you cannot read my mind, then listen to what I say.

Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
<http://www.gentoo.org/proj/en/lisp/>, #gentoo-{lisp,ml} on FreeNode
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpnDXgACgkQp/VmCx0OL2zpcgCgg2QtK7kL5YJCeVP6hpG87h0f
DCMAn3rgkDIk2GYBqnHJ/JRzjsW7ehBw
=e0Hz
-----END PGP SIGNATURE-----

Re: Elisp lexical-let

[Daniel Kraft]

Hi Marijn,

Marijn Schouten (hkBst) wrote:
> Guile also has lexical and dynamic variables; the fluids[1]. Queinnec in his
> book LiSP also describes a system that has (default) lexical and dynamic
> variable, on page 44. In both cases to find the value of a non-default variable
> a function is used. Translated to elisp where the situation is dynamic by
> default you probably want something like `(lexical x)' to dereference the
> lexical variable `x' and also lexical-set(q).
> 
> It seems to me that only the dereferencing of variables is dynamic or lexical,
> not the binding. Thus you don't even need lexical-let and `(lexical x)' would be
> `x' found in the lexical environment (if it isn't found you can generate an
> error) and `x' would be searched for in the dynamic environment. Does that make
> sense?

not only the dereferencing, but also the setting must be of a dynamic or 
lexical value; and at least in my Lisp code, setting is mostly done with 
let's, so at the same time as the "binding".

So what you propose would then be to have a (lexical sym) for 
referencing and (lexical-set! sym value) for setting in lexical scope -- 
and then as a consequence also a lexical-let, that does let but sets to 
the provided values just as lexical-set! does...?  Well, that's how I 
see it, anyways.

And while your arguments seem quite reasonable (decide for each access 
of a symbol between lexical and dynamic), I think this creates a 
superfluous amount of confusion and "just" switching for certain 
variables to lexical binding within the scope of a lexical-let seems to 
me the better solution.

What's about this:

(defun test () a)
(let ((a 1))
   (print a) ; 1
   (print (test)) ; 1
   (lexical-set! a 2)
   (print a) ; 1?
   (print (test)) ; 1
   (print (lexical a)) ; 2
   )

I don't think it's good to have to "completely seperate" variables a and 
(lexical a).  And besides, here there's no way to decide that no fluids 
at all are needed for a, as the let construct itself (except the 
references following) still creates a with-fluids* call.

Yours,
Daniel

PS: From a performance point of view, I guess that the void checks on 
each variable access are far more expensive than the 
fluid/dynamic-scoping business.  But I'll report on this (performance) 
maybe in a seperate posting when I've done some experiments and real 
data as well as suggestions.

-- 
Done:  Arc-Bar-Cav-Ran-Rog-Sam-Tou-Val-Wiz
To go: Hea-Kni-Mon-Pri

Re: Elisp lexical-let

[Ken Raeburn]

On Jul 22, 2009, at 05:11, Daniel Kraft wrote:
> It seemed really hard to me to find at least *basic* information  
> about how the lexbind things works; I did build now an emacs with  
> lexbind from trunk, but so far as I see this is not meant to  
> implement "lexical-let" as the cl package does, but rather allows  
> switching all bindings from dynamic to lexical within one source file.

Oh... I may have seriously misunderstood; sorry about that.  That's  
what comes from not having bothered to look. :-)

In that case, the cl package is probably the thing to look at for now;  
it is used by a bunch of other things in Emacs.  But I do expect  
lexbind is going to get merged in eventually, so keeping that in mind  
for the future would be wise too.

> And checks with the cl package's implementation of lexical-let give  
> the result, that an inner let does the same as if it was another  
> lexical-let; that is, does not revert to dynamic binding but rather  
> sets only the lexical value.
>
> So, what are the opinions regarding lexical-let as an extension  
> construct?  Regarding the behaviour, to me the one described above  
> seems to be a consequence of the implementing with unwind-protect  
> and not necessarily expected -- thus I suggest to implement the  
> version I had in mind, namely that an inner let or argument binding  
> inside a lambda reverts to dynamic binding for that inner scope.   
> This seems more consistent and reasonable to me.

I think you could make an argument for doing it either way.  The  
interesting question would be whether anyone is depending on the  
behavior from the cl package.

Ken

Re: Elisp lexical-let

[Daniel Kraft]

Ken Raeburn wrote:
> On Jul 22, 2009, at 05:11, Daniel Kraft wrote:
>> It seemed really hard to me to find at least *basic* information about 
>> how the lexbind things works; I did build now an emacs with lexbind 
>> from trunk, but so far as I see this is not meant to implement 
>> "lexical-let" as the cl package does, but rather allows switching all 
>> bindings from dynamic to lexical within one source file.
> 
> Oh... I may have seriously misunderstood; sorry about that.  That's what 
> comes from not having bothered to look. :-)

Ok, I think I'll work on a cl like lexical-let now and see what this 
gives in performance related to dynamic let (and just to get it 
available of course).

The implementation will (for now at least) be as I favour it, that is, 
inner lets revert to dynamic binding (as do inner lambdas for their 
arguments) -- this is more reasonable (I think), has more power (as the 
other way can be achieved by using lexical-let for inner lets when 
preferred, which is also clearer to understand) and should be easier to 
implement (because I don't need to mess around with the compilation of 
let constructs depending on the context).

I'll keep in mind also the lexbind idea of optionally making every 
binding lexical.  Andy, can you give me a hint/example/pointer how 
compiler options work?  This would be exactly the place to provide this, 
I think.  Additionally we could add an option to remove the "variable is 
void" error check for a further performance gain.

Yours,
Daniel

-- 
Done:  Arc-Bar-Cav-Ran-Rog-Sam-Tou-Val-Wiz
To go: Hea-Kni-Mon-Pri

Re: Elisp lexical-let

[Marijn Schouten (hkBst)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Daniel Kraft wrote:
> Hi Marijn,
> 
> Marijn Schouten (hkBst) wrote:
>> Guile also has lexical and dynamic variables; the fluids[1]. Queinnec
>> in his
>> book LiSP also describes a system that has (default) lexical and dynamic
>> variable, on page 44. In both cases to find the value of a non-default
>> variable
>> a function is used. Translated to elisp where the situation is dynamic by
>> default you probably want something like `(lexical x)' to dereference the
>> lexical variable `x' and also lexical-set(q).
>>
>> It seems to me that only the dereferencing of variables is dynamic or
>> lexical,
>> not the binding. Thus you don't even need lexical-let and `(lexical
>> x)' would be
>> `x' found in the lexical environment (if it isn't found you can
>> generate an
>> error) and `x' would be searched for in the dynamic environment. Does
>> that make
>> sense?
> 
> not only the dereferencing, but also the setting must be of a dynamic or
> lexical value;

yes and one could argue that to set a variable you need to dereference it first
too or you won't know which location you're set'ting.

> and at least in my Lisp code, setting is mostly done with
> let's, so at the same time as the "binding".

Unless elisp is the exception, let does not set any variables, but only
introduces bindings.

> So what you propose would then be to have a (lexical sym) for
> referencing and (lexical-set! sym value) for setting in lexical scope --

yes.

> and then as a consequence also a lexical-let, that does let but sets to
> the provided values just as lexical-set! does...?  Well, that's how I
> see it, anyways.

no, let does no assignement AFAIK.

> And while your arguments seem quite reasonable (decide for each access
> of a symbol between lexical and dynamic), I think this creates a
> superfluous amount of confusion and "just" switching for certain
> variables to lexical binding within the scope of a lexical-let seems to
> me the better solution.

Aha, now I understand what you want lexical-let to mean.
If you are explicit about each variable access how can that be confusing?
Possibly confusing is if you have to examine the complete enclosing context for
lexical-let's.

> What's about this:
>
> (defun test () a)
> (let ((a 1))
>   (print a) ; 1
>   (print (test)) ; 1
>   (lexical-set! a 2)
there is only one variable `a' in my mental model,
so this changes the value of the only `a' and all
subsequent expressions accessing `a' are thus affected.
The same as if you had written `(set! a 2)'. Dynamic
and lexical only differ in how free variables in
procedures are bound.
>   (print a) ; 1?
no, (print a) => 2
>   (print (test)) ; 1
no, there is only one `a' and its value is 2 here
>   (print (lexical a)) ; 2
>   )
> 
> I don't think it's good to have to "completely seperate" variables a and
> (lexical a).

I don't understand what you mean. My proposal is to have one kind of variable
and two kinds of access.

> And besides, here there's no way to decide that no fluids
> at all are needed for a, as the let construct itself (except the
> references following) still creates a with-fluids* call.

For the calls above all the dynamic accesses can be determined statically.
Lexical accesses can always be determined statically. Thus all accesses in this
example can be determined statically and can be compiled to a location
dereference (either read or write). Nothing in the semantics is inherently
inefficient.

Marijn

- --
If you cannot read my mind, then listen to what I say.

Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
<http://www.gentoo.org/proj/en/lisp/>, #gentoo-{lisp,ml} on FreeNode
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpogK0ACgkQp/VmCx0OL2x4JACgwYR+48aAHTjMu1S6H7Dw4utR
YzoAn3hMwhEadhlrAw7a06gWZBEoDWQX
=DkN6
-----END PGP SIGNATURE-----

Re: Elisp lexical-let

[Mark H Weaver]

On Thu, Jul 23, 2009 at 05:24:30PM +0200, Marijn Schouten (hkBst) wrote:
> For the calls above all the dynamic accesses can be determined statically.
> Lexical accesses can always be determined statically. Thus all accesses in this
> example can be determined statically and can be compiled to a location
> dereference (either read or write). Nothing in the semantics is inherently
> inefficient.

There is at least one inherent loss of efficiency with the semantics
you propose: with a lexical-let binding, the compiler can determine
statically whether the variable is non-mutable, because the only place
it could possibly be set! is within the lexical scope.  Non-mutable
variables can safely be inlined, an especially important optimization,
especially if the value is a procedure.

Also, with lexical-let, the compiler knows statically the entire set
of references, which can be helpful with many analyses, for example
whether continuations or closures can "escape" a particular scope,
whether a particular continuation might be invoked more than once,
etc.

    Regards,
      Mark

Re: Elisp lexical-let

[Daniel Kraft]

Hi,

thanks for your comments; I'm still a bit confused, though :$

Marijn Schouten (hkBst) wrote:
>> What's about this:
>>
>> (defun test () a)
>> (let ((a 1))
>>   (print a) ; 1
>>   (print (test)) ; 1
>>   (lexical-set! a 2)
> there is only one variable `a' in my mental model,
> so this changes the value of the only `a' and all
> subsequent expressions accessing `a' are thus affected.
> The same as if you had written `(set! a 2)'. Dynamic
> and lexical only differ in how free variables in
> procedures are bound.
>>   (print a) ; 1?
> no, (print a) => 2
>>   (print (test)) ; 1
> no, there is only one `a' and its value is 2 here
>>   (print (lexical a)) ; 2
>>   )

Hm...  my problem is trying to understand how you want this implemented; 
my main point about lexical scoping is that it enables us to use Guile's 
built-in lexical mechanisms and we don't have to save the value 
explicitly into some fluids.

But if you require that the second (print (test)) above prints 1 even 
though we have done (lexical-set! a) this means that lexical-set! must 
update the place somehow where a is accessed dynamically (as is done in 
test).  And that seems to imply that this lexical-set! updates the 
fluids, even though it is meant to perform on a lexically bound variable 
a; just in case that "the one" a is at some place referred to dynamically.

>> I don't think it's good to have to "completely seperate" variables a and
>> (lexical a).
> 
> I don't understand what you mean. My proposal is to have one kind of variable
> and two kinds of access.

Can you please elaborate on this?  If there's only one variable and only 
the value 2 after the lexical-set! above (both for (print a) in the 
lexical scope and (print (test)) which accesses a dynamically), what 
would then be the point of writing 'a' or '(lexical a)' and what would 
be the difference between those two?

Yours,
Daniel

-- 
Done:  Arc-Bar-Cav-Ran-Rog-Sam-Tou-Val-Wiz
To go: Hea-Kni-Mon-Pri

Re: Elisp lexical-let

[Andy Wingo]

On Wed 22 Jul 2009 11:11, Daniel Kraft <d@domob.eu> writes:

> And checks with the cl package's implementation of lexical-let give the
> result, that an inner let does the same as if it was another
> lexical-let; that is, does not revert to dynamic binding but rather sets
> only the lexical value.

Interesting -- so it has to codewalk the contained expressions,
replacing let on its variables with appropriate lexical constructs?

> So, what are the opinions regarding lexical-let as an extension
> construct? Regarding the behaviour, to me the one described above seems
> to be a consequence of the implementing with unwind-protect and not
> necessarily expected -- thus I suggest to implement the version I had in
> mind, namely that an inner let or argument binding inside a lambda
> reverts to dynamic binding for that inner scope.  This seems more
> consistent and reasonable to me.

I'm afraid we're going to have to do whatever the existing lexical-let
does, if possible.

Andy
-- 
http://wingolog.org/

Re: Elisp lexical-let

[Andy Wingo]

Hi,

Just for the record :)

On Thu 23 Jul 2009 18:13, Mark H Weaver <mhw@netris.org> writes:

> On Thu, Jul 23, 2009 at 05:24:30PM +0200, Marijn Schouten (hkBst) wrote:
>> For the calls above all the dynamic accesses can be determined statically.
>> Lexical accesses can always be determined statically. Thus all accesses in this
>> example can be determined statically and can be compiled to a location
>> dereference (either read or write). Nothing in the semantics is inherently
>> inefficient.
>
> There is at least one inherent loss of efficiency with the semantics
> you propose: with a lexical-let binding, the compiler can determine
> statically whether the variable is non-mutable, because the only place
> it could possibly be set! is within the lexical scope.  Non-mutable
> variables can safely be inlined, an especially important optimization,
> especially if the value is a procedure.

Guile does detect this. The useful optimization is that variables that
are never set! can be allocated on the stack, and if they're closed
over, they can simply be copied.

> Also, with lexical-let, the compiler knows statically the entire set
> of references, which can be helpful with many analyses, for example
> whether continuations or closures can "escape" a particular scope,
> whether a particular continuation might be invoked more than once,
> etc.

This is a more tricky optimization :)

Andy
-- 
http://wingolog.org/

Re: Elisp lexical-let

[Andy Wingo]

Hi,

On Thu 23 Jul 2009 12:47, Daniel Kraft <d@domob.eu> writes:

> Ken Raeburn wrote:
>> On Jul 22, 2009, at 05:11, Daniel Kraft wrote:
>>> It seemed really hard to me to find at least *basic* information
>>> about how the lexbind things works; I did build now an emacs with
>>> lexbind from trunk, but so far as I see this is not meant to
>>> implement "lexical-let" as the cl package does, but rather allows
>>> switching all bindings from dynamic to lexical within one source
>>> file.
>>
>> Oh... I may have seriously misunderstood; sorry about that.  That's
>> what comes from not having bothered to look. :-)
>
> Ok, I think I'll work on a cl like lexical-let now and see what this
> gives in performance related to dynamic let (and just to get it
> available of course).
>
> The implementation will (for now at least) be as I favour it, that is,
> inner lets revert to dynamic binding (as do inner lambdas for their
> arguments) -- this is more reasonable (I think), has more power (as the
> other way can be achieved by using lexical-let for inner lets when
> preferred, which is also clearer to understand) and should be easier to
> implement (because I don't need to mess around with the compilation of
> let constructs depending on the context).

Fair enough. The other semantics sound pretty crazy.

> I'll keep in mind also the lexbind idea of optionally making every
> binding lexical.  Andy, can you give me a hint/example/pointer how
> compiler options work?  This would be exactly the place to provide this,
> I think.  Additionally we could add an option to remove the "variable is
> void" error check for a further performance gain.

They don't work! Well, basically they're just a value that reaches the
compiler somehow. I think they are a keyword list: (#:foo bar #:baz qux)
etc. They are set via the #:opts argument to compile; I don't know if
you can set them from the command line. I was waiting for a use case :)

Peace,

Andy
-- 
http://wingolog.org/

Re: Elisp lexical-let

[Andy Wingo]

On Tue 21 Jul 2009 21:48, Daniel Kraft <d@domob.eu> writes:

> (defvar x 1)
> (defun foo () x)
> (lexical-let ((x 2))
>   x ; -> 2
>   (foo) ; -> 1
>   (setq x 3)
>   x ; -> 3
>   (foo) ; -> 1
>   (let ((x 4))
>     x ; -> 4?
>     (foo) ; -> 4
>     (setq x 5)
>     x ; -> 5
>     (foo) ; -> 5
>     ) ; end the let
>   x ; -> 3?
>   (foo) ; -> 4
>   )
> x ; -> 4
> (foo) ; -> 4

It's actually fairly simple, imo. Alpha-equivalence says that
(lexical-let ((x a)) x) is the same as (lexical-let ((y a)) y). (Note
that this lexical-let corresponds to Scheme's let.) So your program is
the same as:

> (lexical-let ((y 2))
>   y ; -> 2
>   (foo) ; -> 1
>   (setq y 3)
>   y ; -> 3
>   (foo) ; -> 1
>   (let ((x 4))
>     x ; -> 4?
>     (foo) ; -> 4
>     (setq x 5)
>     x ; -> 5
>     (foo) ; -> 5
>     ) ; end the let
>   y ; -> 3?
>   (foo) ; -> 4
>   )
> x ; -> 4
> (foo) ; -> 4

I haven't reviewed your compiler yet, and for that I apologize. But. You
need to make sure that when you compile, you build up a compile-time
environment, mapping symbols to locations -- if a variable is free,
lexically bound, or dynamically bound. When your compiler sees a lexical
binding, replace that identifier with a gensym, and add an entry to your
compile-time environment to map that identifier to that lexical gensym.
That way you will effectively be processing my example, which is
equivalent to yours, but perhaps clearer because the names have been
uniquified. Then your problems go away.

Peace,

Andy
-- 
http://wingolog.org/

Re: Elisp lexical-let

[Daniel Kraft]

Andy Wingo wrote:
>> I'll keep in mind also the lexbind idea of optionally making every
>> binding lexical.  Andy, can you give me a hint/example/pointer how
>> compiler options work?  This would be exactly the place to provide this,
>> I think.  Additionally we could add an option to remove the "variable is
>> void" error check for a further performance gain.
> 
> They don't work! Well, basically they're just a value that reaches the
> compiler somehow. I think they are a keyword list: (#:foo bar #:baz qux)
> etc. They are set via the #:opts argument to compile; I don't know if
> you can set them from the command line. I was waiting for a use case :)

thanks for the hints, I did some tests yesterday, and it seems that I 
can just provide whatever value I want to (compile ... #:opts <options>) 
and get this as opts of the compiler routine called.

I've never used keyword lists before, but I'll look them up and this 
seems like what we want.  So far I'm thinking about two possibly useful 
options:

1) Declare to disable void checks for certain variables or all (for 
performance, as already written).  Stuff like makunbound/boundp will 
sill work just as before, but accessing a variable that is void will not 
give an error but rather return the special void value.  As most code 
should not really depend on this void-error anyways, it seems reasonable 
to give at least a way to disable this check happening on each and every 
variable access for no good (except debugging maybe).

2) Declare that certain variables or all should always be bound 
lexically rather than dynamically; maybe a defvar construct will retract 
this and make it dynamically bound later on again, then this should be 
like the upcoming lexbind stuff in emacs.  As lexical-let is not 
everything, but function arguments are always dynamically bound (except 
if we provide a lexical-lambda... hm, it might even be useful at times!) 
-- and this spoils tail-calls -- both of this option and lexical-let 
will probably be useful.

And maybe others in the future, but I've no ideas so far.  It seems that 
options work basically, but I guess there's not yet a way to "set" them 
for compilation happening for the elisp-repl?  This means that I can 
just keep on implementing and testing them, but for real use we probably 
want some construct to set them for real uses (besides (compile ...)) as 
well.

Cheers,
Daniel

-- 
Done:  Arc-Bar-Cav-Ran-Rog-Sam-Tou-Val-Wiz
To go: Hea-Kni-Mon-Pri

Re: Elisp lexical-let

[Daniel Kraft]

Hi Andy,

thanks for the clarifications!

> It's actually fairly simple, imo. Alpha-equivalence says that
> (lexical-let ((x a)) x) is the same as (lexical-let ((y a)) y). (Note
> that this lexical-let corresponds to Scheme's let.) So your program is
> the same as:
> 
>> (lexical-let ((y 2))
>>   y ; -> 2
>>   (foo) ; -> 1
>>   (setq y 3)
>>   y ; -> 3
>>   (foo) ; -> 1
>>   (let ((x 4))
>>     x ; -> 4?
>>     (foo) ; -> 4
>>     (setq x 5)
>>     x ; -> 5
>>     (foo) ; -> 5
>>     ) ; end the let
>>   y ; -> 3?
>>   (foo) ; -> 4
>>   )
>> x ; -> 4
>> (foo) ; -> 4

Yes of course, my main question was how the inner let is handled, and 
what dynamic values x will attain (the results of (foo)) -- and here's 
this funny stuff that the inner let will be like lexical-let for those 
variables already lexically bound.

> I haven't reviewed your compiler yet, and for that I apologize. But. You
> need to make sure that when you compile, you build up a compile-time
> environment, mapping symbols to locations -- if a variable is free,
> lexically bound, or dynamically bound. When your compiler sees a lexical
> binding, replace that identifier with a gensym, and add an entry to your
> compile-time environment to map that identifier to that lexical gensym.
> That way you will effectively be processing my example, which is
> equivalent to yours, but perhaps clearer because the names have been
> uniquified. Then your problems go away.

This is how I was thinking about implementing the lexical-let semantics. 
  To give more details:

I want to keep track of identifiers that are lexically bound in the 
current scope while compiling as well as to what gensym they are, and 
then instead of the fluid-ref's just reference/set this 
gensym-identifier when I come across an access.  It seems this is 
exactly what you describe above :)

So, that's it with regards to messages from me :D  Thanks for all your 
comments again,

Daniel

-- 
Done:  Arc-Bar-Cav-Ran-Rog-Sam-Tou-Val-Wiz
To go: Hea-Kni-Mon-Pri

Re: Elisp lexical-let

[Marijn Schouten (hkBst)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Daniel Kraft wrote:
> Hi,
> 
> thanks for your comments; I'm still a bit confused, though :$
> 
> Marijn Schouten (hkBst) wrote:
>>> What's about this:
>>>
>>> (defun test () a)
>>> (let ((a 1))
>>>   (print a) ; 1
>>>   (print (test)) ; 1
>>>   (lexical-set! a 2)
>> there is only one variable `a' in my mental model,
>> so this changes the value of the only `a' and all
>> subsequent expressions accessing `a' are thus affected.
>> The same as if you had written `(set! a 2)'. Dynamic
>> and lexical only differ in how free variables in
>> procedures are bound.
>>>   (print a) ; 1?
>> no, (print a) => 2
>>>   (print (test)) ; 1
>> no, there is only one `a' and its value is 2 here
>>>   (print (lexical a)) ; 2
>>>   )
> 
> Hm...  my problem is trying to understand how you want this implemented;
> my main point about lexical scoping is that it enables us to use Guile's
> built-in lexical mechanisms and we don't have to save the value
> explicitly into some fluids.
> 
> But if you require that the second (print (test)) above prints 1 even
> though we have done (lexical-set! a) this means that lexical-set! must
> update the place somehow where a is accessed dynamically (as is done in
> test).  And that seems to imply that this lexical-set! updates the
> fluids, even though it is meant to perform on a lexically bound variable
> a; just in case that "the one" a is at some place referred to dynamically.

In my proposal it is meaningless to say "lexically bound variable", as all
variables would in principal be accessible both lexically and dynamically.

>>> I don't think it's good to have to "completely seperate" variables a and
>>> (lexical a).
>>
>> I don't understand what you mean. My proposal is to have one kind of
>> variable
>> and two kinds of access.
> 
> Can you please elaborate on this?  If there's only one variable and only
> the value 2 after the lexical-set! above (both for (print a) in the
> lexical scope and (print (test)) which accesses a dynamically), what
> would then be the point of writing 'a' or '(lexical a)' and what would
> be the difference between those two?

(defvar x 3)
(defun dynx () x)
(dynx) ; => 3
(defun lexx () (lexical x))
(lexx) ; => 3
(defun incdynx () (setq x (+ x 1)))
(defun inclexx () (lexical-setq x (+ (lexical x) 1)))
(let ((x 5))
  (dynx) ; => 5
  (lexx) ; => 3
  (lexical-setq x 6)
  (dynx) ; => 6
  (lexx) ; => 3
  (setq x 7)
  (dynx) ; => 7
  (lexx) ; => 3
  (incdynx)
  (dynx) ; => 8
  (lexx) ; => 3
  (inclexx)
  (dynx) ; => 8
  (lexx) ; => 4
)
(dynx) ; => 4
(lexx) ; => 4
(incdynx)
(dynx) ; => 5
(lexx) ; => 5
(inclexx)
(dynx) ; => 6
(lexx) ; => 6

I hope I haven't made any mistakes there and that it explains what I have in mind.

Marijn

- --
If you cannot read my mind, then listen to what I say.

Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
<http://www.gentoo.org/proj/en/lisp/>, #gentoo-{lisp,ml} on FreeNode
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpplk8ACgkQp/VmCx0OL2wFVACfV36PIQRwjQu3gGOc8Rly6h8Z
UKQAnA5yPPMBNJsoRoc70+5znTP10r6G
=SL2/
-----END PGP SIGNATURE-----

Re: Elisp lexical-let

[Andy Wingo]

Hi Daniel,

On Fri 24 Jul 2009 09:08, Daniel Kraft <d@domob.eu> writes:

>> It's actually fairly simple, imo. Alpha-equivalence says that
>> (lexical-let ((x a)) x) is the same as (lexical-let ((y a)) y). (Note
>> that this lexical-let corresponds to Scheme's let.) So your program is
>> the same as:
>>
>>> (lexical-let ((y 2))
>>>   y ; -> 2
>>>   (foo) ; -> 1
>>>   (setq y 3)
>>>   y ; -> 3
>>>   (foo) ; -> 1
>>>   (let ((x 4))
>>>     x ; -> 4?
>>>     (foo) ; -> 4
>>>     (setq x 5)
>>>     x ; -> 5
>>>     (foo) ; -> 5
>>>     ) ; end the let
>>>   y ; -> 3?
>>>   (foo) ; -> 4
>>>   )
>>> x ; -> 4
>>> (foo) ; -> 4
>
> Yes of course, my main question was how the inner let is handled, and
> what dynamic values x will attain (the results of (foo)) -- and here's
> this funny stuff that the inner let will be like lexical-let for those
> variables already lexically bound.

Wow, strange. What were they thinking. But if that's the deal, that's
the deal...

Andy
-- 
http://wingolog.org/