Me no understand scoping

Me no understand scoping

[Maciek Godek]

Hi,
consider the following code (simple iteration
construct invented mainly to cause naming
conflict, as the function 'times' is already
defined in guile)

(define-macro (times n f)
  `(let ((env (the-environment)))
     (let loop ((i 0))
       (if (< i ,n)
           (begin
             (local-eval ,f env)
             (loop (1+ i))
             )
           )
       )
     )
  )

the whole thing with env was made as
a workaround disallowing f to see the
i variable defined in the macro (and to
perhaps see the value that otherwise
would be shadowed)

however,

(times 20 (display i))

yields

012345678910111213141516171819

Why?

--
Regards
M.

Re: Me no understand scoping

[Clinton Ebadi]

"Maciek Godek" <pstrychuj@gmail.com> writes:

> Hi,
> consider the following code (simple iteration
> construct invented mainly to cause naming
> conflict, as the function 'times' is already
> defined in guile)
>
> (define-macro (times n f)
>   `(let ((env (the-environment)))
>      (let loop ((i 0))
>        (if (< i ,n)
>            (begin
>              (local-eval ,f env)
>              (loop (1+ i))
>              )
>            )
>        )
>      )
>   )
>
> the whole thing with env was made as
> a workaround disallowing f to see the
> i variable defined in the macro (and to
> perhaps see the value that otherwise
> would be shadowed)
>
> however,
>
> (times 20 (display i))
>
> yields
>
> 012345678910111213141516171819
>
> Why?

`macroexpand-1' is helpful here -- (local-eval ,f env) is wrong.

As a matter of style, you probably want to avoid local-eval as it will
have to be removed whenever Guile ends up with a faster compiler (one
day when guile-vm is integrated into core and lexical environments
become fixed arrays or similar), and messing with existing lexical
environments is Very Bad (tm). Avoiding all uses of eval and, by
extension, local-eval is a good idea.

-- 
The hubbub of the waking life might close a door 
which in the dreamy Subliminal might remain ajar...

Re: Me no understand scoping

[Maciek Godek]

Clinton Ebadi:

> `macroexpand-1' is helpful here -- (local-eval ,f env) is wrong.
>
> As a matter of style, you probably want to avoid local-eval as it will
> have to be removed whenever Guile ends up with a faster compiler

On the other hand, local-eval is explained in guile manual,
while I couldn't find any documentation for guile's macroexpand-1,
nor for scheme macroexpand-1 (there are some explanations available
for the lisp language). So it's probably equally evil ;]

> (one day when guile-vm is integrated into core and lexical environments
> become fixed arrays or similar), and messing with existing lexical
> environments is Very Bad (tm). Avoiding all uses of eval and, by
> extension, local-eval is a good idea.

as for eval, it will always be there (in RnRS)
besides I don't imagine not messing with lexical
environments :) The bad thing is that scopes aren't
explicitly definable and the report doesn't say
much about their properties. I don't know how
about practical performance, but I'd find it best
if they were implemented as (implicit) hash tables
that could be bound to variables and passed as
arguments. I think that they could perform
equally good as fixed arrays for memoized
procedures.


Thanks,
M.

Re: Me no understand scoping

[Jon Wilson]

Maciek Godek wrote:

> as for eval, it will always be there (in RnRS)

EVAL has not been in every report.  R4RS did not specify EVAL, nor did 
R3RS, RRRS, RRS.

 > besides I don't imagine not messing with lexical
 > environments :) The bad thing is that scopes aren't
 > explicitly definable and the report doesn't say
 > much about their properties. I don't know how
 > about practical performance, but I'd find it best
 > if they were implemented as (implicit) hash tables
 > that could be bound to variables and passed as
 > arguments. I think that they could perform
 > equally good as fixed arrays for memoized
 > procedures.

The view of the scheme community as a whole is that first class lexical 
environments are irreconcilable with the need to compile (optimize) 
code.  In many cases, once you have done some variety of compilation, a 
lot of the lexical environments you think you ought to have will have 
been optimized away; conversely, preserving these environments will 
prevent entire classes of compilation.

I am of the suspicion that in most cases, the environment would never be 
reified, the compiler could prove this, and optimization could proceed 
as desired.  However, I would still agree with Clinton that you should 
avoid messing with environments, first class or otherwise, whenever 
possible.  When you must, you should quarantine it: use it in a very 
limited part of your code, where you wrap it in a more mundane 
abstraction, then use that abstraction elsewhere.  CALL/CC should be 
treated the same way usually.

Broadening the topic a bit, it seems that first class envs and macros 
(and maybe not macros!) are all that would be needed to define a module 
system in scheme, rather than building it into the interpreter.  Then 
the module system could be put in a file which any r5rs+envs (or 
whatever standard you like) scheme could LOAD (most likely implicitly at 
startup) without modification to the interpreter itself.  It might even 
be the case that DEFMACROs and reified environments would be sufficient 
to make low-level hygienic macros a library too (but I'm no macrologist, 
for sure).  You would still have to deal with the compilation issues, 
but as I said, my gut feeling is that these are solvable.
Regards,
Jon

Re: Me no understand scoping

[Klaus Schilling]

From: Jon Wilson <jsw@wilsonjc.us>
Subject: Re: Me no understand scoping
Date: Wed, 30 Jul 2008 10:03:21 -0400
> The view of the scheme community as a whole 

I don't care what the community as a whole says, first-class
environments are the best thing since sliced bread, as proven by
Queinnec and De Roure. They allow for a unified approach for many different
things involving code-sharing.

Klaus Schilling 

Re: Me no understand scoping

[Kjetil S. Matheussen]

Clinton Ebadi:
>
> As a matter of style, you probably want to avoid local-eval as it will
> have to be removed whenever Guile ends up with a faster compiler (one

I've never heard this before, and I think you are way wrong.
first-class environments etc. is what makes Guile special
and it would be insane to remove it since there are xx other very
good scheme implementations. No more first-class environments,
no more Guile for me. Well, I guess someone would fork Guile
very quick in case it happened though.

Re: Me no understand scoping

[Clinton Ebadi]

"Kjetil S. Matheussen" <k.s.matheussen@notam02.no> writes:

> Clinton Ebadi:
>>
>> As a matter of style, you probably want to avoid local-eval as it will
>> have to be removed whenever Guile ends up with a faster compiler (one
>
> I've never heard this before, and I think you are way wrong.
> first-class environments etc. is what makes Guile special
> and it would be insane to remove it since there are xx other very
> good scheme implementations. No more first-class environments,
> no more Guile for me. Well, I guess someone would fork Guile
> very quick in case it happened though.

Mutable first class environments available at runtime prevent a number
of useful optimizations. Immutable environments at runtime ought to be
fine (with nothing more than some space overhead). 

Mutable environments provided at compile time for macros should be
fine however. I think most everything you'd want to do with an
environment at runtime could be done at compile time instead, and
there is not much convenience lost as you could always recompile a
block to effect a change to its lexical environment.

-- 
And did those feet in ancient times
Walk bare upon these lonely streets like mine?
Does God watch us from that penthouse high above
His children down below who live on air and love?

Re: Me no understand scoping

[Maciek Godek]

>> as for eval, it will always be there (in RnRS)
>
> EVAL has not been in every report.  R4RS did not specify EVAL, nor did R3RS,
> RRRS, RRS.

The fact that R5RS does only proves the fact that they should have :)

By the way I have a question (regarding not using eval).
I want to pass the contents of a list to a function that accepts the
so-called "rest" args.
(let ((l '(1 2 3 4 5)))
   (let ((operation (append '(+) l)))
      (primitive-eval operation)
  )
)

How to achieve this effect without using eval?
(I've tried (+ . l) but it didn't work out)

>> besides I don't imagine not messing with lexical
>> environments :) The bad thing is that scopes aren't
>> explicitly definable and the report doesn't say
>> much about their properties. I don't know how
>> about practical performance, but I'd find it best
>> if they were implemented as (implicit) hash tables
>> that could be bound to variables and passed as
>> arguments. I think that they could perform
>> equally good as fixed arrays for memoized
>> procedures.
>
> The view of the scheme community as a whole is that first class lexical
> environments are irreconcilable with the need to compile (optimize) code.

Certainly there's a faction that thinks differently :]

>  In many cases, once you have done some variety of compilation, a lot of the
> lexical environments you think you ought to have will have been optimized
> away; conversely, preserving these environments will prevent entire classes
> of compilation.

Unless some limitations are to be imposed on compiled closures
(and some control over optimization is given to the programmer)
I see no problem with that.

> I am of the suspicion that in most cases, the environment would never be
> reified, the compiler could prove this, and optimization could proceed as
> desired.  However, I would still agree with Clinton that you should avoid
> messing with environments, first class or otherwise, whenever possible.
>  When you must, you should quarantine it: use it in a very limited part of
> your code, where you wrap it in a more mundane abstraction, then use that
> abstraction elsewhere.  CALL/CC should be treated the same way usually.

Yeah, I always write additional layers so the code corresponds to the
way I think. (Everybody does, don't they?)

> Broadening the topic a bit, it seems that first class envs and macros (and
> maybe not macros!) are all that would be needed to define a module system in
> scheme, rather than building it into the interpreter.  Then the module
> system could be put in a file which any r5rs+envs (or whatever standard you
> like) scheme could LOAD (most likely implicitly at startup) without
> modification to the interpreter itself.  It might even be the case that
> DEFMACROs and reified environments would be sufficient to make low-level
> hygienic macros a library too (but I'm no macrologist, for sure).  You would
> still have to deal with the compilation issues, but as I said, my gut
> feeling is that these are solvable.

Such a pretty vision you have :)
("and no religion too..." :P)

The thing is that this would really make *everything* simpler.
(or maybe it's just another collective hallucination)

Re: Me no understand scoping

[Neil Jerram]

2008/7/30 Maciek Godek <pstrychuj@gmail.com>:
>
> By the way I have a question (regarding not using eval).
> I want to pass the contents of a list to a function that accepts the
> so-called "rest" args.
> (let ((l '(1 2 3 4 5)))
>   (let ((operation (append '(+) l)))
>      (primitive-eval operation)
>  )
> )
>
> How to achieve this effect without using eval?
> (I've tried (+ . l) but it didn't work out)

(apply + l)

>> The view of the scheme community as a whole is that first class lexical
>> environments are irreconcilable with the need to compile (optimize) code.
>
> Certainly there's a faction that thinks differently :]

If and when we have an optimized compiler, I imagine there would be
some way of allowing the environment to be optimized away when not
explicitly wanted, but kept available when it is wanted.

> Yeah, I always write additional layers so the code corresponds to the
> way I think. (Everybody does, don't they?)

Yes!

Regards,
      Neil

Re: Me no understand scoping

[Maciek Godek]

Neil Jerram <neiljerram@googlemail.com>:

>> How to achieve this effect without using eval?
>> (I've tried (+ . l) but it didn't work out)
>
> (apply + l)

Correct :)
Here comes another one:
Suppose I want to define a variable, but I don't know its name
-- it is contained in another variable. For example:
(define a 'b)
I want to assign a value to the symbol "contained" in a.
(that would be b in this example). The problem is that "define"
quotes its first argument. How to achieve it?

>> Yeah, I always write additional layers so the code corresponds to the
>> way I think. (Everybody does, don't they?)
>
> Yes!

'(:-)

Re: Me no understand scoping

[Neil Jerram]

2008/7/31 Maciek Godek <pstrychuj@gmail.com>:

> Here comes another one:
> Suppose I want to define a variable, but I don't know its name
> -- it is contained in another variable. For example:
> (define a 'b)
> I want to assign a value to the symbol "contained" in a.
> (that would be b in this example). The problem is that "define"
> quotes its first argument. How to achieve it?

I'm not so sure about this one, but I think that

(eval `(define ,(car a) 1))

should work.

Now, stepping back a bit - and purely out of interest - did you
mention before why you are doing these strange things?  Is it your way
of exploring and learning, or is there (also) a specific requirement?

    Neil

Re: Me no understand scoping

[Maciek Godek]

>> Here comes another one:
>> Suppose I want to define a variable, but I don't know its name
>> -- it is contained in another variable. For example:
>> (define a 'b)
>> I want to assign a value to the symbol "contained" in a.
>> (that would be b in this example). The problem is that "define"
>> quotes its first argument. How to achieve it?
>
> I'm not so sure about this one, but I think that
>
> (eval `(define ,(car a) 1))
>
> should work.

I eventually came up with something similar:
(define (define@ name value) (primitive-eval `(define ,name ,value)))

> Now, stepping back a bit - and purely out of interest - did you
> mention before why you are doing these strange things?  Is it your way
> of exploring and learning, or is there (also) a specific requirement?

I can't tell you precisely (because I don't know myself).
It's just that whenever I try to do something in scheme,
I end up with questions of that sort. (Maybe the thing is
that I try to write as little code as possible -- the prettiness
and simplicity of scheme convinces me to do so).

I'm currently working on a little project (in my spare time).
I can't give you the source right now, because it's in its
infancy and there's definitely nothing to boast with.

However, I wrote the specification about a week ago
(originally it was meant to be a post to this group,
but as I wrote it, it replied to all my doubts and so
there was nothing to ask).

I hereby copy the contents of the specification file (to feed
your curiosity):

=BEGIN

Dear fellows,
For some time, I've been designing a network-oriented
object system, with interface callable from client side
and executed on server.

The toughest issue I have to deal with is the implementation
of RPC protocol.

The idea is fairly simple. Each client connected to
a server may request an object to be created, so the client
side session could look like this:

(noos-init)
(define connection (noos-connect address port))
; on success, noos-connect returns a valid connection handler
; or #f if fails
(if (not connection) (quit))

(noos-get-classes connection)
; noos-get-classes returns the list of valid noos-classes.
=> (class1 class2 class3)

; We can further inquire for the definition of a given class:
(noos-declaration connection 'class1)

; A noos-class declaration consists of:
; - list of public variables (readable by all clients connected)
; - list of private variables (readable by the owner of the class)
; - list of interface function prototypes. From the clients'
;   point of view, they consist of the name of the function and
;   the list of its parameters.
=> ((v1 v2 v3)(h1 h2 h3)((set-private v1 v2 v3)(do-something-else a1)))

; Knowing the interface, we can create instances of classes
(define y (noos-instance 'class1))
; noos-instance returns an object handler. it contains a local
; copy of hidden and public variables for immediate usage:
(y 'public)
=> #(0 0 0)
(y 'private)
=> #(0 0 0)
(y 'v1)
=> 0

; As you can see, the state is kept inside vectors, so they
; can be accessed very quickly.

; We can call the object's member functions:

(y 'set-private 1 2 3)
(y 'do-something-else "like what?")

; Note we can only pass numbers or strings as arguments,
; therefore the following calls would be invalid (regardless
; of the internal method definitions)

(!) (in y (set-public 'symbol1 (the-environment) #f))
(!) (in y (do-something-else (make-hash-table)))

; As the noos can be implemented over the UDP protocol,
; there's no guarrantee that the variables of out obiects
; are updated.

(y 'private)
=> (0 0 0)

; We can force it to do so using the `update'
; procedure if needed:

(update y)
(y 'private)
=> (1 2 3)

; It doesn't mean that the normal invocations of member
; functions are unable to change that state on client
; machine. If the packet is properly delivered to server,
; it immadietly changes the state of an object on the server.
; It also sends an event to a client -- it contains the new
; state of the object

; The update function sends request to a server and
; waits for the event to occur. If it doesn't (within
; specified amount of time), it sends it again and
; again, until the timeout is exceeded and then returns
; #f. If the response is sent properly, it returns #t
; (or something else different from #f)

; There's also another way of making sure that the
; remote member call is performed properly:

(assure (y 'set-public 1 2 3))

; This isn't currently implemented, but it looks tempting :D

         == SERVER SIDE ==

; OK, now that we know how the system looks like from client's
; point of view, it's time to see it from the server's perspective.

; First of all, there are class definitions

(noos-class class1
  (v1 v2 v3)
  (h1 h2 h3)
  (
    (set-private a1 a2 a3) (
      (set! v1 a1)
      (set! v2 a2)
      (set! v3 a3)
      (send-event owner state)
    )
    (do-something-else a1) (
      (send-event owner "do you really want to `" a1 "'")
    )
  )
  ; watch out! the next line defines an update function
  ; -- more about this later
  ((set! v1 (+ v1 h1)) (set! v2 (+ v2 h2)) (set! v3 (+ v3 h3)))
)

; As you can see, there are some variables of special
; meaning: namely 'owner' and 'state' (and a few others,
; like 'private', 'public' and 'neighbors' -- the last
; one to be explained later)

; noos adds a fairly sophisticated concept of meetings to
; the traditional object-oriented view on programming.
; In this concept, the world of objects is divided into
; subspaces (that are nothing but special objects that
; just hold references to their children)

; Obviously, there's a good reason for it. Under certain
; circumstances, two objects can meet. The circumstances
; are specified by a meeting predicate, ie.

(meeting-circumstances ((<sphere> x) (<sphere> y))
  (and
    (< (- (x 'v1) (y 'v1)) 1)
    (< (- (x 'v2) (y 'v2)) 1)
    (< (- (x 'v3) (y 'v3)) 1)
  )
)

       == IMPLEMENTATION ==

1. PROTOCOL

Firstly let's focus on how the underlaying protocol is
implemented. Information about the sender is contained
in the header of UDP packet -- therefore we don't have
to transmit any additional data.

; The server maintains a hash table in which all clients'
; addresses are stored.

(define clients-by-address (make-hash-table))

; When a client sends its first packet, a
; new entry in the hash table is created, assigning
; the client a unique identifier.

(define clients '())

(let ((cl (hash-ref clients-by-address address)))
  (if (not cl)
    (begin
      (set! clients (cons (new-client) clients))
      (hash-set! clients-by-address address (car clients))
    )

  (let ((object (vector-ref cl ob)))
    (let ((function (vector-ref (object 'interface)) fn))
      (local-call object function args)
    )
  )
)

; Is that all? Well, almost. The presented code assumes that
; when a packet is received, it is parsed to the variables
; ob, fn and args that are further processed by us.

; In practice that code is implemented in C (hoping it to
; perform better), as well as our parsing function.

; Current implementation uses the portable SDL_Net library
; to perform data exchange over UDP protocol.

; The protocol is text based and rather simple: client packets
; consist of an object number followed by a function number
; followed by argument list. The packets are therefore very
; lispish, although the types are limited to numbers, strings
; and lists.

; An example packet could look like this:
0 0 (1 (2 3))0+2i 3.4"one""two" (NULL-TERMINATED)

; Officially a single space is a separator, but the parser
; is written liberally and treats any sequence of white spaces
; as a separator. Like in lisp, no additional separator is needed
; after parentheses and double-quotes.

; Certainly this implementation of protocol isn't too efficient
; but an additional compression tunnel can be used in the future
; to provide faster data exchange (my goal now is just to make a
; working system, alright?)

// And this is how the packet is parsed (note we
// take advantage of the fact that the packets are
// meant to be null-terminated strings):

SCM parse(const char *packet) {
    static char *arglist[1024]; // or whatever the limitation of UDP is
    sprintf(arglist, "(quote (%s) )", packet);
    return eval(arglist);
}

// obviously, in this form the code is rather dangerous and
// it requires a more sophisticated control -- you have to be
// aware of this fact when using current implementation of
// noos. (don't say I didn't warn you). On the other hand,
// since the code here is quoted, it won't be executed unless
// you make it do so.

SCM parsed = parse(packet.data);
int ob = scm_to_int(car(parsed));
int fn = scm_to_int(cadr(parsed));
SCM args = cddr(parsed);

// And that's how it's done on server.
; Situation on client machine is to some extent similar.
; In this case, however, the object's method simply invokes
; a `remote-call' function with specified parameters.
; The remote call of a member function consists of:
; - getting an object id from object handler
; - getting a function id from object handler
; - passing these among with the rest arguments to a function
;   that builds a packet
; - sending the packet to the server.

; The other thing that can happen is the server signialising
; an event. This is also a remote-call, but invoked by the
; server and executed by the client.

=END

Surprisingly, this system isn't just a whim of a sick.
It's only a tiny part of a much larger whim: I'm making
a game engine. I already have a customizable input
system (it's the best: you can bind any thunk to any
key or any binary function to mouse move) and I'm
planning to make guile bindings to ogre3d display
engine (and several others) later on.

Obviously everybody's free to join :)
(but I warn you that I'm terribly fussy if it comes
to programming style (esp. my own), very likely
to change function and variable names and that
I don't have much experience in collective
programming)

I can let you know when I get the first working implementation
if you want.

Thanks for attention :)
M.

PS gdm rocks! (thankyouthankyouthankyou!! it really did change
my way of writing the code)

Re: Me no understand scoping

[Clinton Ebadi]

"Maciek Godek" <pstrychuj@gmail.com> writes:

> Neil Jerram <neiljerram@googlemail.com>:
>
>>> How to achieve this effect without using eval?
>>> (I've tried (+ . l) but it didn't work out)
>>
>> (apply + l)
>
> Correct :)
> Here comes another one:
> Suppose I want to define a variable, but I don't know its name
> -- it is contained in another variable. For example:
> (define a 'b)
> I want to assign a value to the symbol "contained" in a.
> (that would be b in this example). The problem is that "define"
> quotes its first argument. How to achieve it?

module-define! MODULE NAME VALUE

e.g. 
(let ((foo 'bar))
 (module-define! (current-module) foo 5)
 bar)
=> 5

Issue: this will be a top level binding, but a solution using `eval'
would also produce a top level binding (as it executes there to allow
lexical environments to be optimized...).

Doing such things, however, is fairly unschemey.

-- 
emacsen: every copy of Emacs comes with a bag of pot and 5 hits of acid
emacsen: and a hotel coffee maker

Re: Me no understand scoping

[Maciek Godek]

2008/8/1 Clinton Ebadi <clinton@unknownlamer.org>:
>
> module-define! MODULE NAME VALUE
>
> e.g.
> (let ((foo 'bar))
>  (module-define! (current-module) foo 5)
>  bar)
> => 5
>
> Issue: this will be a top level binding, but a solution using `eval'
> would also produce a top level binding (as it executes there to allow
> lexical environments to be optimized...).

I've been trying to make it using local-eval to be able
to make a definition within a closure. There's only a
tiny problem: it's not working. When I execute:
(local-eval (list 'define a 5) (the-environment))
everything works fine, but when I make a function of it,
(define (define@ name value) (local-eval (list 'define name value)
(the-environment)))
and when I invoke it, it breaks with "Bad define placement".
The problem gets solved when I make a macro of it:
(define-macro (define@ name value) `(local-eval (list define ,name
,value)(the-environment)))
but it still breaks when called from a nested local-eval.

> Doing such things, however, is fairly unschemey.

What things and why?

What I'm trying to do is to understand the rules of making
and using environments. As for now, these rules seem to be quite
random and unpredictable (so it's rather hard to call them rules
actually), which is sad because closures are a very powerful
yet simple mechanism (definitely simpler than what GOOPS or
CLOS have to offer if it comes to OOP -- both in concepts and
implementation)

See you around :)
M.

Re: Me no understand scoping

[Jon Wilson]

Time to trot out the tired old koan [1] ...

Maciek Godek wrote:
> which is sad because closures are a very powerful
> yet simple mechanism (definitely simpler than what GOOPS or
> CLOS have to offer if it comes to OOP -- both in concepts and
> implementation)

"  The venerable master Qc Na was walking with his student, Anton. 
Hoping to
prompt the master into a discussion, Anton said "Master, I have heard that
objects are a very good thing - is this true?"  Qc Na looked pityingly at
his student and replied, "Foolish pupil - objects are merely a poor man's
closures."

"  Chastised, Anton took his leave from his master and returned to his cell,
intent on studying closures.  He carefully read the entire "Lambda: The
Ultimate..." series of papers and its cousins, and implemented a small
Scheme interpreter with a closure-based object system.  He learned much, and
looked forward to informing his master of his progress.

"  On his next walk with Qc Na, Anton attempted to impress his master by
saying "Master, I have diligently studied the matter, and now understand
that objects are truly a poor man's closures."  Qc Na responded by hitting
Anton with his stick, saying "When will you learn? Closures are a poor man's
object."  At that moment, Anton became enlightened.


[1] http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg03277.html

Re: Me no understand scoping

[Neil Jerram]

2008/8/1 Maciek Godek <pstrychuj@gmail.com>:
>
> I can't tell you precisely (because I don't know myself).
> It's just that whenever I try to do something in scheme,
> I end up with questions of that sort. (Maybe the thing is
> that I try to write as little code as possible -- the prettiness
> and simplicity of scheme convinces me to do so).
>
> I'm currently working on a little project (in my spare time). [...]

Many thanks for providing this description.  The project looks fun, so
I hope it continues to go well.

A few thoughts occurred to me when reading through.

1. IMO this could be really beautifully done in GOOPS, by defining
custom metaclasses and slot types.

2. You say that your protocol is lisp-ish.  I have found it really
useful, both in GDS and in another client/server project, to express
the protocol entirely in lisp forms.  Simply because this means that
you can use `read' to read exactly one protocol instruction; no
parsing required!

3. It's still not obvious to me how this ends up using the-environment
and local-eval.  If class variables never interact directly with the
real environment, they could be just stored in a hash table..

> PS gdm rocks! (thankyouthankyouthankyou!! it really did change
> my way of writing the code)

Do you mean GDS?  If so, thanks for your thanks!

Regards,
         Neil

Re: Me no understand scoping

[Maciek Godek]

> Time to trot out the tired old koan [1] ...
>
> "  The venerable master Qc Na was walking with his student, Anton. Hoping to
> prompt the master into a discussion, Anton said "Master, I have heard that
> objects are a very good thing - is this true?"  Qc Na looked pityingly at
> his student and replied, "Foolish pupil - objects are merely a poor man's
> closures."
>
> "  Chastised, Anton took his leave from his master and returned to his cell,
> intent on studying closures.  He carefully read the entire "Lambda: The
> Ultimate..." series of papers and its cousins, and implemented a small
> Scheme interpreter with a closure-based object system.  He learned much, and
> looked forward to informing his master of his progress.
>
> "  On his next walk with Qc Na, Anton attempted to impress his master by
> saying "Master, I have diligently studied the matter, and now understand
> that objects are truly a poor man's closures."  Qc Na responded by hitting
> Anton with his stick, saying "When will you learn? Closures are a poor man's
> object."  At that moment, Anton became enlightened.
>
>
> [1] http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg03277.html

I only wonder if the idea of explicit closures was floating in the air at the
time when this koan was coined. Does anybody know?
(because I can't loose the impression that explicit closures
could be the rich man's objects)

M.

Re: Me no understand scoping

[Maciek Godek]

>> I'm currently working on a little project (in my spare time). [...]
>
> Many thanks for providing this description.  The project looks fun, so
> I hope it continues to go well.
>
> A few thoughts occurred to me when reading through.
>
> 1. IMO this could be really beautifully done in GOOPS, by defining
> custom metaclasses and slot types.

I've been considering that, and I'm still having doubts.
The main reason is that there's no documented way
of accessing GOOPS objects from C (except from using
scm_c_eval_string etc.), or at least I couldn't find any
documentation for that.
Besides (which is the matter of personal taste), I don't
like the idea of using generics and trashing the global
namespace with them. (I mean, the sole idea of generics
is fine, but I wouldn't want to be forced to use them)

I also get this unpleasant feeling that all these 'getters'
and 'setters' are entities multiplied beyond necessity
(even the infamous C++ doesn't explicate them)

I'm really trying to get close
to the classical OOP notation: object.method() -- and
it's probably why I explore the potential of using these
"poor man's objects"

> 2. You say that your protocol is lisp-ish.  I have found it really
> useful, both in GDS and in another client/server project, to express
> the protocol entirely in lisp forms.  Simply because this means that
> you can use `read' to read exactly one protocol instruction; no
> parsing required!

Yeah, I had a similar idea, although at first I thought of
making a more efficient (but less elastic) binary protocol
that would require some mechanisms to convert various
supported guile types to binary form. I eventually concluded
that it's better to keep everything textual not only to praise
my laziness, but also it would be better to make a statistical
information-theory based compression protocol to speed
it up (if it turns out necessary).

> 3. It's still not obvious to me how this ends up using the-environment
> and local-eval.  If class variables never interact directly with the
> real environment, they could be just stored in a hash table..

I'm just looking for the most expressive way to use the language.
I've first heard of lisp like two years ago or so and even started
to learn that language from David Lamkins' "Successful Lisp"
and caught some of its spirit from Petet Graham's "On Lisp",
but I always kept getting distracted, finding other things to do.
But the point is that I saw that there is a 'make-hash-table' function
available in lisp -- and this lead me to the conclusion that it's probably
because the scopes/closures/environments implicitly use hash
tables to store their bindings (and the same mechanism was given
explicitly to the programmer).
And so I never stopped to believe that (define x 5) is more or
less equivalent to (hash-set! global-scope 'x 5).

>> PS gdm rocks! (thankyouthankyouthankyou!! it really did change
>> my way of writing the code)
>
> Do you mean GDS?  If so, thanks for your thanks!

Yes, it's a great piece of software, regardless of its name :)

There's no need for your thanks for my thanks as they are deserved.


M.

Re: Me no understand scoping

[Neil Jerram]

2008/8/2 Maciek Godek <pstrychuj@gmail.com>:
>> 1. IMO this could be really beautifully done in GOOPS, by defining
>> custom metaclasses and slot types.
>
> I've been considering that, and I'm still having doubts.
> The main reason is that there's no documented way
> of accessing GOOPS objects from C (except from using
> scm_c_eval_string etc.), or at least I couldn't find any
> documentation for that.

Yes, that's a fair point.  And one that keeps coming up.  I hope we
can remedy this some time soon.

> Besides (which is the matter of personal taste), I don't
> like the idea of using generics and trashing the global
> namespace with them. (I mean, the sole idea of generics
> is fine, but I wouldn't want to be forced to use them)

They can of course be confined to modules, like any other names.

> I also get this unpleasant feeling that all these 'getters'
> and 'setters' are entities multiplied beyond necessity
> (even the infamous C++ doesn't explicate them)

I don't understand you here.  And in any case, getters and setters are
optional.  You can just use slot-ref and slot-set! if you prefer.

> I'm really trying to get close
> to the classical OOP notation: object.method() -- and
> it's probably why I explore the potential of using these
> "poor man's objects"

Unfortunately that's only "classical" for non-Lisp-like languages!
Still, I'd take your point if you said you wanted to write something
that would be familiar to OO programmers in non-Lisp languages.

> But the point is that I saw that there is a 'make-hash-table' function
> available in lisp -- and this lead me to the conclusion that it's probably
> because the scopes/closures/environments implicitly use hash
> tables to store their bindings

Right (more or less).

> (and the same mechanism was given
> explicitly to the programmer).

You mean in the sense that can one create and manipulate a hash table
where the keys are symbols?  I'd certainly agree with that.

> And so I never stopped to believe that (define x 5) is more or
> less equivalent to (hash-set! global-scope 'x 5).

Well yes.... but I'm afraid I'm still not sure how that leads us to
local-eval !  Sorry if I'm being dumb about this!

Regards,
         Neil

Re: Me no understand scoping

[Andy Wingo]

Hi Maciek,

On Sat 02 Aug 2008 23:36, "Maciek Godek" <pstrychuj@gmail.com> writes:

> [Neil]
>> 1. IMO this could be really beautifully done in GOOPS, by defining
>> custom metaclasses and slot types.
>
> I've been considering that, and I'm still having doubts.
> The main reason is that there's no documented way
> of accessing GOOPS objects from C (except from using
> scm_c_eval_string etc.), or at least I couldn't find any
> documentation for that.

You can use scm_slot_ref et al. See goops.h.

> Besides (which is the matter of personal taste), I don't
> like the idea of using generics and trashing the global
> namespace with them. (I mean, the sole idea of generics
> is fine, but I wouldn't want to be forced to use them)
[...]
> I'm really trying to get close
> to the classical OOP notation: object.method() -- and
> it's probably why I explore the potential of using these
> "poor man's objects"

I wrote about this notational issue a while back on my blog,
http://wingolog.org/; I'm offline at the moment, so unfortunately I
don't have a link. Search for "slot-ref". I think with-accessors is an
elegant solution to this problem.

> But the point is that I saw that there is a 'make-hash-table' function
> available in lisp -- and this lead me to the conclusion that it's probably
> because the scopes/closures/environments implicitly use hash
> tables to store their bindings (and the same mechanism was given
> explicitly to the programmer).

This is false. Consider the closure:

  (let ((val 0))
    (lambda ()
      (set! val (1+ val))
      val))

Lexical scoping + closures was one of the fundamental ideas of scheme.
You can analyze this code /lexically/ to determine that we only need to
allocate storage for one variable. Wherever you see `val' in the body of
the lambda, you know /lexically/ that you are referring to a location
that is one step out in the stack frame, and the 0th location in that
frame, the frame created by `let'. So there is no need to store the
name, "val", at all!

By way of illustration, look at this disassembly of that closure in
guile-vm:

    scheme@(guile-user)> (let ((val 0)) (lambda () (set! val (1+ val)) val))
    $1 = #<program b6fbf388>
    scheme@(guile-user)> ,x $1
    Disassembly of #<program b6fbf388>:

    nargs = 0  nrest = 0  nlocs = 0  nexts = 0

    Bytecode:

       0    (late-variable-ref 0)
       2    (external-ref 0)
       4    (call 1)
       6    (external-set 0)
       8    (external-ref 0)
      10    (return)

    Objects:

       0    ((guile-user) . 1+)

    Externals:

       0    0

    Sources:

       6    #(1 36 #f)
       8    #(1 26 #f)

The external shown there, "0 0" refers exactly to the outer stack frame,
the zeroeth location. The external-ref and external-set instructions
manipulate that storage location /by index and not by name/.

Do you see now why local-eval can't possibly work in the presence of
efficient compilation? Scheme does not give you the particular kind of
dynamism that you want. There is no hash table lurking inside a closure.

On the other hand, modules do have hash tables, and modules are
evaluation environments; and they have been treated in some literature
as closures. Perhaps consider using modules as your first-class objects?

> And so I never stopped to believe that (define x 5) is more or
> less equivalent to (hash-set! global-scope 'x 5).

At the top level it is; but s/global-scope/the current module/.
But internal defines are completely different.

Happy hacking,

Andy
-- 
http://wingolog.org/

Re: Me no understand scoping

[Maciek Godek]

2008/8/8 Neil Jerram <neiljerram@googlemail.com>:

>> The main reason is that there's no documented way
>> of accessing GOOPS objects from C (except from using
>> scm_c_eval_string etc.), or at least I couldn't find any
>> documentation for that.
>
> Yes, that's a fair point.  And one that keeps coming up.  I hope we
> can remedy this some time soon.

I obviously started to develop my own OO system -- purely
to understand everything. By the way I came up with some
conclusions regarding objects and classes and that is:
- a class can be implemented as a vector of three elements:
  * vector of methods (closures)
  * hash table mapping method names to their closures
  * hash table mapping slot names to vector indices.
- an object (instance) could also be implemented as a
simple vector -- its first field would be a reference to its
class, and the remaining ones would be the slots.

Interfacing such a system from the C level is rather
straightforward (or at least as straightforward as accessing
scm_vectors) and efficient.
(Perhaps goops does it more efficiently, I dunno)

>> I also get this unpleasant feeling that all these 'getters'
>> and 'setters' are entities multiplied beyond necessity
>> (even the infamous C++ doesn't explicate them)
>
> I don't understand you here.  And in any case, getters and setters are
> optional.  You can just use slot-ref and slot-set! if you prefer.

I bet you're right. I'm just learning and trying to justify all the
features that are provided by goops. Anyway, I can't get rid of
a feeling that many of them are redundant and quite random,
but I'm still examining it.

>> I'm really trying to get close
>> to the classical OOP notation: object.method() -- and
>> it's probably why I explore the potential of using these
>> "poor man's objects"
>
> Unfortunately that's only "classical" for non-Lisp-like languages!
> Still, I'd take your point if you said you wanted to write something
> that would be familiar to OO programmers in non-Lisp languages.

Fortunately Andy Wingo recommended his essay about this.

>> And so I never stopped to believe that (define x 5) is more or
>> less equivalent to (hash-set! global-scope 'x 5).
>
> Well yes.... but I'm afraid I'm still not sure how that leads us to
> local-eval !  Sorry if I'm being dumb about this!

Its genesis reaches the Kejtil's suggestion for accessing closure's
scope. The fact that it is possible in guile lead me to the conclusion
that it can be done to implement a simple oo system. (I'm just
beginning to realize how inefficient this can be).


Thanks a lot
M.

Re: Me no understand scoping

[Maciek Godek]

Hello Andy!

>> [Neil]
>>> 1. IMO this could be really beautifully done in GOOPS, by defining
>>> custom metaclasses and slot types.
>>
>> I've been considering that, and I'm still having doubts.
>> The main reason is that there's no documented way
>> of accessing GOOPS objects from C (except from using
>> scm_c_eval_string etc.), or at least I couldn't find any
>> documentation for that.
>
> You can use scm_slot_ref et al. See goops.h.

I took a glimpse. The one thing that stinks to me is an
immanent inefficiency which comes from the fact that
the slots are looked up by their names and therefore
aren't accessed directly. (I wrote a follow-up to Neil's
post in this thread with an efficient solution)

>> Besides (which is the matter of personal taste), I don't
>> like the idea of using generics and trashing the global
>> namespace with them. (I mean, the sole idea of generics
>> is fine, but I wouldn't want to be forced to use them)
> [...]
>> I'm really trying to get close
>> to the classical OOP notation: object.method() -- and
>> it's probably why I explore the potential of using these
>> "poor man's objects"
>
> I wrote about this notational issue a while back on my blog,
> http://wingolog.org/; I'm offline at the moment, so unfortunately I
> don't have a link. Search for "slot-ref". I think with-accessors is an
> elegant solution to this problem.

I've been looking for such thing for a month or so, it's brilliant
(or at least on a good way to be -- I have to examine it in more
detail). BTW I'm placing the link here for the others:
http://wingolog.org/archives/2008/04/22/object-closure-and-the-negative-specification

>> But the point is that I saw that there is a 'make-hash-table' function
>> available in lisp -- and this lead me to the conclusion that it's probably
>> because the scopes/closures/environments implicitly use hash
>> tables to store their bindings (and the same mechanism was given
>> explicitly to the programmer).
>
> This is false. Consider the closure:
>
>  (let ((val 0))
>    (lambda ()
>      (set! val (1+ val))
>      val))
>
> Lexical scoping + closures was one of the fundamental ideas of scheme.
> You can analyze this code /lexically/ to determine that we only need to
> allocate storage for one variable. Wherever you see `val' in the body of
> the lambda, you know /lexically/ that you are referring to a location
> that is one step out in the stack frame, and the 0th location in that
> frame, the frame created by `let'. So there is no need to store the
> name, "val", at all!
[...]
> Do you see now why local-eval can't possibly work in the presence of
> efficient compilation? Scheme does not give you the particular kind of
> dynamism that you want. There is no hash table lurking inside a closure.

Yeah, you've convinced me. I was stupid and irresponsible but now I can
see clearly why local-eval shouldn't be abused.

> On the other hand, modules do have hash tables, and modules are
> evaluation environments; and they have been treated in some literature
> as closures. Perhaps consider using modules as your first-class objects?

Thanks a lot, I think it was the second thing I've been looking for.
I'll read some more on that.

>> And so I never stopped to believe that (define x 5) is more or
>> less equivalent to (hash-set! global-scope 'x 5).
>
> At the top level it is; but s/global-scope/the current module/.
> But internal defines are completely different.

I've heard that in current implementation of guile they are only
different in that they are not storing data in hash-tables but in
assoc lists :) (That's what Ludovic says)

Thanks a lot
M.

Re: Me no understand scoping

[JonWilson]

Hi Andy,

Andy Wingo wrote:

> Do you see now why local-eval can't possibly work in the presence of
> efficient compilation? Scheme does not give you the particular kind of
> dynamism that you want. There is no hash table lurking inside a closure.

Why not compile most closures away, but leave those that are necessary
for the sort of semantics Maciek wants to work?  Maciek more or less
wants to reify environments (it seems), and have those be semantically
the same as the variable lookup mechanism inside closures.

It is certainly true that most continuations are not reified, many are
compiled away, but it is still possible to reify exactly the ones that
are needed and no others.  Is it impossible for program analysis to
reveal which environments need to be reified?  If necessary, one could
even require some sort of explicit tag on lambdas to say "This one gets
reified."

Of course, #scheme is probably a good place to discuss this sort of
thing, but I can't really hold my own once a really technical discussion
gets started there...  Whether that's because I lack the expertise
(likely) or because certain personalities are perhaps overly dominating
(also likely), I don't know.

Regards,
Jon