How does letf work?

How does letf work?

[Florian Beck]

The docstring of letf says: "This is the analogue of `let', but with 
generalized variables (in the sense of `setf') for the PLACEs."

However, with setf, I get this:

(defvar test-x '(KEY 1 2 3 4))

(let ((x (copy-list test-x)))
   (setf (cdr x) '(a b c d))
   x)

=> (KEY a b c d)

(Without copy-list, the global variable is modified.)

Yet this doesn't work with letf:

(letf (((cdr test-x) '(a b c d)))
   test-x)

= > (KEY 1 2 3 4)

So, either "generalized variables" in the sense of setf are different 
from the sense of letf or -- more likely -- I don't understand how they 
work. Any hints?

-- 
Florian

Re: How does letf work?

[Michael Heerdegen]

Florian Beck <fb@miszellen.de> writes:

> (letf (((cdr test-x) '(a b c d)))
>   test-x)
>
> = > (KEY 1 2 3 4)

Though paradoxical, it seems right to me.  You return a list that you
modified temporarily, but when the result is printed in the echo area,
the letf has been left and the structure of the list has changed back.

Note that the variable test-x is not shadowed by your letf:

(let ((test-x-before test-x))
  (letf (((cdr test-x) '(a b c d)))
    (eq test-x test-x-before)))

==> t

But it has the expected value at that "point of time":

(letf (((cdr test-x) '(a b c d)))
  (message "%s" test-x))

"(KEY a b c d)"


Here is a similarly paradoxical example without letf:

(setq test-x '(KEY 1 2 3 4))
(let ((old-cdr (cdr test-x)))
  (prog2
      (setf (cdr test-x) '(a b c d))
      test-x ;returned value
    (setf (cdr test-x) old-cdr)))

==> (KEY 1 2 3 4)



Regards,

Michael.

Re: How does letf work?

[Joost Kremers]

Michael Heerdegen wrote:
> Florian Beck <fb@miszellen.de> writes:
>
>> (letf (((cdr test-x) '(a b c d)))
>>   test-x)
>>
>> = > (KEY 1 2 3 4)
>
> Though paradoxical, it seems right to me.  You return a list that you
> modified temporarily, but when the result is printed in the echo area,
> the letf has been left and the structure of the list has changed back.

In essence, yes, but it's not as simple as all that:

(letf (((cdr test-x) '(a b c d)))
  (cdr test-x))

= > (KEY a b c d)

Taking what you say at face value, one might expect that at the moment
the value of (cdr test-x) is printed, it's already been changed back to
the original value, so it should say (KEY 1 2 3 4), but it doesn't. (I
think that in a way, you are in fact correct, but the issue is more
complicated than your formulation suggests; see below.)

> Note that the variable test-x is not shadowed by your letf:
>
> (let ((test-x-before test-x))
>   (letf (((cdr test-x) '(a b c d)))
>     (eq test-x test-x-before)))
>
>==> t

No, it *is* shadowed (more correctly, its cdr is shadowed):

(let ((test-x-before test-x))
  (letf (((cdr test-x) '(a b c d)))
    (cdr test-x-before)))

=> (a b c d)

test-x and test-x-before are `eq`, which means they point to the
(token-)identical object in memory. 

I suspect what is going on is that what is returned by the letf (and
cl-letf) is not the value of the symbol test-x (which I've kinda assumed
up until now), but a pointer to the object test-x is referring to (i.e.,
the first cons cell of the list). If you would shadow test-x itself,
this object is different from the object the global test-x binding
points to, and so that object is printed. (Even though the printing is
done after the letf has returned, since there is still a pointer to the
object, the object itself is kept alive until it's been printed.)

However, in the OP's example, it's not the binding of test-x that is
shadowed, but its cdr. So inside the letf, test-x still points to the
same (token-identical) cons cell as the global binding of test-x, it's
just that that cons cell has a shadowed cdr. What is returned by the
letf is a pointer to that object. In this case, when the letf returns,
there is no longer any pointer to the shadowed cdr, so it is discarded.

And since the printing is done outside the letf, as you pointed out, the
object that's printed is the one pointed to by the global binding of
test-x. But that's not because outside the letf the object created
inside it is necessarily gone. It's gone because letf doesn't return a
pointer to it. If it does, the object can stay around longer, as
demonstrated by returning the (cdr test-x).



-- 
Joost Kremers                                   joostkremers@fastmail.fm
Selbst in die Unterwelt dringt durch Spalten Licht
EN:SiS(9)

Re: How does letf work?

[Joost Kremers]

Joost Kremers wrote:
> And since the printing is done outside the letf, as you pointed out, the
> object that's printed is the one pointed to by the global binding of
> test-x. But that's not because outside the letf the object created
> inside it is necessarily gone. It's gone because letf doesn't return a
> pointer to it. If it does, the object can stay around longer, as
> demonstrated by returning the (cdr test-x).

[Apologies for the reply-to-self...]

This account also explains why the OP's code with setf works the way it does:

(let ((x (copy-list test-x)))
   (setf (cdr x) '(a b c d))
   x)

=> (KEY a b c d)

Because let creates a new binding (for x), a new pointer is created.
With copy-list, a new object is created to which this binding points.
After modifying the cdr of that object, the pointer is returned.
Outside the let, the binding for x is gone but the pointer to the object
it referred to is still alive, so the object itself is also kept alive
(until it's been printed).

--
Joost Kremers                                   joostkremers@fastmail.fm
Selbst in die Unterwelt dringt durch Spalten Licht
EN:SiS(9)

Re: How does letf work?

[Florian Beck]

Thanks for the answers!


On 29.01.2014 10:14, Joost Kremers wrote:
>> And since the printing is done outside the letf, as you pointed
>> out, the object that's printed is the one pointed to by the global
>> binding of test-x.

Well, there is no local binding for test-x. As far as I understand what
happens is this:

(defvar test-x '(KEY 1 2 3 4))

(letf (((cdr test-x) '(a b c d)))
   test-x)

This returns a pointer to test-x, but its cdr has been restored before
the value is printed.

>> But that's not because outside the letf the object created inside
>> it is necessarily gone. It's gone because letf doesn't return a
>> pointer to it.

But it does return a pointer to test-x, which in turn pointed to the
cdr. This is exactly what happens in the second example:

(let ((x (copy-list test-x)))
   (setf (cdr x) '(a b c d))
   x)

> Because let creates a new binding (for x), a new pointer is created.
> With copy-list, a new object is created to which this binding points.
> After modifying the cdr of that object, the pointer is returned.
> Outside the let, the binding for x is gone but the pointer to the
> object it referred to is still alive, so the object itself is also
> kept alive (until it's been printed).

This makes sense. But it cannot be the whole story:

(letf* ((x (copy-list  test-x))
	((cdr x) '(a b c d)))
   x)

=> (KEY 1 2 3 4)

I'm still confused.

-- 
Florian Beck

Re: How does letf work?

[Nicolas Richard]

Florian Beck <fb@miszellen.de> writes:
>
> (letf* ((x (copy-list  test-x))
> 	((cdr x) '(a b c d)))
>   x)
>
> => (KEY 1 2 3 4)
>
> I'm still confused.

test-x, which I recall is (KEY 1 2 3 4) here, is simply a cons cell. Its
car is KEY, its CDR is (1 2 3 4).

After the first line of the letf*, test-x and x are different cons
cells. In fact their CAR are `eq', and their CDR are not. OTOH nothing
of this is relevant, test-x doesn't matter here.

After the second line, you changed the CDR of x to the list '(a b c d).
But that is a temporary binding that'll be reverted as soon as we exit
the letf* form.

At the third line, before the closing paren, you say: return the value
of x, so you indeed get that cons cell, which is '(KEY a b c d).

But now, the closing paren happens, i.e. letf* does its job and reverts
the variable value of x to what is was before (empty, I presume, making
x an unbound symbol) and the place represented by (cdr x), i.e. the cdr
of the cons cell that just got returned, to its initial value: '(1 2 3
4). So when letf returns, the cons cell that we received changes. And
that's what gets printed.

-- 
Nico.

Re: How does letf work?

[Florian Beck]

Thank you, I think I got it now.

If I understand correctly, it works like this:

{1} means pointer to cons cell 1; <...> means a shadowed cons cell

Simplified example.

(letf*                ; x undefined
     ((x '(A B))       ; x={1} 1=[A|{2}]  2=[B|()]
      ((cdr x) '(C)))  ;                 <2=[C|()]>
   x                   ; x={1} 1=[A|{2}] <2=[C|()]>
   )                   ; returns {1}, i.e. pointer to cell {1}
                       ; which still point to {2}, but {2}'s
                       ; original value has been restored

Same with car:

(letf*                ; x undefined
     ((x '(A B))       ; x={1}  1=[A|{2}]   2=[B|()]
      ((car x) 'D)     ;       <1=[D|{2}]>
      ((cdr x) '(C)))  ;                   <2=[C|()]>
   x                   ; x={1} <1=[A|{2}]> <2=[C|()]>
   )                   ; returns {1}, but both cons cells have
                       ; been restored

But with setf, setcar, etc., there is nothing to restore:

(letf*                ; x undefined
     ((x '(A B)))      ; x={1} 1=[A|{2}] 2=[B|()]
   (setcar x 'C)       ;       1=[C|{2}]
   x)                  ; returns pointer to {1}, which wasn't
                       ; shadowed, so nothing to restore.

(Obviously, we don't need letf in the last example.)

Right? Let returns the "value of the last form". Straightforward in 
hindsight.
-- 
Florian Beck

Re: How does letf work?

[Nicolas Richard]

Joost Kremers <joost.m.kremers@gmail.com> writes:
> I suspect what is going on is that what is returned by the letf (and
> cl-letf) is not the value of the symbol test-x (which I've kinda assumed
> up until now), but a pointer to the object test-x is referring to (i.e.,
> the first cons cell of the list).

How I understand it is : what is returned by the letf *is* the value of
the symbol, but since it is a cons cell it can be modified by setf
(using setcar and setcdr). So part of the job of letf is to *change* the
cons cell upon exiting. The box is the same (and is what is returned),
but the content was changed at the moment the last closing paren of letf
is crossed. IOW:

(defvar test-x '(KEY 1 2 3 4))

(letf (((cdr test-x) '(a b c d)))
  (copy-sequence  test-x))
=> (KEY a b c d)

but

(copy-sequence
 (letf (((cdr test-x) '(a b c d)))
   test-x))
=> (KEY 1 2 3 4)


Returning (copy-sequence test-x) instead of
test-x will produce the expected value.

> And since the printing is done outside the letf, as you pointed out, the
> object that's printed is the one pointed to by the global binding of
> test-x.

The object is the same, but its content changed. Similar to :
(let ((foo (cons 'a 'b))
      (bar))
  (setq bar foo) ;; bar and foo have the same object in their value cell.
  (setcdr foo 'c) ;; change the cdr of that cons cell
  (eq bar foo)) ;; they're still the same.
=> t

> But that's not because outside the letf the object created
> inside it is necessarily gone. It's gone because letf doesn't return a
> pointer to it

I guess it's not gone per se until the garbage collector does its work.

-- 
Nico.

Re: How does letf work?

[Michael Heerdegen]

Nicolas Richard <theonewiththeevillook@yahoo.fr> writes:

> How I understand it is : what is returned by the letf *is* the value of
> the symbol, but since it is a cons cell it can be modified by setf
> (using setcar and setcdr). So part of the job of letf is to *change* the
> cons cell upon exiting. The box is the same (and is what is returned),
> but the content was changed at the moment the last closing paren of letf
> is crossed. IOW:

A good summary, IMHO.

In LISP, lists are passed by reference.  When a list is bound to a
variable, the list is not copied.  The value referenced by the variable
will be the very same object.

> The object is the same, but its content changed. Similar to :
> (let ((foo (cons 'a 'b))
>       (bar))
>   (setq bar foo) ;; bar and foo have the same object in their value cell.
>   (setcdr foo 'c) ;; change the cdr of that cons cell
>   (eq bar foo)) ;; they're still the same.
> => t
>
> > But that's not because outside the letf the object created
> > inside it is necessarily gone. It's gone because letf doesn't return a
> > pointer to it
>
> I guess it's not gone per se until the garbage collector does its
> work.

(a b c d) is garbage collected.  The object equal to (KEY a b c d)
inside letf is never destroyed or garbage collected, since this list is
referenced by test-x and the return value of the function.  But a part
of the object has been changed (restored) by letf.

Regards,

Michael.

Re: How does letf work?

[Michael Heerdegen]

Joost Kremers <joost.m.kremers@gmail.com> writes:

> (letf (((cdr test-x) '(a b c d)))
>   (cdr test-x))
>
> = > (KEY a b c d)
>
> Taking what you say at face value, one might expect that at the moment
> the value of (cdr test-x) is printed, it's already been changed back to
> the original value, so it should say (KEY 1 2 3 4), but it doesn't.

(cdr test-x) here references an object being a list equal to '(a b c d).
But after the letf has been left, that object is not anymore the cdr of
test-x, because its cdr has been restored.  But that doesn't make the
return value "switch" to a different object.

The analogy with boxes is very helpful here.

Michael.

Re: How does letf work?

[Emanuel Berg]

Florian Beck <fb@miszellen.de> writes:

> The docstring of letf says: "This is the analogue of
> let', but with generalized variables (in the sense of
> setf') for the PLACEs."

(letf (((cdr test-x) '(a b c d)))
  (message "%s" test-x) ; says '(KEY a b c d)
  (cdr test-x) )        ; => (a b c d)

(And `... (nth 1 test-x)' => 'a, and so on.)

But:

(letf (((cdr test-x) '(a b c d)))
  (message "%s" test-x) ; says '(KEY a b c d)
  test-x)               ;   => '(KEY 1 2 3 4)

`letf' (or `cl-letf') resets the values when it exits:

> On exit, either normally or because of a `throw' or
> error, the PLACEs are set back to their original
> values.

So perhaps this inconsistency is due to the
implementation of that.

-- 
underground experts united:
http://user.it.uu.se/~embe8573