Re: frames vs. weak hash tables and garbage collection

[Joe Wells <jbw@macs.hw.ac.uk>]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Your explanation doesn't show up in gnu.emacs.bug either.  Can you
>> please resend?
>
> Hmm... not sure what's going on.  There's not much more to send.
>
>> By the way, great work in tracking the problem down!
>
> Thanks.
>
>> I suppose this means frames are never being garbage collected at all?
>
> Not at all: the array of recent input events (as the name implies) only keep
> recent events, so after some number of events (300, more precisely) the
> reference disappears, making it possible to collect the frame.

Ah!

So this is some sort of “reverse heisenbug” which only shows up when
you look for bugs right away?

> The reference from `values' is just due to the fact that you run
> `reproduce-bug' from M-: and that its return value contains the frame, so
> it's a rather unusual circumstance.

I'm confused.  The frame can only show up in values if it fails to be
garbage collected for another reason.  The frame should already have
been garbage collected before the expression finished evaluation and
hence should never get into values.  So in my case it must be the
recent events array which was the cause of the appearance of a failure
to garbage collect.

(I suppose your point is that if one ever tests garbage collection
behavior with M-: (eval-expression), then there is a high likelihood
of getting confused.  Personally, I usually evaluate expressions with
C-x C-e (eval-last-sexp), which does not add to values.)

Anyway, although I can see that the recent events array would be
enough to cause the problem, I'm not sure it is the only cause.  If I
understand correctly, the array you are talking about is recent_keys
which is cleared by clear-this-command-keys.  However, I still get the
bug when I insert a use of clear-this-command-keys into the bug
reproduction code:

  (defun reproduce-bug ()
    (let ((ht (make-hash-table :weakness 'key)))
      (let ((x
             (make-frame)
             ;;(get-buffer-create "xyzzy")
             ))
        (puthash x t ht)
        (delete-frame x)
        ;;(kill-buffer x)
        )
      ;; Give time for various frame related events to be handled, in
      ;; case this is needed.
      (sit-for 1)
      ;; There may be a reference to the frame in the array of recent
      ;; events, so we clear this array.
      (clear-this-command-keys)
      ;; In theory, the only reference to the new frame is now the key
      ;; in the hash table.  Because of the weakness, this key should
      ;; not keep the frame alive.
      (garbage-collect)
      ;; The hash table should now be empty.
      (let (l)
        (maphash (lambda (k v) (push (cons k v) l)) ht)
        l)))

Is there another array of recent events different from recent_keys?

> But it seems that `values' is never flushed, so it is a source of leaks.
> We should probably fix it (I'd remove it since almost noone even knows that
> it exists, but I guess Richard uses it once in a blue moon).

Perhaps it should have a length limit at least?

-- 
Joe