unofficial mirror of bug-gnu-emacs@gnu.org 
 help / color / mirror / code / Atom feed
* frames vs. weak hash tables and garbage collection
@ 2007-08-30  1:27 Joe Wells
  2007-08-30  2:07 ` Eric Hanchrow
                   ` (3 more replies)
  0 siblings, 4 replies; 23+ messages in thread
From: Joe Wells @ 2007-08-30  1:27 UTC (permalink / raw)
  To: bug-gnu-emacs

In a fresh “emacs -Q”, define this function:

  (defun reproduce-bug ()
    (let ((ht (make-hash-table :weakness 'key)))
      (let ((x
	     (make-frame)
	     ;;(get-buffer-create "xyzzy")
	     ))
	(puthash x t ht)
	(delete-frame x)
	;;(kill-buffer x)
	)
      ;; The only reference to the new frame is now the key in the hash
      ;; table.  Because of the weakness, this key should not keep the
      ;; frame alive.
      (garbage-collect)
      ;; The hash table should now be empty.
      (let (l)
	(maphash (lambda (k v) (push (cons k v) l)) ht)
	l)))

Then evaluating (reproduce-bug) will produce a result like this:

  ((#<dead frame emacs@localhost.localdomain 0x88e2278> . t))

It should instead return nil.

If I replace the calls to make-frame and delete-frame by calls to
get-buffer-create and kill-buffer (see commented out lines above),
things work properly.  So this verifies that weak hash tables work at
least for some data types.

There are two possibilities that occur to me:

1. Maybe frames are simply not being garbage collected at all.

2. Maybe the feature of weak keys in hash tables doesn't work for the
   particular case of frames.

I hope this bug report is helpful.

Joe

======================================================================
In GNU Emacs 22.1.1 (i686-pc-linux-gnu, GTK+ Version 2.8.20)
 of 2007-06-27 on artemis
Windowing system distributor `The X.Org Foundation', version 11.0.70000000
configured using `configure  '--prefix=/home/jbw/local2' '--enable-debug' '--disable-nls' '--with-x-toolkit=gtk' 'CFLAGS=-O0 -g3 -ggdb''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: en_US.UTF-8
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: jbw
  value of $LANG: nil
  locale-coding-system: utf-8
  default-enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  unify-8859-on-encoding-mode: t
  utf-translate-cjk-mode: t
  auto-compression-mode: t
  line-number-mode: t

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-08-30  1:27 frames vs. weak hash tables and garbage collection Joe Wells
@ 2007-08-30  2:07 ` Eric Hanchrow
  2007-08-30  3:00 ` Thien-Thi Nguyen
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 23+ messages in thread
From: Eric Hanchrow @ 2007-08-30  2:07 UTC (permalink / raw)
  To: bug-gnu-emacs

I wish all bug reports were this clear.
-- 
When I just need to kick back after a long day of changing 1.s
into 0.s and 0.s into 1.s, nothing satisfies my primal fish urge
better than a can of Genova [tuna].
        -- Ben Collins-Sussman

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-08-30  1:27 frames vs. weak hash tables and garbage collection Joe Wells
  2007-08-30  2:07 ` Eric Hanchrow
@ 2007-08-30  3:00 ` Thien-Thi Nguyen
  2007-08-30  3:07 ` Thien-Thi Nguyen
       [not found] ` <mailman.47.1188442895.18990.bug-gnu-emacs@gnu.org>
  3 siblings, 0 replies; 23+ messages in thread
From: Thien-Thi Nguyen @ 2007-08-30  3:00 UTC (permalink / raw)
  To: Joe Wells; +Cc: bug-gnu-emacs

() Joe Wells <jbw@macs.hw.ac.uk>
() Thu, 30 Aug 2007 02:27:32 +0100

   ;; The only reference to the new frame is now the key in the hash
   ;; table.  Because of the weakness, this key should not keep the
   ;; frame alive.

there's a lot of stuff going on w/ make-frame.

try replacing `(make-frame)' with:

  (x-create-frame-with-faces)

here, i see that `reproduce-bug' returns nil.
what do you see?

thi

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-08-30  1:27 frames vs. weak hash tables and garbage collection Joe Wells
  2007-08-30  2:07 ` Eric Hanchrow
  2007-08-30  3:00 ` Thien-Thi Nguyen
@ 2007-08-30  3:07 ` Thien-Thi Nguyen
       [not found] ` <mailman.47.1188442895.18990.bug-gnu-emacs@gnu.org>
  3 siblings, 0 replies; 23+ messages in thread
From: Thien-Thi Nguyen @ 2007-08-30  3:07 UTC (permalink / raw)
  To: Joe Wells; +Cc: bug-gnu-emacs

() Joe Wells <jbw@macs.hw.ac.uk>
() Thu, 30 Aug 2007 02:27:32 +0100

         (garbage-collect)

another thing: one gc pass may not be enough.
i see (reproduce-bug) => nil only at:

(dotimes (i N)
  (garbage-collect))

where (< 1 N).  my emacs is tweaked, however (ymmv).

thi

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
       [not found] ` <mailman.47.1188442895.18990.bug-gnu-emacs@gnu.org>
@ 2007-08-31 15:15   ` Joe Wells
  2007-08-31 15:42     ` Thien-Thi Nguyen
                       ` (4 more replies)
  0 siblings, 5 replies; 23+ messages in thread
From: Joe Wells @ 2007-08-31 15:15 UTC (permalink / raw)
  To: Thien-Thi Nguyen; +Cc: bug-gnu-emacs

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> () Joe Wells <jbw@macs.hw.ac.uk>
> () Thu, 30 Aug 2007 02:27:32 +0100
>
>    ;; The only reference to the new frame is now the key in the hash
>    ;; table.  Because of the weakness, this key should not keep the
>    ;; frame alive.
>
> there's a lot of stuff going on w/ make-frame.
>
> try replacing `(make-frame)' with:
>
>   (x-create-frame-with-faces)
>
> here, i see that `reproduce-bug' returns nil.
> what do you see?

It doesn't help me to even replace make-frame with x-create-frame (the
primitive used by x-create-frame-with-faces).

Are you able to reproduce the bug at all with the code I originally
sent?

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> another thing: one gc pass may not be enough.
> i see (reproduce-bug) => nil only at:
>
> (dotimes (i N)
>   (garbage-collect))
>
> where (< 1 N).  my emacs is tweaked, however (ymmv).

Running garbage-collect 10 times makes no difference for me.

Here is the code I am now using to reproduce the bug:

  (defun reproduce-bug ()
    (let ((ht (make-hash-table :weakness 'key)))
      (let* (after-make-frame-functions
             (x
              (x-create-frame nil)
              ;;(x-create-frame-with-faces)
              ;;(make-frame)
              ;;(get-buffer-create "xyzzy")
              ))
        (puthash x t ht)
        (delete-frame x)
        ;;(kill-buffer x)
        )
      ;; The only reference to the new frame is now the key in the hash
      ;; table.  Because of the weakness, this key should not keep the
      ;; frame alive.
      (loop repeat 10 do (garbage-collect))
      ;; The hash table should now be empty.
      (let (l)
        (maphash (lambda (k v) (push (cons k v) l)) ht)
        l)))

With the above definition, I still get (reproduce-bug) returning
((#<dead frame emacs@localhost.localdomain 0xa2ae808> . t)) instead of
nil.

-- 
Joe

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-08-31 15:15   ` Joe Wells
@ 2007-08-31 15:42     ` Thien-Thi Nguyen
  2007-08-31 15:50     ` Thien-Thi Nguyen
                       ` (3 subsequent siblings)
  4 siblings, 0 replies; 23+ messages in thread
From: Thien-Thi Nguyen @ 2007-08-31 15:42 UTC (permalink / raw)
  To: Joe Wells; +Cc: bug-gnu-emacs

() Joe Wells <jbw@macs.hw.ac.uk>
() Fri, 31 Aug 2007 16:15:00 +0100

   With the above definition, I still get (reproduce-bug) returning
   ((#<dead frame emacs@localhost.localdomain 0xa2ae808> . t))
   instead of nil.

crazy idea...  have you tried doing this experiment under ratpoison?
for the record:

 $ ratpoison.working-2007-05-04 --version
 ratpoison 1.4.2-CVS (built May  4 2007 11:37:01)
 $ emacs --version
 GNU Emacs 22.0.97.4
 Copyright (C) 2007 Free Software Foundation, Inc.
 GNU Emacs comes with ABSOLUTELY NO WARRANTY.
 You may redistribute copies of Emacs
 under the terms of the GNU General Public License.
 For more information about these matters, see the file named COPYING.
 $ free
              total       used       free     shared    buffers     cached
 Mem:        516616     419580      97036          0      80808     187968
 -/+ buffers/cache:     150804     365812
 Swap:      1510068          0    1510068


i recall emacs21 had weak-hash problems, but i think those were
fixed in emacs22.  such problems when hacking EDB were one reason
i transitioned away from being an emacs21 holdout (about two years
ago).

thi

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-08-31 15:15   ` Joe Wells
  2007-08-31 15:42     ` Thien-Thi Nguyen
@ 2007-08-31 15:50     ` Thien-Thi Nguyen
       [not found]     ` <mailman.116.1188575498.18990.bug-gnu-emacs@gnu.org>
                       ` (2 subsequent siblings)
  4 siblings, 0 replies; 23+ messages in thread
From: Thien-Thi Nguyen @ 2007-08-31 15:50 UTC (permalink / raw)
  To: Joe Wells; +Cc: bug-gnu-emacs

() Joe Wells <jbw@macs.hw.ac.uk>
() Fri, 31 Aug 2007 16:15:00 +0100

sorry i missed this question.

   Are you able to reproduce the bug at all
   with the code I originally sent?

no.  maybe you have "too much" memory.
did you try 100 (1000?) gc passes?

thi

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
       [not found]       ` <xnjir6tyj35.fsf@csb.bu.edu>
@ 2007-09-02  2:00         ` Thien-Thi Nguyen
  0 siblings, 0 replies; 23+ messages in thread
From: Thien-Thi Nguyen @ 2007-09-02  2:00 UTC (permalink / raw)
  To: Joe Wells; +Cc: bug-gnu-emacs

() Joe Wells <jbw@csb.bu.edu>
() 01 Sep 2007 21:03:26 -0400

   > maybe you have "too much" memory.

   What do you mean?

i confesss not looking into emacs' gc innards (yet); the
comment was motivated by fuzzy remembrance of some gc systems
that reclaim "enough" (but not all) to keep going.  if emacs
is like that, then maybe "enough" was reclaimed (but not the
dead frame).  if so, we would need to increase the garbage
creation to get past the threshold.  see following:

   > did you try 100 (1000?) gc passes?

   Yes, I've just now tried with 1000 repetitions [...] makes
   no difference; I still get the bug.

how about doing a lot of consing before or in between?  maybe:

(make-list 100000 (random))
...
(loop repeat 1000 do (progn (make-list 10000 (random))
                            (garbage-collect)))

btw, these are just idle suggestions, please don't take them
too seriously.  i really dislike making them when bug-hunting,
but the ignorantly curious side of me prevails sometimes...

thi

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
       [not found]       ` <xnjhcmdyirv.fsf@csb.bu.edu>
@ 2007-09-02  2:13         ` Thien-Thi Nguyen
  0 siblings, 0 replies; 23+ messages in thread
From: Thien-Thi Nguyen @ 2007-09-02  2:13 UTC (permalink / raw)
  To: Joe Wells; +Cc: bug-gnu-emacs

() Joe Wells <jbw@csb.bu.edu>
() 01 Sep 2007 21:10:12 -0400

   As long as I am not completely filling swap, is there any way
   that Emacs could even be aware of how much memory I have on the
   machine?

well, i could continue wallowing in my ignorance, but source is
available, so i looked in src/alloc.c.  you will probably want to
do (setq gc-cons-threshold SOME-LOWER-NUMBER), based on:

| gc-cons-threshold is a variable defined in `src/alloc.c'.
| Its value is 400000
| 
| *Number of bytes of consing between garbage collections.
| Garbage collection can happen automatically once this many bytes have been
| allocated since the last garbage collection.  All data types count.
| 
| Garbage collection happens automatically only when `eval' is called.
| 
| By binding this temporarily to a large number, you can effectively
| prevent garbage collection during a part of the program.
| See also `gc-cons-percentage'.

basically, by tweaking this (and perhaps some other gc-FOO vars),
you can make "enough" satisfiable only when it includes the dead frame.

   Do you mean the database interface stuff?  That brings back
   memories!  I last hacked on that in 1992 (I think it was 1992),
   before Emacs 19 came out.

yes, i revived it in 2004: <http://www.gnuvola.org/software/edb/>.
there were some very twisted things mernst did, but it's ok now.
perhaps one of these days EDB will find itself in a git repo or two.

thi

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-08-31 15:15   ` Joe Wells
                       ` (3 preceding siblings ...)
       [not found]     ` <mailman.115.1188574978.18990.bug-gnu-emacs@gnu.org>
@ 2007-09-25 23:23     ` Joe Wells
  2007-09-27  7:20       ` Glenn Morris
  4 siblings, 1 reply; 23+ messages in thread
From: Joe Wells @ 2007-09-25 23:23 UTC (permalink / raw)
  To: bug-gnu-emacs

Has anyone else been able to reproduce this bug?

Joe

Joe Wells <jbw@macs.hw.ac.uk> writes:

> Thien-Thi Nguyen <ttn@gnuvola.org> writes:
>
>> () Joe Wells <jbw@macs.hw.ac.uk>
>> () Thu, 30 Aug 2007 02:27:32 +0100
>>
>>    ;; The only reference to the new frame is now the key in the hash
>>    ;; table.  Because of the weakness, this key should not keep the
>>    ;; frame alive.
>>
>> there's a lot of stuff going on w/ make-frame.
>>
>> try replacing `(make-frame)' with:
>>
>>   (x-create-frame-with-faces)
>>
>> here, i see that `reproduce-bug' returns nil.
>> what do you see?
>
> It doesn't help me to even replace make-frame with x-create-frame (the
> primitive used by x-create-frame-with-faces).
>
> Are you able to reproduce the bug at all with the code I originally
> sent?
>
> Thien-Thi Nguyen <ttn@gnuvola.org> writes:
>
>> another thing: one gc pass may not be enough.
>> i see (reproduce-bug) => nil only at:
>>
>> (dotimes (i N)
>>   (garbage-collect))
>>
>> where (< 1 N).  my emacs is tweaked, however (ymmv).
>
> Running garbage-collect 10 times makes no difference for me.
>
> Here is the code I am now using to reproduce the bug:
>
>   (defun reproduce-bug ()
>     (let ((ht (make-hash-table :weakness 'key)))
>       (let* (after-make-frame-functions
>              (x
>               (x-create-frame nil)
>               ;;(x-create-frame-with-faces)
>               ;;(make-frame)
>               ;;(get-buffer-create "xyzzy")
>               ))
>         (puthash x t ht)
>         (delete-frame x)
>         ;;(kill-buffer x)
>         )
>       ;; The only reference to the new frame is now the key in the hash
>       ;; table.  Because of the weakness, this key should not keep the
>       ;; frame alive.
>       (loop repeat 10 do (garbage-collect))
>       ;; The hash table should now be empty.
>       (let (l)
>         (maphash (lambda (k v) (push (cons k v) l)) ht)
>         l)))
>
> With the above definition, I still get (reproduce-bug) returning
> ((#<dead frame emacs@localhost.localdomain 0xa2ae808> . t)) instead of
> nil.
>
> -- 
> Joe




^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-09-25 23:23     ` Joe Wells
@ 2007-09-27  7:20       ` Glenn Morris
  2007-09-27  8:50         ` Thien-Thi Nguyen
  0 siblings, 1 reply; 23+ messages in thread
From: Glenn Morris @ 2007-09-27  7:20 UTC (permalink / raw)
  To: Joe Wells; +Cc: bug-gnu-emacs

Joe Wells wrote:

> Has anyone else been able to reproduce this bug?

Yes, but I doubt that I can do anything about it, except note that it
exists.




^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-09-27  7:20       ` Glenn Morris
@ 2007-09-27  8:50         ` Thien-Thi Nguyen
       [not found]           ` <mailman.1408.1190931512.18990.bug-gnu-emacs@gnu.org>
  0 siblings, 1 reply; 23+ messages in thread
From: Thien-Thi Nguyen @ 2007-09-27  8:50 UTC (permalink / raw)
  To: Glenn Morris; +Cc: bug-gnu-emacs, Joe Wells

() Glenn Morris <rgm@gnu.org>
() Thu, 27 Sep 2007 03:20:41 -0400

   Joe Wells wrote:

   > Has anyone else been able to reproduce this bug?

   Yes, but I doubt that I can do anything about it,
   except note that it exists.

i responded in another thread but perhaps it got dropped.
gist: there is a new variable gc-cons-percentage.  perhaps
frobbing it (and other gc-foo vars) one can re-instate the gc
behavior of previous emacsen.  thus: i don't think this is a bug.

thi




^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
       [not found]           ` <mailman.1408.1190931512.18990.bug-gnu-emacs@gnu.org>
@ 2007-09-28 14:34             ` Stefan Monnier
  2007-09-28 14:56               ` Joe Wells
  0 siblings, 1 reply; 23+ messages in thread
From: Stefan Monnier @ 2007-09-28 14:34 UTC (permalink / raw)
  Cc: bug-gnu-emacs, jbw

The following message is a courtesy copy of an article
that has been posted to gnu.emacs.bug as well.

> Can you try  stepping thru the function sweep_weak_table
> when it operates on this hash table, and see what it does
> when it gets to this line

> 	  int key_known_to_survive_p = survives_gc_p (HASH_KEY (h, i));

> and following lines?

Please check the gnu.emacs.bug newsgroup (apparently the gateway to the
mailing-list isn't working right) where I explained the cause of the problem:
basically it's kept alive by the "recent input events" array and the
`values' variable.


        Stefan




^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-09-28 14:34             ` Stefan Monnier
@ 2007-09-28 14:56               ` Joe Wells
  2007-09-28 16:27                 ` Stefan Monnier
  0 siblings, 1 reply; 23+ messages in thread
From: Joe Wells @ 2007-09-28 14:56 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: bug-gnu-emacs

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> The following message is a courtesy copy of an article
> that has been posted to gnu.emacs.bug as well.
>
>> Can you try  stepping thru the function sweep_weak_table
>> when it operates on this hash table, and see what it does
>> when it gets to this line
>
>> 	  int key_known_to_survive_p = survives_gc_p (HASH_KEY (h, i));
>
>> and following lines?
>
> Please check the gnu.emacs.bug newsgroup (apparently the gateway to the
> mailing-list isn't working right) where I explained the cause of the problem:
> basically it's kept alive by the "recent input events" array and the
> `values' variable.

Your explanation doesn't show up in gnu.emacs.bug either.  Can you
please resend?

By the way, great work in tracking the problem down!

I suppose this means frames are never being garbage collected at all?

-- 
Joe




^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-09-28 14:56               ` Joe Wells
@ 2007-09-28 16:27                 ` Stefan Monnier
  2007-09-28 16:50                   ` Joe Wells
  2007-09-29 16:10                   ` Richard Stallman
  0 siblings, 2 replies; 23+ messages in thread
From: Stefan Monnier @ 2007-09-28 16:27 UTC (permalink / raw)
  To: Joe Wells; +Cc: bug-gnu-emacs

> Your explanation doesn't show up in gnu.emacs.bug either.  Can you
> please resend?

Hmm... not sure what's going on.  There's not much more to send.

> By the way, great work in tracking the problem down!

Thanks.

> I suppose this means frames are never being garbage collected at all?

Not at all: the array of recent input events (as the name implies) only keep
recent events, so after some number of events (300, more precisely) the
reference disappears, making it possible to collect the frame.
The reference from `values' is just due to the fact that you run
`reproduce-bug' from M-: and that its return value contains the frame, so
it's a rather unusual circumstance.

But it seems that `values' is never flushed, so it is a source of leaks.
We should probably fix it (I'd remove it since almost noone even knows that
it exists, but I guess Richard uses it once in a blue moon).


        Stefan




^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-09-28 16:27                 ` Stefan Monnier
@ 2007-09-28 16:50                   ` Joe Wells
  2007-09-28 18:22                     ` Stefan Monnier
  2007-09-29 16:10                   ` Richard Stallman
  1 sibling, 1 reply; 23+ messages in thread
From: Joe Wells @ 2007-09-28 16:50 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: bug-gnu-emacs

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Your explanation doesn't show up in gnu.emacs.bug either.  Can you
>> please resend?
>
> Hmm... not sure what's going on.  There's not much more to send.
>
>> By the way, great work in tracking the problem down!
>
> Thanks.
>
>> I suppose this means frames are never being garbage collected at all?
>
> Not at all: the array of recent input events (as the name implies) only keep
> recent events, so after some number of events (300, more precisely) the
> reference disappears, making it possible to collect the frame.

Ah!

So this is some sort of “reverse heisenbug” which only shows up when
you look for bugs right away?

> The reference from `values' is just due to the fact that you run
> `reproduce-bug' from M-: and that its return value contains the frame, so
> it's a rather unusual circumstance.

I'm confused.  The frame can only show up in values if it fails to be
garbage collected for another reason.  The frame should already have
been garbage collected before the expression finished evaluation and
hence should never get into values.  So in my case it must be the
recent events array which was the cause of the appearance of a failure
to garbage collect.

(I suppose your point is that if one ever tests garbage collection
behavior with M-: (eval-expression), then there is a high likelihood
of getting confused.  Personally, I usually evaluate expressions with
C-x C-e (eval-last-sexp), which does not add to values.)

Anyway, although I can see that the recent events array would be
enough to cause the problem, I'm not sure it is the only cause.  If I
understand correctly, the array you are talking about is recent_keys
which is cleared by clear-this-command-keys.  However, I still get the
bug when I insert a use of clear-this-command-keys into the bug
reproduction code:

  (defun reproduce-bug ()
    (let ((ht (make-hash-table :weakness 'key)))
      (let ((x
             (make-frame)
             ;;(get-buffer-create "xyzzy")
             ))
        (puthash x t ht)
        (delete-frame x)
        ;;(kill-buffer x)
        )
      ;; Give time for various frame related events to be handled, in
      ;; case this is needed.
      (sit-for 1)
      ;; There may be a reference to the frame in the array of recent
      ;; events, so we clear this array.
      (clear-this-command-keys)
      ;; In theory, the only reference to the new frame is now the key
      ;; in the hash table.  Because of the weakness, this key should
      ;; not keep the frame alive.
      (garbage-collect)
      ;; The hash table should now be empty.
      (let (l)
        (maphash (lambda (k v) (push (cons k v) l)) ht)
        l)))

Is there another array of recent events different from recent_keys?

> But it seems that `values' is never flushed, so it is a source of leaks.
> We should probably fix it (I'd remove it since almost noone even knows that
> it exists, but I guess Richard uses it once in a blue moon).

Perhaps it should have a length limit at least?

-- 
Joe




^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-09-28 16:50                   ` Joe Wells
@ 2007-09-28 18:22                     ` Stefan Monnier
  2007-09-28 18:48                       ` Joe Wells
  0 siblings, 1 reply; 23+ messages in thread
From: Stefan Monnier @ 2007-09-28 18:22 UTC (permalink / raw)
  To: Joe Wells; +Cc: bug-gnu-emacs

>> The reference from `values' is just due to the fact that you run
>> `reproduce-bug' from M-: and that its return value contains the frame, so
>> it's a rather unusual circumstance.

> I'm confused.  The frame can only show up in values if it fails to be
> garbage collected for another reason.

Right.  In your case the reason is the recent events array.

> Anyway, although I can see that the recent events array would be
> enough to cause the problem, I'm not sure it is the only cause.

Could be, but at least it matched the behavior I saw:
if I hit a key 300 times, set `values' to nil, and call garbage-collect then
the hash-table entry gets deleted (I modified your test case to make the
hash-table global).

>   (defun reproduce-bug ()
>     (let ((ht (make-hash-table :weakness 'key)))
>       (let ((x
>              (make-frame)
>              ;;(get-buffer-create "xyzzy")
>              ))
>         (puthash x t ht)
>         (delete-frame x)
>         ;;(kill-buffer x)
>         )
>       ;; Give time for various frame related events to be handled, in
>       ;; case this is needed.
>       (sit-for 1)
>       ;; There may be a reference to the frame in the array of recent
>       ;; events, so we clear this array.
>       (clear-this-command-keys)
>       ;; In theory, the only reference to the new frame is now the key
>       ;; in the hash table.  Because of the weakness, this key should
>       ;; not keep the frame alive.
>       (garbage-collect)
>       ;; The hash table should now be empty.
>       (let (l)
>         (maphash (lambda (k v) (push (cons k v) l)) ht)
>         l)))

No idea about this one, but it may very well be due to transient effects
such as the fact that the stack is scanned conservatively, so there may
still be a spurious pointer to the frame left over somewhere at the time you
call `garbage-collect'.

Also the sit-for is enough to cause redisplay and execution of process
filters, but I'm not convinced it's enough to cause the frame events to be
added to the "recent events array", so maybe these will appear after the
call to clear-this-command-keys.

>> But it seems that `values' is never flushed, so it is a source of leaks.
>> We should probably fix it (I'd remove it since almost noone even knows that
>> it exists, but I guess Richard uses it once in a blue moon).

> Perhaps it should have a length limit at least?

Yes.  I.e. it should be a ring.


        Stefan




^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-09-28 18:22                     ` Stefan Monnier
@ 2007-09-28 18:48                       ` Joe Wells
  0 siblings, 0 replies; 23+ messages in thread
From: Joe Wells @ 2007-09-28 18:48 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: bug-gnu-emacs

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> The reference from `values' is just due to the fact that you run
>>> `reproduce-bug' from M-: and that its return value contains the frame, so
>>> it's a rather unusual circumstance.
>
>> I'm confused.  The frame can only show up in values if it fails to be
>> garbage collected for another reason.
>
> Right.  In your case the reason is the recent events array.
>
>> Anyway, although I can see that the recent events array would be
>> enough to cause the problem, I'm not sure it is the only cause.
>
> Could be, but at least it matched the behavior I saw:
> if I hit a key 300 times, set `values' to nil, and call garbage-collect then
> the hash-table entry gets deleted (I modified your test case to make the
> hash-table global).
>
>>   (defun reproduce-bug ()
>>     [ ... new version of code deleted ... ]
>>     ... )
>
> No idea about this one, but it may very well be due to transient effects
> such as the fact that the stack is scanned conservatively, so there may
> still be a spurious pointer to the frame left over somewhere at the time you
> call `garbage-collect'.

It seems it is not the stack.  See below.

> Also the sit-for is enough to cause redisplay and execution of process
> filters, but I'm not convinced it's enough to cause the frame events to be
> added to the "recent events array", so maybe these will appear after the
> call to clear-this-command-keys.

Okay.  Using a recursive-edit seems to be enough:

  (defun reproduce-bug ()
    (let ((ht (make-hash-table :weakness 'key)))
      (let ((x
             (make-frame)
             ;;(get-buffer-create "xyzzy")
             ))
        (puthash x t ht)
        (delete-frame x)
        ;;(kill-buffer x)
        )
      ;; Give time for various frame related events to be handled, in
      ;; case this is needed.
      (recursive-edit)
      ;; There may be a reference to the frame in the array of recent
      ;; events, so we clear this array.
      (clear-this-command-keys)
      ;; In theory, the only reference to the new frame is now the key
      ;; in the hash table.  Because of the weakness, this key should
      ;; not keep the frame alive.
      (garbage-collect)
      ;; The hash table should now be empty.
      (let (l)
        (maphash (lambda (k v) (push (cons k v) l)) ht)
        l)))

Evaluate (reproduce-bug) and type C-M-c (exit-recursive-edit) in the
recursive edit, and it correctly returns nil.

(By the way, I tried doing

  (setq unread-command-events (append (kbd "C-M-c") nil))

just before the recursive edit, but that wasn't enough.  It needs to
have real user interaction.  I'm curious if there is some purely
programmatic way of simulating the effects of a recursive edit with
only C-M-c typed in it, because this would help in building test
cases.)

Conclusion:  There is no bug with garbage collection of deleted
frames, but merely the appearance of a bug, because the recent event
array keeps the frame alive for a while.

Thanks for tracking down the cause of the appearance of a bug!

-- 
Joe




^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-09-28 16:27                 ` Stefan Monnier
  2007-09-28 16:50                   ` Joe Wells
@ 2007-09-29 16:10                   ` Richard Stallman
  2007-09-29 16:20                     ` Joe Wells
  1 sibling, 1 reply; 23+ messages in thread
From: Richard Stallman @ 2007-09-29 16:10 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: bug-gnu-emacs, jbw

    But it seems that `values' is never flushed, so it is a source of leaks.
    We should probably fix it (I'd remove it since almost noone even knows that
    it exists, but I guess Richard uses it once in a blue moon).

I don't use it myself.  I put it in because a history of values
is a useful thing for any read-eval-print loop.  We could delete it,
but it might be better to do more to inform people about it.
For instance, mention it in the text in the *scratch* buffer.

Then they might indeed find it useful.  Or if they don't,
we could delete it.

I agree with the suggestion to truncate it to a specified maximum
length.  How about 40?





^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-09-29 16:10                   ` Richard Stallman
@ 2007-09-29 16:20                     ` Joe Wells
  2007-09-29 18:28                       ` Stefan Monnier
  0 siblings, 1 reply; 23+ messages in thread
From: Joe Wells @ 2007-09-29 16:20 UTC (permalink / raw)
  To: rms; +Cc: bug-gnu-emacs, Stefan Monnier

Richard Stallman <rms@gnu.org> writes:

>     But it seems that `values' is never flushed, so it is a source of leaks.
>     We should probably fix it (I'd remove it since almost noone even knows that
>     it exists, but I guess Richard uses it once in a blue moon).
>
> I don't use it myself.  I put it in because a history of values
> is a useful thing for any read-eval-print loop.  We could delete it,
> but it might be better to do more to inform people about it.
> For instance, mention it in the text in the *scratch* buffer.
>
> Then they might indeed find it useful.  Or if they don't,
> we could delete it.
>
> I agree with the suggestion to truncate it to a specified maximum
> length.  How about 40?

Personally, I can't remember using anything more than (car values).
Maybe once I used something older than the most recent addition to the
list.

So it seems to me a length of 3 would probably be plenty.

By the way, the name “values” is not good.  I would suggest
“eval-expression-recent-values”.

-- 
Joe




^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-09-29 16:20                     ` Joe Wells
@ 2007-09-29 18:28                       ` Stefan Monnier
  2007-09-29 19:25                         ` Drew Adams
  0 siblings, 1 reply; 23+ messages in thread
From: Stefan Monnier @ 2007-09-29 18:28 UTC (permalink / raw)
  To: Joe Wells; +Cc: bug-gnu-emacs, rms

>> But it seems that `values' is never flushed, so it is a source of leaks.
>> We should probably fix it (I'd remove it since almost noone even knows that
>> it exists, but I guess Richard uses it once in a blue moon).
>> 
>> I don't use it myself.  I put it in because a history of values
>> is a useful thing for any read-eval-print loop.  We could delete it,
>> but it might be better to do more to inform people about it.
>> For instance, mention it in the text in the *scratch* buffer.
>> 
>> Then they might indeed find it useful.  Or if they don't,
>> we could delete it.
>> 
>> I agree with the suggestion to truncate it to a specified maximum
>> length.  How about 40?


> Personally, I can't remember using anything more than (car values).
> Maybe once I used something older than the most recent addition to the
> list.

> So it seems to me a length of 3 would probably be plenty.

> By the way, the name “values” is not good.  I would suggest
> “eval-expression-recent-values”.

ielm implements its own mechanism for it and it calls it "*", "**", and
"***", giving access only to the last 3 values.
I do use these occasionally (contrary to `values' which I've never used).

So I guess 3 is plenty indeed.  In many circumstances where I could use this
kind of feature I prefer using (setq sm-tmp <exp>) and then refer to that
value as `sm-tmp' this way I can reuse this value many times easily
(whereas with `values' I'd have to keep track of the position within the
list etc...).


        Stefan




^ permalink raw reply	[flat|nested] 23+ messages in thread

* RE: frames vs. weak hash tables and garbage collection
  2007-09-29 18:28                       ` Stefan Monnier
@ 2007-09-29 19:25                         ` Drew Adams
  2007-09-30 12:55                           ` Richard Stallman
  0 siblings, 1 reply; 23+ messages in thread
From: Drew Adams @ 2007-09-29 19:25 UTC (permalink / raw)
  To: Stefan Monnier, Joe Wells; +Cc: bug-gnu-emacs, rms

> ielm implements its own mechanism for it and it calls it "*", "**", and
> "***", giving access only to the last 3 values.
> I do use these occasionally (contrary to `values' which I've never used).
> 
> So I guess 3 is plenty indeed.  In many circumstances where I 
> could use this
> kind of feature I prefer using (setq sm-tmp <exp>) and then refer to that
> value as `sm-tmp' this way I can reuse this value many times easily
> (whereas with `values' I'd have to keep track of the position within the
> list etc...).

Common Lisp has the same thing: http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node181.html





^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: frames vs. weak hash tables and garbage collection
  2007-09-29 19:25                         ` Drew Adams
@ 2007-09-30 12:55                           ` Richard Stallman
  0 siblings, 0 replies; 23+ messages in thread
From: Richard Stallman @ 2007-09-30 12:55 UTC (permalink / raw)
  To: Drew Adams; +Cc: bug-gnu-emacs, jbw, monnier

I don't mind replacing `values' with `*', `**' and `***'.

However, the name `eval-expression-recent-values' is too long
for such a convenience feature.




^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2007-09-30 12:55 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-08-30  1:27 frames vs. weak hash tables and garbage collection Joe Wells
2007-08-30  2:07 ` Eric Hanchrow
2007-08-30  3:00 ` Thien-Thi Nguyen
2007-08-30  3:07 ` Thien-Thi Nguyen
     [not found] ` <mailman.47.1188442895.18990.bug-gnu-emacs@gnu.org>
2007-08-31 15:15   ` Joe Wells
2007-08-31 15:42     ` Thien-Thi Nguyen
2007-08-31 15:50     ` Thien-Thi Nguyen
     [not found]     ` <mailman.116.1188575498.18990.bug-gnu-emacs@gnu.org>
     [not found]       ` <xnjir6tyj35.fsf@csb.bu.edu>
2007-09-02  2:00         ` Thien-Thi Nguyen
     [not found]     ` <mailman.115.1188574978.18990.bug-gnu-emacs@gnu.org>
     [not found]       ` <xnjhcmdyirv.fsf@csb.bu.edu>
2007-09-02  2:13         ` Thien-Thi Nguyen
2007-09-25 23:23     ` Joe Wells
2007-09-27  7:20       ` Glenn Morris
2007-09-27  8:50         ` Thien-Thi Nguyen
     [not found]           ` <mailman.1408.1190931512.18990.bug-gnu-emacs@gnu.org>
2007-09-28 14:34             ` Stefan Monnier
2007-09-28 14:56               ` Joe Wells
2007-09-28 16:27                 ` Stefan Monnier
2007-09-28 16:50                   ` Joe Wells
2007-09-28 18:22                     ` Stefan Monnier
2007-09-28 18:48                       ` Joe Wells
2007-09-29 16:10                   ` Richard Stallman
2007-09-29 16:20                     ` Joe Wells
2007-09-29 18:28                       ` Stefan Monnier
2007-09-29 19:25                         ` Drew Adams
2007-09-30 12:55                           ` Richard Stallman

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).