* Debugging memory leaks/stale references
@ 2004-09-21 17:38 Florian Weimer
2004-09-21 19:49 ` Simon Josefsson
2004-09-21 19:57 ` Stefan Monnier
0 siblings, 2 replies; 15+ messages in thread
From: Florian Weimer @ 2004-09-21 17:38 UTC (permalink / raw)
I've decided that it's time to track down an annoying bug in Gnus (or
Emacs). The symptom is that memory-limit keeps growing and growing,
but the data segment size (as allocated from the operating system)
remains approximately constant. In my case, this behavior is exposed
mainly when entering large NNTP groups with Gnus.
Is it possible to dump all reachable Lisp objects to a file, in the
format that is returned by princ? I could run diff(1) on two versions
of the file, and this might reveal which objects are leaking (or held
by stale references). garbage-collect output alone is not very
helpful.
(I'm going to test CVS Emacs soon, just to be sure that the behavior
is still the same. It's probably a Gnus bug, but you never know.)
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-21 17:38 Debugging memory leaks/stale references Florian Weimer
@ 2004-09-21 19:49 ` Simon Josefsson
2004-09-27 19:40 ` Florian Weimer
2004-09-21 19:57 ` Stefan Monnier
1 sibling, 1 reply; 15+ messages in thread
From: Simon Josefsson @ 2004-09-21 19:49 UTC (permalink / raw)
Florian Weimer <fw@deneb.enyo.de> writes:
> I've decided that it's time to track down an annoying bug in Gnus (or
> Emacs). The symptom is that memory-limit keeps growing and growing,
> but the data segment size (as allocated from the operating system)
> remains approximately constant. In my case, this behavior is exposed
> mainly when entering large NNTP groups with Gnus.
>
> Is it possible to dump all reachable Lisp objects to a file, in the
> format that is returned by princ? I could run diff(1) on two versions
> of the file, and this might reveal which objects are leaking (or held
> by stale references). garbage-collect output alone is not very
> helpful.
>
> (I'm going to test CVS Emacs soon, just to be sure that the behavior
> is still the same. It's probably a Gnus bug, but you never know.)
I'm using CVS Emacs, and can confirm that the behavior is the same.
When I notice that emacs feel sluggish, it is often the case that ps
report that emacs is using nearly all physical amount memory (1GB
here). Tracking down this would be good. I think someone said that
disabling the gnus agent at least reduce the problem.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-21 17:38 Debugging memory leaks/stale references Florian Weimer
2004-09-21 19:49 ` Simon Josefsson
@ 2004-09-21 19:57 ` Stefan Monnier
2004-09-21 21:01 ` Florian Weimer
1 sibling, 1 reply; 15+ messages in thread
From: Stefan Monnier @ 2004-09-21 19:57 UTC (permalink / raw)
Cc: emacs-devel
> Is it possible to dump all reachable Lisp objects to a file, in the
> format that is returned by princ?
Not that I know.
> I could run diff(1) on two versions of the file, and this might reveal
> which objects are leaking (or held by stale references).
That's an interesting idea.
But one of the problems is that `princ' is not enough. Starting from the
root set (i.e. the stack, and the global vars), princ would fail to print
a lot of info because buffers are just printed #<buffer foo> without
exposing its contents.
> garbage-collect output alone is not very helpful.
Indeed. One thing I was thinking about is a function
`count-exclusive-children' which would take an object X as a parameter
(could be the obarray, a buffer, a symbol, ...) and would return the number
of objects that would die if X died.
I think this can be implemented reasonably easily:
0 - change mark_object so as to increment a counter `objects_marked' each
time it marks an object.
1 - set the mark bit on X (so as ot pretend it and its children were
already marked).
2 - call the gc_mark routine (extracted from garbage_collect).
4 - reset objects_marked to 0.
3 - call `mark_object(X)'.
5 - read the value of objects_marked.
6 - do the gc_sweep.
7 - return the value read.
Obviously it'd be a slow function (it does a full GC just to get a single
integer number), but maybe it's good enough?
You could start by calling (mapcar 'count-exclusive-children (buffer-list))
every once in a while to see how the numbers evolve.
Or maybe (let ((res nil))
(mapatoms (lambda (s)
(push (cons s count-exclusive-children s) res)))
res)
> (I'm going to test CVS Emacs soon, just to be sure that the behavior
> is still the same.
I think it's indeed the same with CVS, based on reports we got here not that
long ago from Kai and others.
> It's probably a Gnus bug, but you never know.)
It might also be in Emacs or in an interaction between the two.
I remember seeing strange things a couple years back where dead buffers
where kept "alive" much longer than I thought (and they can hold on to
a lot of data in their variables and overlays).
Stefan
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-21 19:57 ` Stefan Monnier
@ 2004-09-21 21:01 ` Florian Weimer
0 siblings, 0 replies; 15+ messages in thread
From: Florian Weimer @ 2004-09-21 21:01 UTC (permalink / raw)
Cc: emacs-devel
* Stefan Monnier:
>> Is it possible to dump all reachable Lisp objects to a file, in the
>> format that is returned by princ?
>
> Not that I know.
>
>> I could run diff(1) on two versions of the file, and this might reveal
>> which objects are leaking (or held by stale references).
>
> That's an interesting idea.
> But one of the problems is that `princ' is not enough. Starting from the
> root set (i.e. the stack, and the global vars), princ would fail to print
> a lot of info because buffers are just printed #<buffer foo> without
> exposing its contents.
Hmm, I don't think so. You'd have to mimic a GC traversal, and GC
does descend into buffer objects.
> Indeed. One thing I was thinking about is a function
> `count-exclusive-children' which would take an object X as a parameter
> (could be the obarray, a buffer, a symbol, ...) and would return the number
> of objects that would die if X died.
I'm not sure about the usefulness of this function. If data
structures are cyclic, it might not yield any actionable data at all.
> I think it's indeed the same with CVS, based on reports we got here not that
> long ago from Kai and others.
Indeed. But at least my Emacs doesn't crash at the end of the day.
>> It's probably a Gnus bug, but you never know.)
>
> It might also be in Emacs or in an interaction between the two.
Or even with GNU libc's malloc(). The Emacs allocator disables
allocation using mmap() in some cases, and the corresponding part of
the libc allocator might have been subject to bit rot lately.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-21 19:49 ` Simon Josefsson
@ 2004-09-27 19:40 ` Florian Weimer
2004-09-27 19:52 ` Stefan Monnier
2004-09-28 15:20 ` Richard Stallman
0 siblings, 2 replies; 15+ messages in thread
From: Florian Weimer @ 2004-09-27 19:40 UTC (permalink / raw)
* Simon Josefsson:
> I'm using CVS Emacs, and can confirm that the behavior is the same.
> When I notice that emacs feel sluggish, it is often the case that ps
> report that emacs is using nearly all physical amount memory (1GB
> here). Tracking down this would be good. I think someone said that
> disabling the gnus agent at least reduce the problem.
Okay, I've done some debugging, mainly by instrumenting the garbage
collector. Basically, the patch below is a hook into mark_object()
and prints object types, address, and also contents (for symbols)
while they are traversed. Unfortunately, no major leak turned up,
just a few thousand cons cells which can't cause the massive leak I
see (in the order of dozens of megabytes after entering/exiting a
large NNTP group with Gnus).
So I'm back to the drawing board and have a few further questions.
Is it possible to run a full-featured Emacs (including X11 support) on
a target that does not support dumping? (In case you wonder,
x86/valgrind is such a target. 8-)
Is there a method to determine the (approximate) size of a buffer?
Are there any other objects that can change their size after
allocation? (I'm pretty sure that there are no additional Lisp
objects allocated, but maybe an existing object grows without bounds.)
Oh, and for your amusement, I've appended by debugging patch below.
Basically, this is straight from the "you do not want to need this,
really" department. Of course, it's not intended for inclusion in to
Emacs (except for the first hunk, maybe).
--- orig/src/.arch-inventory
+++ mod/src/.arch-inventory
@@ -4,6 +4,6 @@
# Auto-generated files, which ignore
precious ^(config\.stamp|config\.h|epaths\.h)$
-backup ^(stamp-oldxmenu|prefix-args|temacs|emacs|emacs-[0-9.]*)$
+backup ^(stamp-oldxmenu|prefix-args|temacs|emacs|TAGS-LISP|emacs-[0-9.]*)$
# arch-tag: 277cc7ae-b3f5-44af-abf1-84c073164543
--- orig/src/alloc.c
+++ mod/src/alloc.c
@@ -258,12 +258,15 @@
Lisp_Object Vgc_elapsed; /* accumulated elapsed time in GC */
EMACS_INT gcs_done; /* accumulated GCs */
-static void mark_buffer P_ ((Lisp_Object));
-extern void mark_kboards P_ ((void));
-extern void mark_backtrace P_ ((void));
+/* If non-zero, dump objects to stderr while they are marked. */
+static int do_dump = 0;
+
+static void dump_marker_section (char *name);
+
+static void mark_buffer P_ ((Lisp_Object, unsigned));
static void gc_sweep P_ ((void));
-static void mark_glyph_matrix P_ ((struct glyph_matrix *));
-static void mark_face_cache P_ ((struct face_cache *));
+static void mark_glyph_matrix P_ ((struct glyph_matrix *, unsigned));
+static void mark_face_cache P_ ((struct face_cache *, unsigned));
#ifdef HAVE_WINDOW_SYSTEM
static void mark_image P_ ((struct image *));
@@ -275,8 +278,6 @@
static void free_large_strings P_ ((void));
static void sweep_strings P_ ((void));
-extern int message_enable_multibyte;
-
/* When scanning the C stack for live Lisp objects, Emacs keeps track
of what memory allocated via lisp_malloc is intended for what
purpose. This enumeration specifies the type of memory. */
@@ -389,8 +390,8 @@
static int live_symbol_p P_ ((struct mem_node *, void *));
static int live_float_p P_ ((struct mem_node *, void *));
static int live_misc_p P_ ((struct mem_node *, void *));
-static void mark_maybe_object P_ ((Lisp_Object));
-static void mark_memory P_ ((void *, void *));
+static void mark_maybe_object P_ ((Lisp_Object, unsigned));
+static void mark_memory P_ ((void *, void *, unsigned));
static void mem_init P_ ((void));
static struct mem_node *mem_insert P_ ((void *, void *, enum mem_type));
static void mem_insert_fixup P_ ((struct mem_node *));
@@ -1217,7 +1218,7 @@
{
eassert (!i->gcmarkbit); /* Intervals are never shared. */
i->gcmarkbit = 1;
- mark_object (i->plist);
+ mark_object (i->plist, 0);
}
@@ -3688,8 +3689,9 @@
/* Mark OBJ if we can prove it's a Lisp_Object. */
static INLINE void
-mark_maybe_object (obj)
+mark_maybe_object (obj, depth)
Lisp_Object obj;
+ unsigned depth;
{
void *po = (void *) XPNTR (obj);
struct mem_node *m = mem_find (po);
@@ -3743,7 +3745,7 @@
zombies[nzombies] = obj;
++nzombies;
#endif
- mark_object (obj);
+ mark_object (obj, depth + 1);
}
}
}
@@ -3753,8 +3755,9 @@
marked. */
static INLINE void
-mark_maybe_pointer (p)
+mark_maybe_pointer (p, depth)
void *p;
+ unsigned depth;
{
struct mem_node *m;
@@ -3824,7 +3827,7 @@
}
if (!GC_NILP (obj))
- mark_object (obj);
+ mark_object (obj, depth + 1);
}
}
@@ -3832,8 +3835,9 @@
/* Mark Lisp objects referenced from the address range START..END. */
static void
-mark_memory (start, end)
+mark_memory (start, end, depth)
void *start, *end;
+ unsigned depth;
{
Lisp_Object *p;
void **pp;
@@ -3853,7 +3857,7 @@
/* Mark Lisp_Objects. */
for (p = (Lisp_Object *) start; (void *) p < end; ++p)
- mark_maybe_object (*p);
+ mark_maybe_object (*p, depth + 1);
/* Mark Lisp data pointed to. This is necessary because, in some
situations, the C compiler optimizes Lisp objects away, so that
@@ -4098,7 +4102,7 @@
#endif
#endif
for (i = 0; i < sizeof (Lisp_Object); i += GC_LISP_OBJECT_ALIGNMENT)
- mark_memory ((char *) stack_base + i, end);
+ mark_memory ((char *) stack_base + i, end, 1);
#if GC_MARK_STACK == GC_MARK_STACK_CHECK_GCPROS
check_gcpros ();
@@ -4455,14 +4459,14 @@
/* Mark all the special slots that serve as the roots of accessibility. */
for (i = 0; i < staticidx; i++)
- mark_object (*staticvec[i]);
+ mark_object (*staticvec[i], 1);
for (bind = specpdl; bind != specpdl_ptr; bind++)
{
- mark_object (bind->symbol);
- mark_object (bind->old_value);
+ mark_object (bind->symbol, 1);
+ mark_object (bind->old_value, 1);
}
- mark_kboards ();
+ mark_kboards (1);
#ifdef USE_GTK
{
@@ -4483,18 +4487,22 @@
}
#endif
+ dump_marker_section ("Begin marking byte stack.");
mark_byte_stack ();
+ dump_marker_section ("Begin marking byte stack.");
for (catch = catchlist; catch; catch = catch->next)
{
- mark_object (catch->tag);
- mark_object (catch->val);
+ mark_object (catch->tag, 1);
+ mark_object (catch->val, 1);
}
for (handler = handlerlist; handler; handler = handler->next)
{
- mark_object (handler->handler);
- mark_object (handler->var);
+ mark_object (handler->handler, 1);
+ mark_object (handler->var, 1);
}
+ dump_marker_section ("Begin marking backtrace.");
mark_backtrace ();
+ dump_marker_section ("End marking backtrace.");
#if GC_MARK_STACK == GC_USE_GCPROS_CHECK_ZOMBIES
mark_stack ();
@@ -4542,7 +4550,7 @@
}
/* Now that we have stripped the elements that need not be in the
undo_list any more, we can finally mark the list. */
- mark_object (nextb->undo_list);
+ mark_object (nextb->undo_list, 1);
nextb = nextb->next;
}
@@ -4630,13 +4638,24 @@
return Flist (sizeof total / sizeof *total, total);
}
+DEFUN ("garbage-collect-dump", Fgarbage_collect_dump, Sgarbage_collect_dump, 0, 0, "",
+ doc: /* Run garbage collection and dump objects. */)
+ ()
+{
+ ++do_dump;
+ dump_marker_section ("Begin garbage collection.");
+ Fgarbage_collect ();
+ dump_marker_section ("End garbage collection.");
+ --do_dump;
+}
/* Mark Lisp objects in glyph matrix MATRIX. Currently the
only interesting objects referenced from glyphs are strings. */
static void
-mark_glyph_matrix (matrix)
+mark_glyph_matrix (matrix, depth)
struct glyph_matrix *matrix;
+ unsigned depth;
{
struct glyph_row *row = matrix->rows;
struct glyph_row *end = row + matrix->nrows;
@@ -4653,7 +4672,7 @@
for (; glyph < end_glyph; ++glyph)
if (GC_STRINGP (glyph->object)
&& !STRING_MARKED_P (XSTRING (glyph->object)))
- mark_object (glyph->object);
+ mark_object (glyph->object, depth + 1);
}
}
}
@@ -4662,8 +4681,9 @@
/* Mark Lisp faces in the face cache C. */
static void
-mark_face_cache (c)
+mark_face_cache (c, depth)
struct face_cache *c;
+ unsigned depth;
{
if (c)
{
@@ -4675,7 +4695,7 @@
if (face)
{
for (j = 0; j < LFACE_VECTOR_SIZE; ++j)
- mark_object (face->lface[j]);
+ mark_object (face->lface[j], depth + 1);
}
}
}
@@ -4690,10 +4710,10 @@
mark_image (img)
struct image *img;
{
- mark_object (img->spec);
+ mark_object (img->spec, 1);
if (!NILP (img->data.lisp_val))
- mark_object (img->data.lisp_val);
+ mark_object (img->data.lisp_val, 1);
}
@@ -4725,9 +4745,84 @@
Normally this is zero and the check never goes off. */
int mark_object_loop_halt;
+/* For memory debugging: dumps the string in human-readable form to
+ stderr. */
+static void
+dump_string (char *data, unsigned length)
+{
+ unsigned i;
+
+ for (i = 0; i < length; ++i)
+ {
+ char c = data[i];
+ if (c >= 32 && c <= 126)
+ fputc (c, stderr);
+ else if (c == '"' || c == '\\')
+ {
+ fputc ('\\', stderr);
+ fputc (c, stderr);
+ }
+ else if (c == '\n')
+ fputs ("\\n", stderr);
+ else if (c == '\t')
+ fputs ("\\t", stderr);
+ else if (c == '\r')
+ fputs ("\\r", stderr);
+ else
+ fprintf (stderr, "\\%03o", (unsigned)(unsigned char) c);
+ }
+}
+
+/* For memory debugging: prints OBJ of TYPE, at nesting level
+ DEPTH to stderr. */
+static void
+dump_object (char *type, unsigned depth, Lisp_Object obj, int marked)
+{
+ unsigned i;
+
+ if (!do_dump)
+ return;
+
+ fprintf (stderr, "[%u] %s (%p)%s\n",
+ depth, type, (void *)obj, marked ? " *" : "");
+}
+
+static void
+dump_object_int (unsigned depth, Lisp_Object obj)
+{
+ if (!do_dump)
+ return;
+
+ fprintf (stderr, "[%u] INT %d\n", depth, XINT(obj));
+}
+
+static void
+dump_marker_section(char *name)
+{
+ if (!do_dump)
+ return;
+
+ fprintf (stderr, "%s\n", name);
+}
+
+/* For memory debugging: prints OBJ of TYPE, with NAME, at nesting
+ level DEPTH to stderr. */
+static void
+dump_object_name (char *type, unsigned depth, Lisp_Object obj, struct Lisp_String* name, int marked)
+{
+ if (!do_dump)
+ return;
+
+ fprintf (stderr, "[%u] %s (%p) \"", depth, type, (void *)obj);
+ dump_string (name->data, name->size & ~ARRAY_MARK_FLAG);
+ fprintf (stderr, "\"%s\n", marked ? " *" : "");
+}
+
+
void
-mark_object (arg)
+mark_object (arg, depth)
Lisp_Object arg;
+ unsigned depth;
{
register Lisp_Object obj = arg;
#ifdef GC_CHECK_MARKED_OBJECTS
@@ -4792,6 +4887,7 @@
CHECK_ALLOCATED_AND_LIVE (live_string_p);
MARK_INTERVAL_TREE (ptr->intervals);
MARK_STRING (ptr);
+ dump_object_name ("STRING", depth, obj, ptr, ptr->size & ARRAY_MARK_FLAG);
#ifdef GC_CHECK_STRING_BYTES
/* Check that the string size recorded in the string is the
same as the one recorded in the sdata structure. */
@@ -4811,6 +4907,8 @@
if (GC_BUFFERP (obj))
{
+ dump_object ("BUFFER", depth, obj, VECTOR_MARKED_P (XBUFFER (obj)));
+
if (!VECTOR_MARKED_P (XBUFFER (obj)))
{
#ifdef GC_CHECK_MARKED_OBJECTS
@@ -4823,7 +4921,7 @@
abort ();
}
#endif /* GC_CHECK_MARKED_OBJECTS */
- mark_buffer (obj);
+ mark_buffer (obj, depth + 1);
}
}
else if (GC_SUBRP (obj))
@@ -4837,6 +4935,8 @@
register EMACS_INT size = ptr->size;
register int i;
+ dump_object ("COMPILED", depth, obj, VECTOR_MARKED_P (ptr));
+
if (VECTOR_MARKED_P (ptr))
break; /* Already marked */
@@ -4846,7 +4946,7 @@
for (i = 0; i < size; i++) /* and then mark its elements */
{
if (i != COMPILED_CONSTANTS)
- mark_object (ptr->contents[i]);
+ mark_object (ptr->contents[i], depth + 1);
}
obj = ptr->contents[COMPILED_CONSTANTS];
goto loop;
@@ -4855,40 +4955,48 @@
{
register struct frame *ptr = XFRAME (obj);
+ dump_object ("FRAME", depth, obj, VECTOR_MARKED_P (ptr));
+
if (VECTOR_MARKED_P (ptr)) break; /* Already marked */
+
VECTOR_MARK (ptr); /* Else mark it */
CHECK_LIVE (live_vector_p);
- mark_object (ptr->name);
- mark_object (ptr->icon_name);
- mark_object (ptr->title);
- mark_object (ptr->focus_frame);
- mark_object (ptr->selected_window);
- mark_object (ptr->minibuffer_window);
- mark_object (ptr->param_alist);
- mark_object (ptr->scroll_bars);
- mark_object (ptr->condemned_scroll_bars);
- mark_object (ptr->menu_bar_items);
- mark_object (ptr->face_alist);
- mark_object (ptr->menu_bar_vector);
- mark_object (ptr->buffer_predicate);
- mark_object (ptr->buffer_list);
- mark_object (ptr->menu_bar_window);
- mark_object (ptr->tool_bar_window);
- mark_face_cache (ptr->face_cache);
+ mark_object (ptr->name, depth + 1);
+ mark_object (ptr->icon_name, depth + 1);
+ mark_object (ptr->title, depth + 1);
+ mark_object (ptr->focus_frame, depth + 1);
+ mark_object (ptr->selected_window, depth + 1);
+ mark_object (ptr->minibuffer_window, depth + 1);
+ mark_object (ptr->param_alist, depth + 1);
+ mark_object (ptr->scroll_bars, depth + 1);
+ mark_object (ptr->condemned_scroll_bars, depth + 1);
+ mark_object (ptr->menu_bar_items, depth + 1);
+ mark_object (ptr->face_alist, depth + 1);
+ mark_object (ptr->menu_bar_vector, depth + 1);
+ mark_object (ptr->buffer_predicate, depth + 1);
+ mark_object (ptr->buffer_list, depth + 1);
+ mark_object (ptr->menu_bar_window, depth + 1);
+ mark_object (ptr->tool_bar_window, depth + 1);
+ mark_face_cache (ptr->face_cache, depth + 1);
#ifdef HAVE_WINDOW_SYSTEM
+ dump_marker_section ("Begin marking FRAME images.");
mark_image_cache (ptr);
- mark_object (ptr->tool_bar_items);
- mark_object (ptr->desired_tool_bar_string);
- mark_object (ptr->current_tool_bar_string);
+ dump_marker_section ("End marking FRAME images.");
+ mark_object (ptr->tool_bar_items, depth + 1);
+ mark_object (ptr->desired_tool_bar_string, depth + 1);
+ mark_object (ptr->current_tool_bar_string, depth + 1);
#endif /* HAVE_WINDOW_SYSTEM */
}
else if (GC_BOOL_VECTOR_P (obj))
{
register struct Lisp_Vector *ptr = XVECTOR (obj);
+ dump_object ("BOOL_VECTOR", depth, obj, VECTOR_MARKED_P (ptr));
+
if (VECTOR_MARKED_P (ptr))
break; /* Already marked */
+
CHECK_LIVE (live_vector_p);
VECTOR_MARK (ptr); /* Else mark it */
}
@@ -4898,6 +5006,8 @@
struct window *w = XWINDOW (obj);
register int i;
+ dump_object ("WINDOW", depth, obj, VECTOR_MARKED_P (ptr));
+
/* Stop if already marked. */
if (VECTOR_MARKED_P (ptr))
break;
@@ -4911,7 +5021,7 @@
for (i = 0;
(char *) &ptr->contents[i] < (char *) &w->current_matrix;
i++)
- mark_object (ptr->contents[i]);
+ mark_object (ptr->contents[i], depth + 1);
/* Mark glyphs for leaf windows. Marking window matrices is
sufficient because frame matrices use the same glyph
@@ -4920,14 +5030,16 @@
&& NILP (w->vchild)
&& w->current_matrix)
{
- mark_glyph_matrix (w->current_matrix);
- mark_glyph_matrix (w->desired_matrix);
+ mark_glyph_matrix (w->current_matrix, depth + 1);
+ mark_glyph_matrix (w->desired_matrix, depth + 1);
}
}
else if (GC_HASH_TABLE_P (obj))
{
struct Lisp_Hash_Table *h = XHASH_TABLE (obj);
+ dump_object ("HASH_TABLE", depth, obj, VECTOR_MARKED_P (h));
+
/* Stop if already marked. */
if (VECTOR_MARKED_P (h))
break;
@@ -4941,20 +5053,20 @@
Being in the next_weak chain
should not keep the hash table alive.
No need to mark `count' since it is an integer. */
- mark_object (h->test);
- mark_object (h->weak);
- mark_object (h->rehash_size);
- mark_object (h->rehash_threshold);
- mark_object (h->hash);
- mark_object (h->next);
- mark_object (h->index);
- mark_object (h->user_hash_function);
- mark_object (h->user_cmp_function);
+ mark_object (h->test, depth + 1);
+ mark_object (h->weak, depth + 1);
+ mark_object (h->rehash_size, depth + 1);
+ mark_object (h->rehash_threshold, depth + 1);
+ mark_object (h->hash, depth + 1);
+ mark_object (h->next, depth + 1);
+ mark_object (h->index, depth + 1);
+ mark_object (h->user_hash_function, depth + 1);
+ mark_object (h->user_cmp_function, depth + 1);
/* If hash table is not weak, mark all keys and values.
For weak tables, mark only the vector. */
if (GC_NILP (h->weak))
- mark_object (h->key_and_value);
+ mark_object (h->key_and_value, depth + 1);
else
VECTOR_MARK (XVECTOR (h->key_and_value));
}
@@ -4964,6 +5076,8 @@
register EMACS_INT size = ptr->size;
register int i;
+ dump_object ("VECTOR", depth, obj, VECTOR_MARKED_P (ptr));
+
if (VECTOR_MARKED_P (ptr)) break; /* Already marked */
CHECK_LIVE (live_vector_p);
VECTOR_MARK (ptr); /* Else mark it */
@@ -4971,7 +5085,7 @@
size &= PSEUDOVECTOR_SIZE_MASK;
for (i = 0; i < size; i++) /* and then mark its elements */
- mark_object (ptr->contents[i]);
+ mark_object (ptr->contents[i], depth + 1);
}
break;
@@ -4980,12 +5094,15 @@
register struct Lisp_Symbol *ptr = XSYMBOL (obj);
struct Lisp_Symbol *ptrx;
+ dump_object_name ("SYMBOL", depth, obj, XSTRING (ptr->xname), ptr->gcmarkbit);
+
if (ptr->gcmarkbit) break;
+
CHECK_ALLOCATED_AND_LIVE (live_symbol_p);
ptr->gcmarkbit = 1;
- mark_object (ptr->value);
- mark_object (ptr->function);
- mark_object (ptr->plist);
+ mark_object (ptr->value, depth + 1);
+ mark_object (ptr->function, depth + 1);
+ mark_object (ptr->plist, depth + 1);
if (!PURE_POINTER_P (XSTRING (ptr->xname)))
MARK_STRING (XSTRING (ptr->xname));
@@ -5006,6 +5123,7 @@
case Lisp_Misc:
CHECK_ALLOCATED_AND_LIVE (live_misc_p);
+ dump_object ("MISC", depth, obj, XMARKER (obj)->gcmarkbit);
if (XMARKER (obj)->gcmarkbit)
break;
XMARKER (obj)->gcmarkbit = 1;
@@ -5023,9 +5141,9 @@
obj = ptr->realvalue;
goto loop;
}
- mark_object (ptr->realvalue);
- mark_object (ptr->buffer);
- mark_object (ptr->frame);
+ mark_object (ptr->realvalue, depth + 1);
+ mark_object (ptr->buffer, depth + 1);
+ mark_object (ptr->frame, depth + 1);
obj = ptr->cdr;
goto loop;
}
@@ -5058,18 +5176,17 @@
Lisp_Object *p = (Lisp_Object *) ptr->pointer;
int nelt;
for (nelt = ptr->integer; nelt > 0; nelt--, p++)
- mark_maybe_object (*p);
+ mark_maybe_object (*p, depth + 1);
}
}
#endif
break;
-
case Lisp_Misc_Overlay:
{
struct Lisp_Overlay *ptr = XOVERLAY (obj);
- mark_object (ptr->start);
- mark_object (ptr->end);
- mark_object (ptr->plist);
+ mark_object (ptr->start, depth + 1);
+ mark_object (ptr->end, depth + 1);
+ mark_object (ptr->plist, depth + 1);
if (ptr->next)
{
XSETMISC (obj, ptr->next);
@@ -5086,30 +5203,23 @@
case Lisp_Cons:
{
register struct Lisp_Cons *ptr = XCONS (obj);
+ dump_object ("CONS", depth, obj, CONS_MARKED_P (ptr));
if (CONS_MARKED_P (ptr)) break;
CHECK_ALLOCATED_AND_LIVE (live_cons_p);
CONS_MARK (ptr);
- /* If the cdr is nil, avoid recursion for the car. */
- if (EQ (ptr->cdr, Qnil))
- {
- obj = ptr->car;
- cdr_count = 0;
- goto loop;
- }
- mark_object (ptr->car);
- obj = ptr->cdr;
- cdr_count++;
- if (cdr_count == mark_object_loop_halt)
- abort ();
- goto loop;
+ mark_object (ptr->car, depth + 1);
+ mark_object (ptr->cdr, depth + 1);
+ break;
}
case Lisp_Float:
CHECK_ALLOCATED_AND_LIVE (live_float_p);
+ dump_object ("CONS", depth, obj, FLOAT_MARKED_P (XFLOAT (obj)));
FLOAT_MARK (XFLOAT (obj));
break;
case Lisp_Int:
+ dump_object_int (depth, obj);
break;
default:
@@ -5124,8 +5234,9 @@
/* Mark the pointers in a buffer structure. */
static void
-mark_buffer (buf)
+mark_buffer (buf, depth)
Lisp_Object buf;
+ unsigned depth;
{
register struct buffer *buffer = XBUFFER (buf);
register Lisp_Object *ptr, tmp;
@@ -5142,24 +5253,24 @@
if (buffer->overlays_before)
{
XSETMISC (tmp, buffer->overlays_before);
- mark_object (tmp);
+ mark_object (tmp, depth + 1);
}
if (buffer->overlays_after)
{
XSETMISC (tmp, buffer->overlays_after);
- mark_object (tmp);
+ mark_object (tmp, depth + 1);
}
for (ptr = &buffer->name;
(char *)ptr < (char *)buffer + sizeof (struct buffer);
ptr++)
- mark_object (*ptr);
+ mark_object (*ptr, depth + 1);
/* If this is an indirect buffer, mark its base buffer. */
if (buffer->base_buffer && !VECTOR_MARKED_P (buffer->base_buffer))
{
XSETBUFFER (base_buffer, buffer->base_buffer);
- mark_buffer (base_buffer);
+ mark_buffer (base_buffer, depth + 1);
}
}
@@ -5792,6 +5903,7 @@
defsubr (&Smake_marker);
defsubr (&Spurecopy);
defsubr (&Sgarbage_collect);
+ defsubr (&Sgarbage_collect_dump);
defsubr (&Smemory_limit);
defsubr (&Smemory_use_counts);
--- orig/src/bytecode.c
+++ mod/src/bytecode.c
@@ -289,10 +289,10 @@
eassert (stack->top);
for (obj = stack->bottom; obj <= stack->top; ++obj)
- mark_object (*obj);
+ mark_object (*obj, 1);
- mark_object (stack->byte_string);
- mark_object (stack->constants);
+ mark_object (stack->byte_string, 1);
+ mark_object (stack->constants, 1);
}
}
--- orig/src/eval.c
+++ mod/src/eval.c
@@ -3260,14 +3260,14 @@
for (backlist = backtrace_list; backlist; backlist = backlist->next)
{
- mark_object (*backlist->function);
+ mark_object (*backlist->function, 1);
if (backlist->nargs == UNEVALLED || backlist->nargs == MANY)
i = 0;
else
i = backlist->nargs - 1;
for (; i >= 0; i--)
- mark_object (backlist->args[i]);
+ mark_object (backlist->args[i], 1);
}
}
--- orig/src/fns.c
+++ mod/src/fns.c
@@ -4804,13 +4804,13 @@
/* Make sure key and value survive. */
if (!key_known_to_survive_p)
{
- mark_object (HASH_KEY (h, i));
+ mark_object (HASH_KEY (h, i), 1);
marked = 1;
}
if (!value_known_to_survive_p)
{
- mark_object (HASH_VALUE (h, i));
+ mark_object (HASH_VALUE (h, i), 1);
marked = 1;
}
}
--- orig/src/keyboard.c
+++ mod/src/keyboard.c
@@ -11425,7 +11425,8 @@
/* Mark the pointers in the kboard objects.
Called by the Fgarbage_collector. */
void
-mark_kboards ()
+mark_kboards (depth)
+ unsigned depth;
{
KBOARD *kb;
Lisp_Object *p;
@@ -11433,19 +11434,19 @@
{
if (kb->kbd_macro_buffer)
for (p = kb->kbd_macro_buffer; p < kb->kbd_macro_ptr; p++)
- mark_object (*p);
- mark_object (kb->Voverriding_terminal_local_map);
- mark_object (kb->Vlast_command);
- mark_object (kb->Vreal_last_command);
- mark_object (kb->Vprefix_arg);
- mark_object (kb->Vlast_prefix_arg);
- mark_object (kb->kbd_queue);
- mark_object (kb->defining_kbd_macro);
- mark_object (kb->Vlast_kbd_macro);
- mark_object (kb->Vsystem_key_alist);
- mark_object (kb->system_key_syms);
- mark_object (kb->Vdefault_minibuffer_frame);
- mark_object (kb->echo_string);
+ mark_object (*p, depth + 1);
+ mark_object (kb->Voverriding_terminal_local_map, depth + 1);
+ mark_object (kb->Vlast_command, depth + 1);
+ mark_object (kb->Vreal_last_command, depth + 1);
+ mark_object (kb->Vprefix_arg, depth + 1);
+ mark_object (kb->Vlast_prefix_arg, depth + 1);
+ mark_object (kb->kbd_queue, depth + 1);
+ mark_object (kb->defining_kbd_macro, depth + 1);
+ mark_object (kb->Vlast_kbd_macro, depth + 1);
+ mark_object (kb->Vsystem_key_alist, depth + 1);
+ mark_object (kb->system_key_syms, depth + 1);
+ mark_object (kb->Vdefault_minibuffer_frame, depth + 1);
+ mark_object (kb->echo_string, depth + 1);
}
{
struct input_event *event;
@@ -11455,11 +11456,11 @@
event = kbd_buffer;
if (event->kind != SELECTION_REQUEST_EVENT)
{
- mark_object (event->x);
- mark_object (event->y);
+ mark_object (event->x, depth + 1);
+ mark_object (event->y, depth + 1);
}
- mark_object (event->frame_or_window);
- mark_object (event->arg);
+ mark_object (event->frame_or_window, depth + 1);
+ mark_object (event->arg, depth + 1);
}
}
}
--- orig/src/lisp.h
+++ mod/src/lisp.h
@@ -2440,9 +2440,12 @@
extern void memory_full P_ ((void));
extern void buffer_memory_full P_ ((void));
extern int survives_gc_p P_ ((Lisp_Object));
-extern void mark_object P_ ((Lisp_Object));
+extern void mark_object P_ ((Lisp_Object, unsigned));
+extern void mark_kboards P_ ((unsigned));
+extern void mark_backtrace P_ ((void));
extern Lisp_Object Vpurify_flag;
extern Lisp_Object Vmemory_full;
+extern int message_enable_multibyte;
EXFUN (Fcons, 2);
EXFUN (list2, 2);
EXFUN (list3, 3);
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-27 19:40 ` Florian Weimer
@ 2004-09-27 19:52 ` Stefan Monnier
2004-09-27 20:36 ` Florian Weimer
2004-09-27 20:48 ` Simon Josefsson
2004-09-28 15:20 ` Richard Stallman
1 sibling, 2 replies; 15+ messages in thread
From: Stefan Monnier @ 2004-09-27 19:52 UTC (permalink / raw)
Cc: emacs-devel
> while they are traversed. Unfortunately, no major leak turned up,
> just a few thousand cons cells which can't cause the massive leak I
> see (in the order of dozens of megabytes after entering/exiting a
> large NNTP group with Gnus).
I think it would be worthwhile to start with the basic info returned by
`garbage-collect' and such stuff. See `memory-usage.el' below.
[ I thought you already tried that, but the few lines above make me think
that you don't even know whether the extra MBs are made up of cons cells, or
strings, or non-elisp-data, or ...]
> Is there a method to determine the (approximate) size of a buffer?
See buffer-size-bytes below.
> Are there any other objects that can change their size after
> allocation? (I'm pretty sure that there are no additional Lisp
> objects allocated, but maybe an existing object grows without bounds.)
Yes. Things like the specpdl "stack", the matrices used in display, maybe
the text-property-tree nodes, ...
Stefan
;;; memory-usage.el --- Analyze the memory usage of Emacs in various ways
;; Copyright (C) 2002, 2004 Free Software Foundation, Inc.
;; Author: Stefan Monnier <monnier@cs.yale.edu>
;; Keywords: maint
;; This file is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation; either version 2, or (at your option)
;; any later version.
;; This file is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
;; GNU General Public License for more details.
;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs; see the file COPYING. If not, write to
;; the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
;; Boston, MA 02111-1307, USA.
;;; Commentary:
;;
;;; Code:
(defun buffer-size-bytes (b)
"Return total number of bytes in the buffer contents."
(with-current-buffer b
(save-restriction
(widen)
(- (position-bytes (point-max)) (position-bytes (point-min))))))
(defun buffer-gap-bytes (b)
"Return total number of bytes in the buffer gap."
(with-current-buffer b
(gap-size)))
(defun buffer-total-bytes (b)
"Return total number of ralloc bytes used by buffer."
(with-current-buffer b
(save-restriction
(widen)
(+ (position-bytes (point-max))
(- (position-bytes (point-min)))
(gap-size)))))
;;;###autoload
(defun memory-usage ()
"List all buffers and their memory usage."
(interactive)
(pop-to-buffer (get-buffer-create "*Buffer Details*"))
(erase-buffer)
(let* ((bufs (buffer-list))
(num (length bufs))
(gc-stats (garbage-collect))
(conses (nth 0 gc-stats))
(symbols (nth 1 gc-stats))
(markers (nth 2 gc-stats))
(strings (nth 3 gc-stats))
(vectors (nth 4 gc-stats))
(floats (nth 5 gc-stats))
(intervals (nth 6 gc-stats)))
(insert (format "Garbage collection stats:\n%s\n\n =>" gc-stats))
(insert (format "\t%d bytes in cons cells\n" (* 8 (+ (car conses) (cdr conses)))))
(insert (format "\t%d bytes in symbols\n" (* 24 (+ (car symbols) (cdr symbols)))))
(insert (format "\t%d bytes in markers\n" (* 20 (+ (car markers) (cdr markers)))))
(insert (format "\t%d bytes of string chars\n" strings))
(insert (format "\t%d bytes of vector slots\n" (* 4 vectors)))
(insert (format "\t%d bytes in floats\n" (* 12 (+ (car floats) (cdr floats)))))
(insert (format "\t%d bytes in intervals\n" (* 28 (+ (car intervals) (cdr intervals)))))
(insert (format "\nTotal bytes in lisp objects (not counting string and vector headers): %d\n\n"
(+ (* 8 (+ (car conses) (cdr conses)))
(* 24 (+ (car symbols) (cdr symbols)))
(* 20 (+ (car markers) (cdr markers)))
strings
vectors
(* 12 (+ (car floats) (cdr floats)))
(* 28 (+ (car intervals) (cdr intervals))))))
(insert (format "Buffer ralloc memory usage:\n%d buffers\n%d bytes total (%d in gaps)\n"
num
(apply #'+ (mapcar #'buffer-total-bytes bufs))
(apply #'+ (mapcar #'buffer-gap-bytes bufs))))
(insert (format "%10s\t%s\t%s\n\n" "Size" "Gap" "Name"))
(insert (mapconcat
(lambda (b)
(format "%10d\t%s\t%s"
(buffer-size-bytes b)
(buffer-gap-bytes b)
(buffer-name b)))
(sort bufs (lambda (b1 b2)
(> (buffer-size-bytes b1) (buffer-size-bytes b2))))
"\n"))
(insert "\n"))
(goto-char (point-min)))
(provide 'memory-usage)
;; arch-tag: 04e012f0-3c59-4319-8d1a-e86204671ec5
;;; memory-usage.el ends here
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-27 19:52 ` Stefan Monnier
@ 2004-09-27 20:36 ` Florian Weimer
2004-09-27 20:49 ` Stefan Monnier
2004-09-27 20:48 ` Simon Josefsson
1 sibling, 1 reply; 15+ messages in thread
From: Florian Weimer @ 2004-09-27 20:36 UTC (permalink / raw)
Cc: emacs-devel
* Stefan Monnier:
> I think it would be worthwhile to start with the basic info returned by
> `garbage-collect' and such stuff. See `memory-usage.el' below.
> [ I thought you already tried that, but the few lines above make me think
> that you don't even know whether the extra MBs are made up of cons cells, or
> strings, or non-elisp-data, or ...]
Others already did this. But thanks anyway, I repeated the
experiment: The numbers indeed lack any distinct increase in
magnitude. 8-(
>> Are there any other objects that can change their size after
>> allocation? (I'm pretty sure that there are no additional Lisp
>> objects allocated, but maybe an existing object grows without bounds.)
>
> Yes. Things like the specpdl "stack", the matrices used in display, maybe
> the text-property-tree nodes, ...
Hmm, I'm going to gather a few more stats.
(And running Emacs under valgrind could prove helpful because I could
determine if the low-level allocator leaks memory.)
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-27 19:52 ` Stefan Monnier
2004-09-27 20:36 ` Florian Weimer
@ 2004-09-27 20:48 ` Simon Josefsson
1 sibling, 0 replies; 15+ messages in thread
From: Simon Josefsson @ 2004-09-27 20:48 UTC (permalink / raw)
Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> while they are traversed. Unfortunately, no major leak turned up,
>> just a few thousand cons cells which can't cause the massive leak I
>> see (in the order of dozens of megabytes after entering/exiting a
>> large NNTP group with Gnus).
>
> I think it would be worthwhile to start with the basic info returned by
> `garbage-collect' and such stuff. See `memory-usage.el' below.
> [ I thought you already tried that, but the few lines above make me think
> that you don't even know whether the extra MBs are made up of cons cells, or
> strings, or non-elisp-data, or ...]
Thanks, memory-usage seem useful. The memory growth caused by Gnus
seem to escape it though. This is a fairly fresh emacs (I'm using
Gnus, programming C and reading documentation, but nothing fancy):
20807 jas 15 0 365m 185m 17m S 2.7 18.3 89:12.64 emacs
But there is no indication of where the memory is used (see below).
Perhaps the "top" statistics aren't reliable, but the fact remains
that the sluggishness I feel appear to be correlated to high memory
usage according to "top".
Hm. Is it possible to use 'gdb' to find out where all memory goes?
Seems like it should have the capability to do it, at least.
(FWIW, the above emacs is not sluggish.)
Garbage collection stats:
((1130529 . 100249) (60991 . 170) (8108 . 5450) 3508516 1049895 (322 . 551) (34955 . 28764) (143989 . 25103))
=> 9846224 bytes in cons cells
1467864 bytes in symbols
271160 bytes in markers
3508516 bytes of string chars
4199580 bytes of vector slots
10476 bytes in floats
1784132 bytes in intervals
Total bytes in lisp objects (not counting string and vector headers): 17938267
Buffer ralloc memory usage:
147 buffers
5564167 bytes total (66581 in gaps)
Size Gap Name
1843610 2000 *DOC*
524443 2000 snprintf_2.2.tar.gz
467401 2000 *Messages*
460876 2000 *Gnus Backlog*
...
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-27 20:36 ` Florian Weimer
@ 2004-09-27 20:49 ` Stefan Monnier
0 siblings, 0 replies; 15+ messages in thread
From: Stefan Monnier @ 2004-09-27 20:49 UTC (permalink / raw)
Cc: emacs-devel
>> I think it would be worthwhile to start with the basic info returned by
>> `garbage-collect' and such stuff. See `memory-usage.el' below.
>> [ I thought you already tried that, but the few lines above make me think
>> that you don't even know whether the extra MBs are made up of cons cells, or
>> strings, or non-elisp-data, or ...]
> Others already did this. But thanks anyway, I repeated the
> experiment: The numbers indeed lack any distinct increase in
> magnitude. 8-(
Well, that means that the increase is apparently elsewhere, which is good
because it already tells you that hacking on the gc_marking code probably
won't help you.
Now, that doesn't rule out all of alloc.c :-(
>>> Are there any other objects that can change their size after
>>> allocation? (I'm pretty sure that there are no additional Lisp
>>> objects allocated, but maybe an existing object grows without bounds.)
>>
>> Yes. Things like the specpdl "stack", the matrices used in display, maybe
>> the text-property-tree nodes, ...
> Hmm, I'm going to gather a few more stats.
> (And running Emacs under valgrind could prove helpful because I could
> determine if the low-level allocator leaks memory.)
Another thing you might want to try is:
- check /proc/$$/maps before and after the memory growth.
- based on that, infer the memory range where the new (tho supposedly dead)
data is located.
- from GDB, peek at that memory range to see if you recognize the kind of
data that's there.
Stefan
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-27 19:40 ` Florian Weimer
2004-09-27 19:52 ` Stefan Monnier
@ 2004-09-28 15:20 ` Richard Stallman
2004-09-28 21:00 ` Florian Weimer
2004-09-28 21:40 ` Kim F. Storm
1 sibling, 2 replies; 15+ messages in thread
From: Richard Stallman @ 2004-09-28 15:20 UTC (permalink / raw)
Cc: emacs-devel
It might be interesting to keep track of malloc'ing of large blocks of
memory, recording the last N large blocks allocated in a table.
Or perhaps only those that do not hold conses, miscs, small strings, etc.
This way you might be able to find some large blocks and then
examine them to see what data is in them.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-28 15:20 ` Richard Stallman
@ 2004-09-28 21:00 ` Florian Weimer
2004-09-28 21:51 ` Florian Weimer
` (2 more replies)
2004-09-28 21:40 ` Kim F. Storm
1 sibling, 3 replies; 15+ messages in thread
From: Florian Weimer @ 2004-09-28 21:00 UTC (permalink / raw)
Cc: emacs-devel
* Richard Stallman:
> It might be interesting to keep track of malloc'ing of large blocks of
> memory, recording the last N large blocks allocated in a table.
> Or perhaps only those that do not hold conses, miscs, small strings, etc.
> This way you might be able to find some large blocks and then
> examine them to see what data is in them.
The approach which I have used with great success in the past is to
use GNU libc's malloc() hooks to print complete
malloc()/realloc()/free() traces. The crucial point is to include
information from backtrace() (because otherwise, we only see calls to
xmalloc()). A short Python (or whatever) script is then used to
discover blocks which are never freed, sums up the leakage by
backtrace, and uses addr2line to print a human-readable backtrace.
There is a tool, called valgrind, that uses dynamic recompilation of
x86 code to inject such tracing code without recompilation and
relinking. It can detect memory leaks, but it catches a wide range of
programming mistakes, too. Unfortunately, a dumped Emacs does not run
under valgrind, so I could use it. Therefore, I'm going to post my
enhanced alloc tracing patch in a separate message for future
reference.
Now to the interesting part. The worst offender is
decode-coding-region:
* 42212224
dump_hook_malloc /home/fw/src/gnu/emacs-memory-dump/src/alloc.c:5740
?? ??:0
coding_allocate_composition_data /home/fw/src/gnu/emacs-memory-dump/src/coding.c:1699
code_convert_region /home/fw/src/gnu/emacs-memory-dump/src/coding.c:5650
code_convert_region1 /home/fw/src/gnu/emacs-memory-dump/src/coding.c:6966
Fdecode_coding_region /home/fw/src/gnu/emacs-memory-dump/src/coding.c:6983
Ffuncall /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2743
Fbyte_code /home/fw/src/gnu/emacs-memory-dump/src/bytecode.c:686
funcall_lambda /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2928
Ffuncall /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2803
Fbyte_code /home/fw/src/gnu/emacs-memory-dump/src/bytecode.c:686
Feval /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2097
Fcondition_case /home/fw/src/gnu/emacs-memory-dump/src/eval.c:1293
Fbyte_code /home/fw/src/gnu/emacs-memory-dump/src/bytecode.c:864
Feval /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2097
Fcondition_case /home/fw/src/gnu/emacs-memory-dump/src/eval.c:1293
Fbyte_code /home/fw/src/gnu/emacs-memory-dump/src/bytecode.c:864
funcall_lambda /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2928
Ffuncall /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2803
It turns out that we are returning from code_convert_region() without
freeing the composition data under certain circumstances. This bug is
not new, it has been present since code_convert_region() started
allocating data using xmalloc() in late 1999. The recent
internationalization efforts just uncovered it.
The patch below is for illustration only. It fixes the memory leak,
but maybe other cleanups are missing on this path.
2004-09-28 Florian Weimer <fw@deneb.enyo.de>
* coding.c (code_convert_region): Free composition data to fix
memory leak.
--- emacs-upstream/src/coding.c 2004-09-27 21:32:39.000000000 +0200
+++ emacs-memory-dump/src/coding.c 2004-09-28 22:26:13.000000000 +0200
@@ -5667,6 +5667,7 @@
if (!replace)
/* We must record and adjust for this new text now. */
adjust_after_insert (from, from_byte_orig, to, to_byte_orig, len);
+ coding_free_composition_data (coding);
return 0;
}
(Disclaimer: I haven't got papers on file, but there is just one way
to make this change, so I doubt it is copyrightable.)
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-28 15:20 ` Richard Stallman
2004-09-28 21:00 ` Florian Weimer
@ 2004-09-28 21:40 ` Kim F. Storm
1 sibling, 0 replies; 15+ messages in thread
From: Kim F. Storm @ 2004-09-28 21:40 UTC (permalink / raw)
Cc: Florian Weimer, emacs-devel
Richard Stallman <rms@gnu.org> writes:
> It might be interesting to keep track of malloc'ing of large blocks of
> memory, recording the last N large blocks allocated in a table.
> Or perhaps only those that do not hold conses, miscs, small strings, etc.
> This way you might be able to find some large blocks and then
> examine them to see what data is in them.
Other things that could be potential leaks are:
- realized faces / fonts
- cached images
- buffer text
--
Kim F. Storm <storm@cua.dk> http://www.cua.dk
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-28 21:00 ` Florian Weimer
@ 2004-09-28 21:51 ` Florian Weimer
2004-09-29 16:39 ` Richard Stallman
2004-09-29 23:51 ` Kenichi Handa
2 siblings, 0 replies; 15+ messages in thread
From: Florian Weimer @ 2004-09-28 21:51 UTC (permalink / raw)
Here's the updated patch for future reference.
--- emacs-upstream/src/find-leak.py 1970-01-01 01:00:00.000000000 +0100
+++ emacs-memory-dump/src/find-leak.py 2004-09-28 23:35:37.000000000 +0200
@@ -0,0 +1,108 @@
+# A patched Emacs writes malloc() traces to standard error after user
+# has invoked enable-malloc-tracing. This script processes the output
+# and produces some statistics. It finds non-freed memory blocks and
+# groups them by the backtrace at the point of allocation. After
+# that, the leaking backtraces are sorted by decreasing leakage, and
+# the addresses are resolved using addr2line.
+#
+# You should pass this script the name of the file to which Emacs
+# wrote the malloc trace on the command line. The file might contain
+# some garbled lines with multiple "@" signs (possibly due to some
+# obscure reentrance problems that have not yet been solved). Delete
+# them if this script runs into them.
+#
+# It is a good idea to repeat the action that triggers the memory leak
+# multiple times (so that memory-limit grows by a substantial value),
+# then invoke garbage-collect (to free as much memory as possible),
+# and exit from Emacs. Some Lisp objects will not be freed (and show
+# up in the leakage statistics), but the real memory leak will be
+# substantially larger and easy to spot in the sorted output.
+#
+# Note that if your system's libc library has been compiled with frame
+# pointer omission, the output won't contain usable information. You
+# must use the debugging version of libc. (On Debian systems, it's
+# available in the libc6-dbg package, and you have to set the
+# LD_LIBRARY_PATH environment variable to "/usr/lib/debug".)
+#
+# Note that this approach won't be very helpful if the leak is caused
+# by unused, but still-referenced Lisp data. Use garbage-collect-dump
+# (in the patched Emacs) for that and hope for the best. Its output
+# is more structured and might reveal which Lisp variable is the
+# culprit.
+
+import sys, os
+
+def convert_allocators(allocs):
+ l = []
+ for a in allocs:
+ if a <> '(nil)':
+ l.append(eval(a))
+ else:
+ l.append(0)
+ return tuple(l)
+
+def unfreed_objects(file):
+ objects = {}
+
+ for line in file.readlines():
+ fields = line.split(' ')
+ if fields[0] <> '@':
+ continue
+
+ if fields[1] == 'A':
+ address = eval(fields[2])
+ size = int(fields[3])
+ if fields[4] <> ':':
+ raise ValueError
+ objects[address] = (size, convert_allocators(fields[5:]))
+
+ elif fields[1] == 'R':
+ address = fields[2]
+ if address == '(nil)':
+ address = 0
+ else:
+ address = eval(address)
+
+ new_address = eval(fields[3])
+ size = int(fields[4])
+ if fields[5] <> ':':
+ raise ValueError
+ if objects.has_key(address):
+ del objects[address]
+ objects[new_address] = (size, convert_allocators(fields[6:]))
+
+ elif fields[1] == 'F':
+ address = eval(fields[2])
+ if objects.has_key(address):
+ del objects[address]
+
+ return objects
+
+def leaking_paths(objects):
+ paths = {}
+ for (size, path) in objects.values():
+ paths[path] = paths.get(path, 0) + size
+ return paths
+
+objects = unfreed_objects(open(sys.argv[1]))
+paths = leaking_paths(objects)
+
+resolve_cache = {}
+def resolve(addr):
+ if resolve_cache.has_key(addr):
+ return resolve_cache[addr]
+ address = hex(p)
+ lines = os.popen("addr2line -e ./emacs -f " + address, "r").readlines()
+ result = tuple("".join(lines).split('\n'))
+ resolve_cache[addr] = result
+ return result
+
+l = map(lambda (path, size): (-size, path), paths.items())
+l.sort()
+for (size, path) in l:
+ size = -size
+ print '*', size
+ for p in path:
+ print " " + " ".join(resolve(p))
+
+# arch-tag: 38cf01b5-1b4d-405e-b324-0f79a146826f
--- emacs-upstream/src/lisp.h 2004-09-27 21:32:39.000000000 +0200
+++ emacs-memory-dump/src/lisp.h 2004-09-28 20:49:45.000000000 +0200
@@ -2440,9 +2440,12 @@
extern void memory_full P_ ((void));
extern void buffer_memory_full P_ ((void));
extern int survives_gc_p P_ ((Lisp_Object));
-extern void mark_object P_ ((Lisp_Object));
+extern void mark_object P_ ((Lisp_Object, unsigned));
+extern void mark_kboards P_ ((unsigned));
+extern void mark_backtrace P_ ((void));
extern Lisp_Object Vpurify_flag;
extern Lisp_Object Vmemory_full;
+extern int message_enable_multibyte;
EXFUN (Fcons, 2);
EXFUN (list2, 2);
EXFUN (list3, 3);
--- emacs-upstream/src/alloc.c 2004-09-27 21:33:09.000000000 +0200
+++ emacs-memory-dump/src/alloc.c 2004-09-28 21:10:38.000000000 +0200
@@ -258,12 +258,15 @@
Lisp_Object Vgc_elapsed; /* accumulated elapsed time in GC */
EMACS_INT gcs_done; /* accumulated GCs */
-static void mark_buffer P_ ((Lisp_Object));
-extern void mark_kboards P_ ((void));
-extern void mark_backtrace P_ ((void));
+/* If non-zero, dump objects to stderr while they are marked. */
+static int do_dump = 0;
+
+static void dump_marker_section (char *name);
+
+static void mark_buffer P_ ((Lisp_Object, unsigned));
static void gc_sweep P_ ((void));
-static void mark_glyph_matrix P_ ((struct glyph_matrix *));
-static void mark_face_cache P_ ((struct face_cache *));
+static void mark_glyph_matrix P_ ((struct glyph_matrix *, unsigned));
+static void mark_face_cache P_ ((struct face_cache *, unsigned));
#ifdef HAVE_WINDOW_SYSTEM
static void mark_image P_ ((struct image *));
@@ -275,8 +278,6 @@
static void free_large_strings P_ ((void));
static void sweep_strings P_ ((void));
-extern int message_enable_multibyte;
-
/* When scanning the C stack for live Lisp objects, Emacs keeps track
of what memory allocated via lisp_malloc is intended for what
purpose. This enumeration specifies the type of memory. */
@@ -389,8 +390,8 @@
static int live_symbol_p P_ ((struct mem_node *, void *));
static int live_float_p P_ ((struct mem_node *, void *));
static int live_misc_p P_ ((struct mem_node *, void *));
-static void mark_maybe_object P_ ((Lisp_Object));
-static void mark_memory P_ ((void *, void *));
+static void mark_maybe_object P_ ((Lisp_Object, unsigned));
+static void mark_memory P_ ((void *, void *, unsigned));
static void mem_init P_ ((void));
static struct mem_node *mem_insert P_ ((void *, void *, enum mem_type));
static void mem_insert_fixup P_ ((struct mem_node *));
@@ -1217,7 +1218,7 @@
{
eassert (!i->gcmarkbit); /* Intervals are never shared. */
i->gcmarkbit = 1;
- mark_object (i->plist);
+ mark_object (i->plist, 0);
}
@@ -3688,8 +3689,9 @@
/* Mark OBJ if we can prove it's a Lisp_Object. */
static INLINE void
-mark_maybe_object (obj)
+mark_maybe_object (obj, depth)
Lisp_Object obj;
+ unsigned depth;
{
void *po = (void *) XPNTR (obj);
struct mem_node *m = mem_find (po);
@@ -3743,7 +3745,7 @@
zombies[nzombies] = obj;
++nzombies;
#endif
- mark_object (obj);
+ mark_object (obj, depth + 1);
}
}
}
@@ -3753,8 +3755,9 @@
marked. */
static INLINE void
-mark_maybe_pointer (p)
+mark_maybe_pointer (p, depth)
void *p;
+ unsigned depth;
{
struct mem_node *m;
@@ -3824,7 +3827,7 @@
}
if (!GC_NILP (obj))
- mark_object (obj);
+ mark_object (obj, depth + 1);
}
}
@@ -3832,8 +3835,9 @@
/* Mark Lisp objects referenced from the address range START..END. */
static void
-mark_memory (start, end)
+mark_memory (start, end, depth)
void *start, *end;
+ unsigned depth;
{
Lisp_Object *p;
void **pp;
@@ -3853,7 +3857,7 @@
/* Mark Lisp_Objects. */
for (p = (Lisp_Object *) start; (void *) p < end; ++p)
- mark_maybe_object (*p);
+ mark_maybe_object (*p, depth + 1);
/* Mark Lisp data pointed to. This is necessary because, in some
situations, the C compiler optimizes Lisp objects away, so that
@@ -4098,7 +4102,7 @@
#endif
#endif
for (i = 0; i < sizeof (Lisp_Object); i += GC_LISP_OBJECT_ALIGNMENT)
- mark_memory ((char *) stack_base + i, end);
+ mark_memory ((char *) stack_base + i, end, 1);
#if GC_MARK_STACK == GC_MARK_STACK_CHECK_GCPROS
check_gcpros ();
@@ -4455,14 +4459,14 @@
/* Mark all the special slots that serve as the roots of accessibility. */
for (i = 0; i < staticidx; i++)
- mark_object (*staticvec[i]);
+ mark_object (*staticvec[i], 1);
for (bind = specpdl; bind != specpdl_ptr; bind++)
{
- mark_object (bind->symbol);
- mark_object (bind->old_value);
+ mark_object (bind->symbol, 1);
+ mark_object (bind->old_value, 1);
}
- mark_kboards ();
+ mark_kboards (1);
#ifdef USE_GTK
{
@@ -4483,18 +4487,22 @@
}
#endif
+ dump_marker_section ("Begin marking byte stack.");
mark_byte_stack ();
+ dump_marker_section ("Begin marking byte stack.");
for (catch = catchlist; catch; catch = catch->next)
{
- mark_object (catch->tag);
- mark_object (catch->val);
+ mark_object (catch->tag, 1);
+ mark_object (catch->val, 1);
}
for (handler = handlerlist; handler; handler = handler->next)
{
- mark_object (handler->handler);
- mark_object (handler->var);
+ mark_object (handler->handler, 1);
+ mark_object (handler->var, 1);
}
+ dump_marker_section ("Begin marking backtrace.");
mark_backtrace ();
+ dump_marker_section ("End marking backtrace.");
#if GC_MARK_STACK == GC_USE_GCPROS_CHECK_ZOMBIES
mark_stack ();
@@ -4542,7 +4550,7 @@
}
/* Now that we have stripped the elements that need not be in the
undo_list any more, we can finally mark the list. */
- mark_object (nextb->undo_list);
+ mark_object (nextb->undo_list, 1);
nextb = nextb->next;
}
@@ -4630,13 +4638,24 @@
return Flist (sizeof total / sizeof *total, total);
}
+DEFUN ("garbage-collect-dump", Fgarbage_collect_dump, Sgarbage_collect_dump, 0, 0, "",
+ doc: /* Run garbage collection and dump objects. */)
+ ()
+{
+ ++do_dump;
+ dump_marker_section ("Begin garbage collection.");
+ Fgarbage_collect ();
+ dump_marker_section ("End garbage collection.");
+ --do_dump;
+}
/* Mark Lisp objects in glyph matrix MATRIX. Currently the
only interesting objects referenced from glyphs are strings. */
static void
-mark_glyph_matrix (matrix)
+mark_glyph_matrix (matrix, depth)
struct glyph_matrix *matrix;
+ unsigned depth;
{
struct glyph_row *row = matrix->rows;
struct glyph_row *end = row + matrix->nrows;
@@ -4653,7 +4672,7 @@
for (; glyph < end_glyph; ++glyph)
if (GC_STRINGP (glyph->object)
&& !STRING_MARKED_P (XSTRING (glyph->object)))
- mark_object (glyph->object);
+ mark_object (glyph->object, depth + 1);
}
}
}
@@ -4662,8 +4681,9 @@
/* Mark Lisp faces in the face cache C. */
static void
-mark_face_cache (c)
+mark_face_cache (c, depth)
struct face_cache *c;
+ unsigned depth;
{
if (c)
{
@@ -4675,7 +4695,7 @@
if (face)
{
for (j = 0; j < LFACE_VECTOR_SIZE; ++j)
- mark_object (face->lface[j]);
+ mark_object (face->lface[j], depth + 1);
}
}
}
@@ -4690,10 +4710,10 @@
mark_image (img)
struct image *img;
{
- mark_object (img->spec);
+ mark_object (img->spec, 1);
if (!NILP (img->data.lisp_val))
- mark_object (img->data.lisp_val);
+ mark_object (img->data.lisp_val, 1);
}
@@ -4725,9 +4745,84 @@
Normally this is zero and the check never goes off. */
int mark_object_loop_halt;
+/* For memory debugging: dumps the string in human-readable form to
+ stderr. */
+static void
+dump_string (char *data, unsigned length)
+{
+ unsigned i;
+
+ for (i = 0; i < length; ++i)
+ {
+ char c = data[i];
+ if (c >= 32 && c <= 126)
+ fputc (c, stderr);
+ else if (c == '"' || c == '\\')
+ {
+ fputc ('\\', stderr);
+ fputc (c, stderr);
+ }
+ else if (c == '\n')
+ fputs ("\\n", stderr);
+ else if (c == '\t')
+ fputs ("\\t", stderr);
+ else if (c == '\r')
+ fputs ("\\r", stderr);
+ else
+ fprintf (stderr, "\\%03o", (unsigned)(unsigned char) c);
+ }
+}
+
+/* For memory debugging: prints OBJ of TYPE, at nesting level
+ DEPTH to stderr. */
+static void
+dump_object (char *type, unsigned depth, Lisp_Object obj, int marked)
+{
+ unsigned i;
+
+ if (!do_dump)
+ return;
+
+ fprintf (stderr, "[%u] %s (%p)%s\n",
+ depth, type, (void *)obj, marked ? " *" : "");
+}
+
+static void
+dump_object_int (unsigned depth, Lisp_Object obj)
+{
+ if (!do_dump)
+ return;
+
+ fprintf (stderr, "[%u] INT %d\n", depth, XINT(obj));
+}
+
+static void
+dump_marker_section(char *name)
+{
+ if (!do_dump)
+ return;
+
+ fprintf (stderr, "%s\n", name);
+}
+
+/* For memory debugging: prints OBJ of TYPE, with NAME, at nesting
+ level DEPTH to stderr. */
+static void
+dump_object_name (char *type, unsigned depth, Lisp_Object obj, struct Lisp_String* name, int marked)
+{
+ if (!do_dump)
+ return;
+
+ fprintf (stderr, "[%u] %s (%p) \"", depth, type, (void *)obj);
+ dump_string (name->data, name->size & ~ARRAY_MARK_FLAG);
+ fprintf (stderr, "\"%s\n", marked ? " *" : "");
+}
+
+
void
-mark_object (arg)
+mark_object (arg, depth)
Lisp_Object arg;
+ unsigned depth;
{
register Lisp_Object obj = arg;
#ifdef GC_CHECK_MARKED_OBJECTS
@@ -4792,6 +4887,7 @@
CHECK_ALLOCATED_AND_LIVE (live_string_p);
MARK_INTERVAL_TREE (ptr->intervals);
MARK_STRING (ptr);
+ dump_object_name ("STRING", depth, obj, ptr, ptr->size & ARRAY_MARK_FLAG);
#ifdef GC_CHECK_STRING_BYTES
/* Check that the string size recorded in the string is the
same as the one recorded in the sdata structure. */
@@ -4811,6 +4907,8 @@
if (GC_BUFFERP (obj))
{
+ dump_object ("BUFFER", depth, obj, VECTOR_MARKED_P (XBUFFER (obj)));
+
if (!VECTOR_MARKED_P (XBUFFER (obj)))
{
#ifdef GC_CHECK_MARKED_OBJECTS
@@ -4823,7 +4921,7 @@
abort ();
}
#endif /* GC_CHECK_MARKED_OBJECTS */
- mark_buffer (obj);
+ mark_buffer (obj, depth + 1);
}
}
else if (GC_SUBRP (obj))
@@ -4837,6 +4935,8 @@
register EMACS_INT size = ptr->size;
register int i;
+ dump_object ("COMPILED", depth, obj, VECTOR_MARKED_P (ptr));
+
if (VECTOR_MARKED_P (ptr))
break; /* Already marked */
@@ -4846,7 +4946,7 @@
for (i = 0; i < size; i++) /* and then mark its elements */
{
if (i != COMPILED_CONSTANTS)
- mark_object (ptr->contents[i]);
+ mark_object (ptr->contents[i], depth + 1);
}
obj = ptr->contents[COMPILED_CONSTANTS];
goto loop;
@@ -4855,40 +4955,48 @@
{
register struct frame *ptr = XFRAME (obj);
+ dump_object ("FRAME", depth, obj, VECTOR_MARKED_P (ptr));
+
if (VECTOR_MARKED_P (ptr)) break; /* Already marked */
+
VECTOR_MARK (ptr); /* Else mark it */
CHECK_LIVE (live_vector_p);
- mark_object (ptr->name);
- mark_object (ptr->icon_name);
- mark_object (ptr->title);
- mark_object (ptr->focus_frame);
- mark_object (ptr->selected_window);
- mark_object (ptr->minibuffer_window);
- mark_object (ptr->param_alist);
- mark_object (ptr->scroll_bars);
- mark_object (ptr->condemned_scroll_bars);
- mark_object (ptr->menu_bar_items);
- mark_object (ptr->face_alist);
- mark_object (ptr->menu_bar_vector);
- mark_object (ptr->buffer_predicate);
- mark_object (ptr->buffer_list);
- mark_object (ptr->menu_bar_window);
- mark_object (ptr->tool_bar_window);
- mark_face_cache (ptr->face_cache);
+ mark_object (ptr->name, depth + 1);
+ mark_object (ptr->icon_name, depth + 1);
+ mark_object (ptr->title, depth + 1);
+ mark_object (ptr->focus_frame, depth + 1);
+ mark_object (ptr->selected_window, depth + 1);
+ mark_object (ptr->minibuffer_window, depth + 1);
+ mark_object (ptr->param_alist, depth + 1);
+ mark_object (ptr->scroll_bars, depth + 1);
+ mark_object (ptr->condemned_scroll_bars, depth + 1);
+ mark_object (ptr->menu_bar_items, depth + 1);
+ mark_object (ptr->face_alist, depth + 1);
+ mark_object (ptr->menu_bar_vector, depth + 1);
+ mark_object (ptr->buffer_predicate, depth + 1);
+ mark_object (ptr->buffer_list, depth + 1);
+ mark_object (ptr->menu_bar_window, depth + 1);
+ mark_object (ptr->tool_bar_window, depth + 1);
+ mark_face_cache (ptr->face_cache, depth + 1);
#ifdef HAVE_WINDOW_SYSTEM
+ dump_marker_section ("Begin marking FRAME images.");
mark_image_cache (ptr);
- mark_object (ptr->tool_bar_items);
- mark_object (ptr->desired_tool_bar_string);
- mark_object (ptr->current_tool_bar_string);
+ dump_marker_section ("End marking FRAME images.");
+ mark_object (ptr->tool_bar_items, depth + 1);
+ mark_object (ptr->desired_tool_bar_string, depth + 1);
+ mark_object (ptr->current_tool_bar_string, depth + 1);
#endif /* HAVE_WINDOW_SYSTEM */
}
else if (GC_BOOL_VECTOR_P (obj))
{
register struct Lisp_Vector *ptr = XVECTOR (obj);
+ dump_object ("BOOL_VECTOR", depth, obj, VECTOR_MARKED_P (ptr));
+
if (VECTOR_MARKED_P (ptr))
break; /* Already marked */
+
CHECK_LIVE (live_vector_p);
VECTOR_MARK (ptr); /* Else mark it */
}
@@ -4898,6 +5006,8 @@
struct window *w = XWINDOW (obj);
register int i;
+ dump_object ("WINDOW", depth, obj, VECTOR_MARKED_P (ptr));
+
/* Stop if already marked. */
if (VECTOR_MARKED_P (ptr))
break;
@@ -4911,7 +5021,7 @@
for (i = 0;
(char *) &ptr->contents[i] < (char *) &w->current_matrix;
i++)
- mark_object (ptr->contents[i]);
+ mark_object (ptr->contents[i], depth + 1);
/* Mark glyphs for leaf windows. Marking window matrices is
sufficient because frame matrices use the same glyph
@@ -4920,14 +5030,16 @@
&& NILP (w->vchild)
&& w->current_matrix)
{
- mark_glyph_matrix (w->current_matrix);
- mark_glyph_matrix (w->desired_matrix);
+ mark_glyph_matrix (w->current_matrix, depth + 1);
+ mark_glyph_matrix (w->desired_matrix, depth + 1);
}
}
else if (GC_HASH_TABLE_P (obj))
{
struct Lisp_Hash_Table *h = XHASH_TABLE (obj);
+ dump_object ("HASH_TABLE", depth, obj, VECTOR_MARKED_P (h));
+
/* Stop if already marked. */
if (VECTOR_MARKED_P (h))
break;
@@ -4941,20 +5053,20 @@
Being in the next_weak chain
should not keep the hash table alive.
No need to mark `count' since it is an integer. */
- mark_object (h->test);
- mark_object (h->weak);
- mark_object (h->rehash_size);
- mark_object (h->rehash_threshold);
- mark_object (h->hash);
- mark_object (h->next);
- mark_object (h->index);
- mark_object (h->user_hash_function);
- mark_object (h->user_cmp_function);
+ mark_object (h->test, depth + 1);
+ mark_object (h->weak, depth + 1);
+ mark_object (h->rehash_size, depth + 1);
+ mark_object (h->rehash_threshold, depth + 1);
+ mark_object (h->hash, depth + 1);
+ mark_object (h->next, depth + 1);
+ mark_object (h->index, depth + 1);
+ mark_object (h->user_hash_function, depth + 1);
+ mark_object (h->user_cmp_function, depth + 1);
/* If hash table is not weak, mark all keys and values.
For weak tables, mark only the vector. */
if (GC_NILP (h->weak))
- mark_object (h->key_and_value);
+ mark_object (h->key_and_value, depth + 1);
else
VECTOR_MARK (XVECTOR (h->key_and_value));
}
@@ -4964,6 +5076,8 @@
register EMACS_INT size = ptr->size;
register int i;
+ dump_object ("VECTOR", depth, obj, VECTOR_MARKED_P (ptr));
+
if (VECTOR_MARKED_P (ptr)) break; /* Already marked */
CHECK_LIVE (live_vector_p);
VECTOR_MARK (ptr); /* Else mark it */
@@ -4971,7 +5085,7 @@
size &= PSEUDOVECTOR_SIZE_MASK;
for (i = 0; i < size; i++) /* and then mark its elements */
- mark_object (ptr->contents[i]);
+ mark_object (ptr->contents[i], depth + 1);
}
break;
@@ -4980,12 +5094,15 @@
register struct Lisp_Symbol *ptr = XSYMBOL (obj);
struct Lisp_Symbol *ptrx;
+ dump_object_name ("SYMBOL", depth, obj, XSTRING (ptr->xname), ptr->gcmarkbit);
+
if (ptr->gcmarkbit) break;
+
CHECK_ALLOCATED_AND_LIVE (live_symbol_p);
ptr->gcmarkbit = 1;
- mark_object (ptr->value);
- mark_object (ptr->function);
- mark_object (ptr->plist);
+ mark_object (ptr->value, depth + 1);
+ mark_object (ptr->function, depth + 1);
+ mark_object (ptr->plist, depth + 1);
if (!PURE_POINTER_P (XSTRING (ptr->xname)))
MARK_STRING (XSTRING (ptr->xname));
@@ -5006,6 +5123,7 @@
case Lisp_Misc:
CHECK_ALLOCATED_AND_LIVE (live_misc_p);
+ dump_object ("MISC", depth, obj, XMARKER (obj)->gcmarkbit);
if (XMARKER (obj)->gcmarkbit)
break;
XMARKER (obj)->gcmarkbit = 1;
@@ -5023,9 +5141,9 @@
obj = ptr->realvalue;
goto loop;
}
- mark_object (ptr->realvalue);
- mark_object (ptr->buffer);
- mark_object (ptr->frame);
+ mark_object (ptr->realvalue, depth + 1);
+ mark_object (ptr->buffer, depth + 1);
+ mark_object (ptr->frame, depth + 1);
obj = ptr->cdr;
goto loop;
}
@@ -5058,18 +5176,17 @@
Lisp_Object *p = (Lisp_Object *) ptr->pointer;
int nelt;
for (nelt = ptr->integer; nelt > 0; nelt--, p++)
- mark_maybe_object (*p);
+ mark_maybe_object (*p, depth + 1);
}
}
#endif
break;
-
case Lisp_Misc_Overlay:
{
struct Lisp_Overlay *ptr = XOVERLAY (obj);
- mark_object (ptr->start);
- mark_object (ptr->end);
- mark_object (ptr->plist);
+ mark_object (ptr->start, depth + 1);
+ mark_object (ptr->end, depth + 1);
+ mark_object (ptr->plist, depth + 1);
if (ptr->next)
{
XSETMISC (obj, ptr->next);
@@ -5086,30 +5203,23 @@
case Lisp_Cons:
{
register struct Lisp_Cons *ptr = XCONS (obj);
+ dump_object ("CONS", depth, obj, CONS_MARKED_P (ptr));
if (CONS_MARKED_P (ptr)) break;
CHECK_ALLOCATED_AND_LIVE (live_cons_p);
CONS_MARK (ptr);
- /* If the cdr is nil, avoid recursion for the car. */
- if (EQ (ptr->cdr, Qnil))
- {
- obj = ptr->car;
- cdr_count = 0;
- goto loop;
- }
- mark_object (ptr->car);
- obj = ptr->cdr;
- cdr_count++;
- if (cdr_count == mark_object_loop_halt)
- abort ();
- goto loop;
+ mark_object (ptr->car, depth + 1);
+ mark_object (ptr->cdr, depth + 1);
+ break;
}
case Lisp_Float:
CHECK_ALLOCATED_AND_LIVE (live_float_p);
+ dump_object ("CONS", depth, obj, FLOAT_MARKED_P (XFLOAT (obj)));
FLOAT_MARK (XFLOAT (obj));
break;
case Lisp_Int:
+ dump_object_int (depth, obj);
break;
default:
@@ -5124,8 +5234,9 @@
/* Mark the pointers in a buffer structure. */
static void
-mark_buffer (buf)
+mark_buffer (buf, depth)
Lisp_Object buf;
+ unsigned depth;
{
register struct buffer *buffer = XBUFFER (buf);
register Lisp_Object *ptr, tmp;
@@ -5142,24 +5253,24 @@
if (buffer->overlays_before)
{
XSETMISC (tmp, buffer->overlays_before);
- mark_object (tmp);
+ mark_object (tmp, depth + 1);
}
if (buffer->overlays_after)
{
XSETMISC (tmp, buffer->overlays_after);
- mark_object (tmp);
+ mark_object (tmp, depth + 1);
}
for (ptr = &buffer->name;
(char *)ptr < (char *)buffer + sizeof (struct buffer);
ptr++)
- mark_object (*ptr);
+ mark_object (*ptr, depth + 1);
/* If this is an indirect buffer, mark its base buffer. */
if (buffer->base_buffer && !VECTOR_MARKED_P (buffer->base_buffer))
{
XSETBUFFER (base_buffer, buffer->base_buffer);
- mark_buffer (base_buffer);
+ mark_buffer (base_buffer, depth + 1);
}
}
@@ -5599,6 +5710,71 @@
return Flist (8, consed);
}
+
+static void *(*dump_hook_old_malloc) (size_t, const void *);
+static void *(*dump_hook_old_realloc) (void *, size_t, const void *);
+static void (*dump_hook_old_free) (void *, const void *);
+
+static void
+dump_malloc_backtrace (void)
+{
+ void *trace[20];
+ unsigned depth, i;
+
+ depth = backtrace (trace, 20);
+ for (i = 1; i < depth; ++i)
+ {
+ fprintf (stderr, " %p", trace[i]);
+ }
+ fputs ("\n", stderr);
+}
+
+static void *
+dump_hook_malloc (size_t size, const void *caller)
+{
+ void *result;
+ __malloc_hook = dump_hook_old_malloc;
+ result = malloc(size);
+ fprintf (stderr, "@ A %p %u :", result, size);
+ dump_malloc_backtrace ();
+ __malloc_hook = &dump_hook_malloc;
+ return result;
+}
+
+static void *
+dump_hook_realloc (void *ptr, size_t size, const void *caller)
+{
+ void *result;
+ __realloc_hook = dump_hook_old_realloc;
+ result = realloc(ptr, size);
+ fprintf (stderr, "@ R %p %p %u :", ptr, result, size);
+ dump_malloc_backtrace ();
+ __realloc_hook = &dump_hook_realloc;
+ return result;
+}
+
+static void
+dump_hook_free (void *ptr, const void *caller)
+{
+ __free_hook = dump_hook_old_free;
+ free(ptr);
+ fprintf (stderr, "@ F %p :", ptr);
+ dump_malloc_backtrace ();
+ __free_hook = &dump_hook_free;
+}
+
+DEFUN ("enable-malloc-tracing", Fenable_malloc_tracing, Senable_malloc_tracing, 0, 0, "",
+ doc: /* Enable tracing of memory allocations. */)
+ ()
+{
+ dump_hook_old_malloc = __malloc_hook;
+ dump_hook_old_realloc = __realloc_hook;
+ dump_hook_old_free = __free_hook;
+ __malloc_hook = &dump_hook_malloc;
+ __realloc_hook = &dump_hook_realloc;
+ __free_hook = &dump_hook_free;
+}
+
int suppress_checking;
void
die (msg, file, line)
@@ -5792,8 +5968,10 @@
defsubr (&Smake_marker);
defsubr (&Spurecopy);
defsubr (&Sgarbage_collect);
+ defsubr (&Sgarbage_collect_dump);
defsubr (&Smemory_limit);
defsubr (&Smemory_use_counts);
+ defsubr (&Senable_malloc_tracing);
#if GC_MARK_STACK == GC_USE_GCPROS_CHECK_ZOMBIES
defsubr (&Sgc_status);
--- emacs-upstream/src/bytecode.c 2004-09-27 21:33:01.000000000 +0200
+++ emacs-memory-dump/src/bytecode.c 2004-09-25 19:16:07.000000000 +0200
@@ -289,10 +289,10 @@
eassert (stack->top);
for (obj = stack->bottom; obj <= stack->top; ++obj)
- mark_object (*obj);
+ mark_object (*obj, 1);
- mark_object (stack->byte_string);
- mark_object (stack->constants);
+ mark_object (stack->byte_string, 1);
+ mark_object (stack->constants, 1);
}
}
--- emacs-upstream/src/eval.c 2004-09-27 21:32:39.000000000 +0200
+++ emacs-memory-dump/src/eval.c 2004-09-25 20:23:47.000000000 +0200
@@ -3260,14 +3260,14 @@
for (backlist = backtrace_list; backlist; backlist = backlist->next)
{
- mark_object (*backlist->function);
+ mark_object (*backlist->function, 1);
if (backlist->nargs == UNEVALLED || backlist->nargs == MANY)
i = 0;
else
i = backlist->nargs - 1;
for (; i >= 0; i--)
- mark_object (backlist->args[i]);
+ mark_object (backlist->args[i], 1);
}
}
--- emacs-upstream/src/fns.c 2004-09-27 21:32:39.000000000 +0200
+++ emacs-memory-dump/src/fns.c 2004-09-25 19:15:38.000000000 +0200
@@ -4804,13 +4804,13 @@
/* Make sure key and value survive. */
if (!key_known_to_survive_p)
{
- mark_object (HASH_KEY (h, i));
+ mark_object (HASH_KEY (h, i), 1);
marked = 1;
}
if (!value_known_to_survive_p)
{
- mark_object (HASH_VALUE (h, i));
+ mark_object (HASH_VALUE (h, i), 1);
marked = 1;
}
}
--- emacs-upstream/src/keyboard.c 2004-09-27 21:32:39.000000000 +0200
+++ emacs-memory-dump/src/keyboard.c 2004-09-25 17:36:06.000000000 +0200
@@ -11425,7 +11425,8 @@
/* Mark the pointers in the kboard objects.
Called by the Fgarbage_collector. */
void
-mark_kboards ()
+mark_kboards (depth)
+ unsigned depth;
{
KBOARD *kb;
Lisp_Object *p;
@@ -11433,19 +11434,19 @@
{
if (kb->kbd_macro_buffer)
for (p = kb->kbd_macro_buffer; p < kb->kbd_macro_ptr; p++)
- mark_object (*p);
- mark_object (kb->Voverriding_terminal_local_map);
- mark_object (kb->Vlast_command);
- mark_object (kb->Vreal_last_command);
- mark_object (kb->Vprefix_arg);
- mark_object (kb->Vlast_prefix_arg);
- mark_object (kb->kbd_queue);
- mark_object (kb->defining_kbd_macro);
- mark_object (kb->Vlast_kbd_macro);
- mark_object (kb->Vsystem_key_alist);
- mark_object (kb->system_key_syms);
- mark_object (kb->Vdefault_minibuffer_frame);
- mark_object (kb->echo_string);
+ mark_object (*p, depth + 1);
+ mark_object (kb->Voverriding_terminal_local_map, depth + 1);
+ mark_object (kb->Vlast_command, depth + 1);
+ mark_object (kb->Vreal_last_command, depth + 1);
+ mark_object (kb->Vprefix_arg, depth + 1);
+ mark_object (kb->Vlast_prefix_arg, depth + 1);
+ mark_object (kb->kbd_queue, depth + 1);
+ mark_object (kb->defining_kbd_macro, depth + 1);
+ mark_object (kb->Vlast_kbd_macro, depth + 1);
+ mark_object (kb->Vsystem_key_alist, depth + 1);
+ mark_object (kb->system_key_syms, depth + 1);
+ mark_object (kb->Vdefault_minibuffer_frame, depth + 1);
+ mark_object (kb->echo_string, depth + 1);
}
{
struct input_event *event;
@@ -11455,11 +11456,11 @@
event = kbd_buffer;
if (event->kind != SELECTION_REQUEST_EVENT)
{
- mark_object (event->x);
- mark_object (event->y);
+ mark_object (event->x, depth + 1);
+ mark_object (event->y, depth + 1);
}
- mark_object (event->frame_or_window);
- mark_object (event->arg);
+ mark_object (event->frame_or_window, depth + 1);
+ mark_object (event->arg, depth + 1);
}
}
}
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-28 21:00 ` Florian Weimer
2004-09-28 21:51 ` Florian Weimer
@ 2004-09-29 16:39 ` Richard Stallman
2004-09-29 23:51 ` Kenichi Handa
2 siblings, 0 replies; 15+ messages in thread
From: Richard Stallman @ 2004-09-29 16:39 UTC (permalink / raw)
Cc: emacs-devel
Thank you very much for tracking down a cause of a major
memory leak. We have had a lot of trouble with that in Emacs,
and other people have so far not had much success in debugging them.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Debugging memory leaks/stale references
2004-09-28 21:00 ` Florian Weimer
2004-09-28 21:51 ` Florian Weimer
2004-09-29 16:39 ` Richard Stallman
@ 2004-09-29 23:51 ` Kenichi Handa
2 siblings, 0 replies; 15+ messages in thread
From: Kenichi Handa @ 2004-09-29 23:51 UTC (permalink / raw)
Cc: rms, emacs-devel
In article <87y8iurtvw.fsf@deneb.enyo.de>, Florian Weimer <fw@deneb.enyo.de> writes:
> It turns out that we are returning from code_convert_region() without
> freeing the composition data under certain circumstances. This bug is
> not new, it has been present since code_convert_region() started
> allocating data using xmalloc() in late 1999. The recent
> internationalization efforts just uncovered it.
Thank you very much for finding this bug. I've just
installed your fix. I also checked all calls of
coding_save_composition and
coding_allocate_composition_data, and installed similer
fixes.
---
Ken'ichi HANDA
handa@m17n.org
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2004-09-29 23:51 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2004-09-21 17:38 Debugging memory leaks/stale references Florian Weimer
2004-09-21 19:49 ` Simon Josefsson
2004-09-27 19:40 ` Florian Weimer
2004-09-27 19:52 ` Stefan Monnier
2004-09-27 20:36 ` Florian Weimer
2004-09-27 20:49 ` Stefan Monnier
2004-09-27 20:48 ` Simon Josefsson
2004-09-28 15:20 ` Richard Stallman
2004-09-28 21:00 ` Florian Weimer
2004-09-28 21:51 ` Florian Weimer
2004-09-29 16:39 ` Richard Stallman
2004-09-29 23:51 ` Kenichi Handa
2004-09-28 21:40 ` Kim F. Storm
2004-09-21 19:57 ` Stefan Monnier
2004-09-21 21:01 ` Florian Weimer
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).