all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* Any way of dumping strings?
@ 2021-06-02 17:03 Óscar Fuentes
  2021-06-02 17:24 ` Eli Zaretskii
  2021-06-02 18:38 ` Stefan Monnier
  0 siblings, 2 replies; 10+ messages in thread
From: Óscar Fuentes @ 2021-06-02 17:03 UTC (permalink / raw)
  To: emacs-devel

I'm trying to diagnose a memory leak. As it is used Emacs requires more
memory, an average of a several tens of MB per hour. Right now it is
using ~1GB and `memory-report' shows:

 207.8 MiB  Vectors
  83.7 MiB  Strings

I terminated the previous session after reaching ~2GB with >700MB
vectors and >300MB strings.

In case the culprit is some package, looking at those strings or vectors
could be revealing.

Is there a method for dumping or displaying those strings or vectors?




^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Any way of dumping strings?
  2021-06-02 17:03 Any way of dumping strings? Óscar Fuentes
@ 2021-06-02 17:24 ` Eli Zaretskii
  2021-06-02 17:54   ` Óscar Fuentes
  2021-06-02 18:38 ` Stefan Monnier
  1 sibling, 1 reply; 10+ messages in thread
From: Eli Zaretskii @ 2021-06-02 17:24 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 02 Jun 2021 19:03:06 +0200
> 
> Is there a method for dumping or displaying those strings or vectors?

Set a GDB breakpoint in mark_vectorlike, with commands that dump each
vector, then trigger GC?



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Any way of dumping strings?
  2021-06-02 17:24 ` Eli Zaretskii
@ 2021-06-02 17:54   ` Óscar Fuentes
  2021-06-02 18:16     ` Eli Zaretskii
  0 siblings, 1 reply; 10+ messages in thread
From: Óscar Fuentes @ 2021-06-02 17:54 UTC (permalink / raw)
  To: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Wed, 02 Jun 2021 19:03:06 +0200
>> 
>> Is there a method for dumping or displaying those strings or vectors?
>
> Set a GDB breakpoint in mark_vectorlike, with commands that dump each
> vector, then trigger GC?

I was hoping for something that could show at least thousands of
instances. Going one by one would be extremely time-consuming and
potentially deceptive.

Does Emacs zero-initialize memory when it is garbage-collected? Even if
it doesn't, maybe it is enough to dump the whole image to a file. Some
part would be gc-ed data, but a comparatively small part among all the
rest of hundreds of MB.




^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Any way of dumping strings?
  2021-06-02 17:54   ` Óscar Fuentes
@ 2021-06-02 18:16     ` Eli Zaretskii
  2021-06-02 18:33       ` Óscar Fuentes
  0 siblings, 1 reply; 10+ messages in thread
From: Eli Zaretskii @ 2021-06-02 18:16 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 02 Jun 2021 19:54:20 +0200
> 
> > Set a GDB breakpoint in mark_vectorlike, with commands that dump each
> > vector, then trigger GC?
> 
> I was hoping for something that could show at least thousands of
> instances. Going one by one would be extremely time-consuming and
> potentially deceptive.

That's not what I had in mind.  You can set up breakpoint commands
that dump the contents of the vector being marked, then continue
without stopping.  Then say "set logging on", so that the output goes
to a file, and let Emacs run.  Once GC finishes, you will have a file
with all the vectors in it; then you can examine them at your leisure.

> Does Emacs zero-initialize memory when it is garbage-collected?

You mean, those objects it recycles? it doesn't zero them, but why
would you be interested in the objects that get GCed -- those are not
your problem.  Your problem is those which aren't GCed, and
mark_vector marks each one of them.

> Even if it doesn't, maybe it is enough to dump the whole image to a
> file. Some part would be gc-ed data, but a comparatively small part
> among all the rest of hundreds of MB.

How would you tell the one from the other?



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Any way of dumping strings?
  2021-06-02 18:16     ` Eli Zaretskii
@ 2021-06-02 18:33       ` Óscar Fuentes
  2021-06-02 18:44         ` Eli Zaretskii
  0 siblings, 1 reply; 10+ messages in thread
From: Óscar Fuentes @ 2021-06-02 18:33 UTC (permalink / raw)
  To: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Wed, 02 Jun 2021 19:54:20 +0200
>> 
>> > Set a GDB breakpoint in mark_vectorlike, with commands that dump each
>> > vector, then trigger GC?
>> 
>> I was hoping for something that could show at least thousands of
>> instances. Going one by one would be extremely time-consuming and
>> potentially deceptive.
>
> That's not what I had in mind.  You can set up breakpoint commands
> that dump the contents of the vector being marked, then continue
> without stopping.  Then say "set logging on", so that the output goes
> to a file, and let Emacs run.  Once GC finishes, you will have a file
> with all the vectors in it; then you can examine them at your leisure.

That looks quite useful for this purpose, thanks.

Now that I think of it, my impression on the past session was that
performance degraded quite a bit, which could indicate that a very large
number of objects was live (which implies that the leaked objects are
relatively small, right?).

>> Even if it doesn't, maybe it is enough to dump the whole image to a
>> file. Some part would be gc-ed data, but a comparatively small part
>> among all the rest of hundreds of MB.
>
> How would you tell the one from the other?

I wouldn't. I'll just do some crude frequency analysis expecting that
GCed data is noise among the much larger signal of live data.




^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Any way of dumping strings?
  2021-06-02 17:03 Any way of dumping strings? Óscar Fuentes
  2021-06-02 17:24 ` Eli Zaretskii
@ 2021-06-02 18:38 ` Stefan Monnier
  2021-06-02 18:55   ` Óscar Fuentes
  1 sibling, 1 reply; 10+ messages in thread
From: Stefan Monnier @ 2021-06-02 18:38 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

> I'm trying to diagnose a memory leak. As it is used Emacs requires more
> memory, an average of a several tens of MB per hour. Right now it is
> using ~1GB and `memory-report' shows:
>
>  207.8 MiB  Vectors
>   83.7 MiB  Strings
>
> I terminated the previous session after reaching ~2GB with >700MB
> vectors and >300MB strings.
>
> In case the culprit is some package, looking at those strings or vectors
> could be revealing.
>
> Is there a method for dumping or displaying those strings or vectors?

Have you tried the new `memory-report` function?


        Stefan




^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Any way of dumping strings?
  2021-06-02 18:33       ` Óscar Fuentes
@ 2021-06-02 18:44         ` Eli Zaretskii
  0 siblings, 0 replies; 10+ messages in thread
From: Eli Zaretskii @ 2021-06-02 18:44 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 02 Jun 2021 20:33:22 +0200
> 
> > That's not what I had in mind.  You can set up breakpoint commands
> > that dump the contents of the vector being marked, then continue
> > without stopping.  Then say "set logging on", so that the output goes
> > to a file, and let Emacs run.  Once GC finishes, you will have a file
> > with all the vectors in it; then you can examine them at your leisure.
> 
> That looks quite useful for this purpose, thanks.
> 
> Now that I think of it, my impression on the past session was that
> performance degraded quite a bit, which could indicate that a very large
> number of objects was live (which implies that the leaked objects are
> relatively small, right?).

I don't know.  Maybe you have a significant number of medium-size
objects as well.  In any case, a large Lisp object in most cases also
means a lot of small ones -- these are the elements of that large
object: a list or a large vector or similar.  Only elements that are
atoms, like integers, don't add to GC's complexity, and such elements
are relatively rare in Emacs.



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Any way of dumping strings?
  2021-06-02 18:38 ` Stefan Monnier
@ 2021-06-02 18:55   ` Óscar Fuentes
  2021-06-02 19:30     ` Stefan Monnier
  0 siblings, 1 reply; 10+ messages in thread
From: Óscar Fuentes @ 2021-06-02 18:55 UTC (permalink / raw)
  To: emacs-devel

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Have you tried the new `memory-report` function?

Yes, I mention that on the message you just quoted :-)




^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Any way of dumping strings?
  2021-06-02 18:55   ` Óscar Fuentes
@ 2021-06-02 19:30     ` Stefan Monnier
  2021-06-03  6:08       ` Eli Zaretskii
  0 siblings, 1 reply; 10+ messages in thread
From: Stefan Monnier @ 2021-06-02 19:30 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

Óscar Fuentes [2021-06-02 20:55:07] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> Have you tried the new `memory-report` function?
> Yes, I mention that on the message you just quoted :-)

Duh!  So, IIUC the report about the size of specific variables doesn't
point give any hint about where those many strings&vectors are located?
Too bad.  I hope once you find the origin of the problem we can refine
`memory-report` so that it gives info which would have been helpful now.


        Stefan




^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Any way of dumping strings?
  2021-06-02 19:30     ` Stefan Monnier
@ 2021-06-03  6:08       ` Eli Zaretskii
  0 siblings, 0 replies; 10+ messages in thread
From: Eli Zaretskii @ 2021-06-03  6:08 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: ofv, emacs-devel

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Wed, 02 Jun 2021 15:30:34 -0400
> 
> Óscar Fuentes [2021-06-02 20:55:07] wrote:
> > Stefan Monnier <monnier@iro.umontreal.ca> writes:
> >> Have you tried the new `memory-report` function?
> > Yes, I mention that on the message you just quoted :-)
> 
> Duh!  So, IIUC the report about the size of specific variables doesn't
> point give any hint about where those many strings&vectors are located?
> Too bad.  I hope once you find the origin of the problem we can refine
> `memory-report` so that it gives info which would have been helpful now.

I think a useful extension of memory-report would be optionally to
dump all live objects of a given type to somewhere.  The challenge of
doing that in Lisp is that doing so will generally cons more Lisp
objects, thus skewing both the memory report and possibly the dumped
objects as well.  Patches welcome.



^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2021-06-03  6:08 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-06-02 17:03 Any way of dumping strings? Óscar Fuentes
2021-06-02 17:24 ` Eli Zaretskii
2021-06-02 17:54   ` Óscar Fuentes
2021-06-02 18:16     ` Eli Zaretskii
2021-06-02 18:33       ` Óscar Fuentes
2021-06-02 18:44         ` Eli Zaretskii
2021-06-02 18:38 ` Stefan Monnier
2021-06-02 18:55   ` Óscar Fuentes
2021-06-02 19:30     ` Stefan Monnier
2021-06-03  6:08       ` Eli Zaretskii

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.