bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Donald Tillman]

--text follows this line--
This bug report will be sent to the Bug-GNU-Emacs mailing list
and the GNU bug tracker at debbugs.gnu.org.  Please check that
the From: line contains a valid email address.  After a delay of up
to one day, you should receive an acknowledgment at that address.

Please write in English if possible, as the Emacs maintainers
usually do not have translators for other languages.

Please describe exactly what actions triggered the bug, and
the precise symptoms of the bug.  If you can, give a recipe
starting from `emacs -Q':

----

Hi!

I use Emacs on Mac OS X, Mavericks, Intel MacBook Pro, downloaded from emacsformacosx.com.

Running Emacs, a process named "distnoted" starts around 1% or 2% of the CPU, and after a while, slowly works its way up to 50% to 100% of the CPU.  Yoiks!  Actually there appear to be 3 distnoted processes, but only one eats up the CPU.

Quitting Emacs brings distnoted down to under 1% within seconds. 

  -- Don



If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
/Applications/Emacs.app/Contents/Resources/etc/DEBUG.


In GNU Emacs 24.3.1 (x86_64-apple-darwin, NS apple-appkit-1038.36)
of 2013-03-12 on bob.porkrind.org
Windowing system distributor `Apple', version 10.3.1265
Configured using:
`configure '--host=x86_64-apple-darwin' '--build=i686-apple-darwin'
'--with-ns' 'build_alias=i686-apple-darwin'
'host_alias=x86_64-apple-darwin' 'CC=gcc -mmacosx-version-min=10.7
-isystem
/Users/david/Xcode-10.7_4.5.2/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/usr/include/
-F/Users/david/Xcode-10.7_4.5.2/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks''

Important settings:
  locale-coding-system: nil
  default enable-multibyte-characters: t

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<help-echo> M-x r e p o r t SPC e m a <tab> <retur
n>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr warnings emacsbug message format-spec
rfc822 mml easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rainbow-mode-autoloads
svg-clock-autoloads package rmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel ns-win tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
make-network-process ns multi-tty emacs)


--
Don Tillman
Palo Alto, California
don@till.com
http://www.till.com

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Marc Feeley]

Just want to add that I have the same problem.  After quitting emacs
the distnoted process quiets  down to an acceptable level.

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Jan Djärv]

Hello. 

I can not reproduce this. Does it happen when you start with -Q and then do nothing else?  I tried with trunk and downloaded binaries from the site (release and latest).  I ran them on two computers with 10.9. I once managed to get a distnoted process to use 1.2 percent CPU, but otherwise they where steady at 0 - 0.3 % no matter what I did in Emacs. I monitored the trunk build for a whole workday, about 8 hours. 


    Jan D. 

> 21 nov 2013 kl. 19:18 skrev Donald Tillman <don@till.com>:
> 
> --text follows this line--
> This bug report will be sent to the Bug-GNU-Emacs mailing list
> and the GNU bug tracker at debbugs.gnu.org.  Please check that
> the From: line contains a valid email address.  After a delay of up
> to one day, you should receive an acknowledgment at that address.
> 
> Please write in English if possible, as the Emacs maintainers
> usually do not have translators for other languages.
> 
> Please describe exactly what actions triggered the bug, and
> the precise symptoms of the bug.  If you can, give a recipe
> starting from `emacs -Q':
> 
> ----
> 
> Hi!
> 
> I use Emacs on Mac OS X, Mavericks, Intel MacBook Pro, downloaded from emacsformacosx.com.
> 
> Running Emacs, a process named "distnoted" starts around 1% or 2% of the CPU, and after a while, slowly works its way up to 50% to 100% of the CPU.  Yoiks!  Actually there appear to be 3 distnoted processes, but only one eats up the CPU.
> 
> Quitting Emacs brings distnoted down to under 1% within seconds. 
> 
>  -- Don
> 
> 
> 
> If Emacs crashed, and you have the Emacs process in the gdb debugger,
> please include the output from the following gdb commands:
>    `bt full' and `xbacktrace'.
> For information about debugging Emacs, please read the file
> /Applications/Emacs.app/Contents/Resources/etc/DEBUG.
> 
> 
> In GNU Emacs 24.3.1 (x86_64-apple-darwin, NS apple-appkit-1038.36)
> of 2013-03-12 on bob.porkrind.org
> Windowing system distributor `Apple', version 10.3.1265
> Configured using:
> `configure '--host=x86_64-apple-darwin' '--build=i686-apple-darwin'
> '--with-ns' 'build_alias=i686-apple-darwin'
> 'host_alias=x86_64-apple-darwin' 'CC=gcc -mmacosx-version-min=10.7
> -isystem
> /Users/david/Xcode-10.7_4.5.2/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/usr/include/
> -F/Users/david/Xcode-10.7_4.5.2/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks''
> 
> Important settings:
>  locale-coding-system: nil
>  default enable-multibyte-characters: t
> 
> Major mode: Fundamental
> 
> Minor modes in effect:
>  tooltip-mode: t
>  mouse-wheel-mode: t
>  tool-bar-mode: t
>  menu-bar-mode: t
>  file-name-shadow-mode: t
>  global-font-lock-mode: t
>  blink-cursor-mode: t
>  auto-composition-mode: t
>  auto-encryption-mode: t
>  auto-compression-mode: t
>  buffer-read-only: t
>  line-number-mode: t
>  transient-mark-mode: t
> 
> Recent input:
> <help-echo> M-x r e p o r t SPC e m a <tab> <retur
> n>
> 
> Recent messages:
> For information about GNU Emacs and the GNU system, type C-h C-a.
> 
> Load-path shadows:
> None found.
> 
> Features:
> (shadow sort gnus-util mail-extr warnings emacsbug message format-spec
> rfc822 mml easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse
> rfc2231 mailabbrev gmm-utils mailheader sendmail rainbow-mode-autoloads
> svg-clock-autoloads package rmail rfc2047 rfc2045 ietf-drums mm-util
> mail-prsvr mail-utils time-date tooltip ediff-hook vc-hooks
> lisp-float-type mwheel ns-win tool-bar dnd fontset image regexp-opt
> fringe tabulated-list newcomment lisp-mode register page menu-bar
> rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
> facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
> tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
> czech european ethiopic indian cyrillic chinese case-table epa-hook
> jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
> cus-face macroexp files text-properties overlay sha1 md5 base64 format
> env code-pages mule custom widget hashtable-print-readable backquote
> make-network-process ns multi-tty emacs)
> 
> 
> --
> Don Tillman
> Palo Alto, California
> don@till.com
> http://www.till.com
> 
> 
> 
> 
> 
> 

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Piet Jaspers]

> I can not reproduce this. 

I too have this problem. I'm not quite sure what triggers it, but it
seems to to have something to do with command-tabbing (or at least
that's what it feels like).

If I can pinpoint it exactly I'll let it know.

Re: bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[louisnoel.pouchet]

FYI I have the same problem too. It took me a while to connect it (likely) to emacs on mavericks, but it seems confirmed.

Setup: 
- late 2013 MPB 15" w/ discrete graphics card, CUDA installed
- arrived installed with mavericks, did a migration from my 10.8 MBP, erased the emacs app, downloaded 24.3 from http://emacsformacosx.com
- machine is up 8h/day mini always on, emacs 24.3 always on (used on an hourly basis)

Symptom: 
- after a few days, distnoted start to be noticeable: 15%-25% CPU, system lagging, tens of thousands of threads active
- rebooting the machine removes the problem for a few days
- quitting and restarting emacs but no reboot also removes the problem for a few days

I cannot conclude emacs is the cause of the problem, as I have tens of applications running, but there's an observable correlation.

FYI the problem doesn't arrive in 8h but in a few days for me (2-3). I don't know yet if the speed of occurrence of the problem is connected with my emacs usage pattern, or from the number of hours emacs has been on.

This is an important problem: it can end up making the system lagging, and does consume battery life.

++


On Wednesday, November 27, 2013 1:10:11 PM UTC-8, Piet Jaspers wrote:
> > I can not reproduce this. 
> 
> 
> 
> I too have this problem. I'm not quite sure what triggers it, but it
> 
> seems to to have something to do with command-tabbing (or at least
> 
> that's what it feels like).
> 
> 
> 
> If I can pinpoint it exactly I'll let it know.

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Christopher Smith]

Donald Tillman <don <at> till.com> writes:
> Hi!
> 
> I use Emacs on Mac OS X, Mavericks, Intel MacBook Pro, downloaded from
> emacsformacosx.com.
> 
> Running Emacs, a process named "distnoted" starts around 1% or 2% of the
> CPU, and after a while, slowly works its way up to 50% to 100% of
> the CPU.  Yoiks!  Actually there appear to be 3 distnoted processes,
> but only one eats up the CPU.
> 
> Quitting Emacs brings distnoted down to under 1% within seconds. 

Just wanted to bump this up, as I've now seen this bug myself. Not sure
why Emacs is causing the problem. My emacs was built from macports.

--Chris

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[YAMAMOTO Mitsuharu]

>>>>> On Mon, 9 Dec 2013 04:38:59 +0000 (UTC), Christopher Smith <cbsmith@gmail.com> said:

> Donald Tillman <don <at> till.com> writes:
>> Hi!
>> 
>> I use Emacs on Mac OS X, Mavericks, Intel MacBook Pro, downloaded
>> from emacsformacosx.com.
>> 
>> Running Emacs, a process named "distnoted" starts around 1% or 2%
>> of the CPU, and after a while, slowly works its way up to 50% to
>> 100% of the CPU.  Yoiks!  Actually there appear to be 3 distnoted
>> processes, but only one eats up the CPU.
>> 
>> Quitting Emacs brings distnoted down to under 1% within seconds.

> Just wanted to bump this up, as I've now seen this bug myself. Not
> sure why Emacs is causing the problem. My emacs was built from
> macports.

Which "port" did you use to build Emacs from MacPorts?

I'm asking it because I'd like to know if this problem can also happen
with Emacs Mac port(*).  Users of the MacPorts package system can use
it via the "port" named "emacs-mac-app" (its application bundle name
is EmacsMac.app).

*: http://lists.gnu.org/archive/html/emacs-devel/2013-11/msg00225.html

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Jan Djärv]

Hello. 

> 9 dec 2013 kl. 05:38 skrev Christopher Smith <cbsmith@gmail.com>:
> 
> Donald Tillman <don <at> till.com> writes:
>> Hi!
>> 
>> I use Emacs on Mac OS X, Mavericks, Intel MacBook Pro, downloaded from
>> emacsformacosx.com.
>> 
>> Running Emacs, a process named "distnoted" starts around 1% or 2% of the
>> CPU, and after a while, slowly works its way up to 50% to 100% of
>> the CPU.  Yoiks!  Actually there appear to be 3 distnoted processes,
>> but only one eats up the CPU.
>> 
>> Quitting Emacs brings distnoted down to under 1% within seconds. 
> 
> Just wanted to bump this up, as I've now seen this bug myself. Not sure
> why Emacs is causing the problem. My emacs was built from macports.

It is no use bumping anything that isn't reproducable. More helpful would be if those that have this can do some debugging. For example dtruss on Emacs and distnoted. 

     Jan D. 

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[SB]

Hello, I've investigated this problem since I had similar symptoms
that correlated with everyone else's reports (quit emacs.app and
distnoted quiets down). Also, in my case when I download the vanilla
emacs.app from the emacs for OSX site, the problem does not appear. In
my case, under Mavericks, this problem only occurs when I use
Emacs.app with the inline patch applied (for more native Japanese
input).

Emacs inline patch
http://svn.sourceforge.jp/svnroot/macemacsjp/inline_patch/trunk/emacs-inline.patch

After investigating, it seems that the purpose of distnoted is to
serve as a daemon to facilitate interapplication communication. In the
case of the "inline patch", to capture the moment the IME is changed
so that Emacs can do likewise to change language input. This is done
by adding an observer using NSDistributedNotificationCenter and there
is also a corresponding Core Foundation method. I didn't find any used
of NSDistributedNotificationCenter in the official release (24.3)
source.

https://developer.apple.com/library/mac/documentation/cocoa/reference/foundation/classes/NSDistributedNotificationCenter_Class/Reference/Reference.html

The fix was to add
"suspensionBehavior:NSNotificationSuspensionBehaviorDeliverImmediately"
rather than the default which was
"NSNotificationSuspensionBehaviorCoalesce". For whatever reason,
modifying the inline patch in this manner fixed the issue for me. This
would be consistent with the report above that "Cmd Tabbing" seems to
trigger it (since it would activate the suspension behavior).

Others with the same problem seem to take a sledgehammer approach of
killing distnoted with a cronjob (which I did manually as well).

This is the patch (modified by hand so it may not apply) against emacs 24.3.

https://gist.github.com/anonymous/8142555

The modified line:

  [[NSDistributedNotificationCenter defaultCenter] addObserver: NSApp
                    selector: @selector (changeInputMethod:)
                           name:
@"AppleSelectedInputSourcesChangedNotification" object: nil
suspensionBehavior:NSNotificationSuspensionBehaviorDeliverImmediately];

Suspension Behavior
https://developer.apple.com/library/mac/documentation/cocoa/reference/foundation/classes/NSDistributedNotificationCenter_Class/Reference/Reference.html#//apple_ref/doc/c_ref/NSNotificationSuspensionBehavior

For others who do not use the patch, there may be other applications,
since this is most likely a bug with Mavericks itself and you may want
to log distnoted for additional hints and grep the source of your
emacs build to see if NSDistributedNotificationCenter is being used.

Logging Distnoted
http://www.cocoawithlove.com/2009/02/interprocess-communication-snooping.html

Hope this helps someone.

Cheers,

Sam

On Mon, Dec 9, 2013 at 7:06 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
> Hello.
>
>> 9 dec 2013 kl. 05:38 skrev Christopher Smith <cbsmith@gmail.com>:
>>
>> Donald Tillman <don <at> till.com> writes:
>>> Hi!
>>>
>>> I use Emacs on Mac OS X, Mavericks, Intel MacBook Pro, downloaded from
>>> emacsformacosx.com.
>>>
>>> Running Emacs, a process named "distnoted" starts around 1% or 2% of the
>>> CPU, and after a while, slowly works its way up to 50% to 100% of
>>> the CPU.  Yoiks!  Actually there appear to be 3 distnoted processes,
>>> but only one eats up the CPU.
>>>
>>> Quitting Emacs brings distnoted down to under 1% within seconds.
>>
>> Just wanted to bump this up, as I've now seen this bug myself. Not sure
>> why Emacs is causing the problem. My emacs was built from macports.
>
> It is no use bumping anything that isn't reproducable. More helpful would be if those that have this can do some debugging. For example dtruss on Emacs and distnoted.
>
>      Jan D.
>
>

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Jan Djärv]

Hello.

This is good to know, but others said they used Emacs downloaded from emacsformacosx.com, which is supposed to be unpatched.  So there might be more than one reason for this.

	Jan D.

27 dec 2013 kl. 06:15 skrev SB <richstyles@gmail.com>:

> Hello, I've investigated this problem since I had similar symptoms
> that correlated with everyone else's reports (quit emacs.app and
> distnoted quiets down). Also, in my case when I download the vanilla
> emacs.app from the emacs for OSX site, the problem does not appear. In
> my case, under Mavericks, this problem only occurs when I use
> Emacs.app with the inline patch applied (for more native Japanese
> input).
> 
> Emacs inline patch
> http://svn.sourceforge.jp/svnroot/macemacsjp/inline_patch/trunk/emacs-inline.patch
> 
> After investigating, it seems that the purpose of distnoted is to
> serve as a daemon to facilitate interapplication communication. In the
> case of the "inline patch", to capture the moment the IME is changed
> so that Emacs can do likewise to change language input. This is done
> by adding an observer using NSDistributedNotificationCenter and there
> is also a corresponding Core Foundation method. I didn't find any used
> of NSDistributedNotificationCenter in the official release (24.3)
> source.
> 
> https://developer.apple.com/library/mac/documentation/cocoa/reference/foundation/classes/NSDistributedNotificationCenter_Class/Reference/Reference.html
> 
> The fix was to add
> "suspensionBehavior:NSNotificationSuspensionBehaviorDeliverImmediately"
> rather than the default which was
> "NSNotificationSuspensionBehaviorCoalesce". For whatever reason,
> modifying the inline patch in this manner fixed the issue for me. This
> would be consistent with the report above that "Cmd Tabbing" seems to
> trigger it (since it would activate the suspension behavior).
> 
> Others with the same problem seem to take a sledgehammer approach of
> killing distnoted with a cronjob (which I did manually as well).
> 
> This is the patch (modified by hand so it may not apply) against emacs 24.3.
> 
> https://gist.github.com/anonymous/8142555
> 
> The modified line:
> 
>  [[NSDistributedNotificationCenter defaultCenter] addObserver: NSApp
>                    selector: @selector (changeInputMethod:)
>                           name:
> @"AppleSelectedInputSourcesChangedNotification" object: nil
> suspensionBehavior:NSNotificationSuspensionBehaviorDeliverImmediately];
> 
> Suspension Behavior
> https://developer.apple.com/library/mac/documentation/cocoa/reference/foundation/classes/NSDistributedNotificationCenter_Class/Reference/Reference.html#//apple_ref/doc/c_ref/NSNotificationSuspensionBehavior
> 
> For others who do not use the patch, there may be other applications,
> since this is most likely a bug with Mavericks itself and you may want
> to log distnoted for additional hints and grep the source of your
> emacs build to see if NSDistributedNotificationCenter is being used.
> 
> Logging Distnoted
> http://www.cocoawithlove.com/2009/02/interprocess-communication-snooping.html
> 
> Hope this helps someone.
> 
> Cheers,
> 
> Sam
> 
> On Mon, Dec 9, 2013 at 7:06 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
>> Hello.
>> 
>>> 9 dec 2013 kl. 05:38 skrev Christopher Smith <cbsmith@gmail.com>:
>>> 
>>> Donald Tillman <don <at> till.com> writes:
>>>> Hi!
>>>> 
>>>> I use Emacs on Mac OS X, Mavericks, Intel MacBook Pro, downloaded from
>>>> emacsformacosx.com.
>>>> 
>>>> Running Emacs, a process named "distnoted" starts around 1% or 2% of the
>>>> CPU, and after a while, slowly works its way up to 50% to 100% of
>>>> the CPU.  Yoiks!  Actually there appear to be 3 distnoted processes,
>>>> but only one eats up the CPU.
>>>> 
>>>> Quitting Emacs brings distnoted down to under 1% within seconds.
>>> 
>>> Just wanted to bump this up, as I've now seen this bug myself. Not sure
>>> why Emacs is causing the problem. My emacs was built from macports.
>> 
>> It is no use bumping anything that isn't reproducable. More helpful would be if those that have this can do some debugging. For example dtruss on Emacs and distnoted.
>> 
>>     Jan D.
>> 
>> 

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Josh]

On Tue, Jan 7, 2014 at 2:56 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
> This is good to know, but others said they used Emacs downloaded from emacsformacosx.com, which is supposed to be unpatched.

It does appear to be entirely unpatched, and the tools to produce
it are publicly available[0] so it's easy to check.  There's also
an archive[1] of releases, pretests, and nightlies going back to
2006 (!) in .dmg format which is sometimes handy for bisection.

[0] https://github.com/caldwell/build-emacs
[1] http://emacsformacosx.com/builds/all

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Jan Djärv]

Hello.

8 jan 2014 kl. 04:35 skrev Josh <josh@foxtail.org>:

> On Tue, Jan 7, 2014 at 2:56 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
>> This is good to know, but others said they used Emacs downloaded from emacsformacosx.com, which is supposed to be unpatched.
> 
> It does appear to be entirely unpatched, and the tools to produce
> it are publicly available[0] so it's easy to check.  There's also
> an archive[1] of releases, pretests, and nightlies going back to
> 2006 (!) in .dmg format which is sometimes handy for bisection.
> 
> [0] https://github.com/caldwell/build-emacs
> [1] http://emacsformacosx.com/builds/all

This has to be done by someone who has the problem.  I already tried several of their builds, on three separate machines.  None exhibit the problem.

	Jan D.

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[canoeberry]

I have been observing this since Mavericks was released. I did not make a
connection between distnoted and emacs, other than the following
observation: both are leaking memory.

For the first time ever I am finding that my emacs process slowly grows in
size. I can be editing a small handful of files and be using 500Mb of real
memory. Never seen that before, ever.

Distnoted grows much faster, on the order of 1Gb real memory / day. Killing
that process and having another start up automatically causes some OS X
features to stop working, e.g., it is not possible to enter Time Machine or
even see the status of your Time Machine backups anymore.

I have seen distnoted peg the CPU when Time Machine backups are running or
recently completed. Meanwhile, the menubar no longer animates the
in-progress backup, which I assume is not on purpose and possibly even
related to the issue.

I do not use shells in emacs nowadays.

I am running emacs from emacsformacosx.

I have filed bugs with Apple.

If anybody has any suggestions on how to figure out what is causing Emacs to
grow in size, I will happily run some experiments for you. I tried to figure
out how to profile emacs memory but it was not very obvious to me what to
do.




--
View this message in context: http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310127.html
Sent from the Emacs - Bugs mailing list archive at Nabble.com.

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Jan Djärv]

Hello.

14 jan 2014 kl. 18:15 skrev canoeberry <emacs@jpayne.net>:

> I have been observing this since Mavericks was released. I did not make a
> connection between distnoted and emacs, other than the following
> observation: both are leaking memory.
> 
> For the first time ever I am finding that my emacs process slowly grows in
> size. I can be editing a small handful of files and be using 500Mb of real
> memory. Never seen that before, ever.
> 
> Distnoted grows much faster, on the order of 1Gb real memory / day. Killing
> that process and having another start up automatically causes some OS X
> features to stop working, e.g., it is not possible to enter Time Machine or
> even see the status of your Time Machine backups anymore.
> 
> I have seen distnoted peg the CPU when Time Machine backups are running or
> recently completed. Meanwhile, the menubar no longer animates the
> in-progress backup, which I assume is not on purpose and possibly even
> related to the issue.
> 

I think that is so OSX can save a bit of power.

> I do not use shells in emacs nowadays.
> 
> I am running emacs from emacsformacosx.
> 
> I have filed bugs with Apple.
> 
> If anybody has any suggestions on how to figure out what is causing Emacs to
> grow in size, I will happily run some experiments for you. I tried to figure
> out how to profile emacs memory but it was not very obvious to me what to
> do.

The obvious thing is to run leaks on Emacs (man leaks) after a garbage collection has been made (M-x garbage-collect).  Leaks may report leaks which in fact are not due to uncollected garbage.

	Jan D.

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[canoeberry]

[-- Attachment #1: Type: text/plain, Size: 3557 bytes --]

This is what I see if I run the "leaks" program on my Mac on the emacs process:

Leak: 0x1109f7b20  size=160  zone: DefaultMallocZone_0x100659000   OS_dispatch_source  ObjC  libdispatch.dylib
	0x76964c20 0x00007fff 0x00000001 0x00000000 	 L.v............
	0x89abcdef 0xffffffff 0x769666c0 0x00007fff 	.........f.v....
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000001 0x00000000 0x0002940d 0x00000000 	................
	0x8896290c 0x00007fff 0x013000a0 0x00000001 	.)........0.....
	0x109f7c10 0x00000001 0x00000002 0x0000004c 	.|..........L...
	...

A bazillion of them.

I also ran it on a distnoted which I just killed and restarted because the existing one was over 2Gb of real memory (or so it said). It claimed there were no leaks.

This leaks program is pretty cool. It's like a conservative GC algorithm that assumes every integer could be a pointer. So by definition is is conservative.

Anyway - does this piece of information help any smart hacker types?

JP

On 14 Jan 2014, at 17:47, Jan Djärv [via Emacs] <ml-node+s1067599n310141h49@n5.nabble.com> wrote:

> Hello. 
> 
> 14 jan 2014 kl. 18:15 skrev canoeberry <[hidden email]>: 
> 
> > I have been observing this since Mavericks was released. I did not make a 
> > connection between distnoted and emacs, other than the following 
> > observation: both are leaking memory. 
> > 
> > For the first time ever I am finding that my emacs process slowly grows in 
> > size. I can be editing a small handful of files and be using 500Mb of real 
> > memory. Never seen that before, ever. 
> > 
> > Distnoted grows much faster, on the order of 1Gb real memory / day. Killing 
> > that process and having another start up automatically causes some OS X 
> > features to stop working, e.g., it is not possible to enter Time Machine or 
> > even see the status of your Time Machine backups anymore. 
> > 
> > I have seen distnoted peg the CPU when Time Machine backups are running or 
> > recently completed. Meanwhile, the menubar no longer animates the 
> > in-progress backup, which I assume is not on purpose and possibly even 
> > related to the issue. 
> >
> 
> I think that is so OSX can save a bit of power. 
> 
> > I do not use shells in emacs nowadays. 
> > 
> > I am running emacs from emacsformacosx. 
> > 
> > I have filed bugs with Apple. 
> > 
> > If anybody has any suggestions on how to figure out what is causing Emacs to 
> > grow in size, I will happily run some experiments for you. I tried to figure 
> > out how to profile emacs memory but it was not very obvious to me what to 
> > do.
> 
> The obvious thing is to run leaks on Emacs (man leaks) after a garbage collection has been made (M-x garbage-collect).  Leaks may report leaks which in fact are not due to uncollected garbage. 
> 
>         Jan D. 
> 
> 
> 
> 
> 
> 
> If you reply to this email, your message will be added to the discussion below:
> http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310141.html
> To unsubscribe from bug#15946: 24.3; Mac OS X, Mavericks, distnoted process, click here.
> NAML





-----
Jonathan Payne

--
View this message in context: http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310179.html
Sent from the Emacs - Bugs mailing list archive at Nabble.com.

[-- Attachment #2: Type: text/html, Size: 6881 bytes --]

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[canoeberry]

[-- Attachment #1: Type: text/plain, Size: 5455 bytes --]

I've fired up Emacs with the -q and -nsl options. So my .emacs is not loaded and neither is the site-lisp. It starts leaking memory at a rate ... well here you go:

$ while true
> do
> sudo leaks 12095 | head -20 | grep "leaks for"
> sleep 1
> done
Process 12095: 935 leaks for 165824 total leaked bytes.
Process 12095: 954 leaks for 168864 total leaked bytes.
Process 12095: 972 leaks for 171744 total leaked bytes.
Process 12095: 986 leaks for 173984 total leaked bytes.
Process 12095: 1005 leaks for 177024 total leaked bytes.
Process 12095: 1025 leaks for 180224 total leaked bytes.
Process 12095: 1039 leaks for 182464 total leaked bytes.
Process 12095: 1057 leaks for 185344 total leaked bytes.
Process 12095: 1076 leaks for 188384 total leaked bytes.
Process 12095: 1090 leaks for 190624 total leaked bytes.
Process 12095: 1107 leaks for 193344 total leaked bytes.
Process 12095: 1127 leaks for 196544 total leaked bytes.
Process 12095: 1147 leaks for 199744 total leaked bytes.
Process 12095: 1158 leaks for 201504 total leaked bytes.
Process 12095: 1178 leaks for 204704 total leaked bytes.
Process 12095: 1195 leaks for 207424 total leaked bytes.
Process 12095: 1209 leaks for 209664 total leaked bytes.

So it's basically something like 20 individual 160 byte leaks per second. If I actually use emacs it seems to have spurts of many more leaks. These do seem like proper leaks because my emacs memory is growing at the same rate as this leaks is reporting, and, I am literally running an emacs with nothing in it and not interacting with it.

Output from dtruss shows kevent activity. A bunch of it. Not much else:

workq_kernreturn(0x20, 0x0, 0x1)		 = 0 0
kevent64(0x3, 0x0, 0x0)		 = 1 0
kevent64(0x3, 0x1006CB458, 0x1)		 = 1 0
kevent64(0x3, 0x1006CB458, 0x1)		 = 1 0
kevent64(0x3, 0x0, 0x0)		 = 1 0
kevent64(0x3, 0x1006CB4F8, 0x1)		 = 1 0
kevent64(0x3, 0x0, 0x0)		 = 1 0
kevent64(0x3, 0x0, 0x0)		 = 1 0
kevent64(0x3, 0x1006CB458, 0x1)		 = 1 0
kevent64(0x3, 0x1006CB458, 0x1)		 = 1 0
kevent64(0x3, 0x0, 0x0)		 = 1 0
kevent64(0x3, 0x1006CB3E8, 0x1)		 = 1 0
kevent64(0x3, 0x1006CB4F8, 0x1)		 = 1 0
kevent64(0x3, 0x0, 0x0)		 = 1 0
kevent64(0x3, 0x0, 0x0)		 = 1 0
kevent64(0x3, 0x1006CB458, 0x1)		 = 1 0
kevent64(0x3, 0x1006CB458, 0x1)		 = 1 0
kevent64(0x3, 0x7FFF5FBFD2E8, 0x1)		 = 1 0
kevent64(0x3, 0x0, 0x0)		 = 1 0
kevent64(0x3, 0x1006CB458, 0x1)		 = 1 0
kevent64(0x3, 0x1006CB458, 0x1)		 = 1 0
kevent64(0x3, 0x7FFF5FBFE8B8, 0x1)		 = 1 0
workq_kernreturn(0x20, 0x0, 0x1)		 = 0 0
kevent64(0x3, 0x7FFF5FBFCAF8, 0x1)		 = 1 0
kevent64(0x3, 0x0, 0x0)		 = 1 0
kevent64(0x3, 0x1006CB458, 0x1)		 = 1 0
kevent64(0x3, 0x1006CB458, 0x1)		 = 1 0
kevent64(0x3, 0x0, 0x0)		 = 1 0
kevent64(0x3, 0x1006CB458, 0x1)		 = 1 0
kevent64(0x3, 0x1006CB458, 0x1)		 = 1 0
setitimer(0x0, 0x7FFF5FBFD0B0, 0x0)		 = 0 0
kevent64(0x3, 0x7FFF5FBFE8C8, 0x1)		 = 1 0
workq_kernreturn(0x20, 0x0, 0x1)		 = 0 0
workq_kernreturn(0x20, 0x0, 0x1)		 = 0 0

This is grasping at straws at its worst.

JP

On 14 Jan 2014, at 17:47, Jan Djärv [via Emacs] <ml-node+s1067599n310141h49@n5.nabble.com> wrote:

> Hello. 
> 
> 14 jan 2014 kl. 18:15 skrev canoeberry <[hidden email]>: 
> 
> > I have been observing this since Mavericks was released. I did not make a 
> > connection between distnoted and emacs, other than the following 
> > observation: both are leaking memory. 
> > 
> > For the first time ever I am finding that my emacs process slowly grows in 
> > size. I can be editing a small handful of files and be using 500Mb of real 
> > memory. Never seen that before, ever. 
> > 
> > Distnoted grows much faster, on the order of 1Gb real memory / day. Killing 
> > that process and having another start up automatically causes some OS X 
> > features to stop working, e.g., it is not possible to enter Time Machine or 
> > even see the status of your Time Machine backups anymore. 
> > 
> > I have seen distnoted peg the CPU when Time Machine backups are running or 
> > recently completed. Meanwhile, the menubar no longer animates the 
> > in-progress backup, which I assume is not on purpose and possibly even 
> > related to the issue. 
> >
> 
> I think that is so OSX can save a bit of power. 
> 
> > I do not use shells in emacs nowadays. 
> > 
> > I am running emacs from emacsformacosx. 
> > 
> > I have filed bugs with Apple. 
> > 
> > If anybody has any suggestions on how to figure out what is causing Emacs to 
> > grow in size, I will happily run some experiments for you. I tried to figure 
> > out how to profile emacs memory but it was not very obvious to me what to 
> > do.
> 
> The obvious thing is to run leaks on Emacs (man leaks) after a garbage collection has been made (M-x garbage-collect).  Leaks may report leaks which in fact are not due to uncollected garbage. 
> 
>         Jan D. 
> 
> 
> 
> 
> 
> 
> If you reply to this email, your message will be added to the discussion below:
> http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310141.html
> To unsubscribe from bug#15946: 24.3; Mac OS X, Mavericks, distnoted process, click here.
> NAML





-----
Jonathan Payne

--
View this message in context: http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310185.html
Sent from the Emacs - Bugs mailing list archive at Nabble.com.

[-- Attachment #2: Type: text/html, Size: 9964 bytes --]

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Stefan Monnier]

> If anybody has any suggestions on how to figure out what is causing Emacs to
> grow in size,

It'd be useful to state precisely which Emacs you're running.


        Stefan

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[canoeberry]

[-- Attachment #1: Type: text/plain, Size: 1196 bytes --]

Sorry about that!

	GNU Emacs 24.3.1 (x86_64-apple-darwin, NS apple-appkit-1038.36) of 2013-03-13 on bob.porkrind.org

From emacsformacos.com.

I think I was already using this version before I upgraded to Mavericks and it was sometime after the Mavericks upgrade that I noticed the leak. So it's ... likely an incompatible change or just a bug in Mavericks.

JP

On 14 Jan 2014, at 21:36, Stefan Monnier [via Emacs] <ml-node+s1067599n310197h85@n5.nabble.com> wrote:

> > If anybody has any suggestions on how to figure out what is causing Emacs to 
> > grow in size, 
> 
> It'd be useful to state precisely which Emacs you're running. 
> 
> 
>         Stefan 
> 
> 
> 
> 
> 
> If you reply to this email, your message will be added to the discussion below:
> http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310197.html
> To unsubscribe from bug#15946: 24.3; Mac OS X, Mavericks, distnoted process, click here.
> NAML





-----
Jonathan Payne

--
View this message in context: http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310199.html
Sent from the Emacs - Bugs mailing list archive at Nabble.com.

[-- Attachment #2: Type: text/html, Size: 3180 bytes --]

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Jan Djärv]

[-- Attachment #1: Type: text/plain, Size: 1338 bytes --]

Hello.

14 jan 2014 kl. 22:53 skrev canoeberry <emacs@jpayne.net>:

> Sorry about that!
> 
> 	GNU Emacs 24.3.1 (x86_64-apple-darwin, NS apple-appkit-1038.36) of 2013-03-13 on bob.porkrind.org

You should try a newer version, built from the trunk.

	Jan D.

> 
> From emacsformacos.com.
> 
> I think I was already using this version before I upgraded to Mavericks and it was sometime after the Mavericks upgrade that I noticed the leak. So it's ... likely an incompatible change or just a bug in Mavericks.
> 
> JP
> 
> On 14 Jan 2014, at 21:36, Stefan Monnier [via Emacs] <[hidden email]> wrote:
> 
>> > If anybody has any suggestions on how to figure out what is causing Emacs to 
>> > grow in size, 
>> 
>> It'd be useful to state precisely which Emacs you're running. 
>> 
>> 
>>         Stefan 
>> 
>> 
>> 
>> 
>> 
>> If you reply to this email, your message will be added to the discussion below:
>> http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310197.html
>> To unsubscribe from bug#15946: 24.3; Mac OS X, Mavericks, distnoted process, click here.
>> NAML
> 
> Jonathan Payne 
> 
> View this message in context: Re: bug#15946: 24.3; Mac OS X, Mavericks, distnoted process
> Sent from the Emacs - Bugs mailing list archive at Nabble.com.


[-- Attachment #2: Type: text/html, Size: 3866 bytes --]

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Jan Djärv]

Hello.


14 jan 2014 kl. 21:09 skrev canoeberry <emacs@jpayne.net>:

> This is what I see if I run the "leaks" program on my Mac on the emacs process:
> 
> Leak: 0x1109f7b20  size=160  zone: DefaultMallocZone_0x100659000   OS_dispatch_source  ObjC  libdispatch.dylib
> 	0x76964c20 0x00007fff 0x00000001 0x00000000 	 L.v............
> 	0x89abcdef 0xffffffff 0x769666c0 0x00007fff 	.........f.v....
> 	0x00000000 0x00000000 0x00000000 0x00000000 	................
> 	0x00000000 0x00000000 0x00000000 0x00000000 	................
> 	0x00000000 0x00000000 0x00000000 0x00000000 	................
> 	0x00000001 0x00000000 0x0002940d 0x00000000 	................
> 	0x8896290c 0x00007fff 0x013000a0 0x00000001 	.)........0.....
> 	0x109f7c10 0x00000001 0x00000002 0x0000004c 	.|..........L...
> 	...
> 
> A bazillion of them.
> 
> I also ran it on a distnoted which I just killed and restarted because the existing one was over 2Gb of real memory (or so it said). It claimed there were no leaks.
> 
> This leaks program is pretty cool. It's like a conservative GC algorithm that assumes every integer could be a pointer. So by definition is is conservative.
> 
> Anyway - does this piece of information help any smart hacker types?

They are not all like that, somewhere in there is the real leak.  You need to save all output, and put it somewhere it can be read.  If it is large, better not send it to the list.
But if you are doing this, do it on a more recent version than 24.3.

	Jan D.

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Matthew Leach]

[-- Attachment #1: Type: text/plain, Size: 636 bytes --]


Jan Djärv <jan.h.d@swipnet.se> writes:

> They are not all like that, somewhere in there is the real leak.  You
> need to save all output, and put it somewhere it can be read.  If it
> is large, better not send it to the list.  But if you are doing this,
> do it on a more recent version than 24.3.

Trunk Emacs seems okay, I get a much smaller output from leaks. There
are some leaks in there that I think are worth a look though, see
attached.

I just downloaded Emacs for OS X does and it defiantly has a problem with
leaking, a freshly started Emacs already generates over a MB of output
with leaks.

-- 
Matt


[-- Attachment #2: leaks-trunk.txt --]
[-- Type: text/plain, Size: 5031 bytes --]

Process:         Emacs [21591]
Path:            /Users/matthew/Development/emacs/build/nextstep/Emacs.app/Contents/MacOS/Emacs
Load Address:    0x100000000
Identifier:      org.gnu.Emacs
Version:         Version 24.3.50 (9.0)
Code Type:       X86-64
Parent Process:  launchd [158]

Date/Time:       2014-01-14 22:46:37.788 +0000
OS Version:      Mac OS X 10.9.1 (13B42)
Report Version:  7

leaks Report Version:  2.0
Process 21591: 69442 nodes malloced for 37557 KB
Process 21591: 13 leaks for 17264 total leaked bytes.
Leak: 0x13124d800  size=16384  zone: MallocHelperZone_0x10081a000
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x000b0007 0x00000003 0x00030008 0x00000000 	................
	0x00000000 0x00000000 0x00000020 0x0000003a 	........ ...:...
	0x00000125 0x00000000 0x0d565875 0x00000001 	%.......uXV.....
	0x0010000a 0x00000004 0x00008008 0x0000002f 	............/...
	0x00000000 0x00000000 0x0000004e 0x53256e3c 	........N...<n%S
	0x00000126 0x00000000 0x0d565875 0x00000001 	&.......uXV.....
	0x0010000a 0x00000004 0x13c48008 0x4900002f 	............/..I
	...
Leak: 0x6000000b48e0  size=96  zone: DefaultMallocZone_0x10084a000   NSMenuItem  ObjC  AppKit
	0x7f6407a8 0x00007fff 0x00283250 0x00006000 	..d.....P2(..`..
	0x00652a20 0x00006000 0x005170b0 0x00000001 	 *e..`...pQ.....
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x001eda7a 0x00000001 	........z.......
	0x0996fd90 0x00000001 0x00000010 0x00000001 	................
Leak: 0x6000000b4c40  size=96  zone: DefaultMallocZone_0x10084a000   NSMenuItem  ObjC  AppKit
	0x7f6407a8 0x00007fff 0x00283250 0x00006000 	..d.....P2(..`..
	0x00656c20 0x00006000 0x005170b0 0x00000001 	 le..`...pQ.....
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x001eda7a 0x00000001 	........z.......
	0x00000000 0x00000000 0x00000010 0x00000101 	................
Leak: 0x6000000b58a0  size=96  zone: DefaultMallocZone_0x10084a000   NSMenuItem  ObjC  AppKit
	0x7f6407a8 0x00007fff 0x00283250 0x00006000 	..d.....P2(..`..
	0x7f6565c8 0x00007fff 0x7f6565c8 0x00007fff 	.ee......ee.....
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000010 0x00000301 	................
Leak: 0x6000000b65c0  size=96  zone: DefaultMallocZone_0x10084a000   NSMenuItem  ObjC  AppKit
	0x7f6407a8 0x00007fff 0x00283250 0x00006000 	..d.....P2(..`..
	0x0064f5d0 0x00006000 0x005170b0 0x00000001 	..d..`...pQ.....
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x001eda7a 0x00000001 	........z.......
	0x0996fd50 0x00000001 0x00000010 0x00000001 	P...............
Leak: 0x6000000d4ac0  size=112  zone: DefaultMallocZone_0x10084a000   NSCarbonMenuImpl  ObjC  AppKit
	0x7f63c068 0x00007fff 0x00283250 0x00006000 	h.c.....P2(..`..
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x80000000 0xffffffff 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
Leak: 0x60000023b820  size=32  zone: DefaultMallocZone_0x10084a000
	0x000b4c40 0x00006000 0x000b58a0 0x00006000 	@L...`...X...`..
	0x000b65c0 0x00006000 0x000b48e0 0x00006000 	.e...`...H...`..
Leak: 0x600000283250  size=80  zone: DefaultMallocZone_0x10084a000   EmacsMenu  ObjC  Emacs
	0x001fead0 0x00000001 0x00000000 0x00000000 	................
	0x00656c80 0x00006000 0x00653410 0x00006000 	.le..`...4e..`..
	0x00284650 0x00006000 0x00000001 0x00000000 	PF(..`..........
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00100000 0x00000000 0x00000000 0x00000000 	................
Leak: 0x600000284650  size=80  zone: DefaultMallocZone_0x10084a000   _NSMenuImpl  ObjC  AppKit
	0x7f6406b8 0x00007fff 0x000d4ac0 0x00006000 	..d......J...`..
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
Leak: 0x60000064f5d0  size=48  zone: DefaultMallocZone_0x10084a000   __NSCFString  ObjC  CoreFoundation  "Turn Off minor mode"
Leak: 0x600000653410  size=48  zone: DefaultMallocZone_0x10084a000   __NSArrayM  ObjC  CoreFoundation  item count: 4
Leak: 0x600000656c20  size=48  zone: DefaultMallocZone_0x10084a000   __NSCFString  ObjC  CoreFoundation  " from font-lock.el"
Leak: 0x600000656c80  size=48  zone: DefaultMallocZone_0x10084a000   __NSCFString  ObjC  CoreFoundation  " from font-lock.el"

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Jan Djärv]

Hi.

14 jan 2014 kl. 23:17 skrev Jonathan Payne <emacs@jpayne.net>:

> Hi Jan,
> 
> [Removed the list from this email]

Don't do that unless you are attaching something big.

> 
> Are there already-compiled versions out there? Or do I have to download the toolchain to build it myself? I am not sure I want to go there... even though I think I already have the necessary toolchain installed ...

http://emacsforosx.com/builds has nightlies.  Use the newest of those.

	Jan D.

> 
> Regarding the leaks in my previous message, each line output says "Leak: ...". So I think by definition it's a leak: there are no pointers or "maybe pointers" pointing to the block in question. If the blocks are all pointing to each other, then the leak could be that the first block is not being pointed to, but regardless they are all unreachable by anything looking like a pointer in all of the stacks and heap and static memory of the process.
> 

I forgot, you should start your Emacs like this:

% MallocStackLogging=1 .../pah/to/Emacs.app/Contents/MacOS/Emacs

and then run leaks.  It will then print a stack trace also.  This may slow Emacs down a bit.

	Jan D.

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Jan D.]

Hello.

Matthew Leach skrev 2014-01-14 23:59:
>
> Jan Djärv <jan.h.d@swipnet.se> writes:
>
>> They are not all like that, somewhere in there is the real leak.  You
>> need to save all output, and put it somewhere it can be read.  If it
>> is large, better not send it to the list.  But if you are doing this,
>> do it on a more recent version than 24.3.
>
> Trunk Emacs seems okay, I get a much smaller output from leaks. There
> are some leaks in there that I think are worth a look though, see
> attached.
>
> I just downloaded Emacs for OS X does and it defiantly has a problem with
> leaking, a freshly started Emacs already generates over a MB of output
> with leaks.
>


Can you run this again, but with MallocStackLogging set?  I.e.:

% MallocStackLogging=1 .../path/to/Emacs.app/Contents/MacOS/Emacs

And if you can, briefly describe what you did in Emacs before running
leaks.

Thanks,

	Jan D.

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Matthew Leach]

"Jan D." <jan.h.d@swipnet.se> writes:

> Can you run this again, but with MallocStackLogging set?  I.e.:
>
> % MallocStackLogging=1 .../path/to/Emacs.app/Contents/MacOS/Emacs
>
> And if you can, briefly describe what you did in Emacs before running
> leaks.

Sure, see [1].

I simply did C-h v font-lock <TAB> to bring up a completion buffer, then
added -keywords <RET> to bring up the help buffer and followed the
reference to font-lock.el. That seemed to cause the leaks.

[1]: https://gist.github.com/anonymous/8434281

-- 
Matt

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Jonathan Payne]

[-- Attachment #1: Type: text/plain, Size: 2475 bytes --]

I have downloaded the nightly and my problem is solved. Neither distnoted nor emacs is leaking anymore.

Well - ok there is apparently one leak occurring in emacs. It looks sort of like a string with a regular expression in it:

> leaks Report Version:  2.0
> Process 634: 59789 nodes malloced for 19799 KB
> Process 634: 1 leak for 16384 total leaked bytes.
> Leak: 0x100e20600  size=16384  zone: DefaultMallocZone_0x100687000
> 	0x00000000 0x50000000 0x00000000 0x50000000 	.......P.......P
> 	0x00000000 0x00000000 0x0000004b 0x00000000 	........K.......
> 	0x74696e69 0x4e7c5c79 0x7c5c4e61 0x75677261 	inity\|NaN\|argu
> 	0x746e656d 0x667c5c73 0x65736c61 0x756e7c5c 	ments\|false\|nu
> 	0x7c5c6c6c 0x3f285c74 0x7369683a 0x75727c5c 	ll\|t\(?:his\|ru
> 	0x5c295c65 0x646e757c 0x6e696665 0x295c6465 	e\)\|undefined\)
> 	0x003e5f5c 0x00000000 0x00000000 0x00000000 	\_>.............
> 	0x00000012 0x00000000 0x7079746f 0x5c295c65 	........otype\)\
> 	...


So, I think this was caused by turning on flyspell-mode. The size is 16k which just reeks of two 8k buffers associated with a pipe or something.

Thanks for all the help with this issue.

JP

On 15 Jan 2014, at 06:26, Jan Djärv <jan.h.d@swipnet.se> wrote:

> Hi.
> 
> 14 jan 2014 kl. 23:17 skrev Jonathan Payne <emacs@jpayne.net>:
> 
>> Hi Jan,
>> 
>> [Removed the list from this email]
> 
> Don't do that unless you are attaching something big.
> 
>> 
>> Are there already-compiled versions out there? Or do I have to download the toolchain to build it myself? I am not sure I want to go there... even though I think I already have the necessary toolchain installed ...
> 
> http://emacsforosx.com/builds has nightlies.  Use the newest of those.
> 
> 	Jan D.
> 
>> 
>> Regarding the leaks in my previous message, each line output says "Leak: ...". So I think by definition it's a leak: there are no pointers or "maybe pointers" pointing to the block in question. If the blocks are all pointing to each other, then the leak could be that the first block is not being pointed to, but regardless they are all unreachable by anything looking like a pointer in all of the stacks and heap and static memory of the process.
>> 
> 
> I forgot, you should start your Emacs like this:
> 
> % MallocStackLogging=1 .../pah/to/Emacs.app/Contents/MacOS/Emacs
> 
> and then run leaks.  It will then print a stack trace also.  This may slow Emacs down a bit.
> 
> 	Jan D.
> 


[-- Attachment #2: Type: text/html, Size: 6043 bytes --]

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[vbeffara]

Hi,

Just one thing I noticed, whenever the system is in "leak mode" between
distnoted and emacs: one way to trigger a CPU use spike is to change ambient
light level (say by covering the sensor with your hand). Might be useful in
reproducing / tracing whatever is happening in distnoted ...

(using GNU Emacs 24.3.50.1 from homebrew)

/v



--
View this message in context: http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310630.html
Sent from the Emacs - Bugs mailing list archive at Nabble.com.

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[canoeberry]

I had downloaded a nightly and was so happy to see that the memory leak was
gone. However, the nightly has a ton of other problems with a package that
is near and dear to my heart at the moment: mumamo.

So I have downloaded the source and compiled it and tried to patch it with
the suggestion involving a patch early in this bug report, but that has not
solved the problem. Then I remembered that I could get stack traces, which I
have done:

	Call stack: [thread 0x7fff71dce310]: | 0x1 | start | main emacs.c:1528 |
Frecursive_edit keyboard.c:844 | recursive_edit_1 keyboard.c:1148 |
internal_catch eval.c:1060 | command_loop_2 keyboard.c:1168 |
internal_condition_case eval.c:1290 | command_loop_1 keyboard.c:1459 |
read_key_sequence keyboard.c:9234 | read_char keyboard.c:2059 |
wait_reading_process_output process.c:4743 | detect_input_pending_run_timers
keyboard.c:6680 | readable_events keyboard.c:3355 | timer_check
keyboard.c:4378 | call1 eval.c:2572 | Ffuncall eval.c:2850 | funcall_lambda
eval.c:3010 | exec_byte_code bytecode.c:1096 | internal_lisp_condition_case
eval.c:1243 | eval_sub eval.c:2149 | exec_byte_code bytecode.c:900 |
Ffuncall eval.c:2759 | Fapply eval.c:2255 | Ffuncall eval.c:2850 |
funcall_lambda eval.c:3010 | exec_byte_code bytecode.c:900 | Ffuncall
eval.c:2775 | Finput_pending_p keyboard.c:6687 | gobble_input
keyboard.c:6768 | ns_read_socket nsterm.m:3424 | -[NSApplication run] |
-[NSApplication _installMemoryStatusDispatchSources] |
dispatch_source_create | _os_object_alloc_realized | class_createInstance |
calloc | malloc_zone_calloc 
Leak: 0x1040efe20  size=160  zone: DefaultMallocZone_0x100630000  
OS_dispatch_source  ObjC  libdispatch.dylib
	0x71162c20 0x00007fff 0x00000001 0x00000000 	 ,.q............
	0x89abcdef 0xffffffff 0x71164480 0x00007fff 	.........D.q....
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000000 0x00000000 0x00000000 0x00000000 	................
	0x00000001 0x00000000 0x00000175 0x00000000 	........u.......
	0x8316090c 0x00007fff 0x00804760 0x00000001 	........`G......
	0x040e2120 0x00000001 0x00000002 0x0000004c 	 !..........L...
	...

The last line of code in emacs that produces this leak is:

      if (++apploopnr != 1)
        {
          emacs_abort ();
        }
      [NSApp run];
      --apploopnr;

well it's the --apploopnr according to leaks! A little weird if you ask me.

Anyway - does anybody have any insights into this?



-----
Jonathan Payne

--
View this message in context: http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310818.html
Sent from the Emacs - Bugs mailing list archive at Nabble.com.

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Jan Djärv]

Hello.

18 jan 2014 kl. 23:52 skrev canoeberry <emacs@jpayne.net>:

> I had downloaded a nightly and was so happy to see that the memory leak was
> gone. However, the nightly has a ton of other problems with a package that
> is near and dear to my heart at the moment: mumamo.
> 
> So I have downloaded the source and compiled it and tried to patch it with
> the suggestion involving a patch early in this bug report, but that has not
> solved the problem. Then I remembered that I could get stack traces, which I
> have done:

You are better off trying to get mumamo working with trunk.  There are so many differences between 24.3 and trunk.  It is no trivial task to identify which has memory leak fixes.

> 
> 	Call stack: [thread 0x7fff71dce310]: | 0x1 | start | main emacs.c:1528 |
> Frecursive_edit keyboard.c:844 | recursive_edit_1 keyboard.c:1148 |
> internal_catch eval.c:1060 | command_loop_2 keyboard.c:1168 |
> internal_condition_case eval.c:1290 | command_loop_1 keyboard.c:1459 |
> read_key_sequence keyboard.c:9234 | read_char keyboard.c:2059 |
> wait_reading_process_output process.c:4743 | detect_input_pending_run_timers
> keyboard.c:6680 | readable_events keyboard.c:3355 | timer_check
> keyboard.c:4378 | call1 eval.c:2572 | Ffuncall eval.c:2850 | funcall_lambda
> eval.c:3010 | exec_byte_code bytecode.c:1096 | internal_lisp_condition_case
> eval.c:1243 | eval_sub eval.c:2149 | exec_byte_code bytecode.c:900 |
> Ffuncall eval.c:2759 | Fapply eval.c:2255 | Ffuncall eval.c:2850 |
> funcall_lambda eval.c:3010 | exec_byte_code bytecode.c:900 | Ffuncall
> eval.c:2775 | Finput_pending_p keyboard.c:6687 | gobble_input
> keyboard.c:6768 | ns_read_socket nsterm.m:3424 | -[NSApplication run] |
> -[NSApplication _installMemoryStatusDispatchSources] |
> dispatch_source_create | _os_object_alloc_realized | class_createInstance |
> calloc | malloc_zone_calloc 
> Leak: 0x1040efe20  size=160  zone: DefaultMallocZone_0x100630000  
> OS_dispatch_source  ObjC  libdispatch.dylib
> 	0x71162c20 0x00007fff 0x00000001 0x00000000 	 ,.q............
> 	0x89abcdef 0xffffffff 0x71164480 0x00007fff 	.........D.q....
> 	0x00000000 0x00000000 0x00000000 0x00000000 	................
> 	0x00000000 0x00000000 0x00000000 0x00000000 	................
> 	0x00000000 0x00000000 0x00000000 0x00000000 	................
> 	0x00000001 0x00000000 0x00000175 0x00000000 	........u.......
> 	0x8316090c 0x00007fff 0x00804760 0x00000001 	........`G......
> 	0x040e2120 0x00000001 0x00000002 0x0000004c 	 !..........L...
> 	...
> 
> The last line of code in emacs that produces this leak is:
> 
>      if (++apploopnr != 1)
>        {
>          emacs_abort ();
>        }
>      [NSApp run];
>      --apploopnr;
> 
> well it's the --apploopnr according to leaks! A little weird if you ask me.

The stack trace clearly states that it is in NSApp run (i.e. NSApplication run), so the trace is off by one line.  This happens often.

	Jan D.

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[canoeberry]

[-- Attachment #1: Type: text/plain, Size: 3844 bytes --]

Ironically, the leak was fixed by ... YOU!

On 19 Jan 2014, at 09:34, Jan Djärv [via Emacs] <ml-node+s1067599n310868h86@n5.nabble.com> wrote:

> Hello. 
> 
> 18 jan 2014 kl. 23:52 skrev canoeberry <[hidden email]>: 
> 
> > I had downloaded a nightly and was so happy to see that the memory leak was 
> > gone. However, the nightly has a ton of other problems with a package that 
> > is near and dear to my heart at the moment: mumamo. 
> > 
> > So I have downloaded the source and compiled it and tried to patch it with 
> > the suggestion involving a patch early in this bug report, but that has not 
> > solved the problem. Then I remembered that I could get stack traces, which I 
> > have done: 
> 
> You are better off trying to get mumamo working with trunk.  There are so many differences between 24.3 and trunk.  It is no trivial task to identify which has memory leak fixes. 
> 
> > 
> > Call stack: [thread 0x7fff71dce310]: | 0x1 | start | main emacs.c:1528 | 
> > Frecursive_edit keyboard.c:844 | recursive_edit_1 keyboard.c:1148 | 
> > internal_catch eval.c:1060 | command_loop_2 keyboard.c:1168 | 
> > internal_condition_case eval.c:1290 | command_loop_1 keyboard.c:1459 | 
> > read_key_sequence keyboard.c:9234 | read_char keyboard.c:2059 | 
> > wait_reading_process_output process.c:4743 | detect_input_pending_run_timers 
> > keyboard.c:6680 | readable_events keyboard.c:3355 | timer_check 
> > keyboard.c:4378 | call1 eval.c:2572 | Ffuncall eval.c:2850 | funcall_lambda 
> > eval.c:3010 | exec_byte_code bytecode.c:1096 | internal_lisp_condition_case 
> > eval.c:1243 | eval_sub eval.c:2149 | exec_byte_code bytecode.c:900 | 
> > Ffuncall eval.c:2759 | Fapply eval.c:2255 | Ffuncall eval.c:2850 | 
> > funcall_lambda eval.c:3010 | exec_byte_code bytecode.c:900 | Ffuncall 
> > eval.c:2775 | Finput_pending_p keyboard.c:6687 | gobble_input 
> > keyboard.c:6768 | ns_read_socket nsterm.m:3424 | -[NSApplication run] | 
> > -[NSApplication _installMemoryStatusDispatchSources] | 
> > dispatch_source_create | _os_object_alloc_realized | class_createInstance | 
> > calloc | malloc_zone_calloc 
> > Leak: 0x1040efe20  size=160  zone: DefaultMallocZone_0x100630000   
> > OS_dispatch_source  ObjC  libdispatch.dylib 
> > 0x71162c20 0x00007fff 0x00000001 0x00000000 ,.q............ 
> > 0x89abcdef 0xffffffff 0x71164480 0x00007fff .........D.q.... 
> > 0x00000000 0x00000000 0x00000000 0x00000000 ................ 
> > 0x00000000 0x00000000 0x00000000 0x00000000 ................ 
> > 0x00000000 0x00000000 0x00000000 0x00000000 ................ 
> > 0x00000001 0x00000000 0x00000175 0x00000000 ........u....... 
> > 0x8316090c 0x00007fff 0x00804760 0x00000001 ........`G...... 
> > 0x040e2120 0x00000001 0x00000002 0x0000004c !..........L... 
> > ... 
> > 
> > The last line of code in emacs that produces this leak is: 
> > 
> >      if (++apploopnr != 1) 
> >        { 
> >          emacs_abort (); 
> >        } 
> >      [NSApp run]; 
> >      --apploopnr; 
> > 
> > well it's the --apploopnr according to leaks! A little weird if you ask me.
> 
> The stack trace clearly states that it is in NSApp run (i.e. NSApplication run), so the trace is off by one line.  This happens often. 
> 
>         Jan D. 
> 
> 
> 
> 
> 
> 
> If you reply to this email, your message will be added to the discussion below:
> http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310868.html
> To unsubscribe from bug#15946: 24.3; Mac OS X, Mavericks, distnoted process, click here.
> NAML





-----
Jonathan Payne

--
View this message in context: http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310871.html
Sent from the Emacs - Bugs mailing list archive at Nabble.com.

[-- Attachment #2: Type: text/html, Size: 6095 bytes --]

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[SB]

After reading reports in this thread that it was fixed in the trunk I
went and applied the aforementioned inline patch to the current trunk
and indeed the distnoted issue seems fixed. Whereas with my patch to
the inline patch the problem seemed a little more tolerable and still
required occasionally killing distnoted (and Emacs.app crashed
although less frequently).

If there is a single commit that can make distnoted behave, it may be
in the interest of others using Mavericks to have this incorporated
into an official point release and close this bug (the distnoted bug
for those affected can be quite severe, locking up the entire machine
in my case). Mavericks is inevitable for many mac users and some
people may not be ready to transition to 24.4 for whatever reasons.

The modified inline patch (only relevant if you use Japanese or some
other language for writing text in Emacs and want seamless language
switching while using various keys/commands) for the current trunk:

https://gist.github.com/anonymous/8517045

On Sun, Jan 19, 2014 at 7:49 PM, canoeberry <emacs@jpayne.net> wrote:
> Ironically, the leak was fixed by ... YOU!
>
> On 19 Jan 2014, at 09:34, Jan Djärv [via Emacs] <[hidden email]> wrote:
>
> Hello.
>
> 18 jan 2014 kl. 23:52 skrev canoeberry <<a
> href="x-msg://11/user/SendEmail.jtp?type=node&amp;node=310868&amp;i=0"
> target="_top" rel="nofollow" link="external">[hidden email]>:
>
>> I had downloaded a nightly and was so happy to see that the memory leak
>> was
>> gone. However, the nightly has a ton of other problems with a package that
>> is near and dear to my heart at the moment: mumamo.
>>
>> So I have downloaded the source and compiled it and tried to patch it with
>> the suggestion involving a patch early in this bug report, but that has
>> not
>> solved the problem. Then I remembered that I could get stack traces, which
>> I
>> have done:
>
> You are better off trying to get mumamo working with trunk.  There are so
> many differences between 24.3 and trunk.  It is no trivial task to identify
> which has memory leak fixes.
>
>>
>> Call stack: [thread 0x7fff71dce310]: | 0x1 | start | main emacs.c:1528 |
>> Frecursive_edit keyboard.c:844 | recursive_edit_1 keyboard.c:1148 |
>> internal_catch eval.c:1060 | command_loop_2 keyboard.c:1168 |
>> internal_condition_case eval.c:1290 | command_loop_1 keyboard.c:1459 |
>> read_key_sequence keyboard.c:9234 | read_char keyboard.c:2059 |
>> wait_reading_process_output process.c:4743 |
>> detect_input_pending_run_timers
>> keyboard.c:6680 | readable_events keyboard.c:3355 | timer_check
>> keyboard.c:4378 | call1 eval.c:2572 | Ffuncall eval.c:2850 |
>> funcall_lambda
>> eval.c:3010 | exec_byte_code bytecode.c:1096 |
>> internal_lisp_condition_case
>> eval.c:1243 | eval_sub eval.c:2149 | exec_byte_code bytecode.c:900 |
>> Ffuncall eval.c:2759 | Fapply eval.c:2255 | Ffuncall eval.c:2850 |
>> funcall_lambda eval.c:3010 | exec_byte_code bytecode.c:900 | Ffuncall
>> eval.c:2775 | Finput_pending_p keyboard.c:6687 | gobble_input
>> keyboard.c:6768 | ns_read_socket nsterm.m:3424 | -[NSApplication run] |
>> -[NSApplication _installMemoryStatusDispatchSources] |
>> dispatch_source_create | _os_object_alloc_realized | class_createInstance
>> |
>> calloc | malloc_zone_calloc
>> Leak: 0x1040efe20  size=160  zone: DefaultMallocZone_0x100630000
>> OS_dispatch_source  ObjC  libdispatch.dylib
>> 0x71162c20 0x00007fff 0x00000001 0x00000000 ,.q............
>> 0x89abcdef 0xffffffff 0x71164480 0x00007fff .........D.q....
>> 0x00000000 0x00000000 0x00000000 0x00000000 ................
>> 0x00000000 0x00000000 0x00000000 0x00000000 ................
>> 0x00000000 0x00000000 0x00000000 0x00000000 ................
>> 0x00000001 0x00000000 0x00000175 0x00000000 ........u.......
>> 0x8316090c 0x00007fff 0x00804760 0x00000001 ........`G......
>> 0x040e2120 0x00000001 0x00000002 0x0000004c !..........L...
>> ...
>>
>> The last line of code in emacs that produces this leak is:
>>
>>      if (++apploopnr != 1)
>>        {
>>          emacs_abort ();
>>        }
>>      [NSApp run];
>>      --apploopnr;
>>
>> well it's the --apploopnr according to leaks! A little weird if you ask
>> me.
> The stack trace clearly states that it is in NSApp run (i.e. NSApplication
> run), so the trace is off by one line.  This happens often.
>
>         Jan D.
>
>
>
>
>
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310868.html
> To unsubscribe from bug#15946: 24.3; Mac OS X, Mavericks, distnoted process,
> click here.
> NAML
>
>
> Jonathan Payne
>
> ________________________________
> View this message in context: Re: bug#15946: 24.3; Mac OS X, Mavericks,
> distnoted process
> Sent from the Emacs - Bugs mailing list archive at Nabble.com.

bug#15946: a patch against emacs 24.3.1 for fixing the distnoted memory leak

[Jonathan Payne]

This is FYI for people who do not want to wait for emacs 24.4 and cannot use one of the current nightlies to get around the Mavericks distnoted issue.

I am not a Mac programmer unfortunately, so take this with a grain of salt. My analysis of the problem is this:

1) When Mavericks came out a memory leak was discovered and fixed by Jan D. who is a frequent contributor to ns (Next Step) emacs for Mac OS X.

2) His fix, I think, was to avoid calling Application.run 20 times / second by introducing a variable called "shouldKeepRunning" into the Application.run method.

3) Repeatedly calling Application.run was causing a 160 byte leak each time inside emacs. But I think it was also causing a problem with distnoted, and I am speculating (knowing NOTHING ABOUT IT) that each call to run() was causing some sort of registration with distnoted. That registration, I assume, creates some sort of context in distnoted that is substantially larger than 160 bytes. So the leak in emacs was relatively slow, the one in distnoted was much larger.

This would also explain why distnoted would get slower as time goes by because it has hundreds of thousands of registrations, all for emacs, which I assume it was still trying to deliver notifications to, but really, I have no idea what I am talking about.

So, I applied part of Jan D's patch to emacs-24.3 that addressed this particular problem, and presto-chango, the leak is gone, distnoted is happy, and all my current elisp packages work because it's still emacs-24.3 and not emacs-24.4 which I assume is what's being created in the trunk today.

So - should I post this particular patch some place? Are other people interested in having this patch?

JP

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Jan Djärv]

Hello.

20 jan 2014 kl. 10:53 skrev SB <richstyles@gmail.com>:

> After reading reports in this thread that it was fixed in the trunk I
> went and applied the aforementioned inline patch to the current trunk
> and indeed the distnoted issue seems fixed. Whereas with my patch to
> the inline patch the problem seemed a little more tolerable and still
> required occasionally killing distnoted (and Emacs.app crashed
> although less frequently).
> 
> If there is a single commit that can make distnoted behave, it may be
> in the interest of others using Mavericks to have this incorporated
> into an official point release and close this bug (the distnoted bug
> for those affected can be quite severe, locking up the entire machine
> in my case). Mavericks is inevitable for many mac users and some
> people may not be ready to transition to 24.4 for whatever reasons.

There will not be any release for the 24.3 branch.  24.4 is next.

> 
> The modified inline patch (only relevant if you use Japanese or some
> other language for writing text in Emacs and want seamless language
> switching while using various keys/commands) for the current trunk:
> 
> https://gist.github.com/anonymous/8517045

The patch is way too big to be incorporated into Emacs in the current feature freeze.

	Jan D.

> 
> On Sun, Jan 19, 2014 at 7:49 PM, canoeberry <emacs@jpayne.net> wrote:
>> Ironically, the leak was fixed by ... YOU!
>> 
>> On 19 Jan 2014, at 09:34, Jan Djärv [via Emacs] <[hidden email]> wrote:
>> 
>> Hello.
>> 
>> 18 jan 2014 kl. 23:52 skrev canoeberry <<a
>> href="x-msg://11/user/SendEmail.jtp?type=node&amp;node=310868&amp;i=0"
>> target="_top" rel="nofollow" link="external">[hidden email]>:
>> 
>>> I had downloaded a nightly and was so happy to see that the memory leak
>>> was
>>> gone. However, the nightly has a ton of other problems with a package that
>>> is near and dear to my heart at the moment: mumamo.
>>> 
>>> So I have downloaded the source and compiled it and tried to patch it with
>>> the suggestion involving a patch early in this bug report, but that has
>>> not
>>> solved the problem. Then I remembered that I could get stack traces, which
>>> I
>>> have done:
>> 
>> You are better off trying to get mumamo working with trunk.  There are so
>> many differences between 24.3 and trunk.  It is no trivial task to identify
>> which has memory leak fixes.
>> 
>>> 
>>> Call stack: [thread 0x7fff71dce310]: | 0x1 | start | main emacs.c:1528 |
>>> Frecursive_edit keyboard.c:844 | recursive_edit_1 keyboard.c:1148 |
>>> internal_catch eval.c:1060 | command_loop_2 keyboard.c:1168 |
>>> internal_condition_case eval.c:1290 | command_loop_1 keyboard.c:1459 |
>>> read_key_sequence keyboard.c:9234 | read_char keyboard.c:2059 |
>>> wait_reading_process_output process.c:4743 |
>>> detect_input_pending_run_timers
>>> keyboard.c:6680 | readable_events keyboard.c:3355 | timer_check
>>> keyboard.c:4378 | call1 eval.c:2572 | Ffuncall eval.c:2850 |
>>> funcall_lambda
>>> eval.c:3010 | exec_byte_code bytecode.c:1096 |
>>> internal_lisp_condition_case
>>> eval.c:1243 | eval_sub eval.c:2149 | exec_byte_code bytecode.c:900 |
>>> Ffuncall eval.c:2759 | Fapply eval.c:2255 | Ffuncall eval.c:2850 |
>>> funcall_lambda eval.c:3010 | exec_byte_code bytecode.c:900 | Ffuncall
>>> eval.c:2775 | Finput_pending_p keyboard.c:6687 | gobble_input
>>> keyboard.c:6768 | ns_read_socket nsterm.m:3424 | -[NSApplication run] |
>>> -[NSApplication _installMemoryStatusDispatchSources] |
>>> dispatch_source_create | _os_object_alloc_realized | class_createInstance
>>> |
>>> calloc | malloc_zone_calloc
>>> Leak: 0x1040efe20  size=160  zone: DefaultMallocZone_0x100630000
>>> OS_dispatch_source  ObjC  libdispatch.dylib
>>> 0x71162c20 0x00007fff 0x00000001 0x00000000 ,.q............
>>> 0x89abcdef 0xffffffff 0x71164480 0x00007fff .........D.q....
>>> 0x00000000 0x00000000 0x00000000 0x00000000 ................
>>> 0x00000000 0x00000000 0x00000000 0x00000000 ................
>>> 0x00000000 0x00000000 0x00000000 0x00000000 ................
>>> 0x00000001 0x00000000 0x00000175 0x00000000 ........u.......
>>> 0x8316090c 0x00007fff 0x00804760 0x00000001 ........`G......
>>> 0x040e2120 0x00000001 0x00000002 0x0000004c !..........L...
>>> ...
>>> 
>>> The last line of code in emacs that produces this leak is:
>>> 
>>>     if (++apploopnr != 1)
>>>       {
>>>         emacs_abort ();
>>>       }
>>>     [NSApp run];
>>>     --apploopnr;
>>> 
>>> well it's the --apploopnr according to leaks! A little weird if you ask
>>> me.
>> The stack trace clearly states that it is in NSApp run (i.e. NSApplication
>> run), so the trace is off by one line.  This happens often.
>> 
>>        Jan D.
>> 
>> 
>> 
>> 
>> 
>> 
>> ________________________________
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://emacs.1067599.n5.nabble.com/bug-15946-24-3-Mac-OS-X-Mavericks-distnoted-process-tp303500p310868.html
>> To unsubscribe from bug#15946: 24.3; Mac OS X, Mavericks, distnoted process,
>> click here.
>> NAML
>> 
>> 
>> Jonathan Payne
>> 
>> ________________________________
>> View this message in context: Re: bug#15946: 24.3; Mac OS X, Mavericks,
>> distnoted process
>> Sent from the Emacs - Bugs mailing list archive at Nabble.com.
> 
> 

bug#15946: a patch against emacs 24.3.1 for fixing the distnoted memory leak

[Jonathan Payne]

This is FYI for people who do not want to wait for emacs 24.4 and cannot use one of the current nightlies to get around the Mavericks distnoted issue.

I am not a Mac programmer unfortunately, so take this with a grain of salt. My analysis of the problem is this:

1) When Mavericks came out a memory leak was discovered and fixed by Jan D. who is a frequent contributor to ns (Next Step) emacs for Mac OS X.

2) His fix, I think, was to avoid calling Application.run 20 times / second by introducing a variable called "shouldKeepRunning" into the Application.run method.

3) Repeatedly calling Application.run was causing a 160 byte leak each time inside emacs. But I think it was also causing a problem with distnoted, and I am speculating (knowing NOTHING ABOUT IT) that each call to run() was causing some sort of registration with distnoted. That registration, I assume, creates some sort of context in distnoted that is substantially larger than 160 bytes. So the leak in emacs was relatively slow, the one in distnoted was much larger.

This would also explain why distnoted would get slower as time goes by because it has hundreds of thousands of registrations, all for emacs, which I assume it was still trying to deliver notifications to, but really, I have no idea what I am talking about.

So, I applied part of Jan D's patch to emacs-24.3 that addressed this particular problem, and presto-chango, the leak is gone, distnoted is happy, and all my current elisp packages work because it's still emacs-24.3 and not emacs-24.4 which I assume is what's being created in the trunk today.

So - should I post this particular patch some place? Are other people interested in having this patch?

JP

bug#15946: a patch against emacs 24.3.1 for fixing the distnoted memory leak

[SB]

Great detective work. I'd like to see the patch! ;)

Even if it's superseded by the next release (24.4), some people might
be stuck on 24.3 for whatever reason and right now this bug thread is
the best reference for information.

In the mean time, some more experienced people can look at the patch
and further isolate the relevant changes for a tighter patch or
confirm your work. People with the bug can build with your patch and
confirm whether this bug is fixed by it too.

On Mon, Jan 20, 2014 at 9:01 PM, Jonathan Payne <emacs@jpayne.net> wrote:
> This is FYI for people who do not want to wait for emacs 24.4 and cannot use one of the current nightlies to get around the Mavericks distnoted issue.
>
> I am not a Mac programmer unfortunately, so take this with a grain of salt. My analysis of the problem is this:
>
> 1) When Mavericks came out a memory leak was discovered and fixed by Jan D. who is a frequent contributor to ns (Next Step) emacs for Mac OS X.
>
> 2) His fix, I think, was to avoid calling Application.run 20 times / second by introducing a variable called "shouldKeepRunning" into the Application.run method.
>
> 3) Repeatedly calling Application.run was causing a 160 byte leak each time inside emacs. But I think it was also causing a problem with distnoted, and I am speculating (knowing NOTHING ABOUT IT) that each call to run() was causing some sort of registration with distnoted. That registration, I assume, creates some sort of context in distnoted that is substantially larger than 160 bytes. So the leak in emacs was relatively slow, the one in distnoted was much larger.
>
> This would also explain why distnoted would get slower as time goes by because it has hundreds of thousands of registrations, all for emacs, which I assume it was still trying to deliver notifications to, but really, I have no idea what I am talking about.
>
> So, I applied part of Jan D's patch to emacs-24.3 that addressed this particular problem, and presto-chango, the leak is gone, distnoted is happy, and all my current elisp packages work because it's still emacs-24.3 and not emacs-24.4 which I assume is what's being created in the trunk today.
>
> So - should I post this particular patch some place? Are other people interested in having this patch?
>
> JP
>
>
>
>

Re: bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[tony]

On Monday, 25 November 2013 22:27:43 UTC, Marc Feeley  wrote:
> Just want to add that I have the same problem.  After quitting emacs
> 
> the distnoted process quiets  down to an acceptable level.

I upgraded to 24.4 to try to solve this problem but I've found recent nightly builds from http://emacsformacosx.com/, for example Emacs-2014-05-01_01-33-36-117037-universal-10.6.8.dmg, display the same memory leak behaviour (I'm on OS X 10.9.2). 

An older build of 24.4 from January (Emacs-2014-01-25_01-34-03-116153-universal-10.6.8.dmg) seems fine though.

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Lars Ingebrigtsen]

Jan Djärv <jan.h.d@swipnet.se> writes:

> Hello.
>
> 20 jan 2014 kl. 10:53 skrev SB <richstyles@gmail.com>:
>
>> After reading reports in this thread that it was fixed in the trunk I
>> went and applied the aforementioned inline patch to the current trunk
>> and indeed the distnoted issue seems fixed. Whereas with my patch to
>> the inline patch the problem seemed a little more tolerable and still
>> required occasionally killing distnoted (and Emacs.app crashed
>> although less frequently).
>> 
>> If there is a single commit that can make distnoted behave, it may be
>> in the interest of others using Mavericks to have this incorporated
>> into an official point release and close this bug (the distnoted bug
>> for those affected can be quite severe, locking up the entire machine
>> in my case). Mavericks is inevitable for many mac users and some
>> people may not be ready to transition to 24.4 for whatever reasons.
>
> There will not be any release for the 24.3 branch.  24.4 is next.
>
>> 
>> The modified inline patch (only relevant if you use Japanese or some
>> other language for writing text in Emacs and want seamless language
>> switching while using various keys/commands) for the current trunk:
>> 
>> https://gist.github.com/anonymous/8517045
>
> The patch is way too big to be incorporated into Emacs in the current feature freeze.

Is this still a problem on Mavericks with Emacs 25?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#15946: 24.3; Mac OS X, Mavericks, distnoted process

[Alan Third]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Is this still a problem on Mavericks with Emacs 25?

No response in 29 weeks, I think we can close this one.
-- 
Alan Third

bug#15946: closed (Re: bug#15946: 24.3; Mac OS X, Mavericks, distnoted process)

[Donald Tillman]

[-- Attachment #1: Type: text/plain, Size: 5345 bytes --]

Yes, this was fixed long ago.  I've been using the builds from emacsformacosx.com and the problem has not recurred.

And thanks, 'great job, it's appreciated.

  -- Don

--
Don Tillman
Palo Alto, California


> On Jul 17, 2016, at 1:58 PM, GNU bug Tracking System <help-debbugs@gnu.org> wrote:
> 
> Your bug report
> 
> #15946: 24.3; Mac OS X, Mavericks, distnoted process
> 
> which was filed against the emacs package, has been closed.
> 
> The explanation is attached below, along with your original report.
> If you require more details, please reply to 15946@debbugs.gnu.org.
> 
> -- 
> 15946: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=15946
> GNU Bug Tracking System
> Contact help-debbugs@gnu.org with problems
> 
> From: Alan Third <alan@idiocy.org>
> Subject: Re: bug#15946: 24.3; Mac OS X, Mavericks, distnoted process
> Date: July 17, 2016 at 1:57:34 PM PDT
> To: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: SB <richstyles@gmail.com>, Jan Djärv <jan.h.d@swipnet.se>, 15946-done@debbugs.gnu.org
> 
> 
> Lars Ingebrigtsen <larsi@gnus.org> writes:
> 
>> Is this still a problem on Mavericks with Emacs 25?
> 
> No response in 29 weeks, I think we can close this one.
> -- 
> Alan Third
> 
> 
> 
> 
> From: Donald Tillman <don@till.com>
> Subject: 24.3; Mac OS X, Mavericks, distnoted process
> Date: November 21, 2013 at 10:18:09 AM PST
> To: bug-gnu-emacs@gnu.org
> 
> 
> --text follows this line--
> This bug report will be sent to the Bug-GNU-Emacs mailing list
> and the GNU bug tracker at debbugs.gnu.org.  Please check that
> the From: line contains a valid email address.  After a delay of up
> to one day, you should receive an acknowledgment at that address.
> 
> Please write in English if possible, as the Emacs maintainers
> usually do not have translators for other languages.
> 
> Please describe exactly what actions triggered the bug, and
> the precise symptoms of the bug.  If you can, give a recipe
> starting from `emacs -Q':
> 
> ----
> 
> Hi!
> 
> I use Emacs on Mac OS X, Mavericks, Intel MacBook Pro, downloaded from emacsformacosx.com.
> 
> Running Emacs, a process named "distnoted" starts around 1% or 2% of the CPU, and after a while, slowly works its way up to 50% to 100% of the CPU.  Yoiks!  Actually there appear to be 3 distnoted processes, but only one eats up the CPU.
> 
> Quitting Emacs brings distnoted down to under 1% within seconds. 
> 
>  -- Don
> 
> 
> 
> If Emacs crashed, and you have the Emacs process in the gdb debugger,
> please include the output from the following gdb commands:
>    `bt full' and `xbacktrace'.
> For information about debugging Emacs, please read the file
> /Applications/Emacs.app/Contents/Resources/etc/DEBUG.
> 
> 
> In GNU Emacs 24.3.1 (x86_64-apple-darwin, NS apple-appkit-1038.36)
> of 2013-03-12 on bob.porkrind.org
> Windowing system distributor `Apple', version 10.3.1265
> Configured using:
> `configure '--host=x86_64-apple-darwin' '--build=i686-apple-darwin'
> '--with-ns' 'build_alias=i686-apple-darwin'
> 'host_alias=x86_64-apple-darwin' 'CC=gcc -mmacosx-version-min=10.7
> -isystem
> /Users/david/Xcode-10.7_4.5.2/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/usr/include/
> -F/Users/david/Xcode-10.7_4.5.2/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks''
> 
> Important settings:
>  locale-coding-system: nil
>  default enable-multibyte-characters: t
> 
> Major mode: Fundamental
> 
> Minor modes in effect:
>  tooltip-mode: t
>  mouse-wheel-mode: t
>  tool-bar-mode: t
>  menu-bar-mode: t
>  file-name-shadow-mode: t
>  global-font-lock-mode: t
>  blink-cursor-mode: t
>  auto-composition-mode: t
>  auto-encryption-mode: t
>  auto-compression-mode: t
>  buffer-read-only: t
>  line-number-mode: t
>  transient-mark-mode: t
> 
> Recent input:
> <help-echo> M-x r e p o r t SPC e m a <tab> <retur
> n>
> 
> Recent messages:
> For information about GNU Emacs and the GNU system, type C-h C-a.
> 
> Load-path shadows:
> None found.
> 
> Features:
> (shadow sort gnus-util mail-extr warnings emacsbug message format-spec
> rfc822 mml easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse
> rfc2231 mailabbrev gmm-utils mailheader sendmail rainbow-mode-autoloads
> svg-clock-autoloads package rmail rfc2047 rfc2045 ietf-drums mm-util
> mail-prsvr mail-utils time-date tooltip ediff-hook vc-hooks
> lisp-float-type mwheel ns-win tool-bar dnd fontset image regexp-opt
> fringe tabulated-list newcomment lisp-mode register page menu-bar
> rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
> facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
> tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
> czech european ethiopic indian cyrillic chinese case-table epa-hook
> jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
> cus-face macroexp files text-properties overlay sha1 md5 base64 format
> env code-pages mule custom widget hashtable-print-readable backquote
> make-network-process ns multi-tty emacs)
> 
> 
> --
> Don Tillman
> Palo Alto, California
> don@till.com
> http://www.till.com
> 
> 
> 
> 
> 
> 
> 
> 




[-- Attachment #2: Type: text/html, Size: 13364 bytes --]