bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Achim Gratz]

[-- Attachment #1: Type: text/plain, Size: 2249 bytes --]

In GNU Emacs 25.0.95.1 (x86_64-suse-linux-gnu, GTK+ Version 3.20.6)
 of 2016-07-17 built on Gertrud
Repository revision: 9356fe239281ca15e12a4e5e93d7d00c56df2626
Windowing system distributor 'The X.Org Foundation', version 11.0.11803000
System Description:	openSUSE Tumbleweed (20160710) (x86_64)

Configured using:
 'configure -with-x --with-sound --with-xpm --with-jpeg --with-tiff
 --with-gif --with-png --with-rsvg --with-dbus --without-gpm
 --with-x-toolkit=gtk3 --with-file-notification=inotify
 --x-includes=/usr/include --x-libraries=/usr/lib64 --with-xft
 --with-libotf --with-m17n-flt --with-pop --without-hesiod
 --with-kerberos --with-kerberos5 --with-xim --with-wide-int
 --enable-autodepend 'CFLAGS=-fomit-frame-pointer -fmessage-length=0
 -grecord-gcc-switches -fstack-protector -O2 -Wall -D_FORTIFY_SOURCE=2
 -funwind-tables -fasynchronous-unwind-tables -g -D_GNU_SOURCE
 -std=gnu99 -pipe -Wno-pointer-sign -Wno-unused-variable
 -Wno-unused-label -Wno-unprototyped-calls -fno-optimize-sibling-calls
 -DSYSTEM_PURESIZE_EXTRA=55000 -DSITELOAD_PURESIZE_EXTRA=10000 '
 'LDFLAGS=-Wl,-O2 -Wl,--hash-size=65521'
 '--enable-locallisppath=${datadir}/emacs/${version}/site-lisp:${datadir}/emacs/site-lisp:/usr/share/emacs/site-lisp''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND DBUS GCONF GSETTINGS NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
  value of $LC_MESSAGES: POSIX
  value of $LANG: de_DE.UTF-8
  value of $XMODIFIERS: @im=local
  locale-coding-system: utf-8-unix

Major mode: Text

Minor modes in effect:
  show-paren-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t


The file to reproduce this with contained sensitive information, but
fortunately the replacement of all information with something
inconspicous still produces the same crash (and makes it a lot smaller
after compression):


[-- Attachment #2: crash.txt.xz --]
[-- Type: application/x-xz, Size: 33044 bytes --]

[-- Attachment #3: Type: text/plain, Size: 1384 bytes --]


It is a moderately large (~12500 lines, 585kiB) text
file with UNIX line endings (\n).  Mixed in with the normal text are
protocol logs that have \r\n line endings.  I have confirmed that
replacement of those line endings will stop the bug from happening.  The
bug happens with all 25.x versions I have including master, the earliest
one I can still test is from February 27th this year.

To reproduce, set a mark (C-Space) somewhere in the file.  Then
extend/shrink the selection region by doing Shift-PgUp / Shift-PgDwn,
going back and forth several times.  Once you've crossed a portion of
the file that has the "foreign" line endings, as soon as you take your
finger off the Shift key Emacs becomes unresponsive (the cursor
vanishes).  Several seconds later it starts allocating memory in an
apparently exponential fashion until it has exhausted both physical and
swap, while maxing out one CPU core.  The process must be killed.

Trying the same recipe with Emacs 24.5 yields a short phase of
unresponsiveness with the cursor vanished, then it returns with an error
message "Timed out waiting for property-notify event".

I consider this a blocker for the upcoming 25.1 release.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Sun, 17 Jul 2016 14:39:33 +0200
> 
> To reproduce, set a mark (C-Space) somewhere in the file.  Then
> extend/shrink the selection region by doing Shift-PgUp / Shift-PgDwn,
> going back and forth several times.  Once you've crossed a portion of
> the file that has the "foreign" line endings, as soon as you take your
> finger off the Shift key Emacs becomes unresponsive (the cursor
> vanishes).  Several seconds later it starts allocating memory in an
> apparently exponential fashion until it has exhausted both physical and
> swap, while maxing out one CPU core.  The process must be killed.
> 
> Trying the same recipe with Emacs 24.5 yields a short phase of
> unresponsiveness with the cursor vanished, then it returns with an error
> message "Timed out waiting for property-notify event".

That error message is related to X selections.  Does the problem go
away if you disable putting the selected text into the X selections?
If it does, do you have some agent running that manages X selections?

Also, what is your value of selection-coding-system?

Thanks.

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Achim Gratz]

Eli Zaretskii writes:
>> Trying the same recipe with Emacs 24.5 yields a short phase of
>> unresponsiveness with the cursor vanished, then it returns with an error
>> message "Timed out waiting for property-notify event".
>
> That error message is related to X selections.  Does the problem go
> away if you disable putting the selected text into the X selections?

How do I do that?

> If it does, do you have some agent running that manages X selections?

Not that I know of, but maybe Plasma5 has something built-in that does that.

> Also, what is your value of selection-coding-system?

nil

Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Sun, 17 Jul 2016 16:46:19 +0200
> 
> > That error message is related to X selections.  Does the problem go
> > away if you disable putting the selected text into the X selections?
> 
> How do I do that?

Set select-active-regions to nil.

> > If it does, do you have some agent running that manages X selections?
> 
> Not that I know of, but maybe Plasma5 has something built-in that does that.

If the above solves the issue, try disabling Plasma5 without disabling
select-active-regions.

Thanks.

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Achim Gratz]

Eli Zaretskii writes:
>> How do I do that?
>
> Set select-active-regions to nil.

Works.

>> > If it does, do you have some agent running that manages X selections?
>> 
>> Not that I know of, but maybe Plasma5 has something built-in that does that.
>
> If the above solves the issue, try disabling Plasma5 without disabling
> select-active-regions.

Uh?  That runs my desktop, there's no way of disabling it.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Sun, 17 Jul 2016 18:16:42 +0200
> 
> > Set select-active-regions to nil.
> 
> Works.

OK, so the problem, whatever it is, is directly related to putting
text into the primary selection.

> >> Not that I know of, but maybe Plasma5 has something built-in that does that.
> >
> > If the above solves the issue, try disabling Plasma5 without disabling
> > select-active-regions.
> 
> Uh?  That runs my desktop, there's no way of disabling it.

Ah, sorry, my misunderstanding, I thought Plasma5 was only handling
the selections.

I guess the next step is to see which memory allocation in xselect.c
goes awry, and why.  Unfortunately, I don't have access to a system
where this could be reproduced.

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Achim Gratz]

Eli Zaretskii writes:
>> Not that I know of, but maybe Plasma5 has something built-in that does that.
>
> If the above solves the issue, try disabling Plasma5 without disabling
> select-active-regions.

After a bit of digging, it turns out that "klipper" (which I've chucked
out long ago due to it's uselessness to me) has indeed be replaced by
something integrated directly into Plasma5 (at least I can't find any
conspicously named process)…  So, hunting down the configuration dialog
for the Clipboard and setting the option to "Ignore Selection" seems to
fix the problem.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Sun, 17 Jul 2016 18:43:17 +0200
> 
> After a bit of digging, it turns out that "klipper" (which I've chucked
> out long ago due to it's uselessness to me) has indeed be replaced by
> something integrated directly into Plasma5 (at least I can't find any
> conspicously named process)…  So, hunting down the configuration dialog
> for the Clipboard and setting the option to "Ignore Selection" seems to
> fix the problem.

Thanks.  But does this mean you cannot put active regions into the
primary selection?  Or is that functionality intact after setting the
above option?

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Achim Gratz]

Eli Zaretskii writes:
> Thanks.  But does this mean you cannot put active regions into the
> primary selection?  Or is that functionality intact after setting the
> above option?

It doesn't seem to change anything I care about so far.  In fact it
brings back the behaviour that the clipboard only contains stuff that
gets explicitly copied instead of just about anything that gets
selected.  You'd have to ask someone else how this is supposed to work
in the other mode, I'm afraid.  There are two more options when "Ignore
Selection" is off: one is selected by default ("Text selections only")
and the other is off ("Synchronize contents of selection and
clipboard").


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Sun, 17 Jul 2016 19:58:34 +0200
> 
> Eli Zaretskii writes:
> > Thanks.  But does this mean you cannot put active regions into the
> > primary selection?  Or is that functionality intact after setting the
> > above option?
> 
> It doesn't seem to change anything I care about so far.  In fact it
> brings back the behaviour that the clipboard only contains stuff that
> gets explicitly copied instead of just about anything that gets
> selected.  You'd have to ask someone else how this is supposed to work
> in the other mode, I'm afraid.  There are two more options when "Ignore
> Selection" is off: one is selected by default ("Text selections only")
> and the other is off ("Synchronize contents of selection and
> clipboard").

I see.  Frankly, I feel a bit uneasy that Emacs somehow runs out of
memory in any scenario, but other than that, would you consider
describing this solution in PROBLEMS as good enough to go ahead with
releasing Emacs 25.1?  Perhaps even closing this bug?

Thanks.

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Achim Gratz]

Eli Zaretskii writes:
> I see.  Frankly, I feel a bit uneasy that Emacs somehow runs out of
> memory in any scenario, but other than that, would you consider
> describing this solution in PROBLEMS as good enough to go ahead with
> releasing Emacs 25.1?

I'll make a proposal.

> Perhaps even closing this bug?

Emacs shouldn't bring the machine to a halt in any case, so unless we
can show it's a fault in Qt5/Plasma5, I'd suggest to keep it open.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

[snip]
> would you consider
> describing this solution in PROBLEMS as good enough to go ahead with
> releasing Emacs 25.1?

Yes, please. As a KDE user, I'll like to see this problem described
along the corresponding workaround.

For some reason I also disabled Klipper long time ago and I was unaware
that it was replaced. Looking at the KDE Settings panel it is not
obvious at all how to disable the replacement.

Thanks.

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Stephen Berman]

On Sun, 17 Jul 2016 20:51:33 +0200 Achim Gratz <Stromeko@nexgo.de> wrote:

> Eli Zaretskii writes:
>> I see.  Frankly, I feel a bit uneasy that Emacs somehow runs out of
>> memory in any scenario, but other than that, would you consider
>> describing this solution in PROBLEMS as good enough to go ahead with
>> releasing Emacs 25.1?
>
> I'll make a proposal.
>
>> Perhaps even closing this bug?
>
> Emacs shouldn't bring the machine to a halt in any case, so unless we
> can show it's a fault in Qt5/Plasma5, I'd suggest to keep it open.

FWIW, I'm running Emacs under KDE4/Qt4/Plasma4 (openSUSE 13.2, with
Klipper running) and I could not reproduce the problems you reported
with the file you posted and the recipe you gave: no cursor vanishing,
no rampant memory allocation with Emacs 25.x, and no timeout with 24.5.

Steve Berman

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Date: Sun, 17 Jul 2016 22:07:05 +0200
> Cc: 24013@debbugs.gnu.org
> 
> FWIW, I'm running Emacs under KDE4/Qt4/Plasma4 (openSUSE 13.2, with
> Klipper running) and I could not reproduce the problems you reported
> with the file you posted and the recipe you gave: no cursor vanishing,
> no rampant memory allocation with Emacs 25.x, and no timeout with 24.5.

Maybe it's a matter of specific versions of these packages?

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Achim Gratz]

Stephen Berman writes:
> FWIW, I'm running Emacs under KDE4/Qt4/Plasma4 (openSUSE 13.2, with
> Klipper running) and I could not reproduce the problems you reported
> with the file you posted and the recipe you gave: no cursor vanishing,
> no rampant memory allocation with Emacs 25.x, and no timeout with 24.5.

I can't test KDE4 anymore and as I've said I never used klipper, but I
seem to remember that klipper also had an option to consider all
selections or just something that was copied or cut as something to deal
with.  Only in the case that klipper deals with selections the reported
problem might occur.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Stephen Berman]

On Mon, 18 Jul 2016 20:36:28 +0200 Achim Gratz <Stromeko@nexgo.de> wrote:

> Stephen Berman writes:
>> FWIW, I'm running Emacs under KDE4/Qt4/Plasma4 (openSUSE 13.2, with
>> Klipper running) and I could not reproduce the problems you reported
>> with the file you posted and the recipe you gave: no cursor vanishing,
>> no rampant memory allocation with Emacs 25.x, and no timeout with 24.5.
>
> I can't test KDE4 anymore and as I've said I never used klipper, but I
> seem to remember that klipper also had an option to consider all
> selections or just something that was copied or cut as something to deal
> with.  Only in the case that klipper deals with selections the reported
> problem might occur.

The Klipper General Configuration dialog has these options in the
section "Selection and Clipboard":
  Ignore selection
* Text selection only
  Synchronize contents of the clipboard and the selection
Only the second one is checked (*) in my configuration.  Are you
suggesting to check the third one as well, or instead, or something
else?

Steve Berman

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Achim Gratz]

Stephen Berman writes:
> The Klipper General Configuration dialog has these options in the
> section "Selection and Clipboard":
>   Ignore selection
> * Text selection only
>   Synchronize contents of the clipboard and the selection
> Only the second one is checked (*) in my configuration.  Are you
> suggesting to check the third one as well, or instead, or something
> else?

No, that's the setting I've had the problems with.  So it seems that
it's confined to Qt5 or Plasma5 or something even more specific.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Achim Gratz]

Eli Zaretskii writes:
> I see.  Frankly, I feel a bit uneasy that Emacs somehow runs out of
> memory in any scenario, but other than that, would you consider
> describing this solution in PROBLEMS as good enough to go ahead with
> releasing Emacs 25.1?  Perhaps even closing this bug?

Here's my attempt at getting something into PROBLEMS.

--8<---------------cut here---------------start------------->8---
diff --git a/etc/PROBLEMS b/etc/PROBLEMS
index 2b3becf..b86b2b6 100644
--- a/etc/PROBLEMS
+++ b/etc/PROBLEMS
@@ -1070,6 +1070,28 @@ while, Emacs may print a message:
 A workaround is to not use 'klipper'.  Upgrading 'klipper' to the one
 coming with KDE 3.3 or later also solves the problem.
 
+*** KDE / Plasma 5: Emacs exhausts memory and needs to be killed
+
+This problem occurs when large selections contain mixed line endings
+(i.e the buffer has LF line endings, but in some parts CRLF is used).
+The source of the problem is currently under investigation, Emacs
+versions up to 24.5 just hang for a few seconds and then return with
+the message " Timed out waiting for property-notify event" as
+described in the previous note.  As a workaround, go to the settings
+dialog for the Clipboard widget and select the option "Ignore
+Selection".
+
+Note: Plasma 5 has replaced the separate klipper process from earlier
+KDE versions with functionality directly integrated into plasmashell,
+so even if you've previously did not use klipper this will affect you.
+Also, all configuration you might have done to klipper is not used by
+the new Clipboard widget / plasmoid since it uses it's own settings.
+You can hide the Clipboard widget by removing it's entry from the
+system tray settings "Extra Items", but it's not clear if the
+underlying functionality in plasmashell gets fully disabled as well.
+AT least a restart of plasmashell is required for the clipboard
+history to be cleared.
+
 *** CDE: Frames may cover dialogs they created when using CDE.
 
 This can happen if you have "Allow Primary Windows On Top" enabled which

--8<---------------cut here---------------end--------------->8---

Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

bug#24013: 25.0.95; Emacs 25.x uses up all memory and needs to be killed

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Wed, 20 Jul 2016 20:15:12 +0200
> 
> Here's my attempt at getting something into PROBLEMS.

Perfect, thanks.  I've pushed in your name after fixing a couple of
typos.

bug#24013: bug#28146: 25.1; Timeout waiting for property notify event when select-active-regions is t

[Lars Ingebrigtsen]

Achim Gratz <Stromeko@nexgo.de> writes:

> +*** KDE / Plasma 5: Emacs exhausts memory and needs to be killed

This was almost five years ago, and I can't recall seeing any recent bug
reports about this (and I can't reproduce it myself, but I'm not using
KDE).

Is this problem still an issue in more recent versions of
KDE/Plasma/Emacs?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24013: 25.0.95; [KDE/Plasma 5] Emacs 25.x uses up all memory and needs to be killed

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Achim Gratz <Stromeko@nexgo.de> writes:
>
>> +*** KDE / Plasma 5: Emacs exhausts memory and needs to be killed
>
> This was almost five years ago, and I can't recall seeing any recent bug
> reports about this (and I can't reproduce it myself, but I'm not using
> KDE).
>
> Is this problem still an issue in more recent versions of
> KDE/Plasma/Emacs?

More information was requested, but no response was given within a
month, so I'm closing this bug report.  If the problem still exists,
please respond to this email and we'll reopen the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no