bug#14473: 24.3; emacs locks up when eshell attempts to display a dialog

bug#14473: 24.3; emacs locks up when eshell attempts to display a dialog

[Joseph Mingrone]

When working in Eshell, if a dialog such as
http://www.freebsd.org/doc/en/books/handbook/install/dist-set2.png is to
be displayed, Emacs will lock up and the processor will stay at 100%
until Emacs is killed.  This also happens when starting with Emacs -Q.

In GNU Emacs 24.3.1 (amd64-portbld-freebsd9.1, X toolkit)
 of 2013-03-31 on phe.ath.cx
Windowing system distributor `The X.Org Foundation', version 11.0.11006000
Configured using:
 `configure '--localstatedir=/var' '--without-rsvg'
 '--with-x-toolkit=athena' '--without-xaw3d'
 '--without-toolkit-scroll-bars' '--without-gif' '--with-xft'
 '--without-m17n-flt' '--with-otf' '--without-imagemagick'
 '--without-gsettings' '--without-gconf' '--with-xim' '--with-sound'
 '--without-dbus' '--with-xml2' '--with-gnutls'
 '--x-libraries=/usr/local/lib' '--x-includes=/usr/local/include'
 '--prefix=/usr/local' '--mandir=/usr/local/man'
 '--infodir=/usr/local/info/' '--build=amd64-portbld-freebsd9.1'
 'build_alias=amd64-portbld-freebsd9.1' 'CC=cc' 'CFLAGS=-O2 -pipe
 -fno-strict-aliasing' 'LDFLAGS= -L/usr/local/lib
 -Wl,-rpath=/usr/local/lib' 'CPPFLAGS=-I/usr/local/include' 'CPP=cpp''

Important settings:
  value of $LANG: en_CA.UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  erc-track-mode: t
  erc-spelling-mode: t
  erc-ring-mode: t
  erc-netsplit-mode: t
  erc-menu-mode: t
  erc-match-mode: t
  erc-log-mode: t
  erc-list-mode: t
  erc-pcomplete-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-autojoin-mode: t
  show-paren-mode: t
  global-auto-revert-mode: t
  shell-dirtrack-mode: t
  erc-services-mode: t
  erc-networks-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:

Recent messages:
("emacs")
Loading autorevert...done
Loading paren...done
Starting Emacs daemon.
When done with this frame, type C-x 5 0
Making completion list...

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message idna rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils help-mode
server auctex-autoloads tex-site info dired+-autoloads
exec-path-from-shell-autoloads fill-column-indicator-autoloads
google-translate-autoloads iy-go-to-char-autoloads
multi-eshell-autoloads multi-term-autoloads rainbow-delimiters-autoloads
erc-track erc-spelling flyspell ispell erc-ring erc-netsplit erc-menu
erc-match erc-log erc-pcomplete erc-button erc-fill erc-stamp wid-edit
erc-join paren autorevert cus-start cus-load bbdb-autoloads uniquify tls
stumpwm-mode edmacro kmacro easy-mmode package ido ess-toolbar ess-mouse
mouseme browse-url ess-menu ess-swv ess-noweb ess-noweb-font-lock-mode
ess-bugs-l essd-els ess-sas-d ess-sas-l ess-sas-a shell pcomplete
ess-arc-d ess-vst-d ess-xls-d ess-lsp-l ess-sta-d ess-sta-l cc-vars
cc-defs make-regexp ess-sp6-d ess-sp5-d ess-sp3-d ess-julia ess-r-d
compile ess-tracebug warnings ess-roxy advice cl-lib advice-preload
hideshow ess-help ess-developer ess-r-args eldoc help-fns ess-s-l ess
ess-inf comint ansi-color ring ess-mode ess-noweb-mode ess-utils
ess-custom executable ess-compat ess-site erc-services erc-networks
erc-goodies erc erc-backend erc-compat format-spec auth-source eieio
byte-opt bytecomp byte-compile cconv gnus-util time-date mm-util
mail-prsvr password-cache thingatpt pp epa-file epa derived epg
epg-config doc-view easymenu jka-compr image-mode dired bbdb timezone
tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face macroexp files text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dynamic-setting
font-render-setting x-toolkit x multi-tty emacs)

bug#14473: 24.3; emacs locks up when eshell attempts to display a dialog

[Stefan Kangas]

Joseph Mingrone <jrm@ftfl.ca> writes:

> When working in Eshell, if a dialog such as
> http://www.freebsd.org/doc/en/books/handbook/install/dist-set2.png is to
> be displayed, Emacs will lock up and the processor will stay at 100%
> until Emacs is killed.  This also happens when starting with Emacs -Q.

Are you still seeing this on a modern version of Emacs?  If yes, could
you please provide a recipe for how to reproduce it, starting from
"emacs -Q"?

(BTW, the link above is dead.)

Best regards,
Stefan Kangas

bug#14473: 24.3; emacs locks up when eshell attempts to display a dialog

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Date: Mon, 30 Sep 2019 17:47:48 +0200
> Cc: 14473@debbugs.gnu.org
> 
> Joseph Mingrone <jrm@ftfl.ca> writes:
> 
> > When working in Eshell, if a dialog such as
> > http://www.freebsd.org/doc/en/books/handbook/install/dist-set2.png is to
> > be displayed, Emacs will lock up and the processor will stay at 100%
> > until Emacs is killed.  This also happens when starting with Emacs -Q.
> 
> Are you still seeing this on a modern version of Emacs?  If yes, could
> you please provide a recipe for how to reproduce it, starting from
> "emacs -Q"?
> 
> (BTW, the link above is dead.)

You can still find it in the Internet Archive:

  https://web.archive.org/web/20130801011654/http://www.freebsd.org/doc/en/books/handbook/install/dist-set2.png

bug#14473: 24.3; emacs locks up when eshell attempts to display a dialog

[Joseph Mingrone]

[-- Attachment #1: Type: text/plain, Size: 1594 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Kangas <stefan@marxist.se>
>> Date: Mon, 30 Sep 2019 17:47:48 +0200
>> Cc: 14473@debbugs.gnu.org

>> Joseph Mingrone <jrm@ftfl.ca> writes:

>> > When working in Eshell, if a dialog such as
>> > http://www.freebsd.org/doc/en/books/handbook/install/dist-set2.png is to
>> > be displayed, Emacs will lock up and the processor will stay at 100%
>> > until Emacs is killed.  This also happens when starting with Emacs -Q.

>> Are you still seeing this on a modern version of Emacs?  If yes, could
>> you please provide a recipe for how to reproduce it, starting from
>> "emacs -Q"?

>> (BTW, the link above is dead.)

> You can still find it in the Internet Archive:

>   https://web.archive.org/web/20130801011654/http://www.freebsd.org/doc/en/books/handbook/install/dist-set2.png

It's still a problem (26.3 and 2019-09-15 master-branch build) in that the dialog is not displayed, but the Emacs process no longer consumes 100% CPU.

Here is a simple recipe to reproduce the problem.  It assumes the FreeBSD ports tree is installed in the default location, which is /usr/ports.

1. emacs -Q
2. M-x eshell
3. cd /usr/ports/editors/emacs
4. sudo make config

Instead of the dialog displaying, eshell becomes unusable (blank screen and no keys will return the prompt).  Trying to exit by hitting TAB/Enter reports

Completion function pcomplete-completions-at-point uses a deprecated calling convention
Warning: pcomplete-completions-at-point failed to return valid completion data!

You can switch buffers and kill the eshell process now though.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 979 bytes --]

bug#14473: 24.3; emacs locks up when eshell attempts to display a dialog

[Stefan Kangas]

Joseph Mingrone <jrm@ftfl.ca> writes:

> It's still a problem (26.3 and 2019-09-15 master-branch build) in that the dialog is not displayed, but the Emacs process no longer consumes 100% CPU.

Thanks for reporting back.

> Here is a simple recipe to reproduce the problem.  It assumes the FreeBSD ports tree is installed in the default location, which is /usr/ports.

Can you think of any way to reproduce this if you are not using
FreeBSD?  Is there some particular command run by "make config" that
makes eshell freeze for example?  It seems to me that very few Emacs
developers are using FreeBSD, and I personally don't have access to
any FreeBSD systems for debugging.

> Instead of the dialog displaying, eshell becomes unusable (blank screen and no keys will return the prompt).  Trying to exit by hitting TAB/Enter reports
>
> Completion function pcomplete-completions-at-point uses a deprecated calling convention
> Warning: pcomplete-completions-at-point failed to return valid completion data!
>
> You can switch buffers and kill the eshell process now though.

What happens if you run M-x toggle-debug-on-error before trying to
reproduce?  Do you then get a backtrace?

Best regards,
Stefan Kangas

bug#14473: 24.3; emacs locks up when eshell attempts to display a dialog

[Joseph Mingrone]

[-- Attachment #1: Type: text/plain, Size: 1739 bytes --]

Stefan Kangas <stefan@marxist.se> writes:

> Joseph Mingrone <jrm@ftfl.ca> writes:

>> It's still a problem (26.3 and 2019-09-15 master-branch build) in that the dialog is not displayed, but the Emacs process no longer consumes 100% CPU.

> Thanks for reporting back.

>> Here is a simple recipe to reproduce the problem.  It assumes the FreeBSD ports tree is installed in the default location, which is /usr/ports.

> Can you think of any way to reproduce this if you are not using
> FreeBSD?  Is there some particular command run by "make config" that
> makes eshell freeze for example?  It seems to me that very few Emacs
> developers are using FreeBSD, and I personally don't have access to
> any FreeBSD systems for debugging.

I would guess any GNU/Linux command that also presents these curses
dialogs would have problems.  If you or any Emacs developer wants a
FreeBSD shell account, I can provide one.

https://invisible-island.net/dialog/dialog-figures.html

To be clear, it seems like less of a freeze now and more like an
inability to display the dialog and the point becomes lost requiring
users to kill eshell.  So, it is much less severe of a problem than in
the past.

>> Instead of the dialog displaying, eshell becomes unusable (blank screen and no keys will return the prompt).  Trying to exit by hitting TAB/Enter reports

>> Completion function pcomplete-completions-at-point uses a deprecated calling convention
>> Warning: pcomplete-completions-at-point failed to return valid completion data!

>> You can switch buffers and kill the eshell process now though.

> What happens if you run M-x toggle-debug-on-error before trying to
> reproduce?  Do you then get a backtrace?

There is no backtrace.

Regards,

Joseph

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 979 bytes --]

bug#14473: 24.3; emacs locks up when eshell attempts to display a dialog

[Stefan Kangas]

Hi Joseph,

Joseph Mingrone <jrm@ftfl.ca> writes:

> >> It's still a problem (26.3 and 2019-09-15 master-branch build) in that the dialog is not displayed, but the Emacs process no longer consumes 100% CPU.
>
> > Thanks for reporting back.
>
> >> Here is a simple recipe to reproduce the problem.  It assumes the FreeBSD ports tree is installed in the default location, which is /usr/ports.
>
> > Can you think of any way to reproduce this if you are not using
> > FreeBSD?  Is there some particular command run by "make config" that
> > makes eshell freeze for example?  It seems to me that very few Emacs
> > developers are using FreeBSD, and I personally don't have access to
> > any FreeBSD systems for debugging.
>
> I would guess any GNU/Linux command that also presents these curses
> dialogs would have problems.  If you or any Emacs developer wants a
> FreeBSD shell account, I can provide one.

Thank you, noted.  There are indeed (a small number of) open bugs
regarding *BSD systems, so I'm hoping that someone will take you up on
that offer.  I might if I find the time.

> https://invisible-island.net/dialog/dialog-figures.html
>
> To be clear, it seems like less of a freeze now and more like an
> inability to display the dialog and the point becomes lost requiring
> users to kill eshell.  So, it is much less severe of a problem than in
> the past.

Yes, after installing "dialog", I'm able to reproduce the problem on
my system using this command:

    dialog --yesno "foobar" 10 50

In my case, hitting RET brings me back to the eshell prompt.

I think the problem is that eshell just doesn't support the control
characters that ncurses is producing, meaning that it has to switch to
term-mode to get that to work.  Luckily, there are user options you
could set to make eshell do that automatically.

I created a Makefile with:

    config:
        dialog --yesno "foobar" 10 50

Using that Makefile, saying "make config" in eshell opens it in
term-mode automatically after I evaluate:

    (add-to-list 'eshell-visual-subcommands '("make" "config"))

Does setting that option solve the issues you're seeing too?

If so, I think we can just write this up as a limitation in eshell,
and recommend users to configure this variable.  Also see
eshell-visual-commands and eshell-visual-options for more.

One final thing, is running "make config" common on FreeBSD?  I guess
it's part of the "ports" system that pretty much everyone uses,
including people on OpenBSD?  If so, perhaps it would be worth
changing the default of eshell-visual-subcommands from nil to
something like:

    (when (equal system-type 'berkeley-unix) '(("make" "config")))

Best regards,
Stefan Kangas

bug#14473: 24.3; emacs locks up when eshell attempts to display a dialog

[Joseph Mingrone]

[-- Attachment #1: Type: text/plain, Size: 3586 bytes --]

Stefan Kangas <stefan@marxist.se> writes:

> Hi Joseph,

> Joseph Mingrone <jrm@ftfl.ca> writes:

>> >> It's still a problem (26.3 and 2019-09-15 master-branch build) in that the dialog is not displayed, but the Emacs process no longer consumes 100% CPU.

>> > Thanks for reporting back.

>> >> Here is a simple recipe to reproduce the problem.  It assumes the FreeBSD ports tree is installed in the default location, which is /usr/ports.

>> > Can you think of any way to reproduce this if you are not using
>> > FreeBSD?  Is there some particular command run by "make config" that
>> > makes eshell freeze for example?  It seems to me that very few Emacs
>> > developers are using FreeBSD, and I personally don't have access to
>> > any FreeBSD systems for debugging.

>> I would guess any GNU/Linux command that also presents these curses
>> dialogs would have problems.  If you or any Emacs developer wants a
>> FreeBSD shell account, I can provide one.

> Thank you, noted.  There are indeed (a small number of) open bugs
> regarding *BSD systems, so I'm hoping that someone will take you up on
> that offer.  I might if I find the time.

>> https://invisible-island.net/dialog/dialog-figures.html

>> To be clear, it seems like less of a freeze now and more like an
>> inability to display the dialog and the point becomes lost requiring
>> users to kill eshell.  So, it is much less severe of a problem than in
>> the past.

> Yes, after installing "dialog", I'm able to reproduce the problem on
> my system using this command:

>     dialog --yesno "foobar" 10 50

> In my case, hitting RET brings me back to the eshell prompt.

> I think the problem is that eshell just doesn't support the control
> characters that ncurses is producing, meaning that it has to switch to
> term-mode to get that to work.  Luckily, there are user options you
> could set to make eshell do that automatically.

> I created a Makefile with:

>     config:
>         dialog --yesno "foobar" 10 50

> Using that Makefile, saying "make config" in eshell opens it in
> term-mode automatically after I evaluate:

>     (add-to-list 'eshell-visual-subcommands '("make" "config"))

> Does setting that option solve the issues you're seeing too?

> If so, I think we can just write this up as a limitation in eshell,
> and recommend users to configure this variable.  Also see
> eshell-visual-commands and eshell-visual-options for more.

> One final thing, is running "make config" common on FreeBSD?  I guess
> it's part of the "ports" system that pretty much everyone uses,
> including people on OpenBSD?  If so, perhaps it would be worth
> changing the default of eshell-visual-subcommands from nil to
> something like:

>     (when (equal system-type 'berkeley-unix) '(("make" "config")))

> Best regards,
> Stefan Kangas

Hi Stefan,

RET or TAB RET does not return to the eshell prompt here.

Switching to term mode by adding ("make" "config") `to
eshell-visual-subcommands' does now work, so I think this bug can be
closed. I had this commented out in my config, so assume I tried
it at one point and it didn't work.  I should have checked this again
when you contacted me though.

Running `make config` was more common in the past, but many FreeBSD
users now install pre-built packages or build their own packages with
tools on top of the ports tree.  I am less familiar with OpenBSD, but am
relatively confident that most OpenBSD users install pre-built packages.
Another complication is that many users would run this as `sudo make
config` or `doas make config` on OpenBSD.

Thank you,

Joseph


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

bug#14473: 24.3; emacs locks up when eshell attempts to display a dialog

[Stefan Kangas]

Joseph Mingrone <jrm@ftfl.ca> writes:

> Switching to term mode by adding ("make" "config") `to
> eshell-visual-subcommands' does now work, so I think this bug can be
> closed.

Thanks for verifying that it solves the problem.

> Running `make config` was more common in the past, but many FreeBSD
> users now install pre-built packages or build their own packages with
> tools on top of the ports tree.  I am less familiar with OpenBSD, but am
> relatively confident that most OpenBSD users install pre-built packages.
> Another complication is that many users would run this as `sudo make
> config` or `doas make config` on OpenBSD.

Perhaps we should leave the default alone then.  I'm consequently
closing the bug report.

Best regards,
Stefan Kangas