bug#20202: 24.3; Comint mode sets a bad $EMACS

bug#20202: 24.3; Comint mode sets a bad $EMACS

[Eli Barzilay]

I was surprised to see that compiling some random Emacs code via a
Makefile fails when running inside Emacs with an obscure

  /bin/sh: t: command not found

I found this setting in comint.el:

	  (unless (getenv "EMACS")
	    (list "EMACS=t"))

And that would obviously break such Makefile uses (and IIUC, $EMACS is
a popular choice for specifying which emacs binary to use).

It looks like this was done in this commit:

  commit cfefbbf4404963cdf042fb794e0456503aa8b591
  Author: Chong Yidong <cyd@stupidchicken.com>
  Date:   2006-11-18 21:01:33 +0000

    (comint-exec-1): Set EMACS and INSIDE_EMACS to t.

Before this commit, EMACS was set with

    (concat "EMACS=" invocation-directory invocation-name)

which doesn't look like a good idea either (if you happen to use some
other Emacs version, you still expect the shell to be a plain shell),
but it at least didn't break it.  But this change was done shortly
after:

  commit 4b1aaa8b07cf2797b5a57e2a1fd88f3ec0aa41e2
  Author: Paul Eggert <eggert@twinsun.com>
  Date:   2006-09-12 16:43:25 +0000

    * etc/NEWS: In terminal-oriented subshells, the EMACS environment
    variable now defaults to Emacs's absolute file name, instead of
    to "t".

and before that (all the way to the initial comint version in git), it
was always "t".  It looks like this was intended as a way to tell if
you're running inside emacs, which was superseded by $INSIDE_EMACS --
misc.texi says

  (It also sets the @env{EMACS} environment variable to @code{t}, if
  that environment variable is not already defined.  However, this
  environment variable is deprecated; programs that use it should
  switch to using @env{INSIDE_EMACS} instead.)

and the changelog dates this to the same date as the first commit
above.

So, since it has been deprecated for almost 8 years, it looks fine to
remove it.  If not, then setting it back to the running Emacs would
work too, but better to not do such an unexpected change, so something
like "EMACS=emacs" is probably going to be unobtrusive.

Or if there's some motivation behind intentionally making it some
string that is not an executable, then "EMACS=some-descriptive-text"
would be better.


As a sidenote, misc/efaq.texi uses $EMACS still.  (But for tcsh, so
not that anyone should care...)

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!

bug#20202: 24.3; Comint mode sets a bad $EMACS

[Stefan Monnier]

> So, since it has been deprecated for almost 8 years, it looks fine to
> remove it.

Agreed.

> If not, then setting it back to the running Emacs would work too, but

It's never really been set that way, except for some very brief window
(I don't think it made it into a release, for example).


        Stefan

bug#20202: 24.3; Comint mode sets a bad $EMACS

[Eli Barzilay]

On Wed, Mar 25, 2015 at 8:46 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> So, since it has been deprecated for almost 8 years, it looks fine to
>> remove it.
>
> Agreed.
>
>> If not, then setting it back to the running Emacs would work too, but
>
> It's never really been set that way, except for some very brief window
> (I don't think it made it into a release, for example).

Yeah, that's what I thought.  FWIW, to avoid it for now, I've made my
Emacs set it to "emacs" and it looks like that works fine for builds.

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!

bug#20202: 24.3; Comint mode sets a bad $EMACS

[Stefan Monnier]

> Yeah, that's what I thought.  FWIW, to avoid it for now, I've made my
> Emacs set it to "emacs" and it looks like that works fine for builds.

I've just installed the patch below into master, which should fix this
problem,


        Stefan


diff --git a/lisp/comint.el b/lisp/comint.el
index 31649ff..2769c87 100644
--- a/lisp/comint.el
+++ b/lisp/comint.el
@@ -816,8 +816,6 @@ series of processes in the same Comint buffer.  The hook
 		    (format "COLUMNS=%d" (window-width)))
 	    (list "TERM=emacs"
 		  (format "TERMCAP=emacs:co#%d:tc=unknown:" (window-width))))
-	  (unless (getenv "EMACS")
-	    (list "EMACS=t"))
 	  (list (format "INSIDE_EMACS=%s,comint" emacs-version))
 	  process-environment))
 	(default-directory
diff --git a/lisp/net/tramp-sh.el b/lisp/net/tramp-sh.el
index f59c5fb..3f006e8 100644
--- a/lisp/net/tramp-sh.el
+++ b/lisp/net/tramp-sh.el
@@ -496,7 +496,6 @@ as given in your `~/.profile'."
 (defcustom tramp-remote-process-environment
   `("TMOUT=0" "LC_CTYPE=''"
     ,(format "TERM=%s" tramp-terminal-type)
-    "EMACS=t" ;; Deprecated.
     ,(format "INSIDE_EMACS='%s,tramp:%s'" emacs-version tramp-version)
     "CDPATH=" "HISTORY=" "MAIL=" "MAILCHECK=" "MAILPATH=" "PAGER=cat"
     "autocorrect=" "correct=")
diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
index 362bbf5..9d36e91 100644
--- a/lisp/progmodes/compile.el
+++ b/lisp/progmodes/compile.el
@@ -1666,11 +1666,7 @@ Returns the compilation buffer created."
 		(list "TERM=emacs"
 		      (format "TERMCAP=emacs:co#%d:tc=unknown:"
 			      (window-width))))
-	      ;; Set the EMACS variable, but
-	      ;; don't override users' setting of $EMACS.
-	      (unless (getenv "EMACS")
-		(list "EMACS=t"))
-	      (list "INSIDE_EMACS=t")
+	      (list (format "INSIDE_EMACS=%s,compile" emacs-version))
 	      (copy-sequence process-environment))))
 	(set (make-local-variable 'compilation-arguments)
 	     (list command mode name-function highlight-regexp))
diff --git a/lisp/term.el b/lisp/term.el
index 43138fa..4c82986 100644
--- a/lisp/term.el
+++ b/lisp/term.el
@@ -1505,11 +1505,6 @@ Using \"emacs\" loses, because bash disables editing if $TERM == emacs.")
 	   (format "TERMINFO=%s" data-directory)
 	   (format term-termcap-format "TERMCAP="
 		   term-term-name term-height term-width)
-	   ;; We are going to get rid of the binding for EMACS,
-	   ;; probably in Emacs 23, because it breaks
-	   ;; `./configure' of some packages that expect it to
-	   ;; say where to find EMACS.
-	   (format "EMACS=%s (term:%s)" emacs-version term-protocol-version)
 	   (format "INSIDE_EMACS=%s,term:%s" emacs-version term-protocol-version)
 	   (format "LINES=%d" term-height)
 	   (format "COLUMNS=%d" term-width))

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Jacob Oursland]

[-- Attachment #1: Type: text/plain, Size: 9206 bytes --]

My inferior shell is bash.  I have confirmed that this problem exists
with 'emacs -Q'.  Re-setting my PS1 to the bash default PS1='\h:\w\$ '
and starting Emacs didn't help either.

Steps to reproduce:
1. emacs -Q
2. M-x ansi-term RET RET
3. cd /tmp (or any other directory)
4. C-x C-f

Expected behavior:
Emacs will indicate the shell's working directory (/tmp) in the
find-file minibuffer prompt.

Actual behavior:
Emacs indicates the Emacs working directory working directory ($HOME,
for me) in the find-file minibuffer prompt.

I found that if I revert commit aad65192332dfc4a1df0cd2953554c21da243b51
the problem goes away.



In GNU Emacs 25.0.50.1 (x86_64-pc-linux-gnu, GTK+ Version 3.10.8)
 of 2015-04-19 on lgw01-10
Windowing system distributor `The X.Org Foundation', version 11.0.11501000
System Description: Ubuntu 14.04.2 LTS

Configured using:
 `configure --build=x86_64-linux-gnu --prefix=/usr
 '--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
 '--infodir=${prefix}/share/info' --sysconfdir=/etc --localstatedir=/var
 '--libdir=${prefix}/lib/x86_64-linux-gnu'
 '--libexecdir=${prefix}/lib/x86_64-linux-gnu' --disable-maintainer-mode
 --disable-dependency-tracking --prefix=/usr --sharedstatedir=/var/lib
 --program-suffix=-snapshot --with-x=yes --with-x-toolkit=gtk3
 'CFLAGS=-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat
 -Werror=format-security' CPPFLAGS=-D_FORTIFY_SOURCE=2
 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  TeX-PDF-mode: t
  global-auto-complete-mode: t
  dirtrack-debug-mode: t
  magit-auto-revert-mode: t
  diff-auto-refine-mode: t
  elisp-slime-nav-mode: t
  display-time-mode: t
  shell-dirtrack-mode: t
  global-company-mode: t
  company-mode: t
  xterm-mouse-mode: t
  winner-mode: t
  which-function-mode: t
  show-paren-mode: t
  global-auto-revert-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t

Recent messages:
Creating group entries...done
Creating customization items ...done
Resetting customization items...done
Creating customization setup...done
Auto-saving...done
Save all settings in this buffer? (y or n) y
Saving file /home/jso/.emacs.d/init.el...
Wrote /home/jso/.emacs.d/init.el [2 times]
Mark set [4 times]
Buffer *unsent mail to bug-gnu-emacs@gnu.org* modified; kill anyway? (y or
n) y

Load-path shadows:
/home/jso/.emacs.d/elpa/cmake-mode-20150120.620/cmake-mode hides
/usr/share/emacs/site-lisp/cmake-mode
/usr/share/emacs/site-lisp/rst hides
/usr/share/emacs/25.0.50/lisp/textmodes/rst
/usr/share/emacs/site-lisp/dictionaries-common/ispell hides
/usr/share/emacs/25.0.50/lisp/textmodes/ispell
/usr/share/emacs/site-lisp/dictionaries-common/flyspell hides
/usr/share/emacs/25.0.50/lisp/textmodes/flyspell
~/.emacs.d/cc-mode/cc-guess hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-guess
~/.emacs.d/cc-mode/cc-vars hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-vars
~/.emacs.d/cc-mode/cc-mode hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-mode
~/.emacs.d/cc-mode/cc-bytecomp hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-bytecomp
~/.emacs.d/cc-mode/cc-styles hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-styles
~/.emacs.d/cc-mode/cc-fonts hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-fonts
~/.emacs.d/cc-mode/cc-menus hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-menus
~/.emacs.d/cc-mode/cc-cmds hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-cmds
~/.emacs.d/cc-mode/cc-align hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-align
~/.emacs.d/cc-mode/cc-engine hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-engine
~/.emacs.d/cc-mode/cc-defs hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-defs
~/.emacs.d/cc-mode/cc-awk hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-awk
~/.emacs.d/cc-mode/cc-langs hides
/usr/share/emacs/25.0.50/lisp/progmodes/cc-langs
/usr/share/emacs/site-lisp/latex-cjk-thai/thai-word hides
/usr/share/emacs/25.0.50/lisp/language/thai-word
~/.emacs.d/cc-mode/cc-compat hides
/usr/share/emacs/25.0.50/lisp/obsolete/cc-compat

Features:
(uudecode uce supercite regi spam-report spam spam-stat gnus-uu yenc
gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime smime dig smtpmail
sieve-manage rmail pop3 mspools mh-e mh-compat mh-acros mh-buffers
mh-loaddefs mairix mailclient browse-url mailalias mail-hist imap
hashcash gnus-dired fortune feedmail eudc-vars ecomplete binhex apropos
footnote shadow sort mail-extr emacsbug sendmail mm-archive url-handlers
eieio-opt speedbar sb-image ezimage dframe pcmpl-unix em-unix em-term
em-script em-prompt em-ls em-hist em-pred em-glob em-dirs em-cmpl
em-basic em-banner em-alias magit-blame cmake-mode ruler-mode hl-line
hexl smerge-mode ert timezone texinfo toolbar-x prv-emacs reporter
desktop frameset context plain-tex latex tex-style tex-buf tex dbus xml
crm tempo company-dabbrev hippie-exp debug json irony-cdb jedi-core
python-environment epc ctable concurrent deferred ediff-merg ediff-wind
ediff-diff ediff-mult ediff-help ediff-init ediff-util ediff dired-x
org-rmail org-mhe org-irc org-info org-gnus org-docview org-bibtex
org-bbdb org-w3m rng-loc rng-uri rng-parse rng-match rng-dt rng-util
rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util ox-latex
ox-icalendar ox-html table ox-ascii ox-publish ox org-element org-table
org-agenda esh-var esh-io esh-cmd esh-opt esh-ext esh-proc esh-arg
esh-groups eshell esh-module esh-mode esh-util doc-view jka-compr
image-mode bibtex org-id org org-macro org-footnote org-pcomplete
org-list org-faces org-entities org-version ob-python ob-emacs-lisp
org-loaddefs cal-menu calendar cal-loaddefs gnus-sum gnus-group
gnus-undo gnus-start gnus-cloud nnimap nnmail mail-source utf7 netrc
nnoo parse-time gnus-spec gnus-int gnus-range gnus-win gnus gnus-ems
nnheader ob-octave noutline outline calc calc-loaddefs calc-macs ob
ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys ob-comint
ob-core ob-eval org-compat org-macs rx auto-complete rtags popup repeat
bookmark company-template warnings autoload lisp-mnt tar-mode url-http
url-gw url-cache url-auth url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util url-parse
url-vars mailcap sgml-mode make-mode novice dirtrack sql socks
network-stream nsm starttls tls rlogin proced metamail gud flyspell
ispell pp cus-edit wid-edit magit-key-mode magit view grep epa derived
epg git-rebase-mode git-commit-mode log-edit message dired rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mailabbrev mail-utils gmm-utils mailheader pcvs-util xterm
conf-mode add-log sh-script smie executable thingatpt misearch
multi-isearch vc vc-dispatcher vc-git diff-mode cpp cc-mode cc-fonts
cc-awk cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-langs
cc-vars cc-defs cc-bytecomp tabify man tramp-cache elisp-slime-nav
help-mode company-cmake company-irony irony-completion irony-snippet
server term disp-table ehelp powerline time cl windmove ido tramp
tramp-compat auth-source gnus-util mm-util mail-prsvr password-cache
tramp-loaddefs trampver shell pcomplete format-spec advice help-fns
irony find-func company easy-mmode cl-macs ggtags etags xref eieio
eieio-core cl-generic byte-opt gv bytecomp byte-compile cl-extra seq
cconv compile comint ansi-color ewoc edmacro kmacro cl-loaddefs pcase
cl-lib finder-inf tex-site info easymenu package epg-config saveplace
leuven-theme xt-mouse winner ring which-func imenu paren autorevert
filenotify cus-start cus-load mule-util time-date tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list newcomment elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev minibuffer cl-preloaded nadvice loaddefs button faces cus-face
macroexp files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
make-network-process dbusbind gfilenotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty emacs)

Memory information:
((conses 16 5101727 455519)
 (symbols 48 74114 0)
 (miscs 40 12397 14840)
 (strings 32 207635 114021)
 (string-bytes 1 6335085)
 (vectors 16 79150)
 (vector-slots 8 2022361 83492)
 (floats 8 982 3084)
 (intervals 56 838741 22842)
 (buffers 976 184)
 (heap 1024 216591 159638))

[-- Attachment #2: Type: text/html, Size: 11490 bytes --]

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Glenn Morris]

Jacob Oursland wrote:

> 1. emacs -Q
> 2. M-x ansi-term RET RET
> 3. cd /tmp (or any other directory)
> 4. C-x C-f
>
> Expected behavior:
> Emacs will indicate the shell's working directory (/tmp) in the
> find-file minibuffer prompt.
>
> Actual behavior:
> Emacs indicates the Emacs working directory working directory ($HOME,
> for me) in the find-file minibuffer prompt.

I can reproduce this problem.

> I found that if I revert commit aad65192332dfc4a1df0cd2953554c21da243b51
> the problem goes away.

However, I cannot reproduce this commit being the problem.
Since that commit only touches shell.el, which is not even loaded in the
above recipe, it's hard to see how it could be.

Instead I find commit beaab89896 ("Stop messing with the EMACS env var")
causes this. But that was just a quick experiment, not a proper bisection.

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Glenn Morris]

Glenn Morris wrote:

> Instead I find commit beaab89896 ("Stop messing with the EMACS env var")
> causes this. But that was just a quick experiment, not a proper bisection.

You should be able to verify this by using eg

EMACS='24.5.1 (term:0.96)' emacs -Q -f ansi-term

and seeing that it works again.

I don't get
http://lists.gnu.org/archive/html/emacs-devel/2015-03/msg01034.html

It's just going to break things for anybody running any released version
of bash (as predicted in the preceding message). It will be a long time
before all the systems Emacs supports default to a suitably recent bash.

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Jacob Oursland]

[-- Attachment #1: Type: text/plain, Size: 1424 bytes --]

Hi Glenn,

You are absolutely correct.  I'm not sure how I found the other commit at
fault, user error most definitely.

I'm not entirely sure how to approach this, locally even.  I guess I'll
have to build my own debian packages to revert this change.

Git master for Bash on Savannah doesn't even mention the INSIDE_EMACS
environment variable, so the current software simply will not work.  I
believe that beaab89896 only serves to break things with little gain (to
those who inappropriately used EMACS in their scripts).  It certainly
breaks things for users like me who exist almost entirely in Emacs.  Has
the intent to move away from the EMACS environment variable been
communicated to the Bash folks?

Thanks,
Jake

On Fri, May 1, 2015 at 7:43 PM Glenn Morris <rgm@gnu.org> wrote:

> Glenn Morris wrote:
>
> > Instead I find commit beaab89896 ("Stop messing with the EMACS env var")
> > causes this. But that was just a quick experiment, not a proper
> bisection.
>
> You should be able to verify this by using eg
>
> EMACS='24.5.1 (term:0.96)' emacs -Q -f ansi-term
>
> and seeing that it works again.
>
> I don't get
> http://lists.gnu.org/archive/html/emacs-devel/2015-03/msg01034.html
>
> It's just going to break things for anybody running any released version
> of bash (as predicted in the preceding message). It will be a long time
> before all the systems Emacs supports default to a suitably recent bash.
>

[-- Attachment #2: Type: text/html, Size: 2239 bytes --]

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Stefan Monnier]

> Git master for Bash on Savannah doesn't even mention the INSIDE_EMACS
> environment variable, so the current software simply will not work.  I
> believe that beaab89896 only serves to break things with little gain (to
> those who inappropriately used EMACS in their scripts).  It certainly
> breaks things for users like me who exist almost entirely in Emacs.  Has
> the intent to move away from the EMACS environment variable been
> communicated to the Bash folks?

The old $EMACS setting was declared obsolete many years ago.  It looks
like "the Bash folks" didn't notice yet.

Hopefully beaab89896 will make them notice,


        Stefan

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Jacob Oursland]

[-- Attachment #1: Type: text/plain, Size: 594 bytes --]

Stefan,

It made Emacs users like me notice, mainly by previously-working
functionality in Emacs breaking.  Looking at the old bug report, the $EMACS
setting was determined to be obsolete because a few Bash users incorrectly
used the $EMACS variable in their scripts.

According to the Bash documentation, Bash considers the $EMACS variable to
be special and alters its behavior based on the contents.  I would argue
this indicates that it's the script writers in error and no action
should've been taken besides a kind pointer to refer to the Bash manual to
the script writers.

Regards,
Jake

[-- Attachment #2: Type: text/html, Size: 723 bytes --]

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Hopefully beaab89896 will make them notice,

That is an unfriendly way to treat another free software project (and
this one is a GNU package).

The problem here is a conflict of conventions between Emacs and Bash.
The best way to handle this is by a discussion between the developers
on both sides.  Have you talked with Chet Ramey to look for the right
solution?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! See stallman.org/skype.html.

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Jacob Oursland]

[-- Attachment #1: Type: text/plain, Size: 1004 bytes --]

In the mean time, can we revert this commit so that Bash directory tracking
will work until a coordinated solution is achieved?

Thanks,
Jake


On Sun, May 3, 2015 at 9:29 AM Richard Stallman <rms@gnu.org> wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Hopefully beaab89896 will make them notice,
>
> That is an unfriendly way to treat another free software project (and
> this one is a GNU package).
>
> The problem here is a conflict of conventions between Emacs and Bash.
> The best way to handle this is by a discussion between the developers
> on both sides.  Have you talked with Chet Ramey to look for the right
> solution?
>
> --
> Dr Richard Stallman
> President, Free Software Foundation
> 51 Franklin St
> Boston MA 02110
> USA
> www.fsf.org  www.gnu.org
> Skype: No way! See stallman.org/skype.html.
>
>

[-- Attachment #2: Type: text/html, Size: 1514 bytes --]

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Glenn Morris]

Stefan Monnier wrote:

> The old $EMACS setting was declared obsolete many years ago.  It looks
> like "the Bash folks" didn't notice yet.
>
> Hopefully beaab89896 will make them notice,

Paul mailed them:
http://lists.gnu.org/archive/html/bug-bash/2015-03/msg00179.html

It was applied to the "devel" branch soon after:
http://git.savannah.gnu.org/cgit/bash.git/diff/shell.c?h=devel&id=0385211bb5cb01e0259c64ec2c5cc6337d4e215c

So, the point has been made, but it will take a long time for this to
get into the default bash on all the systems Emacs supports.

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Jacob Oursland]

[-- Attachment #1: Type: text/plain, Size: 980 bytes --]

I see.  I misunderstood the master branch to be development, but it's the
release branch.

Can we compromise a little and revert this patch just until the commit has
made into a Bash release?  I, like other Emacs users, can afford the risks
to run a development Emacs, but I cannot risk a development Bash.

Thanks,
Jake


On Sun, May 3, 2015 at 10:57 AM Glenn Morris <rgm@gnu.org> wrote:

> Stefan Monnier wrote:
>
> > The old $EMACS setting was declared obsolete many years ago.  It looks
> > like "the Bash folks" didn't notice yet.
> >
> > Hopefully beaab89896 will make them notice,
>
> Paul mailed them:
> http://lists.gnu.org/archive/html/bug-bash/2015-03/msg00179.html
>
> It was applied to the "devel" branch soon after:
>
> http://git.savannah.gnu.org/cgit/bash.git/diff/shell.c?h=devel&id=0385211bb5cb01e0259c64ec2c5cc6337d4e215c
>
> So, the point has been made, but it will take a long time for this to
> get into the default bash on all the systems Emacs supports.
>

[-- Attachment #2: Type: text/html, Size: 1658 bytes --]

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Stefan Monnier]

> on both sides.  Have you talked with Chet Ramey to look for the right
> solution?

The right solution is easy: Bash should use $INSIDE_EMACS rather
than $EMACS.

IOW the right thing to do is to file a bug report to the Bash
maintainers telling them they need to use $INSIDE_EMACS rather
than $EMACS because $EMACS is finally being retired.

I see no reason to revert beaab89896 until bash's master code has been
updated to stop using $EMACS.


        Stefan

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Stefan Monnier]

> It was applied to the "devel" branch soon after:
> http://git.savannah.gnu.org/cgit/bash.git/diff/shell.c?h=devel&id=0385211bb5cb01e0259c64ec2c5cc6337d4e215c

> So, the point has been made, but it will take a long time for this to
> get into the default bash on all the systems Emacs supports.

Good.  So we can revert beaab89896 before 25.1.
We should still keep it a bit longer, in case some other project has
failed to notice the obsolescence of $EMACS.


        Stefan

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > on both sides.  Have you talked with Chet Ramey to look for the right
  > > solution?

  > The right solution is easy: Bash should use $INSIDE_EMACS rather
  > than $EMACS.

You may be right, but please talk with him about it.  He may agree and
fix Bash right away.  Or he may show you good reasons for some other
solution.  Please work together with the Bash maintainers.

The way to communicate with other GNU package maintainers is by
talking with them constructively -- not by making changes that will
lead their users to complain to them.  Talking with someone via
the public is not a good way to get people working together.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! See stallman.org/skype.html.

bug#20484: 25.0.50; Directory tracking in ansi-term broken.

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 740 bytes --]

> *From:*  Stefan Monnier <monnier <at> iro.umontreal.ca>
> *Date:*  Sun, 03 May 2015 22:07:57 -0400
>
> >http://git.savannah.gnu.org/cgit/bash.git/diff/shell.c?h=devel&id=0385211bb5cb01e0259c64ec2c5cc6337d4e215c
>
> > So, the point has been made, but it will take a long time for this to
> > get into the default bash on all the systems Emacs supports.
>
> Good.  So we can revert beaab89896 before 25.1.
> We should still keep it a bit longer, in case some other project has
> failed to notice the obsolescence of $EMACS.

OK, it's been ten months and that should be long enough, so I reverted 
it in emacs-25 (see attached patch) and am closing this bug report. Yay! 
One less blocker for Emacs 25! (The fixed Bash isn't out yet, alas.)

[-- Attachment #2: 0001-Comint-term-and-compile-now-set-EMACS.patch --]
[-- Type: application/x-patch, Size: 4164 bytes --]

Considered Harmful 73d213: "Comint, term, and compile new set Emacs"

[Phillip Lord]

I've just found commit 73d213 has been added, which restores the
$EMACS behaviour in comint, shell and compile. In other words $EMACS
gets set to "t".

Unfortunately, this affects both cask and ert-runner which I had altered
to take account of and depend on the new behaviour for Emacs-25. In this
case, cask and ert-runner use $EMACS to mean "the location of the Emacs
to run". This will require explicit setting now.

In an truly entertaining piece of irony, my fixes to cask and ert-runner
were merged on 23 Mar, the same day as commit 73d213, which breaks the
fixes.

Not sure how to fix this, but the commentary prior to commit beaab89
says "we are going to rid of this binding, probably in Emacs 23". It
seems a shame to be three major versions wrong.

Phil

Re: Considered Harmful 73d213: "Comint, term, and compile new set Emacs"

[Paul Eggert]

On 04/05/2016 04:05 AM, Phillip Lord wrote:
> Not sure how to fix this, but the commentary prior to commit beaab89
> says "we are going to rid of this binding, probably in Emacs 23". It
> seems a shame to be three major versions wrong.
Yes, this change has been frustrating to implement (see Bug#20484), 
because we need to wait until the new Bash is released and more widely 
available, as current Bash is incompatible with the change.

It sounds like you have a workaround in mind for cask and ert-runner. Is 
there some problem in installing the workaround? Presumably they need to 
work in older Emacs anyway, so they already have a way to deal with this 
issue.

Re: Considered Harmful 73d213: "Comint, term, and compile new set Emacs"

[Phillip Lord]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 04/05/2016 04:05 AM, Phillip Lord wrote:
>> Not sure how to fix this, but the commentary prior to commit beaab89
>> says "we are going to rid of this binding, probably in Emacs 23". It
>> seems a shame to be three major versions wrong.
> Yes, this change has been frustrating to implement (see Bug#20484), because we
> need to wait until the new Bash is released and more widely available, as
> current Bash is incompatible with the change.

I understand the problem, and did read the bug report.


> It sounds like you have a workaround in mind for cask and ert-runner. Is there
> some problem in installing the workaround? Presumably they need to work in
> older Emacs anyway, so they already have a way to deal with this issue.

Well, it doesn't work now. Cask used to just ditch $EMACS. Now it checks
the value of INSIDE_EMACS and does one thing in Emacs-24 (ditches
$EMACS) and another in Emacs-25 (uses $EMACS). In other words, I don't
have a workaround, I have limited the scope of the existing workaround
to Emacs-24. All of this semantics will now have to be backed out, since
it is all wrong.

The only saving grace is that I added $CASK_EMACS at the same time,
which avoids the name clash entirely. But this also needs to be
percolated to downstream tools like ert-runner.

Is there not a different solution to the bash problem? Passing
--noediting as an option would work, I think, and it should work with
both the old and new $EMACS handling of bash.

Phil

Re: Considered Harmful 73d213: "Comint, term, and compile new set Emacs"

[Paul Eggert]

On 04/05/2016 09:38 AM, Phillip Lord wrote:
> I don't
> have a workaround, I have limited the scope of the existing workaround
> to Emacs-24.

Would something like the following be an adequate workaround? The idea 
is to have code that works with both Emacs 25 and Emacs 24 (and Emacs 23 
etc., for that matter):

     def get_emacs_from_env():
         emacs = ENVB.get(b'CASK_EMACS')
         if emacs:
             return emacs
         emacs = ENVB.get(b'EMACS')
         if emacs != 't':
             return emacs
         return None

> Is there not a different solution to the bash problem? Passing
> --noediting as an option would work, I think, and it should work with
> both the old and new $EMACS handling of bash.
>

Part of the worry here is that programs other than Bash will be 
affected, and that we need to give people some time to adjust to the 
changed behavior of Emacs, by announcing the planned change in Emacs 25 
and then actually making the change in a later Emacs release.

Re: Considered Harmful 73d213: "Comint, term, and compile new set Emacs"

[Phillip Lord]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 04/05/2016 09:38 AM, Phillip Lord wrote:
>> I don't
>> have a workaround, I have limited the scope of the existing workaround
>> to Emacs-24.
>
> Would something like the following be an adequate workaround? The idea is to
> have code that works with both Emacs 25 and Emacs 24 (and Emacs 23 etc., for
> that matter):
>
>     def get_emacs_from_env():
>         emacs = ENVB.get(b'CASK_EMACS')
>         if emacs:
>             return emacs
>         emacs = ENVB.get(b'EMACS')
>         if emacs != 't':
>             return emacs
>         return None

Maybe. Given that I spent several weeks working on the last pull
request, it may take a while for me to get up the energy to try.

The main problem is that use of cask tends to weave in and out of
different Emacs and also Makefiles. So, I use this:

EMACS ?= emacs
CASK ?= cask

-include makefile-local

all: install test

install:
	EMACS=$(EMACS) cask install

Which (did) work fine on Emacs-25 but now fails because EMACS the
invocation ends up as the entirely useless EMACS=t cask install

So, the problem isn't one just for cask, but also make. As other people
have said, the only safe solution is to not use the EMACS variable. This
was suggested for cask, and I have added that also. But, ultimately,
EMACS is the obvious name and CASK_EMACS is not.

Still, all of this has been said before.

>> Is there not a different solution to the bash problem? Passing
>> --noediting as an option would work, I think, and it should work with
>> both the old and new $EMACS handling of bash.
>>
>
> Part of the worry here is that programs other than Bash will be affected, and
> that we need to give people some time to adjust to the changed behavior of
> Emacs, by announcing the planned change in Emacs 25 and then actually making
> the change in a later Emacs release.

Emacs doesn't really have a formal deprecation policy, I think. So, even
if the change is announced in Emacs-25, there will still be breakage in
Emacs-26. As Stefan said "if we wait, nothing is ever going to move".

http://lists.gnu.org/archive/html/emacs-devel/2015-03/msg01034.html

I'm also not convinced that a bug report should be enough to revert a
change during pre-test. Reverting a breaking change that has been applied
deliberately is, effectively a breaking change all of its own.

Anyway, I guess bug #20202 needs re-opening.

Phil

Re: Considered Harmful 73d213: "Comint, term, and compile new set Emacs"

[Paul Eggert]

On 04/05/2016 03:08 PM, Phillip Lord wrote:
> The main problem is that use of cask tends to weave in and out of
> different Emacs and also Makefiles. So, I use this:
>
> EMACS ?= emacs
> CASK ?= cask
>
> -include makefile-local
>
> all: install test
>
> install:
> 	EMACS=$(EMACS) cask install
>
> Which (did) work fine on Emacs-25

But that doesn't work on Emacs 24, right? And if you do something like 
the following:

     EMACS ?= emacs
     CASK ?= cask

     ifeq ($(EMACS),t)
       EMACS = emacs
     endif

     install:
         EMACS=$(EMACS) cask install

This should work with both Emacs 24 and emacs-25 current.

Re: Considered Harmful 73d213: "Comint, term, and compile new set Emacs"

[Phillip Lord]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 04/05/2016 03:08 PM, Phillip Lord wrote:
>> The main problem is that use of cask tends to weave in and out of
>> different Emacs and also Makefiles. So, I use this:
>>
>> EMACS ?= emacs
>> CASK ?= cask
>>
>> -include makefile-local
>>
>> all: install test
>>
>> install:
>> 	EMACS=$(EMACS) cask install
>>
>> Which (did) work fine on Emacs-25
>
> But that doesn't work on Emacs 24, right?

It doesn't have to work on Emacs-24. I don't use Emacs-24 to develop in,
only Emacs-25, at least partly because Emacs-25 doesn't (didn't) set
EMACS=t. 


> And if you do something like the
> following:
>
>     EMACS ?= emacs
>     CASK ?= cask
>
>     ifeq ($(EMACS),t)
>       EMACS = emacs
>     endif
>
>     install:
>         EMACS=$(EMACS) cask install
>
> This should work with both Emacs 24 and emacs-25 current.

At this juncture, I think the sensible thing is to give up on using
EMACS as both an environment variable and a makefile variable. Putting
hacks into my make file to distinguish between running inside and
outside of Emacs is not a solution. The same make file should work.

To repeat, my belief is that a behaviour that was installed
deliberately, and which has been on master/emacs-25 for a year should
not have been removed in pre-test. I think Stefan's patch needs
restoring. Are you prepared to do this?

Failing that, I will back out my changes to cask.

Phil

Re: Considered Harmful 73d213: "Comint, term, and compile new set Emacs"

[Paul Eggert]

On 04/06/2016 10:53 AM, Phillip Lord wrote:
> To repeat, my belief is that a behaviour that was installed
> deliberately, and which has been on master/emacs-25 for a year should
> not have been removed in pre-test. I think Stefan's patch needs
> restoring. Are you prepared to do this?

I think we should be more conservative and stick to the behavior that is 
in the latest stable Emacs release. Stefan wrote in May 2015 that the 
plan was to take this more-conservative approach; see 
<http://bugs.gnu.org/20484#40>. I'm sorry that you didn't know this and 
that it messed you up, but that's part of life when lived on the 
bleeding edge.

Re: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Thu, April 7, 2016 2:05 am, Paul Eggert wrote:
> On 04/06/2016 10:53 AM, Phillip Lord wrote:
>
>> To repeat, my belief is that a behaviour that was installed
>> deliberately, and which has been on master/emacs-25 for a year should not
>> have been removed in pre-test. I think Stefan's patch needs restoring.
>> Are you prepared to do this?
>>
>
> I think we should be more conservative and stick to the behavior that is
> in the latest stable Emacs release. Stefan wrote in May 2015 that the plan
> was to take this more-conservative approach; see
> <http://bugs.gnu.org/20484#40>. I'm sorry that you didn't know this and
> that it messed you up, but that's part of life when lived on the bleeding
> edge.


I disagree. There was a 10 month gap between Stefans email and the
reversion. Making a significant breaking change in pre-test is, I think,
wrong.

Still, if there is nothing that can be done, I will submit a new bug
report to reopen 20202 which is now not fixed.

Phil

Re: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Thu, April 7, 2016 2:05 am, Paul Eggert wrote:
> On 04/06/2016 10:53 AM, Phillip Lord wrote:
>
>> To repeat, my belief is that a behaviour that was installed
>> deliberately, and which has been on master/emacs-25 for a year should not
>> have been removed in pre-test. I think Stefan's patch needs restoring.
>> Are you prepared to do this?
>>
>
> I think we should be more conservative and stick to the behavior that is
> in the latest stable Emacs release. Stefan wrote in May 2015 that the plan
> was to take this more-conservative approach; see
> <http://bugs.gnu.org/20484#40>. I'm sorry that you didn't know this and
> that it messed you up, but that's part of life when lived on the bleeding
> edge.

As a better alternatively, would it be possible to find a solution to
20484 which does not involve EMACS=t?

This would both address 20484 and 20202, rather than robbing Peter to pay
Paul.

Phil

bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Paul Eggert]

On 04/07/2016 12:18 AM, Phillip Lord wrote:
> I will submit a new bug
> report to reopen 20202 which is now not fixed.

I reopened Bug#20202, so there should be no need to submit a new bug report.

For those joining this this via Bug#20202, this message is following up 
on a thread starting here:

http://lists.gnu.org/archive/html/emacs-devel/2016-04/msg00184.html

bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Paul Eggert]

On 04/07/2016 12:35 AM, Phillip Lord wrote:
> As a better alternatively, would it be possible to find a solution to
> 20484 which does not involve EMACS=t?
>
> This would both address 20484 and 20202, rather than robbing Peter to pay
> Paul.
I'm afraid nothing comes to mind. I'll CC: this to both bug reports to 
see if others have ideas.

Re: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Eli Zaretskii]

> Date: Thu, 7 Apr 2016 08:35:23 +0100
> From: "Phillip Lord" <phillip.lord@russet.org.uk>
> Cc: emacs-devel@gnu.org, eggert@penguin.cs.ucla.edu, monnier@iro.umontreal.ca,
> 	Phillip Lord <phillip.lord@russet.org.uk>
> 
> As a better alternatively, would it be possible to find a solution to
> 20484 which does not involve EMACS=t?
> 
> This would both address 20484 and 20202, rather than robbing Peter to pay
> Paul.

Bug#20484 is with a version of Bash that needs $EMACS to be set,
right?  If so, how can we solve it, if the changes to do that must be
made in Bash?

bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Thu, April 7, 2016 4:01 pm, Paul Eggert wrote:
> On 04/07/2016 12:35 AM, Phillip Lord wrote:
>
>> As a better alternatively, would it be possible to find a solution to
>> 20484 which does not involve EMACS=t?
>>
>>
>> This would both address 20484 and 20202, rather than robbing Peter to
>> pay Paul.
>>
> I'm afraid nothing comes to mind. I'll CC: this to both bug reports to
> see if others have ideas.

I can think of several possibilities. In particular, the EMACS=t behaviour
of bash should also be replicable with bash -o emacs. I'm happy to try
these out and see if it works, so long as the EMACS=t is not considered to
be a closed issue for Emacs 25.

Re: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Thu, April 7, 2016 4:07 pm, Eli Zaretskii wrote:
>> Date: Thu, 7 Apr 2016 08:35:23 +0100
>> From: "Phillip Lord" <phillip.lord@russet.org.uk>
>> Cc: emacs-devel@gnu.org, eggert@penguin.cs.ucla.edu,
>> monnier@iro.umontreal.ca, Phillip Lord <phillip.lord@russet.org.uk>
>>
>>
>> As a better alternatively, would it be possible to find a solution to
>> 20484 which does not involve EMACS=t?
>>
>>
>> This would both address 20484 and 20202, rather than robbing Peter to
>> pay Paul.
>>
>
> Bug#20484 is with a version of Bash that needs $EMACS to be set,
> right?

With all versions of bash. It's fixed in trunk, but not a release.

> If so, how can we solve it, if the changes to do that must be made
> in Bash?

There are a variety of ways, which I am happy to investigate. There is
always more than one way to skin a cat.

Phil

bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Paul Eggert]

On 04/07/2016 08:18 AM, Phillip Lord wrote:
> I can think of several possibilities. In particular, the EMACS=t behaviour
> of bash should also be replicable with bash -o emacs.
I expect that this problem affects programs other than bash. For 
example, tcsh 6.19.00 (the latest version of the first other shell that 
I checked) tests whether EMACS is "t".

Re: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Eli Zaretskii]

> Date: Thu, 7 Apr 2016 16:21:43 +0100
> From: "Phillip Lord" <phillip.lord@russet.org.uk>
> Cc: "Phillip Lord" <phillip.lord@russet.org.uk>,
>  eggert@cs.ucla.edu,
>  emacs-devel@gnu.org
> 
> > If so, how can we solve it, if the changes to do that must be made
> > in Bash?
> 
> There are a variety of ways, which I am happy to investigate. There is
> always more than one way to skin a cat.

We do want to be compatible with older versions of Bash.  If you can
suggest a solution that has this important property, please do.

bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Glenn Morris]

Paul Eggert wrote:

> On 04/07/2016 08:18 AM, Phillip Lord wrote:
>> I can think of several possibilities. In particular, the EMACS=t behaviour
>> of bash should also be replicable with bash -o emacs.
> I expect that this problem affects programs other than bash. For
> example, tcsh 6.19.00 (the latest version of the first other shell
> that I checked) tests whether EMACS is "t".

IMO this issue is effectively going to be impossible to change.
Eg is there any sign of tcsh adapting?
How long for that change to make it into the long-term stable release of
all major distributions?
How many other things need similar changes?
Just resign yourself to the fact that EMACS has a not-very useful value
inside comint.

bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Thu, April 7, 2016 4:25 pm, Paul Eggert wrote:
> On 04/07/2016 08:18 AM, Phillip Lord wrote:
>
>> I can think of several possibilities. In particular, the EMACS=t
>> behaviour of bash should also be replicable with bash -o emacs.
> I expect that this problem affects programs other than bash. For
> example, tcsh 6.19.00 (the latest version of the first other shell that I
> checked) tests whether EMACS is "t".

Well, I need to scope this. If the issue is tcsh and bash, then I will
look at both. I cannot, of course, look at any arbitrary program which
might be affected.

Phil

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Paul Eggert]

On 04/07/2016 09:07 AM, Phillip Lord wrote:
> Well, I need to scope this. If the issue is tcsh and bash, then I will
> look at both. I cannot, of course, look at any arbitrary program which
> might be affected.

I think we'd be OK if we work with the "common" shells. But that would 
include zsh, whose current FAQ says the following:

   Probably the most reliable way of dealing with this is to look for
     the environment variable `$EMACS', which is set to `t' in
     Emacs' shell mode.  Putting

       [[ $EMACS = t ]] && unsetopt zle

     in your .zshrc should be sufficient.

So here it's not merely a matter of fixing zsh, it's also fixing all the 
users' .zshrc files that are following this (obsolescent) advice.

So far we've looked at three shells (bash, tcsh, zsh), and found 
compatibility issues with all three. This is not a good sign.

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Stefan Monnier]

>> I can think of several possibilities. In particular, the EMACS=t behaviour
>> of bash should also be replicable with bash -o emacs.
> I expect that this problem affects programs other than bash.  For example,
> tcsh 6.19.00 (the latest version of the first other shell that I checked)
> tests whether EMACS is "t".

It's not clear to me what "affects" means here.
The problem explicitly reported in 20484 is that directory tracking is
"turned off".  This doesn't sound particularly terrible (as long as
there's an easy way to get back the desired directory tracking).

So I think to make an informed decision, we first need to figure out *how*
all those shells are affected by having $EMACS be set to something else
than "t".

Maybe the best solution is to stop messing with $EMACS by default (and
hence change the behavior of sub-shells in negative ways for some
users), and then provide an easy way for those users to get back the
"fully featured" sub-shell they love.


        Stefan

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 07 Apr 2016 12:51:51 -0400
> Cc: Phillip Lord <phillip.lord@russet.org.uk>, 20202@debbugs.gnu.org,
> 	20484@debbugs.gnu.org
> 
> Maybe the best solution is to stop messing with $EMACS by default (and
> hence change the behavior of sub-shells in negative ways for some
> users), and then provide an easy way for those users to get back the
> "fully featured" sub-shell they love.

I don't think this will satisfy users of those shells.

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Stefan Monnier]

>> Maybe the best solution is to stop messing with $EMACS by default (and
>> hence change the behavior of sub-shells in negative ways for some
>> users), and then provide an easy way for those users to get back the
>> "fully featured" sub-shell they love.
> I don't think this will satisfy users of those shells.

I don't think "satisfy" is sufficiently well defined to be useful in
this conversation.

There's clearly a tradeoff to be made between bug#20202 and bug#20484.

IOW, right now people who suffer from bug#20202 are not satisfied either.

The ideal situation (which I think is the one that we should strive for
in the long-run) is to have Emacs not touch the $EMACS var (which will
address bug#20202).  So the question is how to get there.

It will most likely involve a bit of annoyance to some users along
the way.
But I think replacing $EMACS by $INSIDE_EMACS in their ~/.zshrc, or
adding some magic one-liner in their ~/.emacs is a rather
minor annoyance.


        Stefan

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: eggert@cs.ucla.edu,  phillip.lord@russet.org.uk,  20202@debbugs.gnu.org,  20484@debbugs.gnu.org
> Date: Thu, 07 Apr 2016 14:58:07 -0400
> 
> >> Maybe the best solution is to stop messing with $EMACS by default (and
> >> hence change the behavior of sub-shells in negative ways for some
> >> users), and then provide an easy way for those users to get back the
> >> "fully featured" sub-shell they love.
> > I don't think this will satisfy users of those shells.
> 
> I don't think "satisfy" is sufficiently well defined to be useful in
> this conversation.

I think it is.

> There's clearly a tradeoff to be made between bug#20202 and bug#20484.

Experience has taught us that there's no real tradeoff, at least not
in the next few years.  As long as shells are in use which want
EMACS=t, we must leave that in place.

We already do as much as we can to alleviate the problems: set
INSIDE_EMACS as well, and don't set EMACS=t if it is already set.  I
don't see what we can possibly do more.

> IOW, right now people who suffer from bug#20202 are not satisfied either.

They are mostly those who bump into this in Makefile's, where it is
relatively easy to switch to another name.  It's inconvenient, but
easily fixed.  By contrast, users of shells cannot always easily
change what their shells expect in their sources.

> The ideal situation (which I think is the one that we should strive for
> in the long-run) is to have Emacs not touch the $EMACS var (which will
> address bug#20202).  So the question is how to get there.

Waiting is the only way I see.

bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 04/07/2016 09:07 AM, Phillip Lord wrote:
>> Well, I need to scope this. If the issue is tcsh and bash, then I will
>> look at both. I cannot, of course, look at any arbitrary program which
>> might be affected.
>
> I think we'd be OK if we work with the "common" shells. But that would include
> zsh, whose current FAQ says the following:
>
>   Probably the most reliable way of dealing with this is to look for
>     the environment variable `$EMACS', which is set to `t' in
>     Emacs' shell mode.  Putting
>
>       [[ $EMACS = t ]] && unsetopt zle
>
>     in your .zshrc should be sufficient.
>
> So here it's not merely a matter of fixing zsh, it's also fixing all the
> users' .zshrc files that are following this (obsolescent) advice.
>
> So far we've looked at three shells (bash, tcsh, zsh), and found compatibility
> issues with all three. This is not a good sign.

So, bash has a command line option to achieve the same thing as EMACS=t.

I've checked tcsh and as far as I can tell, here, there is no clear
solution. EMACS=t is used, and it's deep in the init code. In my hands,
directory tracking in tcsh does not work in ansi-term either way.

Zsh does not, AFAICT, use EMACS=t (it's hard to be sure searching
through the code, since most instances of "emacs" refer to zsh's
emulation of Emacs). In fact, though, as the FAQ entry you found shows,
zsh actually does this...

    /* unset zle if using zsh under emacs */
    if (!strcmp(term, "emacs"))
	opts[USEZLE] = 0;

which, according to the faq is behaviour from < Emacs-19.29.

So, zsh users already explicitly tell their zsh what to do.


Possible solutions:

1) For bash, launch with -o emacs, or call set -o emacs after launch.
2) zsh, launch and call unsetopt zle.
3) For tcsh, EMACS=t only needs to be correct during init. So, EMACS=t,
launch tcsh, set EMACS=what-ever-it-was-before.
4) Everything else -- move the "call-process" call to a single function
which people can advice as they choose.

Phil

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 04/07/2016 09:07 AM, Phillip Lord wrote:
>> Well, I need to scope this. If the issue is tcsh and bash, then I will
>> look at both. I cannot, of course, look at any arbitrary program which
>> might be affected.
>
> I think we'd be OK if we work with the "common" shells. But that would include
> zsh, whose current FAQ says the following:
>
>   Probably the most reliable way of dealing with this is to look for
>     the environment variable `$EMACS', which is set to `t' in
>     Emacs' shell mode.  Putting
>
>       [[ $EMACS = t ]] && unsetopt zle
>
>     in your .zshrc should be sufficient.
>
> So here it's not merely a matter of fixing zsh, it's also fixing all the
> users' .zshrc files that are following this (obsolescent) advice.
>
> So far we've looked at three shells (bash, tcsh, zsh), and found compatibility
> issues with all three. This is not a good sign.

Incidentally, I have looked again at #20484. What ever it is that is
supporting the directory tracking, it is not the EMACS=t behaviour of 
bash, since in ansi-term we have:

 (format "EMACS=%s (term:%s)" emacs-version term-protocol-version)

This usage will happily not break cask, since it was never supported in
the first place.

On the other hand, directory tracking works just fine in M-x shell in
both Emacs-25.0.91, and Emacs-25 head. So, the EMACS=t setting is not
an issue there either. And, dir tracking is not an issue at all for M-x
compile.

This suggests a simple fix: restore beaab89, except for the bit dealing
with ansi-term, which remains for the sake of future compatability. Both
bug reports are fixed. Anyone launching cask (or the make files Eli
Barzilay talked about in #20202) inside ansi-term may still have
problems.

ansi-term is already exceptional, note, because it does not obey the
"don't fiddle with EMACS if EMACS is already set" semantics.

Phil

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Stefan Monnier]

> in the next few years.  As long as shells are in use which want
> EMACS=t, we must leave that in place.

No, nothing forces us to leave that in place.  What is true is:

   As long as shells are in use which want EMACS=t, we should make sure
   that the affected users can still get the right behavior

> Waiting is the only way I see.

We've tried that for about 10 years now.
I'd be surprised if another 10 years will make any difference in
this respect.


        Stefan

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Stefan Monnier]

> So, bash has a command line option to achieve the same thing as EMACS=t.

or the code can be changed to use $INSIDE_EMACS
or the user could tell Emacs to set EMACS=t
or the user could upgrade to the (upcoming) next release of Bash.

4 options for Bash, one of them will actually become the default in the
future.  So Bash is not a big problem here.

> I've checked tcsh and as far as I can tell, here, there is no clear
> solution.  EMACS=t is used, and it's deep in the init code.  In my hands,
> directory tracking in tcsh does not work in ansi-term either way.

So, does anything break if we don't have EMACS=t?
It sounds like it's a non-issue.

> Zsh does not, AFAICT, use EMACS=t (it's hard to be sure searching
> through the code, since most instances of "emacs" refer to zsh's
> emulation of Emacs). In fact, though, as the FAQ entry you found shows,
> zsh actually does this...
>
>     /* unset zle if using zsh under emacs */
>     if (!strcmp(term, "emacs"))
> 	opts[USEZLE] = 0;
>
> which, according to the faq is behaviour from < Emacs-19.29.

Indeed, this code has been useless for many years already since Emacs
doesn't set $TERM to "emacs" any more.

> So, zsh users already explicitly tell their zsh what to do.

So we don't have any evidence that removing EMACS=t will have any effect
on Zsh users either.


        Stefan

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: eggert@cs.ucla.edu,  phillip.lord@russet.org.uk,  20202@debbugs.gnu.org,  20484@debbugs.gnu.org
> Date: Thu, 07 Apr 2016 18:01:37 -0400
> 
> > in the next few years.  As long as shells are in use which want
> > EMACS=t, we must leave that in place.
> 
> No, nothing forces us to leave that in place.  What is true is:
> 
>    As long as shells are in use which want EMACS=t, we should make sure
>    that the affected users can still get the right behavior

We will never recover from bug reports if the behavior they want is
not the default.

> > Waiting is the only way I see.
> 
> We've tried that for about 10 years now.
> I'd be surprised if another 10 years will make any difference in
> this respect.

Maybe so, but I don't see any problem with that.  We just tried to
make the wait shorter, and see what happened.  We should try again in
few years, maybe the results will be better then.  Hopefully.

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Thu, 07 Apr 2016 22:42:40 +0100
> Cc: 20202@debbugs.gnu.org, 20484@debbugs.gnu.org, monnier@iro.umontreal.ca
> 
> Incidentally, I have looked again at #20484. What ever it is that is
> supporting the directory tracking, it is not the EMACS=t behaviour of 
> bash, since in ansi-term we have:
> 
>  (format "EMACS=%s (term:%s)" emacs-version term-protocol-version)
> 
> This usage will happily not break cask, since it was never supported in
> the first place.
> 
> On the other hand, directory tracking works just fine in M-x shell in
> both Emacs-25.0.91, and Emacs-25 head. So, the EMACS=t setting is not
> an issue there either. And, dir tracking is not an issue at all for M-x
> compile.
> 
> This suggests a simple fix: restore beaab89, except for the bit dealing
> with ansi-term, which remains for the sake of future compatability. Both
> bug reports are fixed. Anyone launching cask (or the make files Eli
> Barzilay talked about in #20202) inside ansi-term may still have
> problems.
> 
> ansi-term is already exceptional, note, because it does not obey the
> "don't fiddle with EMACS if EMACS is already set" semantics.

EMACS=t is for the old versions of Bash and other shells that look at
that.  So your proposal doesn't solve those problems, AFAIU.

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Thu, 07 Apr 2016 20:55:02 +0100
> Cc: 20202@debbugs.gnu.org, 20484@debbugs.gnu.org, monnier@iro.umontreal.ca
> 
> 1) For bash, launch with -o emacs, or call set -o emacs after launch.
> 2) zsh, launch and call unsetopt zle.
> 3) For tcsh, EMACS=t only needs to be correct during init. So, EMACS=t,
> launch tcsh, set EMACS=what-ever-it-was-before.
> 4) Everything else -- move the "call-process" call to a single function
> which people can advice as they choose.

Thanks.  Such  solution, if it indeed works, could be OK for master,
but not for emacs-25.

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 07 Apr 2016 18:20:08 -0400
> Cc: Paul Eggert <eggert@cs.ucla.edu>, 20484@debbugs.gnu.org,
> 	20202@debbugs.gnu.org
> 
> So, does anything break if we don't have EMACS=t?
> It sounds like it's a non-issue.

That's not my conclusion, as I already wrote.

bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Andreas Schwab]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> 1) For bash, launch with -o emacs, or call set -o emacs after launch.

set -o emacs has nothing to do with no_line_editing.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> So, bash has a command line option to achieve the same thing as EMACS=t.
>
> or the code can be changed to use $INSIDE_EMACS
> or the user could tell Emacs to set EMACS=t
> or the user could upgrade to the (upcoming) next release of Bash.
>
> 4 options for Bash, one of them will actually become the default in the
> future.  So Bash is not a big problem here.


So, this turns out to be untrue. Bug #20484 is caused NOT by the lack of
EMACS=t. Ansi term sets 

EMACS="25.0.91.0 (term 0.91)"

which is notable not "t".

The code which checks this looks like this:

      running_under_emacs = (emacs != 0) || (term && STREQN (term, "emacs", 5));
      running_under_emacs += term && STREQN (term, "eterm", 5) && emacs && strstr (emacs, "term");


which sets running_under_emacs=2 in ansi-term.

This is checked later in eval.c.

 if (running_under_emacs == 2)
	send_pwd_to_eterm ();	/* Yuck */


(the /*Yuck*/ is their comment not mine!).

which does this:

static void
send_pwd_to_eterm ()
{
  char *pwd, *f;

  f = 0;
  pwd = get_string_value ("PWD");
  if (pwd == 0)
    f = pwd = get_working_directory ("eterm");
  fprintf (stderr, "\032/%s\n", pwd);
  free (f);
}


In otherwords, bash detects ansi-term specifically, and prints out PWD
every command. The workaround to this is (nearly) documented in the
headers of term.el. Adding this:

cd(){ command cd "$@";printf '\032/%s\n' "$PWD"; }

to .bashrc, and directory tracking works in Emacs-21.0.92 (with no EMACS
set).

The patch to bash that supposedly fixes bash to use INSIDE_EMACS looks
like this...


diff --git a/shell.c b/shell.c
index c0107ff..dc9015d 100644
--- a/shell.c
+++ b/shell.c
@@ -581,12 +581,13 @@ main (argc, argv, env)
       emacs = get_string_value ("EMACS");
 
       /* Not sure any emacs terminal emulator sets TERM=emacs any more */
-      no_line_editing |= term && (STREQ (term, "emacs"));
+      no_line_editing |= STREQ (term, "emacs");
       no_line_editing |= emacs && emacs[0] == 't' && emacs[1] == '\0' && STREQ (term, "dumb");
+      no_line_editing |= get_string_value ("INSIDE_EMACS") != 0;
 
       /* running_under_emacs == 2 for `eterm' */
-      running_under_emacs = (emacs != 0) || (term && STREQN (term, "emacs", 5));
-      running_under_emacs += term && STREQN (term, "eterm", 5) && emacs && strstr (emacs, "term");
+      running_under_emacs = (emacs != 0) || STREQN (term, "emacs", 5);
+      running_under_emacs += STREQN (term, "eterm", 5) && emacs && strstr (emacs, "term");
 
       if (running_under_emacs)
 	gnu_error_format = 1;


Which is not the problem in the first place. Future versions of bash
won't work either with ansi-term either.

There may be outstanding issues -- i.e. "gnu_error_format" is also set,
as is "no_line_editing". What practical implications these have for bash
under Emacs, I do not know. AFAICT, though, we had no bug reports from
users of M-x shell with bash for the year that "EMACS= ". We have a bug
report for ansi-term with bash, is all.



>> I've checked tcsh and as far as I can tell, here, there is no clear
>> solution.  EMACS=t is used, and it's deep in the init code.  In my hands,
>> directory tracking in tcsh does not work in ansi-term either way.
>
> So, does anything break if we don't have EMACS=t?
> It sounds like it's a non-issue.

Not in my hands. To get directory tracking in tcsh would require
porting the bash fix above.


>> Zsh does not, AFAICT, use EMACS=t (it's hard to be sure searching
>> through the code, since most instances of "emacs" refer to zsh's
>> emulation of Emacs). In fact, though, as the FAQ entry you found shows,
>> zsh actually does this...
>>
>>     /* unset zle if using zsh under emacs */
>>     if (!strcmp(term, "emacs"))
>> 	opts[USEZLE] = 0;
>>
>> which, according to the faq is behaviour from < Emacs-19.29.
>
> Indeed, this code has been useless for many years already since Emacs
> doesn't set $TERM to "emacs" any more.
>
>> So, zsh users already explicitly tell their zsh what to do.
>
> So we don't have any evidence that removing EMACS=t will have any effect
> on Zsh users either.


At the moment, I have some evidence from the bash and tcsh that they are
doing something with EMACS=t. But I do not, at the moment, know what the
practical implications of these something are; loss of dir-tracking is
not one of them.

Phil

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

Andreas Schwab <schwab@linux-m68k.org> writes:

> phillip.lord@russet.org.uk (Phillip Lord) writes:
>
>> 1) For bash, launch with -o emacs, or call set -o emacs after launch.
>
> set -o emacs has nothing to do with no_line_editing.

Apologies, you are correct. It is emacs line_editing mode.

As it turns out, but #20202 is not actually related to line_editing_mode
anyway.

Phil

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Cc: eggert@cs.ucla.edu,  phillip.lord@russet.org.uk,  20202@debbugs.gnu.org,  20484@debbugs.gnu.org
>> Date: Thu, 07 Apr 2016 14:58:07 -0400
>> 
>> >> Maybe the best solution is to stop messing with $EMACS by default (and
>> >> hence change the behavior of sub-shells in negative ways for some
>> >> users), and then provide an easy way for those users to get back the
>> >> "fully featured" sub-shell they love.
>> > I don't think this will satisfy users of those shells.
>> 
>> I don't think "satisfy" is sufficiently well defined to be useful in
>> this conversation.
>
> I think it is.
>
>> There's clearly a tradeoff to be made between bug#20202 and bug#20484.
>
> Experience has taught us that there's no real tradeoff, at least not
> in the next few years.  As long as shells are in use which want
> EMACS=t, we must leave that in place.

Both zsh and bash also check TERM=emacs, which isn't set.



> They are mostly those who bump into this in Makefile's, where it is
> relatively easy to switch to another name.  It's inconvenient, but
> easily fixed.  By contrast, users of shells cannot always easily
> change what their shells expect in their sources.

Should we patch the Emacs makefiles to stop making use of $(EMACS) then?

Phil

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  eggert@cs.ucla.edu,  20202@debbugs.gnu.org,  20484@debbugs.gnu.org
> Date: Fri, 08 Apr 2016 14:15:07 +0100
> 
> > They are mostly those who bump into this in Makefile's, where it is
> > relatively easy to switch to another name.  It's inconvenient, but
> > easily fixed.  By contrast, users of shells cannot always easily
> > change what their shells expect in their sources.
> 
> Should we patch the Emacs makefiles to stop making use of $(EMACS) then?

Yes.

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Glenn Morris]

Eli Zaretskii wrote:

> Maybe so, but I don't see any problem with that.  We just tried to
> make the wait shorter, and see what happened.  We should try again in
> few years, maybe the results will be better then.  Hopefully.

Waiting won't do anything. One should actively contact maintainers of
affected packages. It's our fault, not theirs.

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Glenn Morris]

Eli Zaretskii wrote:

>> Should we patch the Emacs makefiles to stop making use of $(EMACS) then?
>
> Yes.

Why? Emacs builds fine with "EMACS=blah" in the environment, since it
explicitly sets EMACS as needed. The only issue here is with Makefiles
that use their environment unsanitised.

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  phillip.lord@russet.org.uk,  eggert@cs.ucla.edu,  20484@debbugs.gnu.org,  20202@debbugs.gnu.org
> Date: Fri, 08 Apr 2016 11:32:08 -0400
> 
> Eli Zaretskii wrote:
> 
> > Maybe so, but I don't see any problem with that.  We just tried to
> > make the wait shorter, and see what happened.  We should try again in
> > few years, maybe the results will be better then.  Hopefully.
> 
> Waiting won't do anything. One should actively contact maintainers of
> affected packages. It's our fault, not theirs.

We already did.  "Waiting" here means wait until the old versions of
the shells die peacefully and people stop using them.

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: phillip.lord@russet.org.uk (Phillip Lord),  20202@debbugs.gnu.org,  eggert@cs.ucla.edu,  20484@debbugs.gnu.org,  monnier@iro.umontreal.ca
> Date: Fri, 08 Apr 2016 11:45:07 -0400
> 
> Eli Zaretskii wrote:
> 
> >> Should we patch the Emacs makefiles to stop making use of $(EMACS) then?
> >
> > Yes.
> 
> Why?

Because it's safer, and doesn't do any harm.

I agree it isn't the utmost urgency, though.

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Stefan Monnier]

> We will never recover from bug reports if the behavior they want is
> not the default.

The problem in bug#20484 is not due to a wrong behavior in Emacs but to
a wrong behavior in shells (or at the very least a bad interaction
between the two), so there's no point trying to "fix it" on our
side only, since that just introduces other bugs (witness bug#20202).

Regarding bug#20484 as pointed out by Philip Lord we can address it
without breaking bug#20484 by only reverting the behavior in ansi-term.


        Stefan

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Stefan Monnier]

> EMACS=t is for the old versions of Bash and other shells that look at
> that.

What change in behavior has been observed by removing EMACS=t from comint?

As far as I can tell, nobody has reported any change of behavior yet.

Bug#20484 is about another removal (the one in ansi-term), so it's
actually not relevant.


        Stefan

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Paul Eggert]

On 04/08/2016 09:46 AM, Stefan Monnier wrote:
> Regarding bug#20484 as pointed out by Philip Lord we can address it
> without breaking bug#20484 by only reverting the behavior in ansi-term.

Yes, this is the direction I'm heading as well. Thanks, Philip, for 
doing all the legwork on other shells. As Philip implies, Bash will need 
another fix before Bash 4.4 goes out, and I'm working on that now. I'll 
CC: here.

What a mess, eh?

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Fri, April 8, 2016 5:46 pm, Stefan Monnier wrote:
>> We will never recover from bug reports if the behavior they want is
>> not the default.
>
> The problem in bug#20484 is not due to a wrong behavior in Emacs but to
> a wrong behavior in shells (or at the very least a bad interaction between
> the two), so there's no point trying to "fix it" on our side only, since
> that just introduces other bugs (witness bug#20202).

I think the problem, at heart, stems from bash trying do the right thing
for Emacs. This is not made any easier by the fact that Emacs needs
different things, depending on whether we are in shell or ansi-term.
Really, Emacs should just explicitly request which features it needs,
rather than saying "I am Emacs, now be right".

> Regarding bug#20484 as pointed out by Philip Lord we can address it
> without breaking bug#20484 by only reverting the behavior in ansi-term.

And there is a simple workaround should we wish for consistent EMACS
behaviour in shell and term, which can be applied with in .bashrc or
having Emacs inject a command into the new shell.

It's "Phillip" actually --- the misspelling is one of the reasons I
normally go by "Phil".

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Fri, April 8, 2016 5:49 pm, Stefan Monnier wrote:
>> EMACS=t is for the old versions of Bash and other shells that look at
>> that.
>
> What change in behavior has been observed by removing EMACS=t from
> comint?
>
> As far as I can tell, nobody has reported any change of behavior yet.
>

I worried about this -- why did line-editing not cause problems, given
that it is this that EMACS=t disables.

So I tested this by doing this:

bind -x '"\C-r"':reset

which gives a warning

bash: bind: warning: line editing not enabled


It turns out that M-x shell gives a warning in emacs-25 head AND
emacs-25.0.91. So it appears that the bash  EMACS=t handling is redundant,
at least with respect to line editing. M-x shell passed --noediting
through explicit-bash-args.

Phil

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Fri, April 8, 2016 6:12 pm, Paul Eggert wrote:
> On 04/08/2016 09:46 AM, Stefan Monnier wrote:
>
>> Regarding bug#20484 as pointed out by Philip Lord we can address it
>> without breaking bug#20484 by only reverting the behavior in ansi-term.
>
> Yes, this is the direction I'm heading as well. Thanks, Philip, for
> doing all the legwork on other shells. As Philip implies, Bash will need
> another fix before Bash 4.4 goes out, and I'm working on that now. I'll
> CC: here.
>
>
> What a mess, eh?

A whole set of decisions which, individually, make sense all conspiring to
produce a disaster.

Phil

bug#20484: Bash 4.4-rc1 incompatibility with future Emacs $EMACS

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 801 bytes --]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu' 
-DCONF_VENDOR='unknown' -DLOCALEDIR='/tmp/prefix/share/locale' 
-DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -DDEBUG -DMALLOC_DEBUG -I. -I. 
-I./include -I./lib   -g -O2 -Wno-parentheses -Wno-format-security
uname output: Linux penguin.cs.ucla.edu 4.4.6-300.fc23.x86_64 #1 SMP Wed 
Mar 16 22:10:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-unknown-linux-gnu

Bash Version: 4.4
Patch Level: 0
Release Status: rc1

Description:
     Please see attached patch.

Repeat-By:
     Please see attached patch.

Fix:
     Please see attached patch.

[-- Attachment #2: 0001-Fix-compatibility-problem-with-INSIDE_EMACS-etc.patch --]
[-- Type: application/x-patch, Size: 3612 bytes --]

bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 565 bytes --]

Thanks for persisting with this, Phillip, and for doing the legwork in 
seeing what other shells do. It does seem that the EMACS='t' setting can 
safely be removed, given what we've seen. We will still need the 
EMACS='25.1 (term:0.96)' setting in term-exec-1 for quite some time, for 
compatibility with Bash 4.3 and earlier. I installed the attached patch 
into emacs-25 to try to get this done for the next release. This should 
not affect Bug#20484, which should still be fixed. Most of Bug#20202 
should be fixed now; the exceptions are M-x term and the like.

[-- Attachment #2: 0001-Comint-and-compile-no-longer-set-EMACS.patch --]
[-- Type: application/x-patch, Size: 3737 bytes --]

bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Thanks for persisting with this, Phillip, and for doing the legwork in seeing
> what other shells do.

No worries, and apologies for being somewhat grumpy at the start. We
seem to have a solution now, which is the main thing.

> It does seem that the EMACS='t' setting can safely be removed, given
> what we've seen. We will still need the EMACS='25.1 (term:0.96)'
> setting in term-exec-1 for quite some time, for compatibility with
> Bash 4.3 and earlier. I installed the attached patch into emacs-25 to
> try to get this done for the next release. This should not affect
> Bug#20484, which should still be fixed. Most of Bug#20202 should be
> fixed now; the exceptions are M-x term and the like.

I think this is a good compromise. Although, it's too late for Emacs
25.1, it would be possible to work around #20484 in other ways, and
maybe get dir-tracking working for other shells also.

Phil

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Glenn Morris]

Eli Zaretskii wrote:

>> Waiting won't do anything. One should actively contact maintainers of
>> affected packages. It's our fault, not theirs.
>
> We already did. 

Please note that the list of affected applications may include things
other than shells; eg

http://lists.gnu.org/archive/html/emacs-pretest-bug/2006-11/msg00035.html
http://lists.gnu.org/archive/html/emacs-pretest-bug/2006-11/msg00084.html

http://www.swi-prolog.org/pldoc/man?section=flags
"SWI-Prolog assumes this is the case if the environment variable EMACS
is t"

Since it is impossible to construct the definitive list of affected
applications, that is why I am personally pessimistic about being able
to change this without use of Emacs SOP: change it and see what breaks.

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Sat, April 9, 2016 3:24 am, Glenn Morris wrote:
> Eli Zaretskii wrote:
>
> Please note that the list of affected applications may include things
> other than shells; eg
>
> http://lists.gnu.org/archive/html/emacs-pretest-bug/2006-11/msg00035.html
>
> http://lists.gnu.org/archive/html/emacs-pretest-bug/2006-11/msg00084.html
>
>
> http://www.swi-prolog.org/pldoc/man?section=flags
> "SWI-Prolog assumes this is the case if the environment variable EMACS
> is t"
>
> Since it is impossible to construct the definitive list of affected
> applications, that is why I am personally pessimistic about being able to
> change this without use of Emacs SOP: change it and see what breaks.

So SWI-Prolog does still do this...

 if ( ((s = Getenv("EMACS", envbuf, sizeof(envbuf))) && s[0]) ||
       ((s = Getenv("INFERIOR", envbuf, sizeof(envbuf))) && streq(s,
"yes")) )

although, AFAICT, it just checks for EMACS=anything at all, so it's
possible to workaround this in a way consistent with
EMACS=location-of-emacs.

I'll contact them.

Phil

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Stefan Monnier]

> http://www.swi-prolog.org/pldoc/man?section=flags
> "SWI-Prolog assumes this is the case if the environment variable EMACS
> is t"

And here again, the page doesn't actually describe what would be the
consequence of a missing EMACS=t setting.  For all we know, it works
just fine without it.


        Stefan

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Sat, April 9, 2016 2:43 pm, Stefan Monnier wrote:
>> http://www.swi-prolog.org/pldoc/man?section=flags
>> "SWI-Prolog assumes this is the case if the environment variable EMACS
>> is t"
>
> And here again, the page doesn't actually describe what would be the
> consequence of a missing EMACS=t setting.  For all we know, it works just
> fine without it.


They've just patched SWI-Prolog to look at INSIDE_EMACS anyway.

I'm guessing that anyone who wants the EMACS=t behaviour should be able to
get it back by adding (setenv "EMACS" "t") to their .emacs?

Phil

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Paul Eggert]

Phillip Lord wrote:
> They've just patched SWI-Prolog to look at INSIDE_EMACS anyway.

Amusingly enough, the SWI-Prolog implementation (both unpatched and patched) 
disagrees with its documentation, which says SWI-Prolog assumes it is running 
under Emacs "if the environment variable EMACS is 't' and INFERIOR is 'yes'."

Note the restriction to 't' in the documentation (which the code does not 
enforce). Note also the "and" in the documentation (the code uses "||").

It sounds like this stuff isn't exercised much....

> I'm guessing that anyone who wants the EMACS=t behaviour should be able to
> get it back by adding (setenv "EMACS" "t") to their .emacs?

Yes, that works. Or in practice for SWI-Prolog (setenv "INFERIOR" "yes") should 
also do the trick.

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Stefan Monnier]

> Yes, that works. Or in practice for SWI-Prolog (setenv "INFERIOR" "yes")
> should also do the trick.

"Do the trick" to get what, exactly?


        Stefan

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1201 bytes --]

Stefan Monnier wrote:
> "Do the trick" to get what, exactly?

It causes the predicate 'current_prolog_flag(emacs_inferior_process, X)' to 
succeed with X=true, and it causes SWI-Prolog to not attempt to put the terminal 
into raw mode. Although the former doesn't seem to do much of anything, latter 
affects the UI, notably the debugger. For example, SWI-Prolog 7.2.3 in Emacs 24 
under M-x run-prolog:

?- trace, member(X, [a,b,c]).
    Call: (8) lists:member(_G2, [a, b, c]) ? a
% Execution Aborted
?-

The same interaction in emacs-25 now:

1 ?- trace, member(X, [a,b,c]).
    Call: (8) lists:member(_G2, [a, b, c]) ? a
abort
% Execution Aborted
2 ?-

I suppose the differences might be annoying to someone who does a lot of Prolog 
debugging. To give some perspective, although I am annoyed whenever I have to 
debug GNU Prolog under Emacs due to gprolog's mishandling of tty modes under 
Emacs, this is something I've never gotten sufficiently annoyed at to fix, even 
though it's been ten years since I first ran afoul of it.

One possible workaround would be something like the attached patch, which relies 
on SWI-Prolog's behavior of assuming GNU Emacs when INFERIOR=yes in the environment.

[-- Attachment #2: prolog.diff --]
[-- Type: text/x-diff, Size: 740 bytes --]

diff --git a/lisp/progmodes/prolog.el b/lisp/progmodes/prolog.el
index 9ee405b..fffa7cc 100644
--- a/lisp/progmodes/prolog.el
+++ b/lisp/progmodes/prolog.el
@@ -1374,8 +1374,10 @@ prolog-ensure-process
       ()
     (with-current-buffer (get-buffer-create "*prolog*")
       (prolog-inferior-mode)
-      (apply 'make-comint-in-buffer "prolog" (current-buffer)
-             (prolog-program-name) nil (prolog-program-switches))
+      (let ((process-environment
+             (cons "INFERIOR=yes" process-environment)))
+        (apply 'make-comint-in-buffer "prolog" (current-buffer)
+               (prolog-program-name) nil (prolog-program-switches)))
       (unless prolog-system
         ;; Setup auto-detection.
         (setq-local

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Michael Albinus]

"Phillip Lord" <phillip.lord@russet.org.uk> writes:

> I'm guessing that anyone who wants the EMACS=t behaviour should be able to
> get it back by adding (setenv "EMACS" "t") to their .emacs?

Not for remote processes.

> Phil

Best regards, Michael.

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Sun, April 10, 2016 12:40 am, Paul Eggert wrote:
> Phillip Lord wrote:
>
>> They've just patched SWI-Prolog to look at INSIDE_EMACS anyway.
>>
>
> Amusingly enough, the SWI-Prolog implementation (both unpatched and
> patched) disagrees with its documentation, which says SWI-Prolog assumes
> it is running under Emacs "if the environment variable EMACS is 't' and
> INFERIOR is 'yes'."
>
>
> Note the restriction to 't' in the documentation (which the code does not
>  enforce). Note also the "and" in the documentation (the code uses "||").

Yes, I noticed that as well. I couldn't patch the documentation because I
couldn't find the source.



> It sounds like this stuff isn't exercised much....


I wouldn't conclude that; more likely, it just works so no one has looked.
The situation with bash is the same -- Emacs requests --noediting, but
bash detects Emacs and enforces noediting anyway.

Phil



>
>> I'm guessing that anyone who wants the EMACS=t behaviour should be able
>> to get it back by adding (setenv "EMACS" "t") to their .emacs?
>
> Yes, that works. Or in practice for SWI-Prolog (setenv "INFERIOR" "yes")
> should also do the trick.
>
>
>
>

bug#20484: bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Sun, April 10, 2016 1:08 am, Stefan Monnier wrote:
>> Yes, that works. Or in practice for SWI-Prolog (setenv "INFERIOR"
>> "yes")
>> should also do the trick.
>
> "Do the trick" to get what, exactly?

Didn't look at the code deeply, but to switch off line editing.

Phil

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

On Sun, April 10, 2016 8:13 am, Michael Albinus wrote:
> "Phillip Lord" <phillip.lord@russet.org.uk> writes:
>
>
>> I'm guessing that anyone who wants the EMACS=t behaviour should be able
>> to get it back by adding (setenv "EMACS" "t") to their .emacs?
>
> Not for remote processes.


Didn't think of that. Can tramp be hooked to add this back, if needed?

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Michael Albinus]

"Phillip Lord" <phillip.lord@russet.org.uk> writes:

>>> I'm guessing that anyone who wants the EMACS=t behaviour should be able
>>> to get it back by adding (setenv "EMACS" "t") to their .emacs?
>>
>> Not for remote processes.
>
> Didn't think of that. Can tramp be hooked to add this back, if needed?

One could add it to tramp-remote-process-environment.

Best regards, Michael.

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Markus Triska]

"Phillip Lord" <phillip.lord@russet.org.uk> writes:

> Yes, I noticed that as well. I couldn't patch the documentation because I
> couldn't find the source.

The source of the SWI documentation is in the "man" directory, see
overview.doc. Please file pull requests for swipl-devel (not swipl).

Thank you for looking into this!

bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Stefan Monnier]

> ?- trace, member(X, [a,b,c]).
>    Call: (8) lists:member(_G2, [a, b, c]) ? a
> % Execution Aborted
> ?-
>
> The same interaction in emacs-25 now:
>
> 1 ?- trace, member(X, [a,b,c]).
>    Call: (8) lists:member(_G2, [a, b, c]) ? a
> abort
> % Execution Aborted
> 2 ?-

Hmm.. what are those "abort" and "2" in the output?

When running swipl inside a terminal I don't see these, so why do we see
them in M-x run-prolog?

> I suppose the differences might be annoying to someone who does a lot of
> Prolog debugging.

Indeed.

> I have to debug GNU Prolog under Emacs due to gprolog's mishandling of tty
> modes under Emacs, this is something I've never gotten sufficiently annoyed
> at to fix, even though it's been ten years since I first ran afoul of it.

FWIW, the current prolog.el has some tweaks for GNU Prolog which might
be relevant.  So you might like to try again and see if the problem is
still present.  If not, report it as a bug (and put me in X-Debbugs-Cc).


        Stefan

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Stefan Monnier]

> Didn't look at the code deeply, but to switch off line editing.

That still doesn't say what changes, concretely.


        Stefan

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Paul Eggert]

Stefan Monnier wrote:
> Hmm.. what are those "abort" and "2" in the output?
>
> When running swipl inside a terminal I don't see these, so why do we see
> them in M-x run-prolog?

I'm afraid I don't know, and don't have the patience to find out. The SWI-Prolog 
source code is not immediately obvious, and it doesn't appear to agree with its 
documentation. Rather than fool around with it I simply installed the 
INFERIOR=yes workaround into emacs-25/lisp/progmodes/prolog.el.

> FWIW, the current prolog.el has some tweaks for GNU Prolog which might
> be relevant.  So you might like to try again and see if the problem is
> still present.  If not, report it as a bug (and put me in X-Debbugs-Cc).

Thanks. Until yesterday I didn't know about M-x run-prolog. I just ran gprolog 
in a M-x shell window, which doesn't work well. I'll try to remember to use M-x 
run-prolog from now on.

bug#20484: bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Didn't look at the code deeply, but to switch off line editing.
>
> That still doesn't say what changes, concretely.


Well, what ever it doesn't do without EMACS=t, it will now, since it's
all been patched. I think I can stop worrying about it.

Phil

bug#20202: bug#20484: bug#20202: Considered Harmful 73d213: 'Comint, term, and compile new set Emacs'

[Phillip Lord]

Markus Triska <triska@metalevel.at> writes:

> "Phillip Lord" <phillip.lord@russet.org.uk> writes:
>
>> Yes, I noticed that as well. I couldn't patch the documentation because I
>> couldn't find the source.
>
> The source of the SWI documentation is in the "man" directory, see
> overview.doc. Please file pull requests for swipl-devel (not swipl).

I sent the PR here, and it's been merged.

https://github.com/SWI-Prolog/swipl/pull/6

Are you saying that this is wrong?

Phil

bug#20202: EMACS=t Joy and Happiness

[Phillip Lord]

Well, it hardly seems like Mon, 11 Apr 2016 since last we discussed this
issue.

If you remember, we managed to blitz, EMACS=t everywhere, then Paul
restored it in term.el because it broke bash otherwise. master currently
looks like this:

  ;; This is for backwards compatibility with Bash 4.3 and earlier.
  ;; Remove this hack once Bash 4.4-or-later is common, because
  ;; it breaks './configure' of some packages that expect it to
  ;; say where to find EMACS.
  (format "EMACS=%s (term:%s)" emacs-version term-protocol-version)

I worry that "Bash 4.4 or later is common" is rather indeterminate; my
desktop is now 4.4.19. but another machine I am using has bash
4.2.46. Given that Emacs-26 is now destined to come out with
EMACS=26.1.0, I would like to propose that we now remove this on master,
i.e. make it happen for Emacs-27 which should be in about 2020.

Phil