bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Dima Kogan]

I'm using a very recent emacs snapshot. shell-mode no longer renders the
ANSI-code colors correctly. Even with a relatively stock bash, All
output is rendered in yellow except for the prompt line. The prompt
isn't yellow, but it's wrong also:

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11883

This is a regression, since both the output in general and the prompt in
particular have been rendered correctly previously. I did a bisection,
and tracked the issues down to two commits. The prompt coloring broke in

http://git.savannah.gnu.org/cgit/emacs.git/commit/lisp/ansi-color.el?id=cd49cea06bb1c492d4d024889f3f320bf4c9b507

After that commit the general output still looks correct, but the prompt
isn't colored. Then later on, the rest of the output rendering broke in

http://git.savannah.gnu.org/cgit/emacs.git/commit/lisp/ansi-color.el?id=1bc85494c3236fa27cd8456635a4109aeed48344

If I back out these two commits, the colors look correct, same as they
did in emacs23. It's still not perfect, but it's 99% right instead of
99% wrong.

Thanks.



In GNU Emacs 24.2.50.1 (x86_64-pc-linux-gnu, X toolkit, Xaw scroll bars)
 of 2012-10-18 on dex, modified by Debian
 (emacs-snapshot package, version 2:20121101-1)
Windowing system distributor `The X.Org Foundation', version 11.0.11204000
System Description:	Debian GNU/Linux unstable (sid)

Configured using:
 `configure '--build' 'x86_64-linux-gnu' '--host' 'x86_64-linux-gnu'
 '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib'
 '--localstatedir=/var' '--infodir=/usr/share/info'
 '--mandir=/usr/share/man' '--with-pop=yes'
 '--enable-locallisppath=/etc/emacs-snapshot:/etc/emacs:/usr/local/share/emacs/24.2.50/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.2.50/site-lisp:/usr/share/emacs/site-lisp'
 '--without-compress-info' '--with-crt-dir=/usr/lib/x86_64-linux-gnu/'
 '--with-x=yes' '--with-x-toolkit=lucid' 'build_alias=x86_64-linux-gnu'
 'host_alias=x86_64-linux-gnu' 'CFLAGS=-DDEBIAN
 -DSITELOAD_PURESIZE_EXTRA=5000 -g -O2' 'LDFLAGS=-g -Wl,--as-needed
 -znocombreloc' 'CPPFLAGS=-D_FORTIFY_SOURCE=2''

Important settings:
  value of $LC_ALL: C
  value of $LANG: C
  locale-coding-system: nil
  default enable-multibyte-characters: t

Major mode: Fundamental

Minor modes in effect:
  which-function-mode: t
  diff-auto-refine-mode: t
  erc-list-mode: t
  erc-menu-mode: t
  erc-autojoin-mode: t
  erc-ring-mode: t
  erc-networks-mode: t
  erc-pcomplete-mode: t
  erc-track-mode: t
  erc-match-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-netsplit-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  global-hl-line-mode: t
  global-subword-mode: t
  subword-mode: t
  show-paren-mode: t
  shell-dirtrack-mode: t
  auto-insert-mode: t
  winner-mode: t
  iswitchb-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <C-kp-end> M-k <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> M-k <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
M-k <down> <down> <M-return> <C-right> <C-tab> <C-right> 
<C-right> <C-left> <C-left> C-s C-w C-w C-w C-w C-w 
C-w C-w C-w C-w <C-tab> C-s C-s C-s C-s C-s C-s C-s 
C-s <down> <down> <up> <C-tab> <down> <right> C-SPC 
<down> <down> <down> M-w <C-tab> C-y <up> <delete> 
<up> <delete> <up> <delete> C-x C-s <up> <up> C-x 1 
C-x C-w ~ / . e m a c s . d <return> C-x C-f <backspace> 
<backspace> <backspace> <return> <up> C-x C-f <up> 
C-g C-x C-f <up> . d <return> g <down> <up> <down> 
<up> C-s C-w C-w C-w C-w <up> C-x b <return> C-s C-s 
C-s C-s <up> C-s a c e - <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <right> <right> 
<right> <f1> f <return> <down> <down> <C-tab> <down> 
<down> <down> <down> C-x k <return> <up> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <up> 
C-s <up> <up> <up> <down> <down> <down> C-s C-w C-w 
<C-S-iso-lefttab> C-s C-s <C-kp-home> C-s C-s <C-tab> 
q <up> <up> <up> <up> <up> <up> M-x r e p o r t <tab> 
e <tab> <return>

Recent messages:
Saving file /home/dima/.emacs.d/ansi-color.el...
Wrote /home/dima/.emacs.d/ansi-color.el
Quit
Mark saved where search started [2 times]
Error during redisplay: (eval (replace-regexp-in-string "%" "%%" (gethash (selected-window) which-func-table which-func-unknown))) signaled (wrong-type-argument arrayp nil) [4 times]
Type C-x 1 to delete the help window.
Mark saved where search started [2 times]
Mark set
Mark saved where search started
When done with this frame, type C-x 5 0

Load-path shadows:
/usr/share/emacs/24.2.50/site-lisp/cscope/xcscope hides /usr/share/emacs/site-lisp/xcscope
/usr/share/emacs/24.2.50/site-lisp/cmake-data/cmake-mode hides /usr/share/emacs/site-lisp/cmake-mode
/usr/share/emacs-snapshot/site-lisp/easypg/epg hides /usr/share/emacs/24.2.50/lisp/epg
/usr/share/emacs-snapshot/site-lisp/easypg/epa-dired hides /usr/share/emacs/24.2.50/lisp/epa-dired
/home/dima/.emacs.d/ansi-color hides /usr/share/emacs/24.2.50/lisp/ansi-color
/usr/share/emacs-snapshot/site-lisp/easypg/epg-config hides /usr/share/emacs/24.2.50/lisp/epg-config
/usr/share/emacs-snapshot/site-lisp/easypg/epa hides /usr/share/emacs/24.2.50/lisp/epa
/usr/share/emacs-snapshot/site-lisp/easypg/epa-file hides /usr/share/emacs/24.2.50/lisp/epa-file
/usr/share/emacs-snapshot/site-lisp/easypg/epa-mail hides /usr/share/emacs/24.2.50/lisp/epa-mail
/usr/share/emacs-snapshot/site-lisp/org-mode/org-list hides /usr/share/emacs/24.2.50/lisp/org/org-list
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-screen hides /usr/share/emacs/24.2.50/lisp/org/ob-screen
/usr/share/emacs-snapshot/site-lisp/org-mode/org-protocol hides /usr/share/emacs/24.2.50/lisp/org/org-protocol
/usr/share/emacs-snapshot/site-lisp/org-mode/org-footnote hides /usr/share/emacs/24.2.50/lisp/org/org-footnote
/usr/share/emacs-snapshot/site-lisp/org-mode/org-docview hides /usr/share/emacs/24.2.50/lisp/org/org-docview
/usr/share/emacs-snapshot/site-lisp/org-mode/org-pcomplete hides /usr/share/emacs/24.2.50/lisp/org/org-pcomplete
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-sh hides /usr/share/emacs/24.2.50/lisp/org/ob-sh
/usr/share/emacs-snapshot/site-lisp/org-mode/org-irc hides /usr/share/emacs/24.2.50/lisp/org/org-irc
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-maxima hides /usr/share/emacs/24.2.50/lisp/org/ob-maxima
/usr/share/emacs-snapshot/site-lisp/org-mode/org-mac-message hides /usr/share/emacs/24.2.50/lisp/org/org-mac-message
/usr/share/emacs-snapshot/site-lisp/org-mode/org-bibtex hides /usr/share/emacs/24.2.50/lisp/org/org-bibtex
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-asymptote hides /usr/share/emacs/24.2.50/lisp/org/ob-asymptote
/usr/share/emacs-snapshot/site-lisp/org-mode/org-bbdb hides /usr/share/emacs/24.2.50/lisp/org/org-bbdb
/usr/share/emacs-snapshot/site-lisp/org-mode/ob hides /usr/share/emacs/24.2.50/lisp/org/ob
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-lilypond hides /usr/share/emacs/24.2.50/lisp/org/ob-lilypond
/usr/share/emacs-snapshot/site-lisp/org-mode/org-exp-blocks hides /usr/share/emacs/24.2.50/lisp/org/org-exp-blocks
/usr/share/emacs-snapshot/site-lisp/org-mode/org-icalendar hides /usr/share/emacs/24.2.50/lisp/org/org-icalendar
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-gnuplot hides /usr/share/emacs/24.2.50/lisp/org/ob-gnuplot
/usr/share/emacs-snapshot/site-lisp/org-mode/org-src hides /usr/share/emacs/24.2.50/lisp/org/org-src
/usr/share/emacs-snapshot/site-lisp/org-mode/org-w3m hides /usr/share/emacs/24.2.50/lisp/org/org-w3m
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-css hides /usr/share/emacs/24.2.50/lisp/org/ob-css
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-ledger hides /usr/share/emacs/24.2.50/lisp/org/ob-ledger
/usr/share/emacs-snapshot/site-lisp/org-mode/org-wl hides /usr/share/emacs/24.2.50/lisp/org/org-wl
/usr/share/emacs-snapshot/site-lisp/org-mode/org-mew hides /usr/share/emacs/24.2.50/lisp/org/org-mew
/usr/share/emacs-snapshot/site-lisp/org-mode/org-mks hides /usr/share/emacs/24.2.50/lisp/org/org-mks
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-python hides /usr/share/emacs/24.2.50/lisp/org/ob-python
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-shen hides /usr/share/emacs/24.2.50/lisp/org/ob-shen
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-tangle hides /usr/share/emacs/24.2.50/lisp/org/ob-tangle
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-haskell hides /usr/share/emacs/24.2.50/lisp/org/ob-haskell
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-eval hides /usr/share/emacs/24.2.50/lisp/org/ob-eval
/usr/share/emacs-snapshot/site-lisp/org-mode/org-rmail hides /usr/share/emacs/24.2.50/lisp/org/org-rmail
/usr/share/emacs-snapshot/site-lisp/org-mode/org-habit hides /usr/share/emacs/24.2.50/lisp/org/org-habit
/usr/share/emacs-snapshot/site-lisp/org-mode/org-exp hides /usr/share/emacs/24.2.50/lisp/org/org-exp
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-mscgen hides /usr/share/emacs/24.2.50/lisp/org/ob-mscgen
/usr/share/emacs-snapshot/site-lisp/org-mode/org-gnus hides /usr/share/emacs/24.2.50/lisp/org/org-gnus
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-plantuml hides /usr/share/emacs/24.2.50/lisp/org/ob-plantuml
/usr/share/emacs-snapshot/site-lisp/org-mode/org-ascii hides /usr/share/emacs/24.2.50/lisp/org/org-ascii
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-octave hides /usr/share/emacs/24.2.50/lisp/org/ob-octave
/usr/share/emacs-snapshot/site-lisp/org-mode/org-entities hides /usr/share/emacs/24.2.50/lisp/org/org-entities
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-dot hides /usr/share/emacs/24.2.50/lisp/org/ob-dot
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-latex hides /usr/share/emacs/24.2.50/lisp/org/ob-latex
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-java hides /usr/share/emacs/24.2.50/lisp/org/ob-java
/usr/share/emacs-snapshot/site-lisp/org-mode/org-indent hides /usr/share/emacs/24.2.50/lisp/org/org-indent
/usr/share/emacs-snapshot/site-lisp/org-mode/org-table hides /usr/share/emacs/24.2.50/lisp/org/org-table
/usr/share/emacs-snapshot/site-lisp/org-mode/org-eshell hides /usr/share/emacs/24.2.50/lisp/org/org-eshell
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-org hides /usr/share/emacs/24.2.50/lisp/org/ob-org
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-exp hides /usr/share/emacs/24.2.50/lisp/org/ob-exp
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-keys hides /usr/share/emacs/24.2.50/lisp/org/ob-keys
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-ref hides /usr/share/emacs/24.2.50/lisp/org/ob-ref
/usr/share/emacs-snapshot/site-lisp/org-mode/org-id hides /usr/share/emacs/24.2.50/lisp/org/org-id
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-sass hides /usr/share/emacs/24.2.50/lisp/org/ob-sass
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-picolisp hides /usr/share/emacs/24.2.50/lisp/org/ob-picolisp
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-sqlite hides /usr/share/emacs/24.2.50/lisp/org/ob-sqlite
/usr/share/emacs-snapshot/site-lisp/org-mode/org-xoxo hides /usr/share/emacs/24.2.50/lisp/org/org-xoxo
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-calc hides /usr/share/emacs/24.2.50/lisp/org/ob-calc
/usr/share/emacs-snapshot/site-lisp/org-mode/org-mhe hides /usr/share/emacs/24.2.50/lisp/org/org-mhe
/usr/share/emacs-snapshot/site-lisp/org-mode/org hides /usr/share/emacs/24.2.50/lisp/org/org
/usr/share/emacs-snapshot/site-lisp/org-mode/org-docbook hides /usr/share/emacs/24.2.50/lisp/org/org-docbook
/usr/share/emacs-snapshot/site-lisp/org-mode/org-mouse hides /usr/share/emacs/24.2.50/lisp/org/org-mouse
/usr/share/emacs-snapshot/site-lisp/org-mode/org-html hides /usr/share/emacs/24.2.50/lisp/org/org-html
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-ditaa hides /usr/share/emacs/24.2.50/lisp/org/ob-ditaa
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-ocaml hides /usr/share/emacs/24.2.50/lisp/org/ob-ocaml
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-lisp hides /usr/share/emacs/24.2.50/lisp/org/ob-lisp
/usr/share/emacs-snapshot/site-lisp/org-mode/org-jsinfo hides /usr/share/emacs/24.2.50/lisp/org/org-jsinfo
/usr/share/emacs-snapshot/site-lisp/org-mode/org-odt hides /usr/share/emacs/24.2.50/lisp/org/org-odt
/usr/share/emacs-snapshot/site-lisp/org-mode/org-macs hides /usr/share/emacs/24.2.50/lisp/org/org-macs
/usr/share/emacs-snapshot/site-lisp/org-mode/org-freemind hides /usr/share/emacs/24.2.50/lisp/org/org-freemind
/usr/share/emacs-snapshot/site-lisp/org-mode/org-archive hides /usr/share/emacs/24.2.50/lisp/org/org-archive
/usr/share/emacs-snapshot/site-lisp/org-mode/org-latex hides /usr/share/emacs/24.2.50/lisp/org/org-latex
/usr/share/emacs-snapshot/site-lisp/org-mode/org-crypt hides /usr/share/emacs/24.2.50/lisp/org/org-crypt
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-emacs-lisp hides /usr/share/emacs/24.2.50/lisp/org/ob-emacs-lisp
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-clojure hides /usr/share/emacs/24.2.50/lisp/org/ob-clojure
/usr/share/emacs-snapshot/site-lisp/org-mode/org-lparse hides /usr/share/emacs/24.2.50/lisp/org/org-lparse
/usr/share/emacs-snapshot/site-lisp/org-mode/org-colview hides /usr/share/emacs/24.2.50/lisp/org/org-colview
/usr/share/emacs-snapshot/site-lisp/org-mode/org-taskjuggler hides /usr/share/emacs/24.2.50/lisp/org/org-taskjuggler
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-matlab hides /usr/share/emacs/24.2.50/lisp/org/ob-matlab
/usr/share/emacs-snapshot/site-lisp/org-mode/org-faces hides /usr/share/emacs/24.2.50/lisp/org/org-faces
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-table hides /usr/share/emacs/24.2.50/lisp/org/ob-table
/usr/share/emacs-snapshot/site-lisp/org-mode/org-capture hides /usr/share/emacs/24.2.50/lisp/org/org-capture
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-js hides /usr/share/emacs/24.2.50/lisp/org/ob-js
/usr/share/emacs-snapshot/site-lisp/org-mode/org-mobile hides /usr/share/emacs/24.2.50/lisp/org/org-mobile
/usr/share/emacs-snapshot/site-lisp/org-mode/org-agenda hides /usr/share/emacs/24.2.50/lisp/org/org-agenda
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-sql hides /usr/share/emacs/24.2.50/lisp/org/ob-sql
/usr/share/emacs-snapshot/site-lisp/org-mode/org-special-blocks hides /usr/share/emacs/24.2.50/lisp/org/org-special-blocks
/usr/share/emacs-snapshot/site-lisp/org-mode/org-beamer hides /usr/share/emacs/24.2.50/lisp/org/org-beamer
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-fortran hides /usr/share/emacs/24.2.50/lisp/org/ob-fortran
/usr/share/emacs-snapshot/site-lisp/org-mode/org-timer hides /usr/share/emacs/24.2.50/lisp/org/org-timer
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-perl hides /usr/share/emacs/24.2.50/lisp/org/ob-perl
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-R hides /usr/share/emacs/24.2.50/lisp/org/ob-R
/usr/share/emacs-snapshot/site-lisp/org-mode/org-ctags hides /usr/share/emacs/24.2.50/lisp/org/org-ctags
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-ruby hides /usr/share/emacs/24.2.50/lisp/org/ob-ruby
/usr/share/emacs-snapshot/site-lisp/org-mode/org-compat hides /usr/share/emacs/24.2.50/lisp/org/org-compat
/usr/share/emacs-snapshot/site-lisp/org-mode/org-feed hides /usr/share/emacs/24.2.50/lisp/org/org-feed
/usr/share/emacs-snapshot/site-lisp/org-mode/org-publish hides /usr/share/emacs/24.2.50/lisp/org/org-publish
/usr/share/emacs-snapshot/site-lisp/org-mode/org-remember hides /usr/share/emacs/24.2.50/lisp/org/org-remember
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-C hides /usr/share/emacs/24.2.50/lisp/org/ob-C
/usr/share/emacs-snapshot/site-lisp/org-mode/org-vm hides /usr/share/emacs/24.2.50/lisp/org/org-vm
/usr/share/emacs-snapshot/site-lisp/org-mode/org-datetree hides /usr/share/emacs/24.2.50/lisp/org/org-datetree
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-awk hides /usr/share/emacs/24.2.50/lisp/org/ob-awk
/usr/share/emacs-snapshot/site-lisp/org-mode/org-plot hides /usr/share/emacs/24.2.50/lisp/org/org-plot
/usr/share/emacs-snapshot/site-lisp/org-mode/org-install hides /usr/share/emacs/24.2.50/lisp/org/org-install
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-scheme hides /usr/share/emacs/24.2.50/lisp/org/ob-scheme
/usr/share/emacs-snapshot/site-lisp/org-mode/org-attach hides /usr/share/emacs/24.2.50/lisp/org/org-attach
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-lob hides /usr/share/emacs/24.2.50/lisp/org/ob-lob
/usr/share/emacs-snapshot/site-lisp/org-mode/ob-comint hides /usr/share/emacs/24.2.50/lisp/org/ob-comint
/usr/share/emacs-snapshot/site-lisp/org-mode/org-inlinetask hides /usr/share/emacs/24.2.50/lisp/org/org-inlinetask
/usr/share/emacs-snapshot/site-lisp/org-mode/org-clock hides /usr/share/emacs/24.2.50/lisp/org/org-clock
/usr/share/emacs-snapshot/site-lisp/org-mode/org-info hides /usr/share/emacs/24.2.50/lisp/org/org-info

Features:
(shadow sort mail-extr emacsbug message rfc822 mml mml-sec mm-decode
mm-bodies mm-encode mailabbrev gmm-utils mailheader whitespace
octave-inf octave-mod thai-util thai-word crm etags todoo gud dirtrack
doc-view image-mode ps-mode sendmail mail-utils electric apropos
cus-theme eieio-custom wid-browse tree-widget cpp cmacexp gnuplot-gui
gnuplot info-look info erc-truncate erc-speedbar speedbar sb-image
ezimage dframe erc-sound erc-services erc-replace erc-page erc-notify
erc-desktop-notifications notifications dbus xml erc-log erc-identd
erc-ibuffer ibuf-ext ibuffer erc-ezbounce erc-dcc erc-capab erc-autoaway
goto-addr view underline boxquote ffap url-parse url-vars autoconf
autoconf-mode tar-mode deb-view tramp-cmds pod-mode debian-copyright
align grep compile debian-bug debian-changelog-mode debian-control-mode
idlwave idlwave-help idlw-help hideif cc-langs cc-mode cc-fonts cc-guess
cc-menus cc-cmds conf-mode skeleton man pcmpl-unix vc-annotate log-view
org-attach org-id jka-compr erc-imenu make-mode mail-parse rfc2231
rfc2047 rfc2045 ietf-drums mailcap tabify sgml-mode sh-script smie
executable which-func imenu tramp-cache tramp-sh tramp tramp-compat
tramp-loaddefs smerge-mode ispell org-table org-archive dabbrev misearch
multi-isearch diff-mode dired-aux epa-dired dired ediff-merg ediff-diff
ediff-wind ediff-help ediff-util ediff-mult ediff-init ediff vc-git
org-wl org-w3m org-vm org-rmail org-mhe org-mew org-irc org-jsinfo
org-infojs org-html org-exp ob-exp org-exp-blocks find-func org-agenda
org-info org-gnus org-docview org-bibtex bibtex org-bbdb org warnings
ob-emacs-lisp ob-tangle ob-ref ob-lob ob-table org-footnote org-src
ob-comint ob-keys ob ob-eval org-pcomplete org-list org-faces org-compat
org-entities org-macs noutline outline cal-menu calendar cal-loaddefs
help-mode home-end browse-url face-remap mule-util network-stream
starttls tls erc-menu erc-join erc-ring erc-networks erc-pcomplete
erc-track erc-match erc-button erc-fill erc-stamp erc-netsplit
erc-goodies erc erc-backend erc-compat format-spec auth-source eieio
byte-opt bytecomp byte-compile cconv gnus-util mm-util mail-prsvr
password-cache thingatpt pp server hl-line cc-styles cc-align cc-engine
cc-vars cc-defs cus-edit wid-edit subword time-date paren cus-start
cus-load shell pcomplete comint ansi-color hideshow autoinsert winner
cycle-buffer commit-patch-buffer log-edit pcvs-util add-log vc
vc-dispatcher edmacro kmacro ace-jump-mode cl-macs gv jump-char cl
cl-lib rect cperl-mode shell-command advice help-fns advice-preload
xcscope ring easymenu iswitchb debian-el debian-el-loaddefs org-install
magit-install emacs-goodies-el emacs-goodies-custom
emacs-goodies-loaddefs easy-mmode epa-setup epa-file epa derived epg
epg-config epg-package-info dpkg-dev-el dpkg-dev-el-loaddefs tooltip
ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face macroexp files text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
dynamic-setting system-font-setting font-render-setting x-toolkit x
multi-tty emacs)

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Chong Yidong]

Dima Kogan <dima@secretsauce.net> writes:

> I'm using a very recent emacs snapshot. shell-mode no longer renders the
> ANSI-code colors correctly. Even with a relatively stock bash, All
> output is rendered in yellow except for the prompt line. The prompt
> isn't yellow, but it's wrong also

I can't reproduce it with

emacs -Q
M-x shell RET
ls --color RET

The output of ls is colorized correctly.  (Latest emacs-24 brach,
x86_64-unknown-linux-gnu, GTK+ Version 3.4.2.)

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Wolfgang Jenkner]

On Sat, Nov 03 2012, Dima Kogan wrote:

> I'm using a very recent emacs snapshot. shell-mode no longer renders the
> ANSI-code colors correctly. Even with a relatively stock bash, All
> output is rendered in yellow except for the prompt line. The prompt
> isn't yellow, but it's wrong also:
>
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11883
>
> This is a regression, since both the output in general and the prompt in
> particular have been rendered correctly previously. I did a bisection,
> and tracked the issues down to two commits. The prompt coloring broke in
>
> http://git.savannah.gnu.org/cgit/emacs.git/commit/lisp/ansi-color.el?id=cd49cea06bb1c492d4d024889f3f320bf4c9b507
>
> After that commit the general output still looks correct, but the prompt
> isn't colored. Then later on, the rest of the output rendering broke in
>
> http://git.savannah.gnu.org/cgit/emacs.git/commit/lisp/ansi-color.el?id=1bc85494c3236fa27cd8456635a4109aeed48344

In bug#11883 you give this example

echo -e "\033[31mtest\033[39m"

but ansi-color has never supported SGR parameter 39.  However, before my
change (the latter commit above), any unknown parameter would have the
same effect as parameter 0 (turn off all rendition aspects), while it is
simply ignored now (well, except for an "Invalid face reference: nil"
message), see `ansi-color-apply-sequence'.  Perhaps, this explains the
problem.

Wolfgang

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Eli Zaretskii]

> From: Wolfgang Jenkner <wjenkner@inode.at>
> Date: Sat, 03 Nov 2012 16:37:59 +0100
> Cc: 12792@debbugs.gnu.org
> 
> In bug#11883 you give this example
> 
> echo -e "\033[31mtest\033[39m"
> 
> but ansi-color has never supported SGR parameter 39.  However, before my
> change (the latter commit above), any unknown parameter would have the
> same effect as parameter 0 (turn off all rendition aspects), while it is
> simply ignored now (well, except for an "Invalid face reference: nil"
> message), see `ansi-color-apply-sequence'.  Perhaps, this explains the
> problem.

SGR parameter 39 means go back to the default foreground color.  How
can ansi-color not support it and still be useful?  IIUC, what the old
code did was exactly the right thing (or at least close, since 0 means
turn off _all_ attributes, not just the foreground color), so removing
it without replacing it with something that restores the default
foreground would be a bug, IMO.

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Wolfgang Jenkner]

On Sat, Nov 03 2012, Eli Zaretskii wrote:

>> In bug#11883 you give this example
>> 
>> echo -e "\033[31mtest\033[39m"
>> 
>> but ansi-color has never supported SGR parameter 39.  However, before my
>> change (the latter commit above), any unknown parameter would have the
>> same effect as parameter 0 (turn off all rendition aspects), while it is
>> simply ignored now (well, except for an "Invalid face reference: nil"
>> message), see `ansi-color-apply-sequence'.  Perhaps, this explains the
>> problem.
>
> SGR parameter 39 means go back to the default foreground color.  How
> can ansi-color not support it and still be useful?

Perhaps because that parameter is but rarely observed in the wild?

> IIUC, what the old
> code did was exactly the right thing (or at least close, since 0 means
> turn off _all_ attributes, not just the foreground color), so removing
> it without replacing it with something that restores the default
> foreground would be a bug, IMO.

I'll fix this omission if nobody beats me to it.

Wolfgang

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Dima Kogan]

> On Sat, 03 Nov 2012 19:16:42 +0800
> Chong Yidong <cyd@gnu.org> wrote:
>
> Dima Kogan <dima@secretsauce.net> writes:
> 
> > I'm using a very recent emacs snapshot. shell-mode no longer renders the
> > ANSI-code colors correctly. Even with a relatively stock bash, All
> > output is rendered in yellow except for the prompt line. The prompt
> > isn't yellow, but it's wrong also
> 
> I can't reproduce it with
> 
> emacs -Q
> M-x shell RET
> ls --color RET
> 
> The output of ls is colorized correctly.  (Latest emacs-24 brach,
> x86_64-unknown-linux-gnu, GTK+ Version 3.4.2.)


Looking at it a bit more, what seems to be happening is that console output that
has coloring (like ls --color) is colored correctly, while anything that is
plain text is yellow. So if I do ls --color, the first few files don't have any
particular colors applied to them, and thus appear yellow. The first directory
is blue, as it should be, and the rest of the ls --color output is ok after that
point. I.e. even the "normal" text appears black instead of yellow. When the ls
is done and I get back to the prompt, things get yellow again.

(describe-face) says that the yellow bits have a nil face. This sounds like the
problem. Apparently I also see lots of "Invalid face reference: nil" complaints
in the *Messages* buffer.

The original bug report had a description of my emacs. It's a very recent build,
and I do see the issue with emacs -Q.

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Wolfgang Jenkner]

On Sat, Nov 03 2012, Wolfgang Jenkner wrote:

> On Sat, Nov 03 2012, Eli Zaretskii wrote:

>> SGR parameter 39 means go back to the default foreground color.  How
>> can ansi-color not support it and still be useful?

> I'll fix this omission if nobody beats me to it.

Here's a rewrite of `ansi-color-apply-sequence' which implements SGR
codes 39 and 49.  It's straightforward but, sadly, uses side-effects all
over the place.  At this point, however, I'd just like to know if its
behaviour meets everybody's expectations, so please test it a bit.

Wolfgang

(defun ansi-color-apply-sequence (escape-sequence codes)
  "Apply ESCAPE-SEQ to CODES and return the new list of codes.

ESCAPE-SEQ is an escape sequence parsed by `ansi-color-parse-sequence'.

For each new code, the following happens: if it is 1-7, add it to
the list of codes; if it is 21-25 or 27, delete appropriate
parameters from the list of codes; if it is 30-37 resp. 39, the
foreground color code is replaced or added resp. deleted; if it
is 40-47 resp. 49, the background color code is replaced or added
resp. deleted; any other code is discarded together with the old
codes.  Finally, the so changed list of codes is returned."
  (let ((new-codes (ansi-color-parse-sequence escape-sequence))
	(fg (and codes (>= (car codes) 30) (<= (car codes) 37)
		 (pop codes)))
	(bg (and codes (>= (car codes) 40) (<= (car codes) 47)
		 (pop codes))))
    (while new-codes
      (let ((new (pop new-codes)))
	(cond ((and (>= new 1) (<= new 7))
	       (unless (memq new codes)
		 (push new codes)))
	      ((and (= new 22)
		    (setq codes (remq 1 codes))
		    nil))
	      ((and (= new 25)
		    (setq codes (remq 6 codes))
		    nil))
	      ;; The standard says `21 doubly underlined' while
	      ;; http://en.wikipedia.org/wiki/ANSI_escape_code claims
	      ;; `21 Bright/Bold: off or Underline: Double'.
	      ((and (>= new 21) (<= new 27) (/= new 26))
	       (setq codes (remq (- new 20) codes)))
	      ((and (>= new 30) (<= new 37))
	       (setq fg new))
	      ((= new 39)
	       (setq fg nil))
	      ((and (>= new 40) (<= new 47))
	       (setq bg new))
	      ((= new 49)
	       (setq bg nil))
	      (t
	       (setq codes nil fg nil bg nil)))))
    (when bg
      (push bg codes))
    (when fg
      (push fg codes))
    codes))

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Dima Kogan]

> On Sun, 04 Nov 2012 04:27:35 +0100
> Wolfgang Jenkner <wjenkner@inode.at> wrote:
>
> On Sat, Nov 03 2012, Wolfgang Jenkner wrote:
> 
> > On Sat, Nov 03 2012, Eli Zaretskii wrote:
> 
> >> SGR parameter 39 means go back to the default foreground color.  How
> >> can ansi-color not support it and still be useful?
> 
> > I'll fix this omission if nobody beats me to it.
> 
> Here's a rewrite of `ansi-color-apply-sequence' which implements SGR
> codes 39 and 49.  It's straightforward but, sadly, uses side-effects all
> over the place.  At this point, however, I'd just like to know if its
> behaviour meets everybody's expectations, so please test it a bit.
> 
> Wolfgang
> 
> (defun ansi-color-apply-sequence (escape-sequence codes)
>   "Apply ESCAPE-SEQ to CODES and return the new list of codes.
> 
> ESCAPE-SEQ is an escape sequence parsed by `ansi-color-parse-sequence'.
> 
> For each new code, the following happens: if it is 1-7, add it to
> the list of codes; if it is 21-25 or 27, delete appropriate
> parameters from the list of codes; if it is 30-37 resp. 39, the
> foreground color code is replaced or added resp. deleted; if it
> is 40-47 resp. 49, the background color code is replaced or added
> resp. deleted; any other code is discarded together with the old
> codes.  Finally, the so changed list of codes is returned."
>   (let ((new-codes (ansi-color-parse-sequence escape-sequence))
> 	(fg (and codes (>= (car codes) 30) (<= (car codes) 37)
> 		 (pop codes)))
> 	(bg (and codes (>= (car codes) 40) (<= (car codes) 47)
> 		 (pop codes))))
>     (while new-codes
>       (let ((new (pop new-codes)))
> 	(cond ((and (>= new 1) (<= new 7))
> 	       (unless (memq new codes)
> 		 (push new codes)))
> 	      ((and (= new 22)
> 		    (setq codes (remq 1 codes))
> 		    nil))
> 	      ((and (= new 25)
> 		    (setq codes (remq 6 codes))
> 		    nil))
> 	      ;; The standard says `21 doubly underlined' while
> 	      ;; http://en.wikipedia.org/wiki/ANSI_escape_code claims
> 	      ;; `21 Bright/Bold: off or Underline: Double'.
> 	      ((and (>= new 21) (<= new 27) (/= new 26))
> 	       (setq codes (remq (- new 20) codes)))
> 	      ((and (>= new 30) (<= new 37))
> 	       (setq fg new))
> 	      ((= new 39)
> 	       (setq fg nil))
> 	      ((and (>= new 40) (<= new 47))
> 	       (setq bg new))
> 	      ((= new 49)
> 	       (setq bg nil))
> 	      (t
> 	       (setq codes nil fg nil bg nil)))))
>     (when bg
>       (push bg codes))
>     (when fg
>       (push fg codes))
>     codes))

I ran a test:

1. I start an 'emacs -Q' with the latest ansi-colors.el
2. I evaluate the defun you pasted
3. M-x shell
4. I run some commands in the shell


Results:

1. The prompt is still not colored at all even though it should be.

2. Everything is still yellow with a 'nil' face UNTIL something with ansi color
   codes is encountered. At that point correct coloring resumes, except for the
   prompt. The prompt also doesn't count as encountering ansi colors.

Are you not able to reproduce this? My setup isn't particularly special.

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Wolfgang Jenkner]

On Sun, Nov 04 2012, Dima Kogan wrote:

> 1. I start an 'emacs -Q' with the latest ansi-colors.el
> 2. I evaluate the defun you pasted

Did you load ansi-colors /before/ step 2?

If so, what is your PS1 set to?  What are the (local) values of
`comint-prompt-regexp' and `comint-use-prompt-regexp' in the "*shell*"
buffer?

Wolfgang

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Dima Kogan]

> On Sun, 04 Nov 2012 14:49:50 +0100
> Wolfgang Jenkner <wjenkner@inode.at> wrote:
>
> On Sun, Nov 04 2012, Dima Kogan wrote:
> 
> > 1. I start an 'emacs -Q' with the latest ansi-colors.el
> > 2. I evaluate the defun you pasted
> 
> Did you load ansi-colors /before/ step 2?
> 
> If so, what is your PS1 set to?  What are the (local) values of
> `comint-prompt-regexp' and `comint-use-prompt-regexp' in the "*shell*"
> buffer?
> 
> Wolfgang


Hi Wolfgang. You're right; I was loading ansi-colors.el after the eval, which
was overwriting the new defun. On top of that, I just ran some more controlled
tests, and I can now see that some of my descriptions were wrong. Stock bash
actually behaves ok (I did test it previously, but must have made a mistake).
Note "behaves ok" here means that the nil faces are gone; the prompts are still
not colored even with bash.

zsh is the main shell I use that looks to be problematic. I can consistently
observe the issues if I delete all ~/.zsh* files, then add just the following to
the .zshrc:

export PS1="%F{red}%n@%m%f:%F{yellow}%~%f%(!.#.$) "

This sets a bash-like prompt. With the latest ansi-colors.el I see the
everything-is-yellow and face-is-nil issues from before. If I also then evaluate
the defun, I observe the behavior I described previously, where everything is
one color, until some ansi code is encountered. After the defun the wrong color
becomes red instead of yellow, suggesting that something isn't being reset
properly.

I'm using the zsh package from Debian version 4.3.17-1.

comint-prompt-regexp is a variable defined in `comint.el.gz'.
Its value is "^[^#$%>
]*[#$%>] *"

comint-use-prompt-regexp is a variable defined in `comint.el.gz'.
Its value is nil

(describe-face) with the point inside the prompt says the prompt face is
'comint-highlight-prompt'.

Sorry for the misniformation

dima

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Wolfgang Jenkner]

On Sun, Nov 04 2012, Dima Kogan wrote:

> zsh is the main shell I use that looks to be problematic. I can consistently
> observe the issues if I delete all ~/.zsh* files, then add just the following to
> the .zshrc:
>
> export PS1="%F{red}%n@%m%f:%F{yellow}%~%f%(!.#.$) "

Could you please do

M-x ansi-color-for-comint-mode-off 

so that we may see the raw escape sequences generated by that PS1?

Wolfgang

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Dima Kogan]

> On Mon, 05 Nov 2012 00:39:46 +0100
> Wolfgang Jenkner <wjenkner@inode.at> wrote:
>
> On Sun, Nov 04 2012, Dima Kogan wrote:
> 
> > zsh is the main shell I use that looks to be problematic. I can consistently
> > observe the issues if I delete all ~/.zsh* files, then add just the following to
> > the .zshrc:
> >
> > export PS1="%F{red}%n@%m%f:%F{yellow}%~%f%(!.#.$) "
> 
> Could you please do
> 
> M-x ansi-color-for-comint-mode-off 
> 
> so that we may see the raw escape sequences generated by that PS1?


To clarify the test even more, I set my .zshrc to

unsetopt zle
unsetopt prompt_cr
unsetopt prompt_subst
export PS1="%F{red}%n@%m%f:%F{yellow}%~%f%(!.#.$) "

This is the complete .zshrc file. The unsetopt stuff turns off zsh's keyboard
processing. Probably doesn't matter, but i'm mentioning this for completeness.

In the shell I pressed enter a few times to see the prompt sequences; then I ran
a command that produces some output without ANSI codes. Then I ran a command
that does produce some ansi codes in its output. Then I ran the plain-output
command again.

The output (with all the 'escape' characters replaced with '\e' to make sure the
email handlers don't get confused):


\e[31mdima@shorty\e[39m:\e[33m~\e[39m$ 
\e[31mdima@shorty\e[39m:\e[33m~\e[39m$ 
\e[31mdima@shorty\e[39m:\e[33m~\e[39m$ seq 5
1
2
3
4
5
\e[31mdima@shorty\e[39m:\e[33m~\e[39m$ ls --color=auto -ld .emacs.d
drwxr-xr-x 6 dima dima 4096 Nov  3 15:31 \e[0m\e[01;34m.emacs.d\e[0m
\e[31mdima@shorty\e[39m:\e[33m~\e[39m$ seq 5
1
2
3
4
5
\e[31mdima@shorty\e[39m:\e[33m~\e[39m$ 



If I run the exact same sequence of commands with ansi-color-for-comint-mode-on
and the latest ansi-color.el, I see everything yellow (nil face), except for
".emacs.d", which is blue and the last prompt, which is all blue also. This
prompt would change to all yellow also if I press enter again. Not that the
prompt shouldn't be all blue.

If I evaluate your defun also, I get everything red (nil face also) until the
".emacs.d". The ".emacs.d" is blue, as it should be. From that point on, all the
plain output is black and all the prompts are blue. This is correct, except the
prompts still shouldn't be all blue.


Sorry for all the confusion.

dima

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Wolfgang Jenkner]

On Mon, Nov 05 2012, Dima Kogan wrote:

> The output (with all the 'escape' characters replaced with '\e' to make sure the
> email handlers don't get confused):
>
>
> \e[31mdima@shorty\e[39m:\e[33m~\e[39m$ 
> \e[31mdima@shorty\e[39m:\e[33m~\e[39m$ 
> \e[31mdima@shorty\e[39m:\e[33m~\e[39m$ seq 5
> 1
> 2
> 3
> 4
> 5
> \e[31mdima@shorty\e[39m:\e[33m~\e[39m$ ls --color=auto -ld .emacs.d
> drwxr-xr-x 6 dima dima 4096 Nov  3 15:31 \e[0m\e[01;34m.emacs.d\e[0m
> \e[31mdima@shorty\e[39m:\e[33m~\e[39m$ seq 5
> 1
> 2
> 3
> 4
> 5
> \e[31mdima@shorty\e[39m:\e[33m~\e[39m$ 
>
>
[...]
> If I evaluate your defun also, I get everything red (nil face also) until the
> ".emacs.d". The ".emacs.d" is blue, as it should be. From that point on, all the
> plain output is black and all the prompts are blue. This is correct, except the
> prompts still shouldn't be all blue.

Thanks for the precise recipe.  I installed zsh 5.0.0 from ports in
FreeBSD 9 and tried it, but I can't reproduce the bogus red colouring
(with the rewritten ansi-color-apply-sequence).

So I would like to ask you to do some more investigation:

As usual, load ansi-color and then eval the new definition of
ansi-color-apply-sequence.

Then

M-x trace-function-background <return> ansi-color-apply-sequence <return> <return>
M-x trace-function-background <return> comint-output-filter <return> <return>

M-x shell and type in the same shell commands as above.

M-x untrace-all

Then please attach the content of "*trace-output*" to your reply.

Wolfgang

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Wolfgang Jenkner]

On Mon, Nov 05 2012, Wolfgang Jenkner wrote:

> As usual, load ansi-color and then eval the new definition of
> ansi-color-apply-sequence.

Also load comint at this point.

And untrace-all seems to be broken in trunk, but never mind...


Wolfgang

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Dima Kogan]

> On Mon, 05 Nov 2012 16:00:34 +0100
> Wolfgang Jenkner <wjenkner@inode.at> wrote:
>
> On Mon, Nov 05 2012, Dima Kogan wrote:
> 
> > The output (with all the 'escape' characters replaced with '\e' to
> > make sure the email handlers don't get confused):
> >
> >
> > \e[31mdima@shorty\e[39m:\e[33m~\e[39m$ 
> > \e[31mdima@shorty\e[39m:\e[33m~\e[39m$ 
> > \e[31mdima@shorty\e[39m:\e[33m~\e[39m$ seq 5
> > 1
> > 2
> > 3
> > 4
> > 5
> > \e[31mdima@shorty\e[39m:\e[33m~\e[39m$ ls --color=auto -ld .emacs.d
> > drwxr-xr-x 6 dima dima 4096 Nov  3 15:31 \e[0m\e[01;34m.emacs.d\e[0m
> > \e[31mdima@shorty\e[39m:\e[33m~\e[39m$ seq 5
> > 1
> > 2
> > 3
> > 4
> > 5
> > \e[31mdima@shorty\e[39m:\e[33m~\e[39m$ 
> >
> >
> [...]
> > If I evaluate your defun also, I get everything red (nil face also)
> > until the ".emacs.d". The ".emacs.d" is blue, as it should be. From
> > that point on, all the plain output is black and all the prompts
> > are blue. This is correct, except the prompts still shouldn't be
> > all blue.
> 
> Thanks for the precise recipe.  I installed zsh 5.0.0 from ports in
> FreeBSD 9 and tried it, but I can't reproduce the bogus red colouring
> (with the rewritten ansi-color-apply-sequence).
> 
> So I would like to ask you to do some more investigation:
> 
> As usual, load ansi-color and then eval the new definition of
> ansi-color-apply-sequence.
> 
> Then
> 
> M-x trace-function-background <return> ansi-color-apply-sequence
> <return> <return> M-x trace-function-background <return>
> comint-output-filter <return> <return>
> 
> M-x shell and type in the same shell commands as above.
> 
> M-x untrace-all
> 
> Then please attach the content of "*trace-output*" to your reply.
> 
> Wolfgang
> 
> 
> 
> 

Hi Wolfgang. I just ran those tests.

Previously, I would do M-x shell, and then evaluate the new defun; this produced
the results I described earlier.

You just asked for a different order: load ansi-color, comint, THEN eval defun,
THEN M-x shell. That produces different colored results: everything looks
correct except the prompt is all blue all the time. Nothing has nil face. Is
this the expected behavior with that defun, or is the prompt supposed to look
right also?

The trace output follows. Note that once again I replaced all escape characters with '\e'.




======================================================================
1 -> comint-output-filter: process=#<process shell> string="\e[32mdima@fatty\e[39m:\e[33m/tmp\e[39m$ "
| 2 -> ansi-color-apply-sequence: escape-sequence=#("32m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (32)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(32)
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("33m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (33)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(33)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 18 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="\e[32mdima@fatty\e[39m:\e[33m/tmp\e[39m$ "
| 2 -> ansi-color-apply-sequence: escape-sequence=#("32m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (32)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(32)
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("33m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (33)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(33)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 36 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="\e[32mdima@fatty\e[39m:\e[33m/tmp\e[39m$ "
| 2 -> ansi-color-apply-sequence: escape-sequence=#("32m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (32)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(32)
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("33m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (33)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(33)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 54 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="\e[32mdima@fatty\e[39m:\e[33m/tmp\e[39m$ "
| 2 -> ansi-color-apply-sequence: escape-sequence=#("32m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (32)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(32)
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("33m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (33)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(33)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 72 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="\e[32mdima@fatty\e[39m:\e[33m/tmp\e[39m$ "
| 2 -> ansi-color-apply-sequence: escape-sequence=#("32m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (32)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(32)
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("33m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (33)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(33)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 90 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="\e[32mdima@fatty\e[39m:\e[33m/tmp\e[39m$ "
| 2 -> ansi-color-apply-sequence: escape-sequence=#("32m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (32)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(32)
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("33m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (33)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(33)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 108 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="\e[32mdima@fatty\e[39m:\e[33m/tmp\e[39m$ "
| 2 -> ansi-color-apply-sequence: escape-sequence=#("32m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (32)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(32)
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("33m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (33)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(33)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 126 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="1
2
3
4
"
1 <- comint-output-filter: #<marker (moves after insertion) at 140 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="5
\e[32mdima@fatty\e[39m:\e[33m/tmp\e[39m$ "
| 2 -> ansi-color-apply-sequence: escape-sequence=#("32m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (32)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(32)
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("33m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (33)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(33)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 159 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="drwxr-xr-x 4 dima dima 4096 Nov  5 00:33 \e[0m\e[01;34m/home/dima/.emacs.d\e[0m
"
| 2 -> ansi-color-apply-sequence: escape-sequence=#("0m" 0 2 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("01;34m" 0 6 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (34 1)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("0m" 0 2 (fontified nil)) codes=(34 1)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 251 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="\e[32mdima@fatty\e[39m:\e[33m/tmp\e[39m$ "
| 2 -> ansi-color-apply-sequence: escape-sequence=#("32m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (32)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(32)
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("33m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (33)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(33)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 268 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="\e[32mdima@fatty\e[39m:\e[33m/tmp\e[39m$ "
| 2 -> ansi-color-apply-sequence: escape-sequence=#("32m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (32)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(32)
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("33m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (33)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(33)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 286 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="\e[32mdima@fatty\e[39m:\e[33m/tmp\e[39m$ "
| 2 -> ansi-color-apply-sequence: escape-sequence=#("32m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (32)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(32)
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("33m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (33)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(33)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 304 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="1
2
3
4
"
1 <- comint-output-filter: #<marker (moves after insertion) at 318 in *shell*>
======================================================================
1 -> comint-output-filter: process=#<process shell> string="5
\e[32mdima@fatty\e[39m:\e[33m/tmp\e[39m$ "
| 2 -> ansi-color-apply-sequence: escape-sequence=#("32m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (32)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(32)
| 2 <- ansi-color-apply-sequence: nil
| 2 -> ansi-color-apply-sequence: escape-sequence=#("33m" 0 3 (fontified nil)) codes=nil
| 2 <- ansi-color-apply-sequence: (33)
| 2 -> ansi-color-apply-sequence: escape-sequence=#("39m" 0 3 (fontified nil)) codes=(33)
| 2 <- ansi-color-apply-sequence: nil
1 <- comint-output-filter: #<marker (moves after insertion) at 337 in *shell*>

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Wolfgang Jenkner]

On Tue, Nov 06 2012, Dima Kogan wrote:

> Previously, I would do M-x shell, and then evaluate the new defun; this produced
> the results I described earlier.

Too late, the function must be redefined before the first prompt is
sent.  I thought it would be simpler to give you the rewritten defun
separately instead of having you patching ansi-color.el ;-)

> You just asked for a different order: load ansi-color, comint, THEN eval defun,
> THEN M-x shell. That produces different colored results: everything looks
> correct except the prompt is all blue all the time. Nothing has nil face. Is
> this the expected behavior with that defun

Yes.  As you found out in your original report the prompt coloring has
been broken for longer.  If you customize the comint-highlight-prompt
face so that it doesn't inherit from minibuffer-prompt (which does the
blue colouring), you'll see that the current prompt is actually coloured
correctly but for the previous prompts, comint-highlight-prompt just
replaces the ansi-color faces.  That's weird.  Somebody should at some
point do something about it ;-)

In any case, you submitted a separate report about the prompt issue,
viz. bug#11883.

So, I'd say that the issue at hand is solved.  I'll just try to rewrite
the new ansi-color-apply-sequence in a less ugly way.

Wolfgang

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Dima Kogan]

> On Tue, 06 Nov 2012 02:24:29 +0100
> Wolfgang Jenkner <wjenkner@inode.at> wrote:
>
> On Tue, Nov 06 2012, Dima Kogan wrote:
> 
> > Previously, I would do M-x shell, and then evaluate the new defun;
> > this produced the results I described earlier.
> 
> Too late, the function must be redefined before the first prompt is
> sent.  I thought it would be simpler to give you the rewritten defun
> separately instead of having you patching ansi-color.el ;-)
> 
> > You just asked for a different order: load ansi-color, comint, THEN
> > eval defun, THEN M-x shell. That produces different colored
> > results: everything looks correct except the prompt is all blue all
> > the time. Nothing has nil face. Is this the expected behavior with
> > that defun
> 
> Yes.  As you found out in your original report the prompt coloring has
> been broken for longer.  If you customize the comint-highlight-prompt
> face so that it doesn't inherit from minibuffer-prompt (which does the
> blue colouring), you'll see that the current prompt is actually
> coloured correctly but for the previous prompts,
> comint-highlight-prompt just replaces the ansi-color faces.  That's
> weird.  Somebody should at some point do something about it ;-)
> 
> In any case, you submitted a separate report about the prompt issue,
> viz. bug#11883.
> 
> So, I'd say that the issue at hand is solved.  I'll just try to
> rewrite the new ansi-color-apply-sequence in a less ugly way.
> 
> Wolfgang

Sounds good. Thanks a lot.

dima

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Wolfgang Jenkner]

On Tue, Nov 06 2012, Dima Kogan wrote:

>> So, I'd say that the issue at hand is solved.  I'll just try to
>> rewrite the new ansi-color-apply-sequence in a less ugly way.
>
> Sounds good. Thanks a lot.

I hope the patch below is less ugly and also correct.  Perhaps you could
try it.

Note to the maintainers:  This fixes a regression for a feature not
actually present in previous versions of emacs ;-)


2012-11-08  Wolfgang Jenkner  <wjenkner@inode.at>

	* ansi-color.el (ansi-color-apply-sequence): Implement SGR codes
	39 and 49 (bug#12792).	Also, restore the behavior of revisions
	prior to 109621, where unimplemented codes are treated like code 0.


=== modified file 'lisp/ansi-color.el'
--- lisp/ansi-color.el	2012-09-25 04:13:02 +0000
+++ lisp/ansi-color.el	2012-11-09 00:19:00 +0000
@@ -534,34 +534,39 @@
 
 ESCAPE-SEQ is an escape sequence parsed by `ansi-color-parse-sequence'.
 
-If the new codes resulting from ESCAPE-SEQ start with 0, then the
-old codes are discarded and the remaining new codes are
-processed.  Otherwise, for each new code: if it is 21-25 or 27-29
-delete appropriate parameters from the list of codes; any other
-code that makes sense is added to the list of codes.  Finally,
-the so changed list of codes is returned."
+For each new code, the following happens: if it is 1-7, add it to
+the list of codes; if it is 21-25 or 27, delete appropriate
+parameters from the list of codes; if it is 30-37 resp. 39, the
+foreground color code is replaced or added resp. deleted; if it
+is 40-47 resp. 49, the background color code is replaced or added
+resp. deleted; any other code is discarded together with the old
+codes.	Finally, the so changed list of codes is returned."
   (let ((new-codes (ansi-color-parse-sequence escape-sequence)))
     (while new-codes
-      (setq codes
-	    (let ((new (pop new-codes)))
-	      (cond ((zerop new)
-		     nil)
-		    ((or (<= new 20)
-			 (>= new 30))
-		     (if (memq new codes)
-			 codes
-		       (cons new codes)))
-		    ;; The standard says `21 doubly underlined' while
-		    ;; http://en.wikipedia.org/wiki/ANSI_escape_code claims
-		    ;; `21 Bright/Bold: off or Underline: Double'.
-		    ((/= new 26)
-		     (remq (- new 20)
-			   (cond ((= new 22)
-				  (remq 1 codes))
-				 ((= new 25)
-				  (remq 6 codes))
-				 (t codes))))
-		    (t codes)))))
+      (let* ((new (pop new-codes))
+	     (q (/ new 10)))
+	(setq codes
+	      (pcase q
+		(0 (unless (memq new '(0 8 9))
+		     (cons new (remq new codes))))
+		(2 (unless (memq new '(20 26 28 29))
+		     ;; The standard says `21 doubly underlined' while
+		     ;; http://en.wikipedia.org/wiki/ANSI_escape_code claims
+		     ;; `21 Bright/Bold: off or Underline: Double'.
+		     (remq (- new 20) (pcase new
+					(22 (remq 1 codes))
+					(25 (remq 6 codes))
+					(_ codes)))))
+		((or 3 4) (let ((r (mod new 10)))
+			    (unless (= r 8)
+			      (let (beg)
+				(while (and codes (/= q (/ (car codes) 10)))
+				  (push (pop codes) beg))
+				(setq codes (nconc (nreverse beg) (cdr codes)))
+				(if (= r 9)
+				    codes
+				  (cons new codes))))))
+		(_ nil)))))
     codes))
 
 (defun ansi-color-make-color-map ()

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Stefan Monnier]

> I hope the patch below is less ugly and also correct.  Perhaps you could
> try it.

It seems to work on the example of bug#11883 (it doesn't solve
bug#11883, mind you, just brings it back to the less broken state that
it was in when the bug was reported).

> Note to the maintainers:  This fixes a regression for a feature not
> actually present in previous versions of emacs ;-)

At least for the example code of bug#11883 it fixes a regression w.r.t
Emacs-24.1.  Please install it.


        Stefan

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Wolfgang Jenkner]

On Fri, Nov 09 2012, Stefan Monnier wrote:

> At least for the example code of bug#11883 it fixes a regression w.r.t
> Emacs-24.1.  Please install it.

To which branch?  Oh, and I don't have a commit bit.

Wolfgang

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Stefan Monnier]

>> At least for the example code of bug#11883 it fixes a regression w.r.t
>> Emacs-24.1.  Please install it.
> To which branch?

emacs-24.

> Oh, and I don't have a commit bit.

That's easy to fix.  Can you request membership in the `emacs' group,
from your savannah account (you may need to create such an account
first, obviously)?


        Stefan

bug#12792: 24.2.50; shell-mode renders all its output in a yellow face

[Paul Eggert]

Thanks for installing that patch.  I'm marking the bug as done.