bug#6691: 23.2; Eshell and ^M

bug#6691: 23.2; Eshell and ^M

[Leo]

[-- Attachment #1: Type: text/plain, Size: 269 bytes --]

1. Emacs -q
2. M-x eshell
3. in eshell enter a git repository and eval 

(let ((eshell-buffer-maximum-lines 0))
    (eshell-truncate-buffer))

4. in eshell run 'git log -1 --stat'
5. run again 'git log -1 --stat'

Step 4 should output lots of ^M as in this screenshot:

[-- Attachment #2: eshell.png --]
[-- Type: image/png, Size: 140325 bytes --]

[-- Attachment #3: Type: text/plain, Size: 164 bytes --]


Step 5 should output cleanly without any ^M.


In GNU Emacs 23.2.10 of 2010-07-09 on Victoria.local
Windowing system distributor `Apple Inc.', version 10.6.4

Leo

bug#6691: 23.2; Eshell and ^M (PATCH attached)

[Leo]

The problem is eshell-last-output-block-begin can point to a wrong
position in the eshell buffer after truncating buffer. I tried to trace
the reason of introducing eshell-last-output-block-begin but the
revision history only goes back to 2000.

It looks like cruft to me. So I propose the following fix.

Leo

commit 1f263a7a7d9d978b68006619a0d2a0d6dcbf39c1
Date:   Thu Jul 22 16:21:30 2010 +0100

    Remove cruft eshell-last-output-block-begin and fix #6691

	Modified lisp/eshell/esh-mode.el
diff --git a/lisp/eshell/esh-mode.el b/lisp/eshell/esh-mode.el
index cc7f0df..ee9b299 100644
--- a/lisp/eshell/esh-mode.el
+++ b/lisp/eshell/esh-mode.el
@@ -213,7 +213,6 @@ This is used by `eshell-watch-for-password-prompt'."
 (defvar eshell-last-input-start nil)
 (defvar eshell-last-input-end nil)
 (defvar eshell-last-output-start nil)
-(defvar eshell-last-output-block-begin nil)
 (defvar eshell-last-output-end nil)
 
 (defvar eshell-currently-handling-window nil)
@@ -381,7 +380,6 @@ This is used by `eshell-watch-for-password-prompt'."
   (set (make-local-variable 'eshell-last-input-end) (point-marker))
   (set (make-local-variable 'eshell-last-output-start) (point-marker))
   (set (make-local-variable 'eshell-last-output-end) (point-marker))
-  (set (make-local-variable 'eshell-last-output-block-begin) (point))
 
   (let ((modules-list (copy-sequence eshell-modules-list)))
     (make-local-variable 'eshell-modules-list)
@@ -611,7 +609,6 @@ If NO-HOOKS is non-nil, then `eshell-post-command-hook' won't be run."
   (setq eshell-last-input-start (point-marker)
 	eshell-last-input-end (point-marker)
 	eshell-last-output-start (point-marker)
-	eshell-last-output-block-begin (point)
 	eshell-last-output-end (point-marker))
   (eshell-begin-on-new-line)
   (unless no-hooks
@@ -808,9 +805,7 @@ This is done after all necessary filtering has been done."
 (defun eshell-run-output-filters ()
   "Run the `eshell-output-filter-functions' on the current output."
   (save-current-buffer
-    (run-hooks 'eshell-output-filter-functions))
-  (setq eshell-last-output-block-begin
-	(marker-position eshell-last-output-end)))
+    (run-hooks 'eshell-output-filter-functions)))
 
 ;;; jww (1999-10-23): this needs testing
 (defun eshell-preinput-scroll-to-bottom ()
@@ -1031,7 +1026,7 @@ buffer's process if STRING contains a password prompt defined by
 This function could be in the list `eshell-output-filter-functions'."
   (when (eshell-interactive-process)
     (save-excursion
-      (goto-char eshell-last-output-block-begin)
+      (goto-char eshell-last-input-end)
       (beginning-of-line)
       (if (re-search-forward eshell-password-prompt-regexp
 			     eshell-last-output-end t)
@@ -1044,7 +1039,7 @@ This function could be in the list `eshell-output-filter-functions'."
   "Act properly when certain control codes are seen."
   (save-excursion
     (let ((orig (point)))
-      (goto-char eshell-last-output-block-begin)
+      (goto-char eshell-last-input-end)
       (unless (eolp)
 	(beginning-of-line))
       (while (< (point) eshell-last-output-end)

bug#6691: 23.2; Eshell and ^M (PATCH attached)

[Lars Magne Ingebrigtsen]

Leo <sdl.web@gmail.com> writes:

> The problem is eshell-last-output-block-begin can point to a wrong
> position in the eshell buffer after truncating buffer. I tried to trace
> the reason of introducing eshell-last-output-block-begin but the
> revision history only goes back to 2000.

This looks like a good patch to me, but I don't use eshell, so I can't
test.

Does anybody know whether the `eshell-last-output-block-begin' stuff
really is cruft that can be removed?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#6691: 23.2; Eshell and ^M (PATCH attached)

[John Wiegley]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Does anybody know whether the `eshell-last-output-block-begin' stuff really
> is cruft that can be removed?

The original meaning of that variable was for performance: so that output
filter would only be run again on newly accumulated output text, not on text
that had already been filtered.

John

bug#6691: 23.2; Eshell and ^M

[Glenn Morris]

Leo wrote:

> Step 4 should output lots of ^M as in this screenshot:
[...]
> In GNU Emacs 23.2.10 of 2010-07-09 on Victoria.local
> Windowing system distributor `Apple Inc.', version 10.6.4

Are you sure this is related to `eshell-last-output-block-begin'?
IIRC, there are various reports to do with extraneous ^M characters on
Macs (eg with gdb). FWIW, I don't seem to see the problem you describe
on GNU/Linux.

bug#6691: 23.2; Eshell and ^M (PATCH attached)

[Lars Magne Ingebrigtsen]

John Wiegley <johnw@newartisans.com> writes:

> The original meaning of that variable was for performance: so that output
> filter would only be run again on newly accumulated output text, not on text
> that had already been filtered.

If this patch leads to a significant slow-down, then it can't be
applied, and the reporter should try to determine when the variable is
mis-updated, I guess?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#6691: 23.2; Eshell and ^M (PATCH attached)

[John Wiegley]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> If this patch leads to a significant slow-down, then it can't be applied,
> and the reporter should try to determine when the variable is mis-updated, I
> guess?

I would think so, yes.  I don't believe that I put it in for no reason. :)

John

bug#6691: 23.2; Eshell and ^M

[Leo]

On 2012-04-12 09:23 +0800, Glenn Morris wrote:
> Are you sure this is related to `eshell-last-output-block-begin'?
> IIRC, there are various reports to do with extraneous ^M characters on
> Macs (eg with gdb). FWIW, I don't seem to see the problem you describe
> on GNU/Linux.

I have forgotten much about the reasoning behind the patch. So I am not
sure it is the right fix.

The known fact is if I remove eshell-handle-control-codes from
eshell-output-filter-functions, then I can observe outputting ^M in both
GNU/Linux and OSX.

So I think eshell-handle-control-codes might be where the problem is.

BTW, why process coding system for input and output are set to
utf-8-unix? I bind coding-system-for-read to utf-8 and observe correct
output with nearly no ^M, i.e. no need to remove ^M after the fact.

Leo

bug#6691: 23.2; Eshell and ^M

[Andreas Schwab]

Leo <sdl.web@gmail.com> writes:

> The known fact is if I remove eshell-handle-control-codes from
> eshell-output-filter-functions, then I can observe outputting ^M in both
> GNU/Linux and OSX.
>
> So I think eshell-handle-control-codes might be where the problem is.

The point of eshell-handle-control-codes is to handle ^M (and other
control chars) like a terminal does.

(That its implementation is pretty inefficient is different issue.)

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#6691: 23.2; Eshell and ^M

[Andreas Schwab]

Leo <sdl.web@gmail.com> writes:

> 4. in eshell run 'git log -1 --stat'
> 5. run again 'git log -1 --stat'

What is your pager?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#6691: 23.2; Eshell and ^M

[Leo]

On 2012-04-14 16:27 +0800, Andreas Schwab wrote:
> What is your pager?
>
> Andreas.

I have 'pager = more -R' in .gitconfig.

Leo

bug#6691: 23.2; Eshell and ^M

[Andreas Schwab]

Leo <sdl.web@gmail.com> writes:

> I have 'pager = more -R' in .gitconfig.

What does "more -R"?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#6691: 23.2; Eshell and ^M

[Eli Zaretskii]

> From: Leo <sdl.web@gmail.com>
> Date: Sat, 14 Apr 2012 15:48:32 +0800
> Cc: 6691@debbugs.gnu.org, John Wiegley <johnw@newartisans.com>,
> 	Lars Magne Ingebrigtsen <larsi@gnus.org>
> 
> BTW, why process coding system for input and output are set to
> utf-8-unix?

I'm not entirely sure (my failing memory is ... well, failing), but I
think that's because guessing the EOL format in process output is
unreliable (due to partial reads) and can cause trouble.  Again, don't
take this for granted; perhaps someone else can confirm or refute.

bug#6691: 23.2; Eshell and ^M

[Andreas Schwab]

Leo <sdl.web@gmail.com> writes:

> BTW, why process coding system for input and output are set to
> utf-8-unix?

Because the terminal is set to -icrnl -onlcr, so no CRLF are expected.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#6691: 23.2; Eshell and ^M

[Leo]

On 2012-04-14 16:53 +0800, Andreas Schwab wrote:
> What does "more -R"?
>
> Andreas.

Sorry for the late reply.

The switch -R seems BSD only.

,----[ man more ]
|        -R or --RAW-CONTROL-CHARS
|               Like -r, but only ANSI "color" escape sequences  are  output  in
|               "raw" form.  Unlike -r, the screen appearance is maintained cor-
|               rectly  in  most  cases.   ANSI  "color"  escape  sequences  are
|               sequences of the form:
| 
|                    ESC [ ... m
| 
|               where  the  "..." is zero or more color specification characters
|               For the purpose of keeping  track  of  screen  appearance,  ANSI
|               color  escape sequences are assumed to not move the cursor.  You
|               can make less think that characters other than "m" can end  ANSI
|               color  escape  sequences  by  setting  the  environment variable
|               LESSANSIENDCHARS to the list of characters which can end a color
|               escape  sequence.   And  you can make less think that characters
|               other than the standard ones may appear between the ESC and  the
|               m  by  setting  the environment variable LESSANSIMIDCHARS to the
|               list of characters which can appear.
`----

Leo

bug#6691: 23.2; Eshell and ^M

[Andreas Schwab]

Leo <sdl.web@gmail.com> writes:

> |               LESSANSIENDCHARS to the list of characters which can end a color

Are you sure you are using more, not less?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#6691: 23.2; Eshell and ^M

[Leo]

On 2012-04-16 21:45 +0800, Andreas Schwab wrote:
> Are you sure you are using more, not less?
>
> Andreas.

I have only just realised 'more' is hardlinked to 'less'.

Leo

bug#6691: 23.2; Eshell and ^M

[Glenn Morris]

As far as I can make out, this was a local problem rather than an Emacs one.

> I have only just realised 'more' is hardlinked to 'less'.