yet more term.el fixes #2

yet more term.el fixes #2

[Dan Nicolaescu]

[-- Attachment #1: Type: text/plain, Size: 23583 bytes --]


I wrote: 
"This round of term.el changes will make it work flawlessly (AFAICT)
for 7bit ASCII platforms."

That was too soon, another problem showed up right after I sent the
first message. Line-wrapping did not work properly for example when
using very long command lines. Now it's fixed and included in the
patch below together with all the earlier fixes. 


A few problems were fixed: 
- term-mode did not set indent-tabs-mode to nil, so tabs were
sometimes inserted, and that confused the display engine. 
- handling of underline and reverse was inverted
- The cursor_up (cuu1) terminfo capability was not working because
term-erase-in-line was not handling its parameter correctly.
- The functions to start/stop logging the terminal output (for
debugging), had confusing names: term-set-output-log and
term-stop-photo. They have been renamed. 
- The smcup/rmcup handling was not working. The terminfo manpage on my
GNU/Linux system says that these capabilities are optional, so it's
better not to have to deal with them. So all that functionality was
commented out in both the elisp code and the eterm TERMINFO entry. 
- The delete and backspace keys did not work properly in "less" or in
an emacs -nw session running in a M-x term terminal by default. The
strings sent by delete and backspace have been changed to what xterm
sends on my GNU/Linux system. Now they work as expected. 

- xterm uses  S-prior and S-next for scrolling up and down, and S-insert
for pasting. These bindings have been added to term.el

One remaining problem in term.el is the handling of characters in the
128-255 range (and probably the non 8-bit encodings too). 
term.el uses move-to-column to position the cursor at certain
positions in the buffer. But the char-width of a character >127 is 4 
(at least in my setup), this confuses the cursor positioning and the
display will be incorrect (to illustrate this: try running
"lynx www.emacswiki.org"). A very simple _kludge_ to alleviate this
problem is to replace all the characters >128 with "?", i.e. just
replace  (insert (substring str i funny)) in
`term-emulate-terminal' with:
(insert (mapconcat (lambda (arg) (if (< arg 128)  (char-to-string arg) "?")) 
                   (substring str i funny) ""))
If someone could come up with a proper solution, please do, I am not
very familiar with how character encodings and display are supposed to
work.

Please is install this patch if it's OK. 

for etc/ChangeLog

2004-09-21  Dan Nicolaescu  <dann@ics.uci.edu>

	* e/eterm.ti: Comment out smcup, rmcup. Add kbs, kdch1.
	Reformat.
        * e/eterm: Regenerate.


for lisp/ChangeLog

2004-09-21  Dan Nicolaescu  <dann@ics.uci.edu>

	* term.el (term-ansi-at-eval-string, term-ansi-default-fg)
	(term-ansi-default-bg, term-ansi-current-temp): Delete unused
	vars.
	(map): Bind S-prior, S-next and S-insert.
	(term-mode): Set `indent-tabs-mode' to nil.
	(term-paste): New function to be bound to S-insert.
	(term-send-del, term-send-backspace): Change the strings sent.
	(term-termcap-format): Synchronyze with etc/e/eterm.ti.
	(term-handle-colors-array): Fix handling of underline and reverse.
	(term-handle-ansi-escape): Do not handle smcup/rmcup. Add
	comments. 
	(term-erase-in-line): Fix comparison.
        (term-emulate-terminal): Fix wrap handling.
	(term-start-output-log): Renamed from `term-set-output-log'.
	(term-stop-output-log): Renamed from `term-stop-photo'.
        (term-switch-to-alternate-sub-buffer): Comment out, unused. 


Index: term.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/term.el,v
retrieving revision 1.58
diff -c -3 -p -c -r1.58 term.el
*** term.el	20 Sep 2004 15:59:31 -0000	1.58
--- term.el	22 Sep 2004 03:09:51 -0000
***************
*** 1,6 ****
  ;;; term.el --- general command interpreter in a window stuff
  
! ;;; Copyright (C) 1988, 1990, 1992, 1994, 1995 Free Software Foundation, Inc.
  
  ;; Author: Per Bothner <bothner@cygnus.com>
  ;; Based on comint mode written by: Olin Shivers <shivers@cs.cmu.edu>
--- 1,6 ----
  ;;; term.el --- general command interpreter in a window stuff
  
! ;;; Copyright (C) 1988, 1990, 1992, 1994, 1995, 2004 Free Software Foundation, Inc.
  
  ;; Author: Per Bothner <bothner@cygnus.com>
  ;; Based on comint mode written by: Olin Shivers <shivers@cs.cmu.edu>
*************** Buffer local variable.")
*** 676,682 ****
  (defvar term-terminal-menu)
  
  ;;; Let's silence the byte-compiler -mm
- (defvar term-ansi-at-eval-string nil)
  (defvar term-ansi-at-host nil)
  (defvar term-ansi-at-dir nil)
  (defvar term-ansi-at-user nil)
--- 676,681 ----
*************** Buffer local variable.")
*** 692,700 ****
  (defvar term-ansi-current-highlight 0)
  (defvar term-ansi-current-reverse 0)
  (defvar term-ansi-current-invisible 0)
- (defvar term-ansi-default-fg 0)
- (defvar term-ansi-default-bg 0)
- (defvar term-ansi-current-temp 0)
  
  ;;; Four should be enough, if you want more, just add. -mm
  (defvar term-terminal-more-parameters 0)
--- 691,696 ----
*************** is buffer-local.")
*** 917,922 ****
--- 913,921 ----
      (define-key term-raw-map [backspace] 'term-send-backspace)
      (define-key term-raw-map [home] 'term-send-home)
      (define-key term-raw-map [end] 'term-send-end)
+     (define-key term-raw-map [S-prior] 'scroll-down)
+     (define-key term-raw-map [S-next] 'scroll-up)
+     (define-key term-raw-map [S-insert] 'term-paste)
      (define-key term-raw-map [prior] 'term-send-prior)
      (define-key term-raw-map [next] 'term-send-next)))
  
*************** Entry to this mode runs the hooks on `te
*** 981,986 ****
--- 980,988 ----
    (setq major-mode 'term-mode)
    (setq mode-name "Term")
    (use-local-map term-mode-map)
+   ;; we do not want indent to sneak in any tabs
+   (setq indent-tabs-mode nil)
+ 
    (make-local-variable 'term-home-marker)
    (setq term-home-marker (copy-marker 0))
    (make-local-variable 'term-saved-home-marker)
*************** without any interpretation."
*** 1184,1189 ****
--- 1186,1196 ----
  					((eq arg '-) -1)
  					(t (1- arg)))))))
  
+ (defun term-paste ()
+   "Insert the last stretch of killed text at point."
+   (interactive)
+    (term-send-raw-string (current-kill 0)))
+ 
  ;; Which would be better:  "\e[A" or "\eOA"? readline accepts either.
  ;; For my configuration it's definitely better \eOA but YMMV. -mm
  ;; For example: vi works with \eOA while elm wants \e[A ...
*************** without any interpretation."
*** 1195,1202 ****
  (defun term-send-end   () (interactive) (term-send-raw-string "\e[4~"))
  (defun term-send-prior () (interactive) (term-send-raw-string "\e[5~"))
  (defun term-send-next  () (interactive) (term-send-raw-string "\e[6~"))
! (defun term-send-del   () (interactive) (term-send-raw-string "\C-?"))
! (defun term-send-backspace  () (interactive) (term-send-raw-string "\C-H"))
  \f
  (defun term-char-mode ()
    "Switch to char (\"raw\") sub-mode of term mode.
--- 1202,1209 ----
  (defun term-send-end   () (interactive) (term-send-raw-string "\e[4~"))
  (defun term-send-prior () (interactive) (term-send-raw-string "\e[5~"))
  (defun term-send-next  () (interactive) (term-send-raw-string "\e[6~"))
! (defun term-send-del   () (interactive) (term-send-raw-string "\e[3~"))
! (defun term-send-backspace  () (interactive) (term-send-raw-string "\C-?"))
  \f
  (defun term-char-mode ()
    "Switch to char (\"raw\") sub-mode of term mode.
*************** The main purpose is to get rid of the lo
*** 1366,1379 ****
    "%s%s:li#%d:co#%d:cl=\\E[H\\E[J:cd=\\E[J:bs:am:xn:cm=\\E[%%i%%d;%%dH\
  :nd=\\E[C:up=\\E[A:ce=\\E[K:ho=\\E[H:pt\
  :al=\\E[L:dl=\\E[M:DL=\\E[%%dM:AL=\\E[%%dL:cs=\\E[%%i%%d;%%dr:sf=^J\
- :te=\\E[2J\\E[?47l\\E8:ti=\\E7\\E[?47h\
  :dc=\\E[P:DC=\\E[%%dP:IC=\\E[%%d@:im=\\E[4h:ei=\\E[4l:mi:\
  :so=\\E[7m:se=\\E[m:us=\\E[4m:ue=\\E[m:md=\\E[1m:mr=\\E[7m:me=\\E[m\
  :UP=\\E[%%dA:DO=\\E[%%dB:LE=\\E[%%dD:RI=\\E[%%dC\
  :kl=\\EOD:kd=\\EOB:kr=\\EOC:ku=\\EOA:kN=\\E[6~:kP=\\E[5~:@7=\\E[4~:kh=\\E[1~\
  :mk=\\E[8m:cb=\\E[1K:op=\\E[39;49m:Co#8:pa#64:AB=\\E[4%%dm:AF=\\E[3%%dm:cr=^M\
! :bl=^G:do=^J:le=^H:ta=^I:se=\E[27m:ue=\E24m:"
  ;;; : -undefine ic
    "termcap capabilities supported")
  
  ;;; This auxiliary function cranks up the process for term-exec in
--- 1373,1387 ----
    "%s%s:li#%d:co#%d:cl=\\E[H\\E[J:cd=\\E[J:bs:am:xn:cm=\\E[%%i%%d;%%dH\
  :nd=\\E[C:up=\\E[A:ce=\\E[K:ho=\\E[H:pt\
  :al=\\E[L:dl=\\E[M:DL=\\E[%%dM:AL=\\E[%%dL:cs=\\E[%%i%%d;%%dr:sf=^J\
  :dc=\\E[P:DC=\\E[%%dP:IC=\\E[%%d@:im=\\E[4h:ei=\\E[4l:mi:\
  :so=\\E[7m:se=\\E[m:us=\\E[4m:ue=\\E[m:md=\\E[1m:mr=\\E[7m:me=\\E[m\
  :UP=\\E[%%dA:DO=\\E[%%dB:LE=\\E[%%dD:RI=\\E[%%dC\
  :kl=\\EOD:kd=\\EOB:kr=\\EOC:ku=\\EOA:kN=\\E[6~:kP=\\E[5~:@7=\\E[4~:kh=\\E[1~\
  :mk=\\E[8m:cb=\\E[1K:op=\\E[39;49m:Co#8:pa#64:AB=\\E[4%%dm:AF=\\E[3%%dm:cr=^M\
! :bl=^G:do=^J:le=^H:ta=^I:se=\E[27m:ue=\E24m\
! :kb=^?:kD=^[[3~:"
  ;;; : -undefine ic
+ ;;; don't define :te=\\E[2J\\E[?47l\\E8:ti=\\E7\\E[?47h\
    "termcap capabilities supported")
  
  ;;; This auxiliary function cranks up the process for term-exec in
*************** See `term-prompt-regexp'."
*** 2693,2699 ****
  		   (if (not funny) (setq funny str-length))
  		   (cond ((> funny i)
  			  (cond ((eq term-terminal-state 1)
! 				 (term-move-columns 1)
  				 (setq term-terminal-state 0)))
  			  (setq count (- funny i))
  			  (setq temp (- (+ (term-horizontal-column) count)
--- 2701,2711 ----
  		   (if (not funny) (setq funny str-length))
  		   (cond ((> funny i)
  			  (cond ((eq term-terminal-state 1)
! 				 ;; We are in state 1, we need to wrap
! 				 ;; around.  Go to the beginning of
! 				 ;; the next line and switch to state
! 				 ;; 0.
! 				 (term-goto (1+ (term-current-row)) 0)
  				 (setq term-terminal-state 0)))
  			  (setq count (- funny i))
  			  (setq temp (- (+ (term-horizontal-column) count)
*************** See `term-prompt-regexp'."
*** 2702,2707 ****
--- 2714,2720 ----
  				((> count temp)	;; Some chars fit.
  				 ;; This iteration, handle only what fits.
  				 (setq count (- count temp))
+ 				 (setq temp 0)
  				 (setq funny (+ count i)))
  				((or (not (or term-pager-count
  					      term-scroll-with-delete))
*************** See `term-prompt-regexp'."
*** 2741,2747 ****
  				 (setq term-terminal-state 1)))
  			  (setq i (1- funny)))
  			 ((and (setq term-terminal-state 0)
! 			       (eq char ?\^I)) ; TAB
  			  ;; FIXME:  Does not handle line wrap!
  			  (setq count (term-current-column))
  			  (setq count (+ count 8 (- (mod count 8))))
--- 2754,2760 ----
  				 (setq term-terminal-state 1)))
  			  (setq i (1- funny)))
  			 ((and (setq term-terminal-state 0)
! 			       (eq char ?\^I)) ; TAB (terminfo: ht)
  			  ;; FIXME:  Does not handle line wrap!
  			  (setq count (term-current-column))
  			  (setq count (+ count 8 (- (mod count 8))))
*************** See `term-prompt-regexp'."
*** 2768,2774 ****
  			  (if (not (and term-kill-echo-list
  					(term-check-kill-echo-list)))
  			      (term-down 1 t)))
! 			 ((eq char ?\b)
  			  (term-move-columns -1))
  			 ((eq char ?\033) ; Escape
  			  (setq term-terminal-state 2))
--- 2781,2787 ----
  			  (if (not (and term-kill-echo-list
  					(term-check-kill-echo-list)))
  			      (term-down 1 t)))
! 			 ((eq char ?\b)  ;; (terminfo: cub1)
  			  (term-move-columns -1))
  			 ((eq char ?\033) ; Escape
  			  (setq term-terminal-state 2))
*************** See `term-prompt-regexp'."
*** 2818,2830 ****
  			 ((eq char ?M) ;; scroll reversed
  			  (term-insert-lines 1)
  			  (setq term-terminal-state 0))
! 			 ((eq char ?7) ;; Save cursor
  			  (term-handle-deferred-scroll)
  			  (setq term-saved-cursor
  				(cons (term-current-row)
  				      (term-horizontal-column)))
  			  (setq term-terminal-state 0))
! 			 ((eq char ?8) ;; Restore cursor
  			  (if term-saved-cursor
  			      (term-goto (car term-saved-cursor)
  					 (cdr term-saved-cursor)))
--- 2831,2843 ----
  			 ((eq char ?M) ;; scroll reversed
  			  (term-insert-lines 1)
  			  (setq term-terminal-state 0))
! 			 ((eq char ?7) ;; Save cursor (terminfo: sc)
  			  (term-handle-deferred-scroll)
  			  (setq term-saved-cursor
  				(cons (term-current-row)
  				      (term-horizontal-column)))
  			  (setq term-terminal-state 0))
! 			 ((eq char ?8) ;; Restore cursor (terminfo: rc)
  			  (if term-saved-cursor
  			      (term-goto (car term-saved-cursor)
  					 (cdr term-saved-cursor)))
*************** See `term-prompt-regexp'."
*** 2976,2988 ****
     ((eq parameter 8)
      (setq term-ansi-current-invisible 1))
  
! ;;; Reset reverse (i.e. terminfo rmso)
     ((eq parameter 24)
!     (setq term-ansi-current-reverse 0))
  
! ;;; Reset underline (i.e. terminfo rmul)
     ((eq parameter 27)
!     (setq term-ansi-current-underline 0))
  
  ;;; Foreground
     ((and (>= parameter 30) (<= parameter 37))
--- 2989,3001 ----
     ((eq parameter 8)
      (setq term-ansi-current-invisible 1))
  
! ;;; Reset underline (i.e. terminfo rmul)
     ((eq parameter 24)
!     (setq term-ansi-current-underline 0))
  
! ;;; Reset reverse (i.e. terminfo rmso)
     ((eq parameter 27)
!     (setq term-ansi-current-reverse 0))
  
  ;;; Foreground
     ((and (>= parameter 30) (<= parameter 37))
*************** See `term-prompt-regexp'."
*** 3097,3103 ****
      (term-goto
       (1- term-terminal-previous-parameter)
       (1- term-terminal-parameter)))
!    ;; \E[A - cursor up
     ((eq char ?A)
      (term-handle-deferred-scroll)
      (term-down (- (max 1 term-terminal-parameter)) t))
--- 3110,3116 ----
      (term-goto
       (1- term-terminal-previous-parameter)
       (1- term-terminal-parameter)))
!    ;; \E[A - cursor up (terminfo: cuu1)
     ((eq char ?A)
      (term-handle-deferred-scroll)
      (term-down (- (max 1 term-terminal-parameter)) t))
*************** See `term-prompt-regexp'."
*** 3110,3122 ****
     ;; \E[D - cursor left
     ((eq char ?D)
      (term-move-columns (- (max 1 term-terminal-parameter))))
!    ;; \E[J - clear to end of screen
     ((eq char ?J)
      (term-erase-in-display term-terminal-parameter))
!    ;; \E[K - clear to end of line
     ((eq char ?K)
      (term-erase-in-line term-terminal-parameter))
!    ;; \E[L - insert lines
     ((eq char ?L)
      (term-insert-lines (max 1 term-terminal-parameter)))
     ;; \E[M - delete lines
--- 3123,3135 ----
     ;; \E[D - cursor left
     ((eq char ?D)
      (term-move-columns (- (max 1 term-terminal-parameter))))
!    ;; \E[J - clear to end of screen (terminfo: ed, clear)
     ((eq char ?J)
      (term-erase-in-display term-terminal-parameter))
!    ;; \E[K - clear to end of line (terminfo: el, el1)
     ((eq char ?K)
      (term-erase-in-line term-terminal-parameter))
!    ;; \E[L - insert lines (terminfo: il, il1)
     ((eq char ?L)
      (term-insert-lines (max 1 term-terminal-parameter)))
     ;; \E[M - delete lines
*************** See `term-prompt-regexp'."
*** 3130,3148 ****
      (term-insert-spaces (max 1 term-terminal-parameter)))
     ;; \E[?h - DEC Private Mode Set
     ((eq char ?h)
!     (cond ((eq term-terminal-parameter 4)
  	   (setq term-insert-mode t))
! 	  ((eq term-terminal-parameter 47)
! 	   (term-switch-to-alternate-sub-buffer t))))
     ;; \E[?l - DEC Private Mode Reset
     ((eq char ?l)
!     (cond ((eq term-terminal-parameter 4)
  	   (setq term-insert-mode nil))
! 	  ((eq term-terminal-parameter 47)
! 	   (term-switch-to-alternate-sub-buffer nil))))
  
  ;;; Modified to allow ansi coloring -mm
!    ;; \E[m - Set/reset standard mode
     ((eq char ?m)
      (when (= term-terminal-more-parameters 1)
        (if (>= term-terminal-previous-parameter-4 0)
--- 3143,3164 ----
      (term-insert-spaces (max 1 term-terminal-parameter)))
     ;; \E[?h - DEC Private Mode Set
     ((eq char ?h)
!     (cond ((eq term-terminal-parameter 4)  ;; (terminfo: smir)
  	   (setq term-insert-mode t))
! 	  ;; ((eq term-terminal-parameter 47) ;; (terminfo: smcup)
! 	  ;; (term-switch-to-alternate-sub-buffer t))
! 	  ))
     ;; \E[?l - DEC Private Mode Reset
     ((eq char ?l)
!     (cond ((eq term-terminal-parameter 4)  ;; (terminfo: rmir)
  	   (setq term-insert-mode nil))
! 	  ;; ((eq term-terminal-parameter 47) ;; (terminfo: rmcup)
! 	  ;; (term-switch-to-alternate-sub-buffer nil))
! 	  ))
  
  ;;; Modified to allow ansi coloring -mm
!    ;; \E[m - Set/reset modes, set bg/fg 
!    ;;(terminfo: smso,rmso,smul,rmul,rev,bold,sgr0,invis,op,setab,setaf)
     ((eq char ?m)
      (when (= term-terminal-more-parameters 1)
        (if (>= term-terminal-previous-parameter-4 0)
*************** The top-most line is line 0."
*** 3186,3217 ****
  	    (not (and (= term-scroll-start 0)
  		      (= term-scroll-end term-height))))))
  
! (defun term-switch-to-alternate-sub-buffer (set)
!   ;; If asked to switch to (from) the alternate sub-buffer, and already (not)
!   ;; using it, do nothing.  This test is needed for some programs (including
!   ;; Emacs) that emit the ti termcap string twice, for unknown reason.
!   (term-handle-deferred-scroll)
!   (if (eq set (not (term-using-alternate-sub-buffer)))
!       (let ((row (term-current-row))
! 	    (col (term-horizontal-column)))
! 	(cond (set
! 	       (goto-char (point-max))
! 	       (if (not (eq (preceding-char) ?\n))
! 		   (term-insert-char ?\n 1))
! 	       (setq term-scroll-with-delete t)
! 	       (setq term-saved-home-marker (copy-marker term-home-marker))
! 	       (set-marker term-home-marker (point)))
! 	      (t
! 	       (setq term-scroll-with-delete
! 		     (not (and (= term-scroll-start 0)
! 			       (= term-scroll-end term-height))))
! 	       (set-marker term-home-marker term-saved-home-marker)
! 	       (set-marker term-saved-home-marker nil)
! 	       (setq term-saved-home-marker nil)
! 	       (goto-char term-home-marker)))
! 	(setq term-current-column nil)
! 	(setq term-current-row 0)
! 	(term-goto row col))))
  
  ;; Default value for the symbol term-command-hook.
  
--- 3202,3233 ----
  	    (not (and (= term-scroll-start 0)
  		      (= term-scroll-end term-height))))))
  
! ;; (defun term-switch-to-alternate-sub-buffer (set)
! ;;   ;; If asked to switch to (from) the alternate sub-buffer, and already (not)
! ;;   ;; using it, do nothing.  This test is needed for some programs (including
! ;;   ;; Emacs) that emit the ti termcap string twice, for unknown reason.
! ;;   (term-handle-deferred-scroll)
! ;;   (if (eq set (not (term-using-alternate-sub-buffer)))
! ;;       (let ((row (term-current-row))
! ;; 	    (col (term-horizontal-column)))
! ;; 	(cond (set
! ;; 	       (goto-char (point-max))
! ;; 	       (if (not (eq (preceding-char) ?\n))
! ;; 		   (term-insert-char ?\n 1))
! ;; 	       (setq term-scroll-with-delete t)
! ;; 	       (setq term-saved-home-marker (copy-marker term-home-marker))
! ;; 	       (set-marker term-home-marker (point)))
! ;; 	      (t
! ;; 	       (setq term-scroll-with-delete
! ;; 		     (not (and (= term-scroll-start 0)
! ;; 			       (= term-scroll-end term-height))))
! ;; 	       (set-marker term-home-marker term-saved-home-marker)
! ;; 	       (set-marker term-saved-home-marker nil)
! ;; 	       (setq term-saved-home-marker nil)
! ;; 	       (goto-char term-home-marker)))
! ;; 	(setq term-current-column nil)
! ;; 	(setq term-current-row 0)
! ;; 	(term-goto row col))))
  
  ;; Default value for the symbol term-command-hook.
  
*************** all pending output has been dealt with."
*** 3521,3531 ****
    (if (not (bolp)) (insert-before-markers ?\n)))
  
  (defun term-erase-in-line (kind)
!   (if (> kind 1) ;; erase left of point
        (let ((cols (term-horizontal-column)) (saved-point (point)))
  	(term-vertical-motion 0)
  	(delete-region (point) saved-point)
! 	(term-insert-char ?\n cols)))
    (if (not (eq kind 1)) ;; erase right of point
        (let ((saved-point (point))
  	    (wrapped (and (zerop (term-horizontal-column))
--- 3537,3547 ----
    (if (not (bolp)) (insert-before-markers ?\n)))
  
  (defun term-erase-in-line (kind)
!   (if (= kind 1) ;; erase left of point
        (let ((cols (term-horizontal-column)) (saved-point (point)))
  	(term-vertical-motion 0)
  	(delete-region (point) saved-point)
! 	(term-insert-char ?  cols)))
    (if (not (eq kind 1)) ;; erase right of point
        (let ((saved-point (point))
  	    (wrapped (and (zerop (term-horizontal-column))
*************** Should only be called when point is at t
*** 3624,3630 ****
      (term-insert-char ?\n lines)
      (goto-char start)))
  \f
! (defun term-set-output-log (name)
    "Record raw inferior process output in a buffer."
    (interactive (list (if term-log-buffer
  			 nil
--- 3640,3646 ----
      (term-insert-char ?\n lines)
      (goto-char start)))
  \f
! (defun term-start-output-log (name)
    "Record raw inferior process output in a buffer."
    (interactive (list (if term-log-buffer
  			 nil
*************** Should only be called when point is at t
*** 3646,3655 ****
      (message "Recording terminal emulator output into buffer \"%s\""
  	     (buffer-name term-log-buffer))))
  
! (defun term-stop-photo ()
    "Discontinue raw inferior process logging."
    (interactive)
!   (term-set-output-log nil))
  
  (defun term-show-maximum-output ()
    "Put the end of the buffer at the bottom of the window."
--- 3662,3671 ----
      (message "Recording terminal emulator output into buffer \"%s\""
  	     (buffer-name term-log-buffer))))
  
! (defun term-stop-output-log ()
    "Discontinue raw inferior process logging."
    (interactive)
!   (term-start-output-log nil))
  
  (defun term-show-maximum-output ()
    "Put the end of the buffer at the bottom of the window."



Index: eterm.ti
===================================================================
RCS file: /cvsroot/emacs/emacs/etc/e/eterm.ti,v
retrieving revision 1.3
diff -c -3 -p -c -r1.3 eterm.ti
*** eterm.ti	17 Sep 2004 21:08:45 -0000	1.3
--- eterm.ti	21 Sep 2004 20:47:42 -0000
***************
*** 1,21 ****
  eterm,
! 	lines#24,cols#80,
! 	colors#8,pairs#64,
! 	cuu1=\E[A,cud1=\n,cub1=\b,cuf1=\E[C,home=\E[H,cr=\r,
! 	cuu=\E[%p1%dA,cud=\E[%p1%dB,cub=\E[%p1%dD,cuf=\E[%p1%dC,
  	cup=\E[%i%p1%d;%p2%dH,
! 	ind=\n,csr=\E[%i%p1%d;%p2%dr,
! 	il1=\E[L,il=\E[%p1%dL,
! 	clear=\E[H\E[J,ed=\E[J,el=\E[K,el1=\E[1K,
! 	dl1=\E[M,dl=\E[%p1%dM,dch1=\E[P,dch=\E[%p1%dP,
! 	smir=\E[4h,rmir=\E[4l,ich=\E[%p1%d@,mir,
! 	smcup=\E7\E[?47h,rmcup=\E[2J\E[?47l\E8,
! 	ht=\t,khome=\E[1~,kend=\E[4~,knp=\E[6~,kpp=\E[5~,
! 	kcub1=\EOD, kcud1=\EOB, kcuf1=\EOC, kcuu1=\EOA,
!  	smso=\E[7m,rmso=\E[27m,
! 	smul=\E[4m,rmul=\E[24m,
! 	rev=\E[7m,bold=\E[1m,sgr0=\E[m,
! 	invis=\E[8m,op=\E[39;49m,
! 	setab=\E[%p1%{40}%+%dm, setaf=\E[%p1%{30}%+%dm,
! 	bel=^G,xenl,am,
! 
--- 1,59 ----
  eterm,
! 	lines#24,
! 	cols#80,
! 	colors#8,
! 	pairs#64,
! 	am,
! 	mir,
! 	xenl,
! 	bel=^G,
! 	bold=\E[1m,
! 	clear=\E[H\E[J,
! 	cr=\r,
! 	csr=\E[%i%p1%d;%p2%dr,
! 	cub1=\b,
! 	cub=\E[%p1%dD,
! 	cud1=\n,
! 	cud=\E[%p1%dB,
! 	cuf1=\E[C,
! 	cuf=\E[%p1%dC,
  	cup=\E[%i%p1%d;%p2%dH,
! 	cuu1=\E[A,
! 	cuu=\E[%p1%dA,
! 	dch1=\E[P,
! 	dch=\E[%p1%dP,
! 	dl1=\E[M,
! 	dl=\E[%p1%dM,
! 	ed=\E[J,
! 	el1=\E[1K,
! 	el=\E[K,
! 	home=\E[H,
! 	ht=\t,
! 	ich=\E[%p1%d@,
! 	il1=\E[L,
! 	il=\E[%p1%dL,
! 	ind=\n,
! 	invis=\E[8m,
! 	kbs=^?,
! 	kcub1=\EOD,
! 	kcud1=\EOB,
! 	kcuf1=\EOC,
! 	kcuu1=\EOA,
! 	kdch1=\E[3~,
! 	kend=\E[4~,
! 	khome=\E[1~,
! 	knp=\E[6~,
! 	kpp=\E[5~,
! 	op=\E[39;49m,
! 	rev=\E[7m,
! 	rmir=\E[4l,
! 	rmso=\E[27m,
! 	rmul=\E[24m,
! 	setab=\E[%p1%{40}%+%dm,
! 	setaf=\E[%p1%{30}%+%dm,
! 	sgr0=\E[m,
! 	smir=\E[4h,
! 	smul=\E[4m,
!  	smso=\E[7m,
! #	smcup=\E[?47h,
! #	rmcup=\E[?47l,



[-- Attachment #2: eterm --]
[-- Type: application/octet-stream, Size: 1080 bytes --]

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: yet more term.el fixes #2

[Miles Bader]

Dan Nicolaescu <dann@ics.uci.edu> writes:
> One remaining problem in term.el is the handling of characters in the
> 128-255 range (and probably the non 8-bit encodings too).  term.el
> uses move-to-column to position the cursor at certain positions in the
> buffer. But the char-width of a character >127 is 4 (at least in my
> setup), this confuses the cursor positioning

Seems odd... in *scratch* in an `emacs -q' session:

   (char-width ?\217)
   1
   (char-width ?á)
   1
   (char-width ?字)
   2

Does term.el use some funny buffer settings?

-Miles
-- 
Somebody has to do something, and it's just incredibly pathetic that it
has to be us.  -- Jerry Garcia

Re: yet more term.el fixes #2

[Dan Nicolaescu]

Miles Bader <miles@lsi.nec.co.jp> writes:

  > Dan Nicolaescu <dann@ics.uci.edu> writes:
  > > One remaining problem in term.el is the handling of characters in the
  > > 128-255 range (and probably the non 8-bit encodings too).  term.el
  > > uses move-to-column to position the cursor at certain positions in the
  > > buffer. But the char-width of a character >127 is 4 (at least in my
  > > setup), this confuses the cursor positioning
  > 
  > Seems odd... in *scratch* in an `emacs -q' session:
  > 
  >    (char-width ?\217)
  >    1
  >    (char-width ?á)
  >    1
  >    (char-width ?å­—)
  >    2

Looking closer, the characters > 158 decimal have char-width 4 in my
setup. 

Use this to print all the widths: 
(defun print-width ()
  (interactive)
  (let ((i 0))
    (while (< i 256)
      (insert (format "%d %d %c\n" i (char-width i) i) )
      (setq i (1+ i)))))

Anything with != 1 will confuse term.el if sent to the terminal as
such.

  > Does term.el use some funny buffer settings?

Not AFAIK. 

Re: yet more term.el fixes #2

[Miles Bader]

Dan Nicolaescu <dann@ics.uci.edu> writes:
> Looking closer, the characters > 158 decimal have char-width 4 in my
> setup. 
>
> Use this to print all the widths: 

Hmmm, using that code I get simular (odd) results:  the result of
`char-width' seems to be completely unrelated to the actual displayed
width of the characters -- some values are displayed as a single column
character, but have a `char-width' value of 4, and some are displayed as
a 4-column escape-sequence, but have a `char-width' value of 1!!

Here's a version that uses `decode-coding-string' to get correct
results; probably term.el should be using something like that:

   (defun print-width ()
     (interactive)
     (let ((i 0))
       (while (< i 256)
         (let ((decoded (decode-coding-string (string i) 'iso-8859-1)))
           (insert (format "%d %d %s\n" i (string-width decoded) decoded)))
         (setq i (1+ i)))))

Using this, all the char-width values match the displayed representation.

When simply inserting raw character values in to the buffer, it seems
that some sort of default decoding takes place at insert/display time,
but an equivalent default decoding is not done by `char-width'; whether
that represents a bug, and if so, where, I don't know, but I think it's
probably always safer to be explicit as in the above code.

-Miles
-- 
The secret to creativity is knowing how to hide your sources.
  --Albert Einstein

Re: yet more term.el fixes #2

[Stefan Monnier]

> When simply inserting raw character values in to the buffer, it seems
> that some sort of default decoding takes place at insert/display time,

Yes, `insert' says it uses string-make-multibyte which itself uses
unibyte-char-to-multibyte.

By the way, `insert' also says "if you want to preserve binary data as
binary, use string-as-multibyte", shouldn't that be "string-TO-multibyte"?


        Stefan

Re: yet more term.el fixes #2

[Dan Nicolaescu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

  > > When simply inserting raw character values in to the buffer, it seems
  > > that some sort of default decoding takes place at insert/display time,
  > 
  > Yes, `insert' says it uses string-make-multibyte which itself uses
  > unibyte-char-to-multibyte.
  > 
  > By the way, `insert' also says "if you want to preserve binary data as
  > binary, use string-as-multibyte", shouldn't that be "string-TO-multibyte"?

Well, I managed to make it work, I needed to add: 

  (set-buffer-multibyte t) 
to `term-mode'

and change the (insert (substring str i funny)) call in
`term-emulate-terminal' to: 

(insert (string-as-unibyte (substring str i funny)))

This works with both:

EMACS_UNIBYTE=1 emacs
EMACS_UNIBYTE=0 emacs

One way to test it is to run lynx www.emacswiki.org in the term.el
terminal and check that the language names are printed correctly.

I don't know if the above changes are strictly correct, so I'd
appreciate is somebody could confirm it. 

Thanks.
                --dan

Re: yet more term.el fixes #2

[Stefan Monnier]

> (insert (string-as-unibyte (substring str i funny)))

I think string-as-unibyte is also an extremely poor choice here and will
likely lead to unexpected errors in some cases (it's a function that should
have big warning signs all around it, like string-as-multibyte).

Also calling set-buffer-multibyte is to be avoided if possible.

Can you try with

  (insert (decode-coding-string (substring str i funny) locale-coding-system))

Also based on your above code, it seems that `str' is a multibyte string,
which sounds wrong.  I think that the coding-system used for the process's
output should be `binary'.  Can you see what coding-system is used for
the process?

I suggest a patch like the one below,


        Stefan


--- orig/lisp/term.el
+++ mod/lisp/term.el
@@ -1290,7 +1290,8 @@
     (let ((proc (get-buffer-process buffer)))	; Blast any old process.
       (if proc (delete-process proc)))
     ;; Crank up a new process
-    (let ((proc (term-exec-1 name buffer command switches)))
+    (let ((proc (term-exec-1 name buffer command switches))
+	  (coding-system-for-read 'binary))
       (make-local-variable 'term-ptyp)
       (setq term-ptyp process-connection-type) ; t if pty, nil if pipe.
       ;; Jump to the end, and set the process mark.
@@ -2722,7 +2724,7 @@
 			  ;; following point if not eob nor insert-mode.
 			  (let ((old-column (current-column))
 				columns pos)
-			    (insert (substring str i funny))
+			    (insert (decode-coding-string (substring str i funny) locale-coding-system))
 			    (setq term-current-column (current-column)
 				  columns (- term-current-column old-column))
 			    (when (not (or (eobp) term-insert-mode))

Re: yet more term.el fixes #2

[Dan Nicolaescu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

  > > (insert (string-as-unibyte (substring str i funny)))
  > 
  > I think string-as-unibyte is also an extremely poor choice here and will
  > likely lead to unexpected errors in some cases (it's a function that should
  > have big warning signs all around it, like string-as-multibyte).
  > 
  > Also calling set-buffer-multibyte is to be avoided if possible.
  > 
  > Can you try with
  > 
  >   (insert (decode-coding-string (substring str i funny) locale-coding-system))
  > Also based on your above code, it seems that `str' is a multibyte string,
  > which sounds wrong.  I think that the coding-system used for the process's
  > output should be `binary'.  Can you see what coding-system is used for
  > the process?

In term-exec-1:
 	(coding-system-for-read 'unknown-unix)

  > I suggest a patch like the one below,
[snip]

Did not work. 

It can be made work by binding: 

(default-enable-multibyte-characters t)

when creating the buffer in make-term  (so as to do what
set-buffer-multibyte used to do)

and binding 
(default-enable-multibyte-characters nil)
in term-exec-1. 

Now that this stuff work I found out that pasting non-ASCII text with
the mouse does not work. Pasting works by doing process-send-string
with the data from the kill ring. It seems that the string must be
encoded in some way. Any idea how to deal with that? 

Re: yet more term.el fixes #2

[Stefan]

>> > (insert (string-as-unibyte (substring str i funny)))
>> 
>> I think string-as-unibyte is also an extremely poor choice here and will
>> likely lead to unexpected errors in some cases (it's a function that should
>> have big warning signs all around it, like string-as-multibyte).
>> 
>> Also calling set-buffer-multibyte is to be avoided if possible.
>> 
>> Can you try with
>> 
>> (insert (decode-coding-string (substring str i funny) locale-coding-system))
>> Also based on your above code, it seems that `str' is a multibyte string,
>> which sounds wrong.  I think that the coding-system used for the process's
>> output should be `binary'.  Can you see what coding-system is used for
>> the process?

> In term-exec-1:
>  	(coding-system-for-read 'unknown-unix)

Sorry I miss that term-exec-1.
Please add to my previous patch the one below.

> Did not work.

What did it do?

> It can be made work by binding: 
> (default-enable-multibyte-characters t)
> when creating the buffer in make-term  (so as to do what
> set-buffer-multibyte used to do)

But fiddling with enable-multibyte-characters is only ever going to hide
the bug.


        Stefan

Re: yet more term.el fixes #2

[Stefan Monnier]

> Sorry I miss that term-exec-1.
> Please add to my previous patch the one below.

Duh!
Alright, just try the patch below.

But in any case I don't really understand your problem.  I don't see any
call to `char-width', so I guess you're probably not referring to the
`char-width' function but just to the on-screen width of some characters in
the 128-25 range.  Most likely it's 4 because they are displayed as "\NNN".
When I try M-x term and then I cat a file with latin-1 accents they
get displayed correctly.  Is that the case for you as well?
If not can you better describe your setup (things like locale and stuff)?

As for those \NNN thingies, we should indeed get rid of them in term-mode
because they screw things up.  For that, we need something like:

(defvar term-display-table
  (let ((dt (or (copy-sequence standard-display-table)
		(make-display-table)))
        i)
    (setq i 0)
    (while (< i 32)
      (aset dt i (vector i))
      (setq i (1+ i)))
    (setq i 128)
    (while (< i 256)
      (aset dt i (vector i))
      (setq i (1+ i)))
    dt))

and in term-mode we need to (setq buffer-display-table term-display-table).


        Stefan


--- orig/lisp/term.el
+++ mod/lisp/term.el
@@ -1398,9 +1396,10 @@
 	   (format "COLUMNS=%d" term-width))
 	  process-environment))
 	(process-connection-type t)
-	;; inhibit-eol-conversion doesn't seem to do the job, but this does.
-	(coding-system-for-read 'binary)
-	)
+	;; The process's output contains not just chars but also binary
+	;; escape codes, so we need to see the raw output.  We will have to
+	;; do the decoding by hand on the parts that are made of chars.
+	(coding-system-for-read 'binary))
     (apply 'start-process name buffer
 	   "/bin/sh" "-c"
 	   (format "stty -nl echo rows %d columns %d sane 2>/dev/null;\
@@ -2720,7 +2718,8 @@
 			  ;; following point if not eob nor insert-mode.
 			  (let ((old-column (current-column))
 				columns pos)
-			    (insert (substring str i funny))
+			    (insert (decode-coding-string
+				     (substring str i funny) locale-coding-system))
 			    (setq term-current-column (current-column)
 				  columns (- term-current-column old-column))
 			    (when (not (or (eobp) term-insert-mode))