bug#29918: 26.0.90; serial-term error in process filter

bug#29918: 26.0.90; serial-term error in process filter

[Gemini Lasswell]

When a serial-term buffer is connected to a device with bad wiring or
hardware problems, it spams the minibuffer with messages like this:

error in process filter: Args out of range: "\361", -1 [2 times]
error in process filter: Args out of range: "\377\375", -1 [2 times]
error in process filter: Args out of range: "\370\340\374\374", -1 [2 times]
error in process filter: Args out of range: "\376\340\360\370", -1 [2 times]
error in process filter: Args out of range: "\377\377\374\376\371\376\301\377", -1 [2 times]
error in process filter: Args out of range: "\376", -1 [2 times]

Emacs 25.3 is silent under the same conditions.

I'm using an FTDI USB to serial converter and a Raspberry pi. I
discovered the bug after I had wired things up with a flaky
breadboard, but it's quick to reproduce by disconnecting ground.

With emacs -Q:

Connect USB from your Emacs machine to the FTDI device.
Connect GND, TX and RX from the FTDI to the Raspberry Pi.
Turn on the Pi.
M-x serial-term RET /dev/your-device-name-here RET 115200 RET
Now disconnect GND.

Result: The serial-term buffer starts to fill with gibberish, which is
expected and also happens in 25.3, and the process filter errors start
showing up at irregular intervals.


In GNU Emacs 26.0.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.21)
 of 2017-12-31 built on chinook
Repository revision: 312c5655669a882186884626f0cf361de70e679d
Windowing system distributor 'The X.Org Foundation', version 11.0.11905000
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --prefix=/home/gem/src/emacs/emacs-26/bin --with-modules
 --with-x-toolkit=gtk3 --with-xft --config-cache'

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS GSETTINGS NOTIFY LIBSELINUX
GNUTLS LIBXML2 FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 MODULES

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort map mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils elec-pair
time-date mule-util tooltip eldoc electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet
lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese composite charscript
charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
minibuffer cl-preloaded nadvice loaddefs button faces cus-face
macroexp files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
dbusbind inotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 95859 7544)
 (symbols 48 20453 1)
 (miscs 40 44 104)
 (strings 32 28532 1259)
 (string-bytes 1 795727)
 (vectors 16 14820)
 (vector-slots 8 502732 10570)
 (floats 8 49 68)
 (intervals 56 205 0)
 (buffers 992 11))

bug#29918: 26.0.90; serial-term error in process filter

[Lars Ingebrigtsen]

Gemini Lasswell <gazally@runbox.com> writes:

> When a serial-term buffer is connected to a device with bad wiring or
> hardware problems, it spams the minibuffer with messages like this:
>
> error in process filter: Args out of range: "\361", -1 [2 times]
> error in process filter: Args out of range: "\377\375", -1 [2 times]
> error in process filter: Args out of range: "\370\340\374\374", -1 [2 times]
> error in process filter: Args out of range: "\376\340\360\370", -1 [2 times]
> error in process filter: Args out of range: "\377\377\374\376\371\376\301\377", -1 [2 times]
> error in process filter: Args out of range: "\376", -1 [2 times]
>
> Emacs 25.3 is silent under the same conditions.

Does setting (setq debug-on-error t) give you a backtrace for these
errors?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#29918: 26.0.90; serial-term error in process filter

[Gemini Lasswell]

[-- Attachment #1: Type: text/plain, Size: 418 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Does setting (setq debug-on-error t) give you a backtrace for these
> errors?

No, but I just made one happen by removing the internal_condition_case_1
wrapper on the read_process_output_call in read_and_dispose_of_process_output.
Then after I got the first backtrace showing that the error was in
term-emulate-terminal I eval-defun'd it to make a more useful backtrace:


[-- Attachment #2: term-emulate-terminal.txt --]
[-- Type: text/plain, Size: 59801 bytes --]

Debugger entered--Lisp error: (args-out-of-range "\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376" -1)
  aref("\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376" -1)
  (char-charset (aref decoded-substring (- count 1 partial)))
  (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit)
  (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial)))
  (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial)))))
    proc = #<process /dev/ftdi0>
    str = "\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376"
    i = 0
    funny = 25
    decoded-substring = "\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376"
    save-point = #<marker at 25166 in /dev/ftdi0>
    save-marker = #<marker at 20731 in /dev/ftdi0>
    win = nil
    selected = #<window 10>
    last-win = nil
    str-length = 25
    save-selected-window--state = (#<window 10> (#<frame emacs@sockeye 0x1b81630> . #<window 10>))
    ctl-params = nil
    ctl-params-end = nil
    ctl-end = 26
    partial = 25
    count = 25
  (progn (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial))))))
  (if (= funny str-length) (progn (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial)))))))
  (progn (if term-do-line-wrapping (progn (term-down 1 t) (term-move-to-column 0) (setq term-do-line-wrapping nil))) (setq decoded-substring (decode-coding-string (substring str i funny) locale-coding-system t)) (if (= funny str-length) (progn (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial))))))) (let ((old-column (term-horizontal-column)) (old-point (point)) columns) (if term-suppress-hard-newline nil (while (> (+ (length decoded-substring) old-column) term-width) (insert (substring decoded-substring 0 (- term-width old-column))) (delete-region (point) (line-end-position)) (term-down 1 t) (term-move-columns (- (term-current-column))) (add-text-properties (1- (point)) (point) '(term-line-wrap t rear-nonsticky t)) (setq decoded-substring (substring decoded-substring (- term-width old-column))) (setq old-column 0))) (insert decoded-substring) (setq term-current-column (current-column) columns (- term-current-column old-column)) (if (not (or (eobp) term-insert-mode)) (progn (let ((pos (point))) (term-move-columns columns) (delete-region pos (point)) (setq term-current-column nil)))) (if term-insert-mode (progn (let ((pos (point))) (end-of-line) (if (> (current-column) term-width) (progn (delete-region (- (point) (- (current-column) term-width)) (point)))) (goto-char pos)))) (put-text-property old-point (point) 'font-lock-face term-current-face)) (if (eq (term-current-column) term-width) (progn (term-move-columns -1) (setq term-do-line-wrapping (point)))) (setq term-current-column nil) (setq i funny))
  (if (> funny i) (progn (if term-do-line-wrapping (progn (term-down 1 t) (term-move-to-column 0) (setq term-do-line-wrapping nil))) (setq decoded-substring (decode-coding-string (substring str i funny) locale-coding-system t)) (if (= funny str-length) (progn (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial))))))) (let ((old-column (term-horizontal-column)) (old-point (point)) columns) (if term-suppress-hard-newline nil (while (> (+ (length decoded-substring) old-column) term-width) (insert (substring decoded-substring 0 (- term-width old-column))) (delete-region (point) (line-end-position)) (term-down 1 t) (term-move-columns (- (term-current-column))) (add-text-properties (1- (point)) (point) '(term-line-wrap t rear-nonsticky t)) (setq decoded-substring (substring decoded-substring (- term-width old-column))) (setq old-column 0))) (insert decoded-substring) (setq term-current-column (current-column) columns (- term-current-column old-column)) (if (not (or (eobp) term-insert-mode)) (progn (let ((pos (point))) (term-move-columns columns) (delete-region pos (point)) (setq term-current-column nil)))) (if term-insert-mode (progn (let ((pos (point))) (end-of-line) (if (> (current-column) term-width) (progn (delete-region (- (point) (- (current-column) term-width)) (point)))) (goto-char pos)))) (put-text-property old-point (point) 'font-lock-face term-current-face)) (if (eq (term-current-column) term-width) (progn (term-move-columns -1) (setq term-do-line-wrapping (point)))) (setq term-current-column nil) (setq i funny)))
  (let ((ctl-params (and funny (match-string 1 str))) (ctl-params-end (and funny (match-end 1))) (ctl-end (if funny (match-end 0) (setq funny (string-match term-control-seq-prefix-regexp str i)) (if funny (setq term-terminal-undecoded-bytes (substring str funny)) (setq funny str-length)) (1+ str-length)))) (if (> funny i) (progn (if term-do-line-wrapping (progn (term-down 1 t) (term-move-to-column 0) (setq term-do-line-wrapping nil))) (setq decoded-substring (decode-coding-string (substring str i funny) locale-coding-system t)) (if (= funny str-length) (progn (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial))))))) (let ((old-column (term-horizontal-column)) (old-point (point)) columns) (if term-suppress-hard-newline nil (while (> (+ (length decoded-substring) old-column) term-width) (insert (substring decoded-substring 0 (- term-width old-column))) (delete-region (point) (line-end-position)) (term-down 1 t) (term-move-columns (- (term-current-column))) (add-text-properties (1- (point)) (point) '(term-line-wrap t rear-nonsticky t)) (setq decoded-substring (substring decoded-substring (- term-width old-column))) (setq old-column 0))) (insert decoded-substring) (setq term-current-column (current-column) columns (- term-current-column old-column)) (if (not (or (eobp) term-insert-mode)) (progn (let ((pos (point))) (term-move-columns columns) (delete-region pos (point)) (setq term-current-column nil)))) (if term-insert-mode (progn (let ((pos (point))) (end-of-line) (if (> (current-column) term-width) (progn (delete-region (- (point) (- (current-column) term-width)) (point)))) (goto-char pos)))) (put-text-property old-point (point) 'font-lock-face term-current-face)) (if (eq (term-current-column) term-width) (progn (term-move-columns -1) (setq term-do-line-wrapping (point)))) (setq term-current-column nil) (setq i funny))) (let* ((val (and (<= ctl-end str-length) (aref str i)))) (cond ((eql val 9) (let ((col (term-current-column))) (term-move-to-column (min (1- term-width) (+ col 8 (- (mod col 8))))))) ((eql val 13) (funcall term-vertical-motion 0) (setq term-current-column term-start-line-column)) ((eql val 10) (if (and term-kill-echo-list (term-check-kill-echo-list)) nil (term-down 1 t))) ((eql val 8) (term-move-columns -1)) ((eql val 7) (beep t)) ((eql val 26) (funcall term-command-function (decode-coding-string (substring str (1+ i) (- ctl-end (if (eq (aref str (- ctl-end 2)) 13) 2 1))) locale-coding-system t))) ((eql val 27) (let* ((val (aref str (1+ i)))) (cond ((eql val 91) (if (eq ctl-params-end (1- ctl-end)) (progn (term-handle-ansi-escape proc (mapcar #'string-to-number (split-string ctl-params ";")) (aref str (1- ctl-end)))))) ((eql val 68) (term-handle-deferred-scroll) (term-down 1 t)) ((eql val 77) (if (or (< (term-current-row) term-scroll-start) (>= (1- (term-current-row)) term-scroll-start)) (term-down -1) (term-down -1 t))) ((eql val 55) (term-handle-deferred-scroll) (setq term-saved-cursor (list (term-current-row) (term-horizontal-column) term-ansi-current-bg-color term-ansi-current-bold term-ansi-current-color term-ansi-current-invisible term-ansi-current-reverse term-ansi-current-underline term-current-face))) ((eql val 56) (if term-saved-cursor (progn (term-goto (nth 0 term-saved-cursor) (nth 1 term-saved-cursor)) (setq term-ansi-current-bg-color (nth 2 term-saved-cursor) term-ansi-current-bold (nth 3 term-saved-cursor) term-ansi-current-color (nth 4 term-saved-cursor) term-ansi-current-invisible (nth 5 term-saved-cursor) term-ansi-current-reverse (nth 6 term-saved-cursor) term-ansi-current-underline (nth 7 term-saved-cursor) term-current-face (nth 8 term-saved-cursor))))) ((eql val 99) (term-reset-terminal)) ((eql val 65) (term-handle-ansi-terminal-messages (substring str i ctl-end))) (t nil)))) ((memql val '(nil 15 14 0)) nil) (t (let ((x37 val)) (error "No clause matching `%S'" x37))))) (if (eq term-do-line-wrapping (point)) nil (setq term-do-line-wrapping nil)) (if term-pager-old-local-map (progn (if (> (% (current-column) term-width) 0) (setq term-terminal-undecoded-bytes (substring str i)) (if (= 0 i) (setq term-terminal-undecoded-bytes (concat "\15" (substring str i))) (setq term-terminal-undecoded-bytes (substring str (1- i))) (aset term-terminal-undecoded-bytes 0 13)) (goto-char (point-max))) (make-local-variable 'term-pager-old-filter) (setq term-pager-old-filter (process-filter proc)) (set-process-filter proc term-pager-filter) (setq i str-length)) (setq i ctl-end)))
  (while (< i str-length) (setq funny (string-match term-control-seq-regexp str i)) (let ((ctl-params (and funny (match-string 1 str))) (ctl-params-end (and funny (match-end 1))) (ctl-end (if funny (match-end 0) (setq funny (string-match term-control-seq-prefix-regexp str i)) (if funny (setq term-terminal-undecoded-bytes (substring str funny)) (setq funny str-length)) (1+ str-length)))) (if (> funny i) (progn (if term-do-line-wrapping (progn (term-down 1 t) (term-move-to-column 0) (setq term-do-line-wrapping nil))) (setq decoded-substring (decode-coding-string (substring str i funny) locale-coding-system t)) (if (= funny str-length) (progn (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial))))))) (let ((old-column (term-horizontal-column)) (old-point (point)) columns) (if term-suppress-hard-newline nil (while (> (+ (length decoded-substring) old-column) term-width) (insert (substring decoded-substring 0 (- term-width old-column))) (delete-region (point) (line-end-position)) (term-down 1 t) (term-move-columns (- (term-current-column))) (add-text-properties (1- (point)) (point) '(term-line-wrap t rear-nonsticky t)) (setq decoded-substring (substring decoded-substring (- term-width old-column))) (setq old-column 0))) (insert decoded-substring) (setq term-current-column (current-column) columns (- term-current-column old-column)) (if (not (or (eobp) term-insert-mode)) (progn (let ((pos (point))) (term-move-columns columns) (delete-region pos (point)) (setq term-current-column nil)))) (if term-insert-mode (progn (let ((pos (point))) (end-of-line) (if (> (current-column) term-width) (progn (delete-region (- (point) (- (current-column) term-width)) (point)))) (goto-char pos)))) (put-text-property old-point (point) 'font-lock-face term-current-face)) (if (eq (term-current-column) term-width) (progn (term-move-columns -1) (setq term-do-line-wrapping (point)))) (setq term-current-column nil) (setq i funny))) (let* ((val (and (<= ctl-end str-length) (aref str i)))) (cond ((eql val 9) (let ((col (term-current-column))) (term-move-to-column (min (1- term-width) (+ col 8 (- (mod col 8))))))) ((eql val 13) (funcall term-vertical-motion 0) (setq term-current-column term-start-line-column)) ((eql val 10) (if (and term-kill-echo-list (term-check-kill-echo-list)) nil (term-down 1 t))) ((eql val 8) (term-move-columns -1)) ((eql val 7) (beep t)) ((eql val 26) (funcall term-command-function (decode-coding-string (substring str (1+ i) (- ctl-end (if (eq (aref str (- ctl-end 2)) 13) 2 1))) locale-coding-system t))) ((eql val 27) (let* ((val (aref str (1+ i)))) (cond ((eql val 91) (if (eq ctl-params-end (1- ctl-end)) (progn (term-handle-ansi-escape proc (mapcar #'string-to-number (split-string ctl-params ";")) (aref str (1- ctl-end)))))) ((eql val 68) (term-handle-deferred-scroll) (term-down 1 t)) ((eql val 77) (if (or (< (term-current-row) term-scroll-start) (>= (1- (term-current-row)) term-scroll-start)) (term-down -1) (term-down -1 t))) ((eql val 55) (term-handle-deferred-scroll) (setq term-saved-cursor (list (term-current-row) (term-horizontal-column) term-ansi-current-bg-color term-ansi-current-bold term-ansi-current-color term-ansi-current-invisible term-ansi-current-reverse term-ansi-current-underline term-current-face))) ((eql val 56) (if term-saved-cursor (progn (term-goto (nth 0 term-saved-cursor) (nth 1 term-saved-cursor)) (setq term-ansi-current-bg-color (nth 2 term-saved-cursor) term-ansi-current-bold (nth 3 term-saved-cursor) term-ansi-current-color (nth 4 term-saved-cursor) term-ansi-current-invisible (nth 5 term-saved-cursor) term-ansi-current-reverse (nth 6 term-saved-cursor) term-ansi-current-underline (nth 7 term-saved-cursor) term-current-face (nth 8 term-saved-cursor))))) ((eql val 99) (term-reset-terminal)) ((eql val 65) (term-handle-ansi-terminal-messages (substring str i ctl-end))) (t nil)))) ((memql val '(nil 15 14 0)) nil) (t (let ((x37 val)) (error "No clause matching `%S'" x37))))) (if (eq term-do-line-wrapping (point)) nil (setq term-do-line-wrapping nil)) (if term-pager-old-local-map (progn (if (> (% (current-column) term-width) 0) (setq term-terminal-undecoded-bytes (substring str i)) (if (= 0 i) (setq term-terminal-undecoded-bytes (concat "\15" (substring str i))) (setq term-terminal-undecoded-bytes (substring str (1- i))) (aset term-terminal-undecoded-bytes 0 13)) (goto-char (point-max))) (make-local-variable 'term-pager-old-filter) (setq term-pager-old-filter (process-filter proc)) (set-process-filter proc term-pager-filter) (setq i str-length)) (setq i ctl-end))))
  (save-restriction (if (and (> (point-max) (process-mark proc)) (not (eq (current-local-map) term-raw-map))) (progn (narrow-to-region (point-min) (process-mark proc)))) (if term-log-buffer (progn (princ str term-log-buffer))) (if term-terminal-undecoded-bytes (progn (setq str (concat term-terminal-undecoded-bytes str)) (setq str-length (length str)) (setq term-terminal-undecoded-bytes nil))) (while (< i str-length) (setq funny (string-match term-control-seq-regexp str i)) (let ((ctl-params (and funny (match-string 1 str))) (ctl-params-end (and funny (match-end 1))) (ctl-end (if funny (match-end 0) (setq funny (string-match term-control-seq-prefix-regexp str i)) (if funny (setq term-terminal-undecoded-bytes (substring str funny)) (setq funny str-length)) (1+ str-length)))) (if (> funny i) (progn (if term-do-line-wrapping (progn (term-down 1 t) (term-move-to-column 0) (setq term-do-line-wrapping nil))) (setq decoded-substring (decode-coding-string (substring str i funny) locale-coding-system t)) (if (= funny str-length) (progn (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial))))))) (let ((old-column (term-horizontal-column)) (old-point (point)) columns) (if term-suppress-hard-newline nil (while (> (+ (length decoded-substring) old-column) term-width) (insert (substring decoded-substring 0 (- term-width old-column))) (delete-region (point) (line-end-position)) (term-down 1 t) (term-move-columns (- (term-current-column))) (add-text-properties (1- (point)) (point) '(term-line-wrap t rear-nonsticky t)) (setq decoded-substring (substring decoded-substring (- term-width old-column))) (setq old-column 0))) (insert decoded-substring) (setq term-current-column (current-column) columns (- term-current-column old-column)) (if (not (or (eobp) term-insert-mode)) (progn (let ((pos (point))) (term-move-columns columns) (delete-region pos (point)) (setq term-current-column nil)))) (if term-insert-mode (progn (let ((pos (point))) (end-of-line) (if (> (current-column) term-width) (progn (delete-region (- (point) (- (current-column) term-width)) (point)))) (goto-char pos)))) (put-text-property old-point (point) 'font-lock-face term-current-face)) (if (eq (term-current-column) term-width) (progn (term-move-columns -1) (setq term-do-line-wrapping (point)))) (setq term-current-column nil) (setq i funny))) (let* ((val (and (<= ctl-end str-length) (aref str i)))) (cond ((eql val 9) (let ((col (term-current-column))) (term-move-to-column (min (1- term-width) (+ col 8 (- (mod col 8))))))) ((eql val 13) (funcall term-vertical-motion 0) (setq term-current-column term-start-line-column)) ((eql val 10) (if (and term-kill-echo-list (term-check-kill-echo-list)) nil (term-down 1 t))) ((eql val 8) (term-move-columns -1)) ((eql val 7) (beep t)) ((eql val 26) (funcall term-command-function (decode-coding-string (substring str (1+ i) (- ctl-end (if (eq (aref str (- ctl-end 2)) 13) 2 1))) locale-coding-system t))) ((eql val 27) (let* ((val (aref str (1+ i)))) (cond ((eql val 91) (if (eq ctl-params-end (1- ctl-end)) (progn (term-handle-ansi-escape proc (mapcar #'string-to-number (split-string ctl-params ";")) (aref str (1- ctl-end)))))) ((eql val 68) (term-handle-deferred-scroll) (term-down 1 t)) ((eql val 77) (if (or (< (term-current-row) term-scroll-start) (>= (1- (term-current-row)) term-scroll-start)) (term-down -1) (term-down -1 t))) ((eql val 55) (term-handle-deferred-scroll) (setq term-saved-cursor (list (term-current-row) (term-horizontal-column) term-ansi-current-bg-color term-ansi-current-bold term-ansi-current-color term-ansi-current-invisible term-ansi-current-reverse term-ansi-current-underline term-current-face))) ((eql val 56) (if term-saved-cursor (progn (term-goto (nth 0 term-saved-cursor) (nth 1 term-saved-cursor)) (setq term-ansi-current-bg-color (nth 2 term-saved-cursor) term-ansi-current-bold (nth 3 term-saved-cursor) term-ansi-current-color (nth 4 term-saved-cursor) term-ansi-current-invisible (nth 5 term-saved-cursor) term-ansi-current-reverse (nth 6 term-saved-cursor) term-ansi-current-underline (nth 7 term-saved-cursor) term-current-face (nth 8 term-saved-cursor))))) ((eql val 99) (term-reset-terminal)) ((eql val 65) (term-handle-ansi-terminal-messages (substring str i ctl-end))) (t nil)))) ((memql val '(nil 15 14 0)) nil) (t (let ((x37 val)) (error "No clause matching `%S'" x37))))) (if (eq term-do-line-wrapping (point)) nil (setq term-do-line-wrapping nil)) (if term-pager-old-local-map (progn (if (> (% (current-column) term-width) 0) (setq term-terminal-undecoded-bytes (substring str i)) (if (= 0 i) (setq term-terminal-undecoded-bytes (concat "\15" (substring str i))) (setq term-terminal-undecoded-bytes (substring str (1- i))) (aset term-terminal-undecoded-bytes 0 13)) (goto-char (point-max))) (make-local-variable 'term-pager-old-filter) (setq term-pager-old-filter (process-filter proc)) (set-process-filter proc term-pager-filter) (setq i str-length)) (setq i ctl-end)))))
  (progn (if (marker-buffer term-pending-delete-marker) (progn (delete-region term-pending-delete-marker (process-mark proc)) (set-marker term-pending-delete-marker nil))) (if (/= (point) (process-mark proc)) (progn (setq save-point (point-marker)))) (setq term-vertical-motion (if (eq (window-buffer) (current-buffer)) 'vertical-motion 'term-buffer-vertical-motion)) (setq save-marker (copy-marker (process-mark proc))) (goto-char (process-mark proc)) (save-restriction (if (and (> (point-max) (process-mark proc)) (not (eq (current-local-map) term-raw-map))) (progn (narrow-to-region (point-min) (process-mark proc)))) (if term-log-buffer (progn (princ str term-log-buffer))) (if term-terminal-undecoded-bytes (progn (setq str (concat term-terminal-undecoded-bytes str)) (setq str-length (length str)) (setq term-terminal-undecoded-bytes nil))) (while (< i str-length) (setq funny (string-match term-control-seq-regexp str i)) (let ((ctl-params (and funny (match-string 1 str))) (ctl-params-end (and funny (match-end 1))) (ctl-end (if funny (match-end 0) (setq funny (string-match term-control-seq-prefix-regexp str i)) (if funny (setq term-terminal-undecoded-bytes (substring str funny)) (setq funny str-length)) (1+ str-length)))) (if (> funny i) (progn (if term-do-line-wrapping (progn (term-down 1 t) (term-move-to-column 0) (setq term-do-line-wrapping nil))) (setq decoded-substring (decode-coding-string (substring str i funny) locale-coding-system t)) (if (= funny str-length) (progn (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial))))))) (let ((old-column (term-horizontal-column)) (old-point (point)) columns) (if term-suppress-hard-newline nil (while (> (+ (length decoded-substring) old-column) term-width) (insert (substring decoded-substring 0 (- term-width old-column))) (delete-region (point) (line-end-position)) (term-down 1 t) (term-move-columns (- (term-current-column))) (add-text-properties (1- (point)) (point) '(term-line-wrap t rear-nonsticky t)) (setq decoded-substring (substring decoded-substring (- term-width old-column))) (setq old-column 0))) (insert decoded-substring) (setq term-current-column (current-column) columns (- term-current-column old-column)) (if (not (or (eobp) term-insert-mode)) (progn (let ((pos (point))) (term-move-columns columns) (delete-region pos (point)) (setq term-current-column nil)))) (if term-insert-mode (progn (let ((pos (point))) (end-of-line) (if (> (current-column) term-width) (progn (delete-region (- (point) (- (current-column) term-width)) (point)))) (goto-char pos)))) (put-text-property old-point (point) 'font-lock-face term-current-face)) (if (eq (term-current-column) term-width) (progn (term-move-columns -1) (setq term-do-line-wrapping (point)))) (setq term-current-column nil) (setq i funny))) (let* ((val (and (<= ctl-end str-length) (aref str i)))) (cond ((eql val 9) (let ((col (term-current-column))) (term-move-to-column (min (1- term-width) (+ col 8 (- (mod col 8))))))) ((eql val 13) (funcall term-vertical-motion 0) (setq term-current-column term-start-line-column)) ((eql val 10) (if (and term-kill-echo-list (term-check-kill-echo-list)) nil (term-down 1 t))) ((eql val 8) (term-move-columns -1)) ((eql val 7) (beep t)) ((eql val 26) (funcall term-command-function (decode-coding-string (substring str (1+ i) (- ctl-end (if (eq (aref str (- ctl-end 2)) 13) 2 1))) locale-coding-system t))) ((eql val 27) (let* ((val (aref str (1+ i)))) (cond ((eql val 91) (if (eq ctl-params-end (1- ctl-end)) (progn (term-handle-ansi-escape proc (mapcar #'string-to-number (split-string ctl-params ";")) (aref str (1- ctl-end)))))) ((eql val 68) (term-handle-deferred-scroll) (term-down 1 t)) ((eql val 77) (if (or (< (term-current-row) term-scroll-start) (>= (1- (term-current-row)) term-scroll-start)) (term-down -1) (term-down -1 t))) ((eql val 55) (term-handle-deferred-scroll) (setq term-saved-cursor (list (term-current-row) (term-horizontal-column) term-ansi-current-bg-color term-ansi-current-bold term-ansi-current-color term-ansi-current-invisible term-ansi-current-reverse term-ansi-current-underline term-current-face))) ((eql val 56) (if term-saved-cursor (progn (term-goto (nth 0 term-saved-cursor) (nth 1 term-saved-cursor)) (setq term-ansi-current-bg-color (nth 2 term-saved-cursor) term-ansi-current-bold (nth 3 term-saved-cursor) term-ansi-current-color (nth 4 term-saved-cursor) term-ansi-current-invisible (nth 5 term-saved-cursor) term-ansi-current-reverse (nth 6 term-saved-cursor) term-ansi-current-underline (nth 7 term-saved-cursor) term-current-face (nth 8 term-saved-cursor))))) ((eql val 99) (term-reset-terminal)) ((eql val 65) (term-handle-ansi-terminal-messages (substring str i ctl-end))) (t nil)))) ((memql val '(nil 15 14 0)) nil) (t (let ((x37 val)) (error "No clause matching `%S'" x37))))) (if (eq term-do-line-wrapping (point)) nil (setq term-do-line-wrapping nil)) (if term-pager-old-local-map (progn (if (> (% (current-column) term-width) 0) (setq term-terminal-undecoded-bytes (substring str i)) (if (= 0 i) (setq term-terminal-undecoded-bytes (concat "\15" (substring str i))) (setq term-terminal-undecoded-bytes (substring str (1- i))) (aset term-terminal-undecoded-bytes 0 13)) (goto-char (point-max))) (make-local-variable 'term-pager-old-filter) (setq term-pager-old-filter (process-filter proc)) (set-process-filter proc term-pager-filter) (setq i str-length)) (setq i ctl-end))))) (if (>= (term-current-row) term-height) (progn (term-handle-deferred-scroll))) (set-marker (process-mark proc) (point)) (if (stringp decoded-substring) (progn (term-watch-for-password-prompt decoded-substring))) (if save-point (progn (goto-char save-point) (set-marker save-point nil))) (if (and term-pending-frame (eq (window-buffer selected) (current-buffer))) (progn (term-display-line (car term-pending-frame) (cdr term-pending-frame)) (setq term-pending-frame nil))) (setq win selected) (while (window-minibuffer-p win) (setq win (next-window win nil t))) (setq last-win win) (while (progn (setq win (next-window win nil t)) (if (eq (window-buffer win) (process-buffer proc)) (progn (let ((scroll term-scroll-to-bottom-on-output)) (select-window win) (if (or (= (point) save-marker) (eq scroll t) (eq scroll 'all) (and (eq selected win) (or (eq scroll 'this) (not save-point))) (and (eq scroll 'others) (not (eq selected win)))) (progn (goto-char term-home-marker) (recenter 0) (goto-char (process-mark proc)) (if (not (pos-visible-in-window-p (point) win)) (recenter -1)))) (if (and term-scroll-show-maximum-output (>= (point) (process-mark proc))) (progn (save-excursion (goto-char (point-max)) (recenter -1))))))) (not (eq win last-win)))) (if (> term-buffer-maximum-size 0) (progn (save-excursion (goto-char (process-mark (get-buffer-process (current-buffer)))) (forward-line (- term-buffer-maximum-size)) (beginning-of-line) (delete-region (point-min) (point))))) (set-marker save-marker nil))
  (unwind-protect (progn (if (marker-buffer term-pending-delete-marker) (progn (delete-region term-pending-delete-marker (process-mark proc)) (set-marker term-pending-delete-marker nil))) (if (/= (point) (process-mark proc)) (progn (setq save-point (point-marker)))) (setq term-vertical-motion (if (eq (window-buffer) (current-buffer)) 'vertical-motion 'term-buffer-vertical-motion)) (setq save-marker (copy-marker (process-mark proc))) (goto-char (process-mark proc)) (save-restriction (if (and (> (point-max) (process-mark proc)) (not (eq (current-local-map) term-raw-map))) (progn (narrow-to-region (point-min) (process-mark proc)))) (if term-log-buffer (progn (princ str term-log-buffer))) (if term-terminal-undecoded-bytes (progn (setq str (concat term-terminal-undecoded-bytes str)) (setq str-length (length str)) (setq term-terminal-undecoded-bytes nil))) (while (< i str-length) (setq funny (string-match term-control-seq-regexp str i)) (let ((ctl-params (and funny (match-string 1 str))) (ctl-params-end (and funny (match-end 1))) (ctl-end (if funny (match-end 0) (setq funny (string-match term-control-seq-prefix-regexp str i)) (if funny (setq term-terminal-undecoded-bytes (substring str funny)) (setq funny str-length)) (1+ str-length)))) (if (> funny i) (progn (if term-do-line-wrapping (progn (term-down 1 t) (term-move-to-column 0) (setq term-do-line-wrapping nil))) (setq decoded-substring (decode-coding-string (substring str i funny) locale-coding-system t)) (if (= funny str-length) (progn (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial))))))) (let ((old-column (term-horizontal-column)) (old-point (point)) columns) (if term-suppress-hard-newline nil (while (> (+ (length decoded-substring) old-column) term-width) (insert (substring decoded-substring 0 (- term-width old-column))) (delete-region (point) (line-end-position)) (term-down 1 t) (term-move-columns (- (term-current-column))) (add-text-properties (1- (point)) (point) '(term-line-wrap t rear-nonsticky t)) (setq decoded-substring (substring decoded-substring (- term-width old-column))) (setq old-column 0))) (insert decoded-substring) (setq term-current-column (current-column) columns (- term-current-column old-column)) (if (not (or (eobp) term-insert-mode)) (progn (let ((pos (point))) (term-move-columns columns) (delete-region pos (point)) (setq term-current-column nil)))) (if term-insert-mode (progn (let ((pos (point))) (end-of-line) (if (> (current-column) term-width) (progn (delete-region (- (point) (- (current-column) term-width)) (point)))) (goto-char pos)))) (put-text-property old-point (point) 'font-lock-face term-current-face)) (if (eq (term-current-column) term-width) (progn (term-move-columns -1) (setq term-do-line-wrapping (point)))) (setq term-current-column nil) (setq i funny))) (let* ((val (and (<= ctl-end str-length) (aref str i)))) (cond ((eql val 9) (let ((col (term-current-column))) (term-move-to-column (min (1- term-width) (+ col 8 (- (mod col 8))))))) ((eql val 13) (funcall term-vertical-motion 0) (setq term-current-column term-start-line-column)) ((eql val 10) (if (and term-kill-echo-list (term-check-kill-echo-list)) nil (term-down 1 t))) ((eql val 8) (term-move-columns -1)) ((eql val 7) (beep t)) ((eql val 26) (funcall term-command-function (decode-coding-string (substring str (1+ i) (- ctl-end (if (eq (aref str (- ctl-end 2)) 13) 2 1))) locale-coding-system t))) ((eql val 27) (let* ((val (aref str (1+ i)))) (cond ((eql val 91) (if (eq ctl-params-end (1- ctl-end)) (progn (term-handle-ansi-escape proc (mapcar #'string-to-number (split-string ctl-params ";")) (aref str (1- ctl-end)))))) ((eql val 68) (term-handle-deferred-scroll) (term-down 1 t)) ((eql val 77) (if (or (< (term-current-row) term-scroll-start) (>= (1- (term-current-row)) term-scroll-start)) (term-down -1) (term-down -1 t))) ((eql val 55) (term-handle-deferred-scroll) (setq term-saved-cursor (list (term-current-row) (term-horizontal-column) term-ansi-current-bg-color term-ansi-current-bold term-ansi-current-color term-ansi-current-invisible term-ansi-current-reverse term-ansi-current-underline term-current-face))) ((eql val 56) (if term-saved-cursor (progn (term-goto (nth 0 term-saved-cursor) (nth 1 term-saved-cursor)) (setq term-ansi-current-bg-color (nth 2 term-saved-cursor) term-ansi-current-bold (nth 3 term-saved-cursor) term-ansi-current-color (nth 4 term-saved-cursor) term-ansi-current-invisible (nth 5 term-saved-cursor) term-ansi-current-reverse (nth 6 term-saved-cursor) term-ansi-current-underline (nth 7 term-saved-cursor) term-current-face (nth 8 term-saved-cursor))))) ((eql val 99) (term-reset-terminal)) ((eql val 65) (term-handle-ansi-terminal-messages (substring str i ctl-end))) (t nil)))) ((memql val '(nil 15 14 0)) nil) (t (let ((x37 val)) (error "No clause matching `%S'" x37))))) (if (eq term-do-line-wrapping (point)) nil (setq term-do-line-wrapping nil)) (if term-pager-old-local-map (progn (if (> (% (current-column) term-width) 0) (setq term-terminal-undecoded-bytes (substring str i)) (if (= 0 i) (setq term-terminal-undecoded-bytes (concat "\15" (substring str i))) (setq term-terminal-undecoded-bytes (substring str (1- i))) (aset term-terminal-undecoded-bytes 0 13)) (goto-char (point-max))) (make-local-variable 'term-pager-old-filter) (setq term-pager-old-filter (process-filter proc)) (set-process-filter proc term-pager-filter) (setq i str-length)) (setq i ctl-end))))) (if (>= (term-current-row) term-height) (progn (term-handle-deferred-scroll))) (set-marker (process-mark proc) (point)) (if (stringp decoded-substring) (progn (term-watch-for-password-prompt decoded-substring))) (if save-point (progn (goto-char save-point) (set-marker save-point nil))) (if (and term-pending-frame (eq (window-buffer selected) (current-buffer))) (progn (term-display-line (car term-pending-frame) (cdr term-pending-frame)) (setq term-pending-frame nil))) (setq win selected) (while (window-minibuffer-p win) (setq win (next-window win nil t))) (setq last-win win) (while (progn (setq win (next-window win nil t)) (if (eq (window-buffer win) (process-buffer proc)) (progn (let ((scroll term-scroll-to-bottom-on-output)) (select-window win) (if (or (= (point) save-marker) (eq scroll t) (eq scroll 'all) (and (eq selected win) (or (eq scroll 'this) (not save-point))) (and (eq scroll 'others) (not (eq selected win)))) (progn (goto-char term-home-marker) (recenter 0) (goto-char (process-mark proc)) (if (not (pos-visible-in-window-p (point) win)) (recenter -1)))) (if (and term-scroll-show-maximum-output (>= (point) (process-mark proc))) (progn (save-excursion (goto-char (point-max)) (recenter -1))))))) (not (eq win last-win)))) (if (> term-buffer-maximum-size 0) (progn (save-excursion (goto-char (process-mark (get-buffer-process (current-buffer)))) (forward-line (- term-buffer-maximum-size)) (beginning-of-line) (delete-region (point-min) (point))))) (set-marker save-marker nil)) (internal--after-save-selected-window save-selected-window--state))
  (save-current-buffer (unwind-protect (progn (if (marker-buffer term-pending-delete-marker) (progn (delete-region term-pending-delete-marker (process-mark proc)) (set-marker term-pending-delete-marker nil))) (if (/= (point) (process-mark proc)) (progn (setq save-point (point-marker)))) (setq term-vertical-motion (if (eq (window-buffer) (current-buffer)) 'vertical-motion 'term-buffer-vertical-motion)) (setq save-marker (copy-marker (process-mark proc))) (goto-char (process-mark proc)) (save-restriction (if (and (> (point-max) (process-mark proc)) (not (eq (current-local-map) term-raw-map))) (progn (narrow-to-region (point-min) (process-mark proc)))) (if term-log-buffer (progn (princ str term-log-buffer))) (if term-terminal-undecoded-bytes (progn (setq str (concat term-terminal-undecoded-bytes str)) (setq str-length (length str)) (setq term-terminal-undecoded-bytes nil))) (while (< i str-length) (setq funny (string-match term-control-seq-regexp str i)) (let ((ctl-params (and funny (match-string 1 str))) (ctl-params-end (and funny (match-end 1))) (ctl-end (if funny (match-end 0) (setq funny (string-match term-control-seq-prefix-regexp str i)) (if funny (setq term-terminal-undecoded-bytes (substring str funny)) (setq funny str-length)) (1+ str-length)))) (if (> funny i) (progn (if term-do-line-wrapping (progn (term-down 1 t) (term-move-to-column 0) (setq term-do-line-wrapping nil))) (setq decoded-substring (decode-coding-string (substring str i funny) locale-coding-system t)) (if (= funny str-length) (progn (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial))))))) (let ((old-column (term-horizontal-column)) (old-point (point)) columns) (if term-suppress-hard-newline nil (while (> (+ (length decoded-substring) old-column) term-width) (insert (substring decoded-substring 0 (- term-width old-column))) (delete-region (point) (line-end-position)) (term-down 1 t) (term-move-columns (- (term-current-column))) (add-text-properties (1- (point)) (point) '(term-line-wrap t rear-nonsticky t)) (setq decoded-substring (substring decoded-substring (- term-width old-column))) (setq old-column 0))) (insert decoded-substring) (setq term-current-column (current-column) columns (- term-current-column old-column)) (if (not (or (eobp) term-insert-mode)) (progn (let ((pos (point))) (term-move-columns columns) (delete-region pos (point)) (setq term-current-column nil)))) (if term-insert-mode (progn (let ((pos (point))) (end-of-line) (if (> (current-column) term-width) (progn (delete-region (- (point) (- (current-column) term-width)) (point)))) (goto-char pos)))) (put-text-property old-point (point) 'font-lock-face term-current-face)) (if (eq (term-current-column) term-width) (progn (term-move-columns -1) (setq term-do-line-wrapping (point)))) (setq term-current-column nil) (setq i funny))) (let* ((val (and (<= ctl-end str-length) (aref str i)))) (cond ((eql val 9) (let ((col (term-current-column))) (term-move-to-column (min (1- term-width) (+ col 8 (- (mod col 8))))))) ((eql val 13) (funcall term-vertical-motion 0) (setq term-current-column term-start-line-column)) ((eql val 10) (if (and term-kill-echo-list (term-check-kill-echo-list)) nil (term-down 1 t))) ((eql val 8) (term-move-columns -1)) ((eql val 7) (beep t)) ((eql val 26) (funcall term-command-function (decode-coding-string (substring str (1+ i) (- ctl-end (if (eq (aref str (- ctl-end 2)) 13) 2 1))) locale-coding-system t))) ((eql val 27) (let* ((val (aref str (1+ i)))) (cond ((eql val 91) (if (eq ctl-params-end (1- ctl-end)) (progn (term-handle-ansi-escape proc (mapcar #'string-to-number (split-string ctl-params ";")) (aref str (1- ctl-end)))))) ((eql val 68) (term-handle-deferred-scroll) (term-down 1 t)) ((eql val 77) (if (or (< (term-current-row) term-scroll-start) (>= (1- (term-current-row)) term-scroll-start)) (term-down -1) (term-down -1 t))) ((eql val 55) (term-handle-deferred-scroll) (setq term-saved-cursor (list (term-current-row) (term-horizontal-column) term-ansi-current-bg-color term-ansi-current-bold term-ansi-current-color term-ansi-current-invisible term-ansi-current-reverse term-ansi-current-underline term-current-face))) ((eql val 56) (if term-saved-cursor (progn (term-goto (nth 0 term-saved-cursor) (nth 1 term-saved-cursor)) (setq term-ansi-current-bg-color (nth 2 term-saved-cursor) term-ansi-current-bold (nth 3 term-saved-cursor) term-ansi-current-color (nth 4 term-saved-cursor) term-ansi-current-invisible (nth 5 term-saved-cursor) term-ansi-current-reverse (nth 6 term-saved-cursor) term-ansi-current-underline (nth 7 term-saved-cursor) term-current-face (nth 8 term-saved-cursor))))) ((eql val 99) (term-reset-terminal)) ((eql val 65) (term-handle-ansi-terminal-messages (substring str i ctl-end))) (t nil)))) ((memql val '(nil 15 14 0)) nil) (t (let ((x37 val)) (error "No clause matching `%S'" x37))))) (if (eq term-do-line-wrapping (point)) nil (setq term-do-line-wrapping nil)) (if term-pager-old-local-map (progn (if (> (% (current-column) term-width) 0) (setq term-terminal-undecoded-bytes (substring str i)) (if (= 0 i) (setq term-terminal-undecoded-bytes (concat "\15" (substring str i))) (setq term-terminal-undecoded-bytes (substring str (1- i))) (aset term-terminal-undecoded-bytes 0 13)) (goto-char (point-max))) (make-local-variable 'term-pager-old-filter) (setq term-pager-old-filter (process-filter proc)) (set-process-filter proc term-pager-filter) (setq i str-length)) (setq i ctl-end))))) (if (>= (term-current-row) term-height) (progn (term-handle-deferred-scroll))) (set-marker (process-mark proc) (point)) (if (stringp decoded-substring) (progn (term-watch-for-password-prompt decoded-substring))) (if save-point (progn (goto-char save-point) (set-marker save-point nil))) (if (and term-pending-frame (eq (window-buffer selected) (current-buffer))) (progn (term-display-line (car term-pending-frame) (cdr term-pending-frame)) (setq term-pending-frame nil))) (setq win selected) (while (window-minibuffer-p win) (setq win (next-window win nil t))) (setq last-win win) (while (progn (setq win (next-window win nil t)) (if (eq (window-buffer win) (process-buffer proc)) (progn (let ((scroll term-scroll-to-bottom-on-output)) (select-window win) (if (or (= (point) save-marker) (eq scroll t) (eq scroll 'all) (and (eq selected win) (or (eq scroll 'this) (not save-point))) (and (eq scroll 'others) (not (eq selected win)))) (progn (goto-char term-home-marker) (recenter 0) (goto-char (process-mark proc)) (if (not (pos-visible-in-window-p (point) win)) (recenter -1)))) (if (and term-scroll-show-maximum-output (>= (point) (process-mark proc))) (progn (save-excursion (goto-char (point-max)) (recenter -1))))))) (not (eq win last-win)))) (if (> term-buffer-maximum-size 0) (progn (save-excursion (goto-char (process-mark (get-buffer-process (current-buffer)))) (forward-line (- term-buffer-maximum-size)) (beginning-of-line) (delete-region (point-min) (point))))) (set-marker save-marker nil)) (internal--after-save-selected-window save-selected-window--state)))
  (let ((save-selected-window--state (internal--before-save-selected-window))) (save-current-buffer (unwind-protect (progn (if (marker-buffer term-pending-delete-marker) (progn (delete-region term-pending-delete-marker (process-mark proc)) (set-marker term-pending-delete-marker nil))) (if (/= (point) (process-mark proc)) (progn (setq save-point (point-marker)))) (setq term-vertical-motion (if (eq (window-buffer) (current-buffer)) 'vertical-motion 'term-buffer-vertical-motion)) (setq save-marker (copy-marker (process-mark proc))) (goto-char (process-mark proc)) (save-restriction (if (and (> (point-max) (process-mark proc)) (not (eq (current-local-map) term-raw-map))) (progn (narrow-to-region (point-min) (process-mark proc)))) (if term-log-buffer (progn (princ str term-log-buffer))) (if term-terminal-undecoded-bytes (progn (setq str (concat term-terminal-undecoded-bytes str)) (setq str-length (length str)) (setq term-terminal-undecoded-bytes nil))) (while (< i str-length) (setq funny (string-match term-control-seq-regexp str i)) (let ((ctl-params (and funny (match-string 1 str))) (ctl-params-end (and funny (match-end 1))) (ctl-end (if funny (match-end 0) (setq funny (string-match term-control-seq-prefix-regexp str i)) (if funny (setq term-terminal-undecoded-bytes (substring str funny)) (setq funny str-length)) (1+ str-length)))) (if (> funny i) (progn (if term-do-line-wrapping (progn (term-down 1 t) (term-move-to-column 0) (setq term-do-line-wrapping nil))) (setq decoded-substring (decode-coding-string (substring str i funny) locale-coding-system t)) (if (= funny str-length) (progn (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial))))))) (let ((old-column (term-horizontal-column)) (old-point (point)) columns) (if term-suppress-hard-newline nil (while (> (+ (length decoded-substring) old-column) term-width) (insert (substring decoded-substring 0 (- term-width old-column))) (delete-region (point) (line-end-position)) (term-down 1 t) (term-move-columns (- (term-current-column))) (add-text-properties (1- (point)) (point) '(term-line-wrap t rear-nonsticky t)) (setq decoded-substring (substring decoded-substring (- term-width old-column))) (setq old-column 0))) (insert decoded-substring) (setq term-current-column (current-column) columns (- term-current-column old-column)) (if (not (or (eobp) term-insert-mode)) (progn (let ((pos (point))) (term-move-columns columns) (delete-region pos (point)) (setq term-current-column nil)))) (if term-insert-mode (progn (let ((pos (point))) (end-of-line) (if (> (current-column) term-width) (progn (delete-region (- (point) (- (current-column) term-width)) (point)))) (goto-char pos)))) (put-text-property old-point (point) 'font-lock-face term-current-face)) (if (eq (term-current-column) term-width) (progn (term-move-columns -1) (setq term-do-line-wrapping (point)))) (setq term-current-column nil) (setq i funny))) (let* ((val (and (<= ctl-end str-length) (aref str i)))) (cond ((eql val 9) (let ((col (term-current-column))) (term-move-to-column (min (1- term-width) (+ col 8 (- (mod col 8))))))) ((eql val 13) (funcall term-vertical-motion 0) (setq term-current-column term-start-line-column)) ((eql val 10) (if (and term-kill-echo-list (term-check-kill-echo-list)) nil (term-down 1 t))) ((eql val 8) (term-move-columns -1)) ((eql val 7) (beep t)) ((eql val 26) (funcall term-command-function (decode-coding-string (substring str (1+ i) (- ctl-end (if (eq (aref str (- ctl-end 2)) 13) 2 1))) locale-coding-system t))) ((eql val 27) (let* ((val (aref str (1+ i)))) (cond ((eql val 91) (if (eq ctl-params-end (1- ctl-end)) (progn (term-handle-ansi-escape proc (mapcar #'string-to-number (split-string ctl-params ";")) (aref str (1- ctl-end)))))) ((eql val 68) (term-handle-deferred-scroll) (term-down 1 t)) ((eql val 77) (if (or (< (term-current-row) term-scroll-start) (>= (1- (term-current-row)) term-scroll-start)) (term-down -1) (term-down -1 t))) ((eql val 55) (term-handle-deferred-scroll) (setq term-saved-cursor (list (term-current-row) (term-horizontal-column) term-ansi-current-bg-color term-ansi-current-bold term-ansi-current-color term-ansi-current-invisible term-ansi-current-reverse term-ansi-current-underline term-current-face))) ((eql val 56) (if term-saved-cursor (progn (term-goto (nth 0 term-saved-cursor) (nth 1 term-saved-cursor)) (setq term-ansi-current-bg-color (nth 2 term-saved-cursor) term-ansi-current-bold (nth 3 term-saved-cursor) term-ansi-current-color (nth 4 term-saved-cursor) term-ansi-current-invisible (nth 5 term-saved-cursor) term-ansi-current-reverse (nth 6 term-saved-cursor) term-ansi-current-underline (nth 7 term-saved-cursor) term-current-face (nth 8 term-saved-cursor))))) ((eql val 99) (term-reset-terminal)) ((eql val 65) (term-handle-ansi-terminal-messages (substring str i ctl-end))) (t nil)))) ((memql val '(nil 15 14 0)) nil) (t (let ((x37 val)) (error "No clause matching `%S'" x37))))) (if (eq term-do-line-wrapping (point)) nil (setq term-do-line-wrapping nil)) (if term-pager-old-local-map (progn (if (> (% (current-column) term-width) 0) (setq term-terminal-undecoded-bytes (substring str i)) (if (= 0 i) (setq term-terminal-undecoded-bytes (concat "\15" (substring str i))) (setq term-terminal-undecoded-bytes (substring str (1- i))) (aset term-terminal-undecoded-bytes 0 13)) (goto-char (point-max))) (make-local-variable 'term-pager-old-filter) (setq term-pager-old-filter (process-filter proc)) (set-process-filter proc term-pager-filter) (setq i str-length)) (setq i ctl-end))))) (if (>= (term-current-row) term-height) (progn (term-handle-deferred-scroll))) (set-marker (process-mark proc) (point)) (if (stringp decoded-substring) (progn (term-watch-for-password-prompt decoded-substring))) (if save-point (progn (goto-char save-point) (set-marker save-point nil))) (if (and term-pending-frame (eq (window-buffer selected) (current-buffer))) (progn (term-display-line (car term-pending-frame) (cdr term-pending-frame)) (setq term-pending-frame nil))) (setq win selected) (while (window-minibuffer-p win) (setq win (next-window win nil t))) (setq last-win win) (while (progn (setq win (next-window win nil t)) (if (eq (window-buffer win) (process-buffer proc)) (progn (let ((scroll term-scroll-to-bottom-on-output)) (select-window win) (if (or (= (point) save-marker) (eq scroll t) (eq scroll 'all) (and (eq selected win) (or (eq scroll 'this) (not save-point))) (and (eq scroll 'others) (not (eq selected win)))) (progn (goto-char term-home-marker) (recenter 0) (goto-char (process-mark proc)) (if (not (pos-visible-in-window-p (point) win)) (recenter -1)))) (if (and term-scroll-show-maximum-output (>= (point) (process-mark proc))) (progn (save-excursion (goto-char (point-max)) (recenter -1))))))) (not (eq win last-win)))) (if (> term-buffer-maximum-size 0) (progn (save-excursion (goto-char (process-mark (get-buffer-process (current-buffer)))) (forward-line (- term-buffer-maximum-size)) (beginning-of-line) (delete-region (point-min) (point))))) (set-marker save-marker nil)) (internal--after-save-selected-window save-selected-window--state))))
  (let* ((i 0) funny decoded-substring save-point save-marker win (inhibit-read-only t) (buffer-undo-list t) (selected (selected-window)) last-win (str-length (length str))) (let ((save-selected-window--state (internal--before-save-selected-window))) (save-current-buffer (unwind-protect (progn (if (marker-buffer term-pending-delete-marker) (progn (delete-region term-pending-delete-marker (process-mark proc)) (set-marker term-pending-delete-marker nil))) (if (/= (point) (process-mark proc)) (progn (setq save-point (point-marker)))) (setq term-vertical-motion (if (eq (window-buffer) (current-buffer)) 'vertical-motion 'term-buffer-vertical-motion)) (setq save-marker (copy-marker (process-mark proc))) (goto-char (process-mark proc)) (save-restriction (if (and (> (point-max) (process-mark proc)) (not (eq (current-local-map) term-raw-map))) (progn (narrow-to-region (point-min) (process-mark proc)))) (if term-log-buffer (progn (princ str term-log-buffer))) (if term-terminal-undecoded-bytes (progn (setq str (concat term-terminal-undecoded-bytes str)) (setq str-length (length str)) (setq term-terminal-undecoded-bytes nil))) (while (< i str-length) (setq funny (string-match term-control-seq-regexp str i)) (let ((ctl-params (and funny (match-string 1 str))) (ctl-params-end (and funny (match-end 1))) (ctl-end (if funny (match-end 0) (setq funny (string-match term-control-seq-prefix-regexp str i)) (if funny (setq term-terminal-undecoded-bytes (substring str funny)) (setq funny str-length)) (1+ str-length)))) (if (> funny i) (progn (if term-do-line-wrapping (progn (term-down 1 t) (term-move-to-column 0) (setq term-do-line-wrapping nil))) (setq decoded-substring (decode-coding-string (substring str i funny) locale-coding-system t)) (if (= funny str-length) (progn (let ((partial 0) (count (length decoded-substring))) (while (eq (char-charset (aref decoded-substring (- count 1 partial))) 'eight-bit) (setq partial (1+ partial))) (if (> partial 0) (progn (setq term-terminal-undecoded-bytes (substring decoded-substring (- partial))) (setq decoded-substring (substring decoded-substring 0 (- partial))) (setq str-length (- str-length partial)) (setq funny (- funny partial))))))) (let ((old-column (term-horizontal-column)) (old-point (point)) columns) (if term-suppress-hard-newline nil (while (> (+ (length decoded-substring) old-column) term-width) (insert (substring decoded-substring 0 (- term-width old-column))) (delete-region (point) (line-end-position)) (term-down 1 t) (term-move-columns (- (term-current-column))) (add-text-properties (1- (point)) (point) '(term-line-wrap t rear-nonsticky t)) (setq decoded-substring (substring decoded-substring (- term-width old-column))) (setq old-column 0))) (insert decoded-substring) (setq term-current-column (current-column) columns (- term-current-column old-column)) (if (not (or (eobp) term-insert-mode)) (progn (let ((pos (point))) (term-move-columns columns) (delete-region pos (point)) (setq term-current-column nil)))) (if term-insert-mode (progn (let ((pos (point))) (end-of-line) (if (> (current-column) term-width) (progn (delete-region (- (point) (- (current-column) term-width)) (point)))) (goto-char pos)))) (put-text-property old-point (point) 'font-lock-face term-current-face)) (if (eq (term-current-column) term-width) (progn (term-move-columns -1) (setq term-do-line-wrapping (point)))) (setq term-current-column nil) (setq i funny))) (let* ((val (and (<= ctl-end str-length) (aref str i)))) (cond ((eql val 9) (let ((col (term-current-column))) (term-move-to-column (min (1- term-width) (+ col 8 (- (mod col 8))))))) ((eql val 13) (funcall term-vertical-motion 0) (setq term-current-column term-start-line-column)) ((eql val 10) (if (and term-kill-echo-list (term-check-kill-echo-list)) nil (term-down 1 t))) ((eql val 8) (term-move-columns -1)) ((eql val 7) (beep t)) ((eql val 26) (funcall term-command-function (decode-coding-string (substring str (1+ i) (- ctl-end (if (eq (aref str (- ctl-end 2)) 13) 2 1))) locale-coding-system t))) ((eql val 27) (let* ((val (aref str (1+ i)))) (cond ((eql val 91) (if (eq ctl-params-end (1- ctl-end)) (progn (term-handle-ansi-escape proc (mapcar #'string-to-number (split-string ctl-params ";")) (aref str (1- ctl-end)))))) ((eql val 68) (term-handle-deferred-scroll) (term-down 1 t)) ((eql val 77) (if (or (< (term-current-row) term-scroll-start) (>= (1- (term-current-row)) term-scroll-start)) (term-down -1) (term-down -1 t))) ((eql val 55) (term-handle-deferred-scroll) (setq term-saved-cursor (list (term-current-row) (term-horizontal-column) term-ansi-current-bg-color term-ansi-current-bold term-ansi-current-color term-ansi-current-invisible term-ansi-current-reverse term-ansi-current-underline term-current-face))) ((eql val 56) (if term-saved-cursor (progn (term-goto (nth 0 term-saved-cursor) (nth 1 term-saved-cursor)) (setq term-ansi-current-bg-color (nth 2 term-saved-cursor) term-ansi-current-bold (nth 3 term-saved-cursor) term-ansi-current-color (nth 4 term-saved-cursor) term-ansi-current-invisible (nth 5 term-saved-cursor) term-ansi-current-reverse (nth 6 term-saved-cursor) term-ansi-current-underline (nth 7 term-saved-cursor) term-current-face (nth 8 term-saved-cursor))))) ((eql val 99) (term-reset-terminal)) ((eql val 65) (term-handle-ansi-terminal-messages (substring str i ctl-end))) (t nil)))) ((memql val '(nil 15 14 0)) nil) (t (let ((x37 val)) (error "No clause matching `%S'" x37))))) (if (eq term-do-line-wrapping (point)) nil (setq term-do-line-wrapping nil)) (if term-pager-old-local-map (progn (if (> (% (current-column) term-width) 0) (setq term-terminal-undecoded-bytes (substring str i)) (if (= 0 i) (setq term-terminal-undecoded-bytes (concat "\15" (substring str i))) (setq term-terminal-undecoded-bytes (substring str (1- i))) (aset term-terminal-undecoded-bytes 0 13)) (goto-char (point-max))) (make-local-variable 'term-pager-old-filter) (setq term-pager-old-filter (process-filter proc)) (set-process-filter proc term-pager-filter) (setq i str-length)) (setq i ctl-end))))) (if (>= (term-current-row) term-height) (progn (term-handle-deferred-scroll))) (set-marker (process-mark proc) (point)) (if (stringp decoded-substring) (progn (term-watch-for-password-prompt decoded-substring))) (if save-point (progn (goto-char save-point) (set-marker save-point nil))) (if (and term-pending-frame (eq (window-buffer selected) (current-buffer))) (progn (term-display-line (car term-pending-frame) (cdr term-pending-frame)) (setq term-pending-frame nil))) (setq win selected) (while (window-minibuffer-p win) (setq win (next-window win nil t))) (setq last-win win) (while (progn (setq win (next-window win nil t)) (if (eq (window-buffer win) (process-buffer proc)) (progn (let ((scroll term-scroll-to-bottom-on-output)) (select-window win) (if (or (= (point) save-marker) (eq scroll t) (eq scroll 'all) (and (eq selected win) (or (eq scroll 'this) (not save-point))) (and (eq scroll 'others) (not (eq selected win)))) (progn (goto-char term-home-marker) (recenter 0) (goto-char (process-mark proc)) (if (not (pos-visible-in-window-p (point) win)) (recenter -1)))) (if (and term-scroll-show-maximum-output (>= (point) (process-mark proc))) (progn (save-excursion (goto-char (point-max)) (recenter -1))))))) (not (eq win last-win)))) (if (> term-buffer-maximum-size 0) (progn (save-excursion (goto-char (process-mark (get-buffer-process (current-buffer)))) (forward-line (- term-buffer-maximum-size)) (beginning-of-line) (delete-region (point-min) (point))))) (set-marker save-marker nil)) (internal--after-save-selected-window save-selected-window--state)))))
  (save-current-buffer (set-buffer (process-buffer proc)) (let* ((i 0) funny decoded-substring save-point save-marker win (inhibit-read-only t) (buffer-undo-list t) (selected (selected-window)) last-win (str-length (length str))) (let ((save-selected-window--state (internal--before-save-selected-window))) (save-current-buffer (unwind-protect (progn (if (marker-buffer term-pending-delete-marker) (progn ... ...)) (if (/= ... ...) (progn ...)) (setq term-vertical-motion (if ... ... ...)) (setq save-marker (copy-marker ...)) (goto-char (process-mark proc)) (save-restriction (if ... ...) (if term-log-buffer ...) (if term-terminal-undecoded-bytes ...) (while ... ... ...)) (if (>= ... term-height) (progn ...)) (set-marker (process-mark proc) (point)) (if (stringp decoded-substring) (progn ...)) (if save-point (progn ... ...)) (if (and term-pending-frame ...) (progn ... ...)) (setq win selected) (while (window-minibuffer-p win) (setq win ...)) (setq last-win win) (while (progn ... ... ...)) (if (> term-buffer-maximum-size 0) (progn ...)) (set-marker save-marker nil)) (internal--after-save-selected-window save-selected-window--state))))) (if (get-buffer-window (current-buffer)) (progn (redisplay))))
  term-emulate-terminal(#<process /dev/ftdi0> "\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376")

[-- Attachment #3: Type: text/plain, Size: 57 bytes --]


This is from master.

Thanks for taking a look at this.

bug#29918: 26.0.90; serial-term error in process filter

[Lars Ingebrigtsen]

Gemini Lasswell <gazally@runbox.com> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> Does setting (setq debug-on-error t) give you a backtrace for these
>> errors?
>
> No, but I just made one happen by removing the internal_condition_case_1
> wrapper on the read_process_output_call in read_and_dispose_of_process_output.
> Then after I got the first backtrace showing that the error was in
> term-emulate-terminal I eval-defun'd it to make a more useful backtrace:
>
> Debugger entered--Lisp error: (args-out-of-range "\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376" -1)
>   aref("\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376" -1)
>   (char-charset (aref decoded-substring (- count 1 partial)))

I'm guessing this bug was introduced by:

commit 47019a521f774fbd13441e178a6a82c9989b9912
Author: Noam Postavsky <npostavs@gmail.com>
Date:   Thu Jan 18 08:22:47 2018 -0500

    Switch term.el to lexical binding, and clean up code a bit
    
It's a huge patch, though, and it's difficult to follow the changes in
logic.

The old code had:

-			  (setq count (length decoded-substring))
-                          ;; Check for multibyte characters that ends
-                          ;; before end of string, and save it for
-                          ;; next time.
-                          (when (= funny str-length)
-                            (let ((partial 0))
-                              (while (eq (char-charset (aref decoded-substring
-                                                             (- count 1 partial)))
-                                         'eight-bit)
-                                (cl-incf partial))

(And it's that aref that's failing with (- count 1 partial) being -1,
and the new one is

+                (when (= funny str-length)
+                  (let ((partial 0)
+                        (count (length decoded-substring)))
+                    (while (eq (char-charset (aref decoded-substring
+                                                   (- count 1 partial)))
+                               'eight-bit)
+                      (cl-incf partial))

with that length being recalculated that way...

Noam?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#29918: 26.0.90; serial-term error in process filter

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 1824 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Gemini Lasswell <gazally@runbox.com> writes:
>
>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>>
>>> Does setting (setq debug-on-error t) give you a backtrace for these
>>> errors?
>>
>> No, but I just made one happen by removing the internal_condition_case_1
>> wrapper on the read_process_output_call in read_and_dispose_of_process_output.
>> Then after I got the first backtrace showing that the error was in
>> term-emulate-terminal I eval-defun'd it to make a more useful backtrace:
>>
>> Debugger entered--Lisp error: (args-out-of-range "\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376" -1)
>>   aref("\376\374\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376\364\376" -1)
>>   (char-charset (aref decoded-substring (- count 1 partial)))
>
> I'm guessing this bug was introduced by:
>
> commit 47019a521f774fbd13441e178a6a82c9989b9912
> Author: Noam Postavsky <npostavs@gmail.com>
> Date:   Thu Jan 18 08:22:47 2018 -0500
>
>     Switch term.el to lexical binding, and clean up code a bit

It can't be that patch, because it's only in master, not emacs-26.  I
think it's rather [1: 134e86b360] (also mine).

[1: 134e86b360]: 2017-01-03 08:58:40 -0500
  Handle multibyte chars spanning chunks in term.el
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=134e86b360cab0d0a5cb634b71a4b06ec26c5f1f

If I understand the backtrace correctly, then the problem is that I
didn't think of the case where the whole string fails to decode.  I
think the patch below should fix it; there is a possibility it prevents
correct decoding in case the terminal receives a utf-8 character in
single byte chunks (though I'm not sure if it's even valid for a program
to emit single bytes like that).


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 1608 bytes --]

From 3e2663f8366ea82f52f47019b63fca243e3f88d9 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Wed, 17 Jul 2019 07:20:20 -0400
Subject: [PATCH] Handle completely undecoded input in term (Bug#29918)

* lisp/term.el (term-emulate-terminal): Avoid errors if the whole
decoded string is eight-bit characters.
---
 lisp/term.el | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/lisp/term.el b/lisp/term.el
index cbef68dc0a..9785ce3024 100644
--- a/lisp/term.el
+++ b/lisp/term.el
@@ -2900,11 +2900,12 @@ term-emulate-terminal
                           ;; next time.
                           (when (= funny str-length)
                             (let ((partial 0))
-                              (while (eq (char-charset (aref decoded-substring
-                                                             (- count 1 partial)))
-                                         'eight-bit)
+                              (while (and (< partial count)
+                                          (eq (char-charset (aref decoded-substring
+                                                                  (- count 1 partial)))
+                                              'eight-bit))
                                 (cl-incf partial))
-                              (when (> partial 0)
+                              (when (> count partial 0)
                                 (setq term-terminal-undecoded-bytes
                                       (substring decoded-substring (- partial)))
                                 (setq decoded-substring
-- 
2.11.0

bug#29918: 26.0.90; serial-term error in process filter

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@gmail.com>
> Date: Wed, 17 Jul 2019 07:36:34 -0400
> Cc: 29918@debbugs.gnu.org, Gemini Lasswell <gazally@runbox.com>
> 
> If I understand the backtrace correctly, then the problem is that I
> didn't think of the case where the whole string fails to decode.  I
> think the patch below should fix it

Yes, I think you are right.

> there is a possibility it prevents correct decoding in case the
> terminal receives a utf-8 character in single byte chunks (though
> I'm not sure if it's even valid for a program to emit single bytes
> like that).

I agree.

Thanks.

bug#29918: 26.0.90; serial-term error in process filter

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 426 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> there is a possibility it prevents correct decoding in case the
>> terminal receives a utf-8 character in single byte chunks (though
>> I'm not sure if it's even valid for a program to emit single bytes
>> like that).
>
> I agree.

I thought of limiting partial to 4, to avoid this possibility, since
utf-8 characters can be 4 bytes at most (not sure about other encodings
though).  


[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 1506 bytes --]

From bad839ca47ac58bd4967b95f447a596940e832f1 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Wed, 17 Jul 2019 07:20:20 -0400
Subject: [PATCH] Handle completely undecoded input in term (Bug#29918)

* lisp/term.el (term-emulate-terminal): Avoid errors if the whole
decoded string is eight-bit characters.  Accumulate at most 4
undecoded bytes.
---
 lisp/term.el | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/lisp/term.el b/lisp/term.el
index cbef68dc0a..2fdd3253ec 100644
--- a/lisp/term.el
+++ b/lisp/term.el
@@ -2900,9 +2900,11 @@ term-emulate-terminal
                           ;; next time.
                           (when (= funny str-length)
                             (let ((partial 0))
-                              (while (eq (char-charset (aref decoded-substring
-                                                             (- count 1 partial)))
-                                         'eight-bit)
+                              (while (and (< partial count)
+                                          (< partial 4)
+                                          (eq (char-charset (aref decoded-substring
+                                                                  (- count 1 partial)))
+                                              'eight-bit))
                                 (cl-incf partial))
                               (when (> partial 0)
                                 (setq term-terminal-undecoded-bytes
-- 
2.11.0

bug#29918: 26.0.90; serial-term error in process filter

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@gmail.com>
> Cc: 29918@debbugs.gnu.org,  gazally@runbox.com,  larsi@gnus.org
> Date: Thu, 18 Jul 2019 07:22:10 -0400
> 
> >> there is a possibility it prevents correct decoding in case the
> >> terminal receives a utf-8 character in single byte chunks (though
> >> I'm not sure if it's even valid for a program to emit single bytes
> >> like that).
> >
> > I agree.
> 
> I thought of limiting partial to 4, to avoid this possibility, since
> utf-8 characters can be 4 bytes at most (not sure about other encodings
> though).  

I don't understand why limit, if we assume that no program will write
UTF-8 encoded sequences byte-byte.  What am I missing?

In general, I'd prefer the simplest fix, as this is going to be pushed
to the release branch, yes?

bug#29918: 26.0.90; serial-term error in process filter

[Noam Postavsky]

Eli Zaretskii <eliz@gnu.org> writes:

> I don't understand why limit, if we assume that no program will write
> UTF-8 encoded sequences byte-byte.  What am I missing?

I wasn't assuming that, I was wondering whether it was true or not.

> In general, I'd prefer the simplest fix, as this is going to be pushed
> to the release branch, yes?

I wasn't sure about pushing to the release branch either, but yes, I
think my first patch is safer for the release branch (since it only
affects cases where we are currently signaling an error).

Gemini, could you try it and confirm that it fixes your problem please?

bug#29918: 26.0.90; serial-term error in process filter

[Gemini Lasswell]

Noam Postavsky <npostavs@gmail.com> writes:

> Gemini, could you try it and confirm that it fixes your problem please?

I tried both patches and both solve the problem.

bug#29918: 26.0.90; serial-term error in process filter

[Noam Postavsky]

tags 29918 fixed
close 29918 26.3
quit

Gemini Lasswell <gazally@runbox.com> writes:

> Noam Postavsky <npostavs@gmail.com> writes:
>
>> Gemini, could you try it and confirm that it fixes your problem please?
>
> I tried both patches and both solve the problem.

Thanks, I added a test and pushed the first patch to emacs-26.

150bdfe43a 2019-07-20T21:35:21-04:00 "Handle completely undecoded input in term (Bug#29918)"
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=150bdfe43acde8423612cbff4eafbbb88878b497

bug#29918: 26.0.90; serial-term error in process filter

[Stefan Monnier]

> I don't understand why limit, if we assume that no program will write
> UTF-8 encoded sequences byte-byte.

I don't think we can assume that, in general.


        Stefan

bug#29918: 26.0.90; serial-term error in process filter

[Stefan Monnier]

>>> there is a possibility it prevents correct decoding in case the
>>> terminal receives a utf-8 character in single byte chunks (though
>>> I'm not sure if it's even valid for a program to emit single bytes
>>> like that).
>> I agree.
> I thought of limiting partial to 4, to avoid this possibility, since
> utf-8 characters can be 4 bytes at most (not sure about other encodings
> though).

That sounds like a good idea.

Just a side note to mention that there are other multibyte encodings
than utf-8 and I have no reason to believe that they're never used for
terminals, so we should look for the maximum over all coding-systems
rather than only utf-8.


        Stefan

bug#29918: 26.0.90; serial-term error in process filter

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Noam Postavsky <npostavs@gmail.com>,  29918@debbugs.gnu.org,
>   gazally@runbox.com,  larsi@gnus.org
> Date: Sun, 21 Jul 2019 09:49:02 -0400
> 
> > I don't understand why limit, if we assume that no program will write
> > UTF-8 encoded sequences byte-byte.
> 
> I don't think we can assume that, in general.

What would be a situation where it is false?

bug#29918: 26.0.90; serial-term error in process filter

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Cc: Noam Postavsky <npostavs@gmail.com>,  29918@debbugs.gnu.org,
>>   gazally@runbox.com,  larsi@gnus.org
>> Date: Sun, 21 Jul 2019 09:49:02 -0400
>> 
>> > I don't understand why limit, if we assume that no program will write
>> > UTF-8 encoded sequences byte-byte.
>> 
>> I don't think we can assume that, in general.
>
> What would be a situation where it is false?

Isn't sending bytes one-by-one a pretty common thing for (primitive)
programs to do?  I'd assume so, especially for serial communication
programs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#29918: 26.0.90; serial-term error in process filter

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  npostavs@gmail.com,
>   29918@debbugs.gnu.org,  gazally@runbox.com
> Date: Sun, 21 Jul 2019 16:45:33 +0200
> 
> >> > I don't understand why limit, if we assume that no program will write
> >> > UTF-8 encoded sequences byte-byte.
> >> 
> >> I don't think we can assume that, in general.
> >
> > What would be a situation where it is false?
> 
> Isn't sending bytes one-by-one a pretty common thing for (primitive)
> programs to do?  I'd assume so, especially for serial communication
> programs.

We are talking about term.el, don't we?  So the programs relevant here
are those which run in a terminal, whether serial or not, and they are
generally expected to talk in characters, not bytes.