bug#33959:

[Carlos Pita <carlosjosepita@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]

> but I see no reason to not call (erase-buffer) altogether.
>
> Do you?

Yes, obviously, multiline input, I forgot.

Nevertheless I found the origin of the problem. Somehow when company
is enabled in the shell buffer
python-shell-font-lock-comint-output-filter-function is getting empty
strings as output and then the ";; Otherwise just add a newline."
clause is activated.

No matter the reason why empty output is being passed to the filter,
it's wrong for the filter to add a new line to the font lock buffer if
this happens.

I'm attaching two patches, since
python-shell-font-lock-comint-output-filter-function is an old friend
of mine and we have spent many nights together now:

1. The first one (Avoid-spurious...) strictly fixes this issue by
moving the not-empty condition to the top and doing everything else
unless not-empty. Period. That works.

2. The second one (Fix-font-lock...) combines 1 with my previous patch
for fixing ipython multiline input (#32390) since both changes
overlap.

I suggest to directly apply patch 2 since it improves prompt detection
heuristic and fixes two bugs. Then you can close this and #32390.

Regards
--
Carlos

[-- Attachment #2: 0001-Avoid-spurious-empty-lines-in-font-lock-buffer-fixes.patch --]
[-- Type: text/x-patch, Size: 2290 bytes --]

From d816a29f5a9cdcebd8f69ad0a6555eaca051587e Mon Sep 17 00:00:00 2001
From: memeplex <carlosjosepita@gmail.com>
Date: Thu, 3 Jan 2019 02:28:30 -0300
Subject: [PATCH] Avoid spurious empty lines in font lock buffer, fixes #33959

---
 lisp/progmodes/python.el | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 27d31ab..81a6e89 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -2556,19 +2556,19 @@ python-shell-font-lock-cleanup-buffer
 
 (defun python-shell-font-lock-comint-output-filter-function (output)
   "Clean up the font-lock buffer after any OUTPUT."
-  (if (and (not (string= "" output))
-           ;; Is end of output and is not just a prompt.
-           (not (member
-                 (python-shell-comint-end-of-output-p
-                  (ansi-color-filter-apply output))
-                 '(nil 0))))
-      ;; If output is other than an input prompt then "real" output has
-      ;; been received and the font-lock buffer must be cleaned up.
-      (python-shell-font-lock-cleanup-buffer)
-    ;; Otherwise just add a newline.
-    (python-shell-font-lock-with-font-lock-buffer
-      (goto-char (point-max))
-      (newline)))
+  (unless (string= "" output)
+    (if ;; Is end of output and is not just a prompt.
+        (not (member
+              (python-shell-comint-end-of-output-p
+               (ansi-color-filter-apply output))
+              '(nil 0)))
+        ;; If output is other than an input prompt then "real" output has
+        ;; been received and the font-lock buffer must be cleaned up.
+        (python-shell-font-lock-cleanup-buffer)
+      ;; Otherwise just add a newline.
+      (python-shell-font-lock-with-font-lock-buffer
+        (goto-char (point-max))
+        (newline))))
   output)
 
 (defun python-shell-font-lock-post-command-hook ()
@@ -2597,6 +2597,7 @@ python-shell-font-lock-post-command-hook
                                   (point-max))))
              (replacement-length (length replacement))
              (i 0))
+
         ;; Inject text properties to get input fontified.
         (while (not (= i replacement-length))
           (let* ((plist (text-properties-at i replacement))
-- 
2.20.1


[-- Attachment #3: 0001-Fix-font-lock-output-filter-bugs-33959-and-32390.patch --]
[-- Type: text/x-patch, Size: 2484 bytes --]

From dc80df310933f9f5f9b9bf065e72ee79a6b16134 Mon Sep 17 00:00:00 2001
From: memeplex <carlosjosepita@gmail.com>
Date: Thu, 3 Jan 2019 02:31:52 -0300
Subject: [PATCH] Fix font lock output filter bugs #33959 and #32390

Improves multiline input detection heuristic.
Fixes multiline input for ipython.
Avoid adding spurious empty lines to the font lock buffer.
Filter ansi color from output before matching.
---
 lisp/progmodes/python.el | 27 +++++++++++++--------------
 1 file changed, 13 insertions(+), 14 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 27d31ab..e98d88e 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -2556,20 +2556,18 @@ python-shell-font-lock-cleanup-buffer
 
 (defun python-shell-font-lock-comint-output-filter-function (output)
   "Clean up the font-lock buffer after any OUTPUT."
-  (if (and (not (string= "" output))
-           ;; Is end of output and is not just a prompt.
-           (not (member
-                 (python-shell-comint-end-of-output-p
-                  (ansi-color-filter-apply output))
-                 '(nil 0))))
-      ;; If output is other than an input prompt then "real" output has
-      ;; been received and the font-lock buffer must be cleaned up.
-      (python-shell-font-lock-cleanup-buffer)
-    ;; Otherwise just add a newline.
-    (python-shell-font-lock-with-font-lock-buffer
-      (goto-char (point-max))
-      (newline)))
-  output)
+  (unless (string= output "")
+    (if (let ((output (ansi-color-filter-apply output)))
+          (and (python-shell-comint-end-of-output-p output)
+               (not (string-match "\\.\\.\\." output))))
+        ;; If output ends with an initial (not continuation) input prompt
+        ;; then the font-lock buffer must be cleaned up.
+        (python-shell-font-lock-cleanup-buffer)
+      ;; Otherwise just add a newline.
+      (python-shell-font-lock-with-font-lock-buffer
+        (goto-char (point-max))
+        (newline)))
+    output))
 
 (defun python-shell-font-lock-post-command-hook ()
   "Fontifies current line in shell buffer."
@@ -2597,6 +2595,7 @@ python-shell-font-lock-post-command-hook
                                   (point-max))))
              (replacement-length (length replacement))
              (i 0))
+
         ;; Inject text properties to get input fontified.
         (while (not (= i replacement-length))
           (let* ((plist (text-properties-at i replacement))
-- 
2.20.1