bug#50459: 28.0.50; Python shell completion is incompatible with flex, orderless, etc.

[Augusto Stoffel <arstoffel@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

After more discussions elsewhere [1], I noticed another problem with
the dynamic completion table from python.el.

Each call to the completion table requires communication with the
inferior process.  If called frequently enough, the comint input
filters can get confused, and some garbage gets printed to the shell.
If the inferior is running over Tramp, the problem gets amplified.

Now, this is not likely to cause much trouble when calling
`completion-at-point' manually, but it is a problem with any of
continuously-updating completion UIs like Company or Corfu.

To alleviate this, the completion-at-point function could implement some
sort of caching.  The difficult question is when to invalidate the
cache.  I've attached one possiblility as a draft patch.  If the
approach seems reasonable, then I'll format it properly.

As a side effect, the patch also solves the original issue described in
this ticket.

Please let me know what you think.

[1] https://github.com/oantolin/orderless/issues/79


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: python-capf.diff --]
[-- Type: text/x-patch, Size: 1471 bytes --]

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 4f222b4cf5..3239722981 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -3841,6 +3841,8 @@ python-shell-completion-get-completions
         (split-string completions
                       "^'\\|^\"\\|;\\|'$\\|\"$" t)))))
 
+(defvar-local python-shell--capf-cache nil)
+
 (defun python-shell-completion-at-point (&optional process)
   "Function for `completion-at-point-functions' in `inferior-python-mode'.
 Optional argument PROCESS forces completions to be retrieved
@@ -3895,11 +3897,13 @@ python-shell-completion-at-point
                        #'ignore
                      #'python-shell-completion-get-completions))
                   (t #'python-shell-completion-native-get-completions)))))
-    (list start end
-          (completion-table-dynamic
-           (apply-partially
-            completion-fn
-            process import-statement)))))
+    (let  ((re (or (car python-shell--capf-cache) regexp-unmatchable))
+           (prefix (buffer-substring-no-properties start end)))
+      (unless (string-match re prefix)
+        (setq python-shell--capf-cache
+              (cons (concat "\\`" (regexp-quote prefix) "\\(?:\\sw\\|\\s_\\)*\\'")
+                    (funcall completion-fn process import-statement prefix)))))
+    (list start end (cdr python-shell--capf-cache))))
 
 (define-obsolete-function-alias
   'python-shell-completion-complete-at-point