bug#68559: [PATCH] Improve Python shell completion

[Liu Hui <liuhui1610@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1731 bytes --]

On Fri, Feb 23, 2024 at 10:39 PM kobarity <kobarity@gmail.com> wrote:
>
> Hi Liu,
>
> I noticed that with Non-native completions, Jedi completion is not
> enabled even if PYTHONSTARTUP is set and "REPL completion using Jedi
> 0.18.2" is shown in inferior Python. Is this expected behavior?  Jedi
> completion is enabled regardless of PYTHONSTARTUP setting when I use
> IPython with Non-native completions.

Thanks for pointing out the problem! The attached patch should enable
Jedi completion for the non-native case.

On Fri, Feb 16, 2024 at 3:41 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > The Python shell completion relies on the readline module, which is
> > not available for Python on MS-Windows. According to the instruction
> > in python.el:
> >
> > ;; readline based shell (it's known to work with PyPy).  If your
> > ;; Python installation lacks readline (like CPython for Windows),
> > ;; installing pyreadline (URL `https://ipython.org/pyreadline.html')
> > ;; should suffice.  To troubleshoot why you are not getting any
> > ;; completions, you can try the following in your Python shell:
> >
> > ;; >>> import readline, rlcompleter
> >
> > ;; If you see an error, then you need to either install pyreadline or
> > ;; setup custom code that avoids that dependency.
>
> I don't know if I have CPython, but the above does show an error
> message.
>
> > It may be necessary to install pyreadline (for Python 2.7) or
> > pyreadline3 (for Python 3).
>
> I will see if I can do that, thanks.
>
> Regardless, patches to the test suite to skip the tests which rely on
> those modules, if they aren't installed, will be welcome.

Such tests should be skipped now with this patch.

[-- Attachment #2: 0001-Detect-the-readline-support-for-Python-shell-complet.patch --]
[-- Type: text/x-patch, Size: 8213 bytes --]

From e02e55192e68b6272dea2a23096f90cdee517e74 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Mon, 26 Feb 2024 18:46:36 +0800
Subject: [PATCH] Detect the readline support for Python shell completion

* lisp/progmodes/python.el
(python-shell-readline-completer-delims): Update docstring.
(python-shell-completion-native-setup): Move the detection of
readline to ...
(python-shell-readline-detect): ... new function.
(python-shell-completion-native-turn-on-maybe): Skip if Python
has no readline support.
(python-shell-completion-at-point): Respect the delimiter of
readline completer in non-native completion.
* test/lisp/progmodes/python-tests.el
(python-shell-completion-at-point-1)
(python-shell-completion-at-point-native-1)
(python-completion-at-point-1, python-completion-at-point-2)
(python-completion-at-point-pdb-1)
(python-completion-at-point-while-running-1)
(python-completion-at-point-native-1)
(python-completion-at-point-native-2)
(python-completion-at-point-native-with-ffap-1)
(python-completion-at-point-native-with-eldoc-1): Skip tests if
Python has no readline support.
(python-shell-completion-at-point-jedi-completer): Add test for
non-native Python shell completion. (bug#68559)
---
 lisp/progmodes/python.el            | 29 ++++++++++++++++++++++-------
 test/lisp/progmodes/python-tests.el | 23 ++++++++++++++++++++++-
 2 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 587d0b36304..6c73ceba4ef 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -4439,7 +4439,24 @@ python-shell-completion-native-try-output-timeout
 
 (defvar python-shell-readline-completer-delims nil
   "Word delimiters used by the readline completer.
-It is automatically set by Python shell.")
+It is automatically set by Python shell.  A value of nil means the
+Python shell has no readline support.")
+
+(defun python-shell-readline-detect ()
+  "Detect the readline support for Python shell completion."
+  (let* ((process (python-shell-get-process))
+         (output (python-shell-send-string-no-output "
+try:
+    import readline
+    print(readline.get_completer_delims())
+except:
+    print('No readline support')" process)))
+    (unless (string-match-p "No readline support" output)
+      (setq-local python-shell-readline-completer-delims
+                  (string-trim-right output)))))
+
+(add-hook 'python-shell-first-prompt-hook
+          'python-shell-readline-detect -90)
 
 (defvar python-shell-completion-native-redirect-buffer
   " *Python completions redirect*"
@@ -4579,10 +4596,6 @@ python-shell-completion-native-setup
 __PYTHON_EL_native_completion_setup()" process)))
     (when (string-match-p "python\\.el: native completion setup loaded"
                           output)
-      (setq-local python-shell-readline-completer-delims
-                  (string-trim-right
-                   (python-shell-send-string-no-output
-                    "import readline; print(readline.get_completer_delims())")))
       (python-shell-completion-native-try))))
 
 (defun python-shell-completion-native-turn-off (&optional msg)
@@ -4611,7 +4624,8 @@ python-shell-completion-native-turn-on-maybe
       (cond
        ((python-shell-completion-native-interpreter-disabled-p)
         (python-shell-completion-native-turn-off msg))
-       ((python-shell-completion-native-setup)
+       ((and python-shell-readline-completer-delims
+             (python-shell-completion-native-setup))
         (when msg
           (message "Shell native completion is enabled.")))
        (t
@@ -4783,7 +4797,8 @@ python-shell-completion-at-point
                (with-current-buffer (process-buffer process)
                  (if python-shell-completion-native-enable
                      (string= python-shell-readline-completer-delims "")
-                   (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
+                   (or (string-match-p "ipython[23]?\\'" python-shell-interpreter)
+                       (string= python-shell-readline-completer-delims ""))))))
          (start
           (if (< (point) line-start)
               (point)
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 6c6cd9eee2b..e0584f78f82 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4777,6 +4777,7 @@ python-shell-completion-at-point-1
   (python-tests-with-temp-buffer-with-shell
    ""
    (python-shell-with-shell-buffer
+     (skip-when (null python-shell-readline-completer-delims))
      (insert "import abc")
      (comint-send-input)
      (python-tests-shell-wait-for-prompt)
@@ -4791,6 +4792,7 @@ python-shell-completion-at-point-native-1
    ""
    (python-shell-completion-native-turn-on)
    (python-shell-with-shell-buffer
+     (skip-when (null python-shell-readline-completer-delims))
      (insert "import abc")
      (comint-send-input)
      (python-tests-shell-wait-for-prompt)
@@ -4866,8 +4868,11 @@ python-shell-completion-at-point-jedi-completer
   (python-tests-with-temp-buffer-with-shell
    ""
    (python-shell-with-shell-buffer
-     (python-shell-completion-native-turn-on)
      (skip-unless (string= python-shell-readline-completer-delims ""))
+     (python-shell-completion-native-turn-off)
+     (python-tests--completion-module)
+     (python-tests--completion-parameters)
+     (python-shell-completion-native-turn-on)
      (python-tests--completion-module)
      (python-tests--completion-parameters)
      (python-tests--completion-extra-context))))
@@ -4905,6 +4910,8 @@ python-completion-at-point-1
 import abc
 "
    (let ((inhibit-message t))
+     (python-shell-with-shell-buffer
+       (skip-when (null python-shell-readline-completer-delims)))
      (python-shell-send-buffer)
      (python-tests-shell-wait-for-prompt)
      (goto-char (point-max))
@@ -4921,6 +4928,8 @@ python-completion-at-point-2
 import abc
 "
    (let ((inhibit-message t))
+     (python-shell-with-shell-buffer
+       (skip-when (null python-shell-readline-completer-delims)))
      (python-shell-send-buffer)
      (python-tests-shell-wait-for-prompt)
      (python-shell-with-shell-buffer
@@ -4940,6 +4949,8 @@ python-completion-at-point-pdb-1
 print('Hello')
 "
    (let ((inhibit-message t))
+     (python-shell-with-shell-buffer
+       (skip-when (null python-shell-readline-completer-delims)))
      (python-shell-send-buffer)
      (python-tests-shell-wait-for-prompt)
      (goto-char (point-max))
@@ -4956,6 +4967,8 @@ python-completion-at-point-while-running-1
 time.sleep(3)
 "
    (let ((inhibit-message t))
+     (python-shell-with-shell-buffer
+       (skip-when (null python-shell-readline-completer-delims)))
      (python-shell-send-buffer)
      (goto-char (point-max))
      (insert "time.")
@@ -4968,6 +4981,8 @@ python-completion-at-point-native-1
 import abc
 "
    (let ((inhibit-message t))
+     (python-shell-with-shell-buffer
+       (skip-when (null python-shell-readline-completer-delims)))
      (python-shell-completion-native-turn-on)
      (python-shell-send-buffer)
      (python-tests-shell-wait-for-prompt)
@@ -4985,6 +5000,8 @@ python-completion-at-point-native-2
 import abc
 "
    (let ((inhibit-message t))
+     (python-shell-with-shell-buffer
+       (skip-when (null python-shell-readline-completer-delims)))
      (python-shell-completion-native-turn-on)
      (python-shell-send-buffer)
      (python-tests-shell-wait-for-prompt)
@@ -5001,6 +5018,8 @@ python-completion-at-point-native-with-ffap-1
 import abc
 "
    (let ((inhibit-message t))
+     (python-shell-with-shell-buffer
+       (skip-when (null python-shell-readline-completer-delims)))
      (python-shell-completion-native-turn-on)
      (python-shell-send-buffer)
      (python-tests-shell-wait-for-prompt)
@@ -5017,6 +5036,8 @@ python-completion-at-point-native-with-eldoc-1
 import abc
 "
    (let ((inhibit-message t))
+     (python-shell-with-shell-buffer
+       (skip-when (null python-shell-readline-completer-delims)))
      (python-shell-completion-native-turn-on)
      (python-shell-send-buffer)
      (python-tests-shell-wait-for-prompt)
-- 
2.25.1