bug#68559: [PATCH] Improve Python shell completion

[Liu Hui <liuhui1610@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1782 bytes --]

On Tue, Feb 20, 2024 at 9:15 PM Basil L. Contovounesios
<basil@contovou.net> wrote:
>
> Liu Hui [2024-02-20 12:46 +0800] wrote:
>
> > On Mon, Feb 19, 2024 at 9:18 PM Basil L. Contovounesios
> > <basil@contovou.net> wrote:
> >>
> >> BTW, I have been seeing an IPython test failure on GNU/Linux:
> >> Any pointers?
> >
> > I cannot reproduce the problem with Python 3.12 and IPython 8.21. Can
> > you check the results with the following steps? Thanks.
>
> Thanks.  These all work as expected in 'emacs -Q', which made me realise
> what may be the problem:
>
> - the Emacs test suite runs under HOME=/nonexistent
> - but PATH is unchanged
> - I have the latest IPython installed locally using pipx:
>   $ ls -l $(which ipython)
>   lrwxrwxrwx 1 blc blc 53 Feb 20 11:46
>     /home/blc/.local/bin/ipython ->
>       /home/blc/.local/share/pipx/venvs/ipython/bin/ipython
>
> This is confirmed by the following experiment:
>
>
> Running 'make TEST_LOAD_EL=no test/python-tests' now results in the
> following output file:
>
>
> In this case list(get_ipython().Completer.completions(...)) is empty.

Thank you for the investigation! I didn't realize the problem is
related to HOME=/nonexistent in 'make test'. Now I can also reproduce
it. In fact, I found that the test failure was caused by Jedi, as it
attempts to write cache to a non-existent directory.

When Jedi is directly used as the completion backend, more test
failures will be triggered:

PYTHONSTARTUP="$(python -m jedi repl)" make TEST_LOAD_EL=no test/python-tests

If the cache directory is writable, all tests will pass:

PYTHONSTARTUP="$(python -m jedi repl)" XDG_CACHE_HOME=~/.cache make
TEST_LOAD_EL=no test/python-tests

The attached patch should fix the problem.

[-- Attachment #2: 0001-Fix-Python-shell-completion-test-failures.patch --]
[-- Type: text/x-patch, Size: 1760 bytes --]

From a6773258cf6fe1acc5520f56ef948ba6975cbe32 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Wed, 21 Feb 2024 12:40:06 +0800
Subject: [PATCH] Fix Python shell completion test failures

* test/lisp/progmodes/python-tests.el
(python-tests-with-temp-buffer-with-shell): Set XDG_CACHE_HOME
to a temporary directory. (bug#68559)
---
 test/lisp/progmodes/python-tests.el | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index af6c199b5bd..6454cfcc2b7 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -60,12 +60,17 @@ python-tests-with-temp-buffer-with-shell
            (python-shell-completion-native-enable nil))
        (python-mode)
        (unwind-protect
-           (progn
-             (run-python nil t)
-             (insert ,contents)
-             (goto-char (point-min))
-             (python-tests-shell-wait-for-prompt)
-             ,@body)
+           ;; Prevent test failures when Jedi is used as a completion
+           ;; backend, either directly or indirectly (e.g., via
+           ;; IPython).  Jedi needs to store cache, but the
+           ;; "/nonexistent" HOME directory is not writable.
+           (ert-with-temp-directory cache-dir
+             (with-environment-variables (("XDG_CACHE_HOME" cache-dir))
+               (run-python nil t)
+               (insert ,contents)
+               (goto-char (point-min))
+               (python-tests-shell-wait-for-prompt)
+               ,@body))
          (when (python-shell-get-buffer)
            (python-shell-with-shell-buffer
              (let (kill-buffer-hook kill-buffer-query-functions)
-- 
2.25.1