* bug#68559: [PATCH] Improve Python shell completion
@ 2024-01-18 4:48 Liu Hui
2024-01-18 6:39 ` Eli Zaretskii
2024-02-15 14:43 ` Mattias Engdegård
0 siblings, 2 replies; 68+ messages in thread
From: Liu Hui @ 2024-01-18 4:48 UTC (permalink / raw)
To: 68559
[-- Attachment #1: Type: text/plain, Size: 856 bytes --]
Hi,
Currently python-shell-completion-at-point doesn't respect the
delimiter setting of readline completer and always split the text,
resulting in poor completions when completers that don't need word
splitting, e.g. jedi[1], is used. This patch fixes the problem.
Meanwhile, this patch adds an option 'python-shell-readline-completer'
to make Python shell provide better completion experience (e.g.
completing function parameters, dictionary keys) by default if users
are using vanilla Python interpreter with jedi or using IPython, where
the completion support for IPython has been enhanced (e.g. showing
type annotations and function signatures).
BTW, I think it may make sense to move Python snippets to a separate
file to make them easier to be maintained.
Thanks.
Best,
[1] https://jedi.readthedocs.io/en/latest/docs/usage.html#repl-completion
[-- Attachment #2: 0001-Improve-Python-shell-completion.patch --]
[-- Type: text/x-patch, Size: 21278 bytes --]
From 63a0f172bcd3a3ac882d76c7788c3637acc6ae39 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Thu, 18 Jan 2024 12:00:00 +0800
Subject: [PATCH] Improve Python shell completion
* lisp/progmodes/python.el (python-shell-completion-setup-code): Fix
the completion code of IPython. Change the return value to JSON
string and ...
(python-shell-completion-get-completions): ... simplify parsing.
(inferior-python-mode): Update docstring.
(python-shell-readline-completer): New option.
(python-shell-readline-completer-delims)
(python-shell-readline-jedi-setup-code)
(python-shell-readline-ipython-setup-code): New variables.
(python-shell-completion-native-setup): Setup a suitable readline
completer and set the completer delimiter.
(python-shell-completion-native-get-completions): Convert output
string to completions properly.
(python-shell-completion-at-point): Send text beginning from the line
start if the completion backend does not need word splitting. Remove
the detection of import statement because it is not needed anymore.
Create proper completion table based on completions returned from
different backends.
* test/lisp/progmodes/python-tests.el (python-shell-completion-module):
(python-shell-completion-parameters):
(python-shell-completion-at-point-jedi-completer):
(python-shell-completion-at-point-ipython): New tests.
* etc/NEWS: Announce the change.
---
etc/NEWS | 6 +
lisp/progmodes/python.el | 234 ++++++++++++++++++++++------
test/lisp/progmodes/python-tests.el | 66 ++++++++
3 files changed, 255 insertions(+), 51 deletions(-)
diff --git a/etc/NEWS b/etc/NEWS
index 03b8c3b517a..6fd337727c5 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1029,6 +1029,12 @@ instead of:
This allows the user to specify command line arguments to the non
interactive Python interpreter specified by 'python-interpreter'.
+*** New user option 'python-shell-readline-completer'.
+This allows the user to specify the readline completer used for Python
+shell completion. The default is 'auto', which means a suitable
+completer will be configured automatically according to the Python
+interpreter.
+
** use-package
+++
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index ff799e1e662..2d87eb59ede 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -128,9 +128,9 @@
;; receiving escape sequences (with some limitations, i.e. completion
;; in blocks does not work). The code executed for the "fallback"
;; completion can be found in `python-shell-completion-setup-code' and
-;; `python-shell-completion-string-code' variables. Their default
-;; values enable completion for both CPython and IPython, and probably
-;; any readline based shell (it's known to work with PyPy). If your
+;; `python-shell-completion-get-completions'. Their default values
+;; enable completion for both CPython and IPython, and probably any
+;; readline based shell (it's known to work with PyPy). If your
;; Python installation lacks readline (like CPython for Windows),
;; installing pyreadline (URL `https://ipython.org/pyreadline.html')
;; should suffice. To troubleshoot why you are not getting any
@@ -3601,7 +3601,6 @@ inferior-python-mode
`python-shell-prompt-block-regexp',
`python-shell-font-lock-enable',
`python-shell-completion-setup-code',
-`python-shell-completion-string-code',
`python-eldoc-setup-code',
`python-ffap-setup-code' can
customize this mode for different Python interpreters.
@@ -4319,8 +4318,9 @@ python-shell-completion-setup-code
completions = []
completer = None
+ import json
try:
- import readline
+ import readline, re
try:
import __builtin__
@@ -4331,16 +4331,29 @@ python-shell-completion-setup-code
is_ipython = ('__IPYTHON__' in builtins or
'__IPYTHON__active' in builtins)
- splits = text.split()
- is_module = splits and splits[0] in ('from', 'import')
-
- if is_ipython and is_module:
- from IPython.core.completerlib import module_completion
- completions = module_completion(text.strip())
- elif is_ipython and '__IP' in builtins:
- completions = __IP.complete(text)
- elif is_ipython and 'get_ipython' in builtins:
- completions = get_ipython().Completer.all_completions(text)
+
+ if is_ipython and 'get_ipython' in builtins:
+ def filter_c(prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ import IPython
+ try:
+ if IPython.version_info[0] >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ completions = [
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if filter_c(text, c.text)]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ completions = [text + m[len(part):] for m in matches if filter_c(text, m)]
+ except:
+ pass
else:
# Try to reuse current completer.
completer = readline.get_completer()
@@ -4363,7 +4376,7 @@ python-shell-completion-setup-code
finally:
if getattr(completer, 'PYTHON_EL_WRAPPED', False):
completer.print_mode = True
- return completions"
+ return json.dumps(completions)"
"Code used to setup completion in inferior Python processes."
:type 'string)
@@ -4404,6 +4417,79 @@ python-shell-completion-native-try-output-timeout
:version "25.1"
:type 'float)
+(defcustom python-shell-readline-completer 'auto
+ "The readline completer used for Python shell completion.
+If the value is non-nil, Python shell will setup the readline
+completer unless it has been set elsewhere (e.g. in the
+PYTHONSTARTUP file). Below are possible values:
+- `auto': the completer is determined according to the
+interpreter. Specifically, the IPython completer, defined in
+`python-shell-readline-ipython-setup-code', is used when the
+interpreter is ipython, otherwise the Jedi completer is used.
+- a string: Python code to setup the readline. It should define
+the function `__PYTHON_EL_setup_readline_completer'. See
+`python-shell-readline-jedi-setup-code' for reference.
+- `nil': Python shell will do nothing.
+
+In any case, if the completer is not set successfully in the end,
+fallback to the built-in rlcompleter."
+ :type '(choice (const :tag "Automatic" auto)
+ (const :tag "No configuration" nil)
+ (string :tag "Python setup code"))
+ :version "30.1")
+
+(defvar python-shell-readline-completer-delims nil
+ "Word delimiters used by the readline completer.
+It is automatically set by Python shell.")
+
+(defconst python-shell-readline-jedi-setup-code
+ "
+def __PYTHON_EL_setup_readline_completer():
+ from jedi.utils import setup_readline
+ setup_readline()"
+ "Code used to setup readline completer with Jedi.")
+
+(defconst python-shell-readline-ipython-setup-code
+ "
+def __PYTHON_EL_setup_readline_completer():
+ import readline, re, json, IPython
+
+ class __ipython_RL:
+ def __init__(self, v):
+ self.version = v
+
+ def filter(self, prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ def complete(self, text, state):
+ if state == 0:
+ try:
+ if self.version >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ self.matches = [json.dumps([
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if self.filter(text, c.text)])]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ self.matches = [text + m[len(part):] for m in matches
+ if self.filter(text, m)]
+ except Exception:
+ pass
+ try:
+ return self.matches[state]
+ except IndexError:
+ return None
+
+ readline.set_completer(__ipython_RL(IPython.version_info[0]).complete)
+ readline.set_completer_delims('')"
+ "Code used to setup readline completer for IPython.")
+
(defvar python-shell-completion-native-redirect-buffer
" *Python completions redirect*"
"Buffer to be used to redirect output of readline commands.")
@@ -4427,7 +4513,20 @@ python-shell-completion-native-try
(defun python-shell-completion-native-setup ()
"Try to setup native completion, return non-nil on success."
(let* ((process (python-shell-get-process))
- (output (python-shell-send-string-no-output "
+ (completer (pcase python-shell-readline-completer
+ ('auto
+ (if (string-match-p "ipython[23]?\\'" python-shell-interpreter)
+ python-shell-readline-ipython-setup-code
+ python-shell-readline-jedi-setup-code))
+ ((pred stringp) python-shell-readline-completer)
+ (_ "")))
+ (output (python-shell-send-string-no-output
+ (concat "
+try:
+ del __PYTHON_EL_setup_readline_completer
+except:
+ pass
+" completer "
def __PYTHON_EL_native_completion_setup():
try:
import readline
@@ -4503,8 +4602,11 @@ python-shell-completion-native-setup
completer = readline.get_completer()
if not completer:
- # Used as last resort to avoid breaking customizations.
- import rlcompleter
+ try:
+ __PYTHON_EL_setup_readline_completer()
+ except:
+ # Used as last resort to avoid breaking customizations.
+ import rlcompleter
completer = readline.get_completer()
if completer and not getattr(completer, 'PYTHON_EL_WRAPPED', False):
@@ -4539,9 +4641,13 @@ python-shell-completion-native-setup
print ('python.el: native completion setup failed, %s: %s'
% sys.exc_info()[:2])
-__PYTHON_EL_native_completion_setup()" process)))
+__PYTHON_EL_native_completion_setup()") process)))
(when (string-match-p "python\\.el: native completion setup loaded"
output)
+ (setq-local python-shell-readline-completer-delims
+ (string-trim-right
+ (python-shell-send-string-no-output
+ "import readline; print(readline.get_completer_delims())")))
(python-shell-completion-native-try))))
(defun python-shell-completion-native-turn-off (&optional msg)
@@ -4609,6 +4715,8 @@ python-shell-completion-native-get-completions
(let* ((original-filter-fn (process-filter process))
(redirect-buffer (get-buffer-create
python-shell-completion-native-redirect-buffer))
+ (sep (if (string= python-shell-readline-completer-delims "")
+ "[\n\r]+" "[ \f\t\n\r\v()]+"))
(trigger "\t")
(new-input (concat input trigger))
(input-length
@@ -4651,28 +4759,26 @@ python-shell-completion-native-get-completions
process python-shell-completion-native-output-timeout
comint-redirect-finished-regexp)
(re-search-backward "0__dummy_completion__" nil t)
- (cl-remove-duplicates
- (split-string
- (buffer-substring-no-properties
- (line-beginning-position) (point-min))
- "[ \f\t\n\r\v()]+" t)
- :test #'string=))))
+ (let ((str (buffer-substring-no-properties
+ (line-beginning-position) (point-min))))
+ (if (string= "[" (substring str 0 1))
+ (condition-case nil
+ (python--parse-json-array str)
+ (t (cl-remove-duplicates (split-string str sep t)
+ :test #'string=)))
+ (cl-remove-duplicates (split-string str sep t)
+ :test #'string=))))))
(set-process-filter process original-filter-fn)))))
(defun python-shell-completion-get-completions (process input)
"Get completions of INPUT using PROCESS."
(with-current-buffer (process-buffer process)
- (let ((completions
- (python-util-strip-string
- (python-shell-send-string-no-output
- (format
- "%s\nprint(';'.join(__PYTHON_EL_get_completions(%s)))"
+ (python--parse-json-array
+ (python-shell-send-string-no-output
+ (format "%s\nprint(__PYTHON_EL_get_completions(%s))"
python-shell-completion-setup-code
(python-shell--encode-string input))
- process))))
- (when (> (length completions) 2)
- (split-string completions
- "^'\\|^\"\\|;\\|'$\\|\"$" t)))))
+ process))))
(defvar-local python-shell--capf-cache nil
"Variable to store cached completions and invalidation keys.")
@@ -4687,21 +4793,21 @@ python-shell-completion-at-point
;; Working on a shell buffer: use prompt end.
(cdr (python-util-comint-last-prompt))
(line-beginning-position)))
- (import-statement
- (when (string-match-p
- (rx (* space) word-start (or "from" "import") word-end space)
- (buffer-substring-no-properties line-start (point)))
- (buffer-substring-no-properties line-start (point))))
+ (no-delims (with-current-buffer (process-buffer process)
+ (if python-shell-completion-native-enable
+ (string= python-shell-readline-completer-delims "")
+ (string-match-p "ipython[23]?\\'" python-shell-interpreter))))
(start
(if (< (point) line-start)
(point)
(save-excursion
- (if (not (re-search-backward
- (python-rx
- (or whitespace open-paren close-paren
- string-delimiter simple-operator))
- line-start
- t 1))
+ (if (or no-delims
+ (not (re-search-backward
+ (python-rx
+ (or whitespace open-paren close-paren
+ string-delimiter simple-operator))
+ line-start
+ t 1)))
line-start
(forward-char (length (match-string-no-properties 0)))
(point)))))
@@ -4741,18 +4847,44 @@ python-shell-completion-at-point
(t #'python-shell-completion-native-get-completions))))
(prev-prompt (car python-shell--capf-cache))
(re (or (cadr python-shell--capf-cache) regexp-unmatchable))
- (prefix (buffer-substring-no-properties start end)))
+ (prefix (buffer-substring-no-properties start end))
+ (prefix-offset 0))
;; To invalidate the cache, we check if the prompt position or the
;; completion prefix changed.
(unless (and (equal prev-prompt (car prompt-boundaries))
- (string-match re prefix))
+ (string-match re prefix)
+ (setq prefix-offset (- (length prefix) (match-end 1))))
(setq python-shell--capf-cache
`(,(car prompt-boundaries)
,(if (string-empty-p prefix)
regexp-unmatchable
- (concat "\\`" (regexp-quote prefix) "\\(?:\\sw\\|\\s_\\)*\\'"))
- ,@(funcall completion-fn process (or import-statement prefix)))))
- (list start end (cddr python-shell--capf-cache))))
+ (concat "\\`\\(" (regexp-quote prefix) "\\)\\(?:\\sw\\|\\s_\\)*\\'"))
+ ,@(funcall completion-fn process prefix))))
+ (let ((cands (cddr python-shell--capf-cache)))
+ (cond
+ ((stringp (car cands))
+ (if no-delims
+ ;; Reduce completion candidates due to long prefix.
+ (if-let ((Lp (length prefix))
+ ((string-match "\\(\\sw\\|\\s_\\)+\\'" prefix))
+ (L (match-beginning 0)))
+ (list (+ start L) end (mapcar (lambda (s) (substring s L)) cands))
+ (list end end (mapcar (lambda (s) (substring s Lp)) cands)))
+ (list start end cands)))
+ ;; python-shell-completion(-native)-get-completions may produce
+ ;; a list of (text start end type signature) for completion.
+ ;; See `python-shell-readline-ipython-setup-code' and
+ ;; `python-shell-completion-setup-code'.
+ ((consp (car cands))
+ (list (+ start (nth 1 (car cands)))
+ ;; Candidates may be cached, so the end position should
+ ;; be adjusted according to current completion prefix.
+ (+ start (nth 2 (car cands)) prefix-offset)
+ cands
+ :annotation-function
+ (lambda (c) (concat " " (nth 3 (assoc c cands))))
+ :company-docsig
+ (lambda (c) (nth 4 (assoc c cands)))))))))
(define-obsolete-function-alias
'python-shell-completion-complete-at-point
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 97ffd5fe20f..975baf9e576 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4787,6 +4787,72 @@ python-shell-completion-at-point-native-1
(end-of-line 0)
(should-not (nth 2 (python-shell-completion-at-point))))))
+(defun python-shell-completion-module ()
+ "Check if modules can be completed in Python shell."
+ (insert "import datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "import datetime"))
+ (kill-line)
+ (insert "from datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime"))
+ (end-of-line)
+ (insert " import timed")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime import timedelta"))
+ (kill-line))
+
+(defun python-shell-completion-parameters ()
+ "Check if parameters can be completed in Python shell."
+ (insert "import re")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "re.split('b', 'abc', maxs")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit="
+ (buffer-substring (line-beginning-position) (point))))
+ (insert "0, ")
+ (should (python-shell-completion-at-point))
+ ;; Test if cache is used.
+ (cl-letf (((symbol-function 'python-shell-completion-get-completions)
+ 'ignore)
+ ((symbol-function 'python-shell-completion-native-get-completions)
+ 'ignore))
+ (insert "fla")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit=0, flags="
+ (buffer-substring (line-beginning-position) (point)))))
+ (beginning-of-line)
+ (kill-line))
+
+(ert-deftest python-shell-completion-at-point-jedi-completer ()
+ "Check if Python shell completion works with Jedi."
+ (skip-unless (executable-find python-tests-shell-interpreter))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-shell-completion-module)
+ (python-shell-completion-parameters))))
+
+(ert-deftest python-shell-completion-at-point-ipython ()
+ "Check if Python shell completion works for IPython."
+ (let ((python-shell-interpreter "ipython")
+ (python-shell-interpreter-args "-i --simple-prompt"))
+ (skip-unless (executable-find python-shell-interpreter))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-off)
+ (python-shell-completion-module)
+ (python-shell-completion-parameters)
+ (python-shell-completion-native-turn-on)
+ (python-shell-completion-module)
+ (python-shell-completion-parameters)))))
\f
;;; PDB Track integration
--
2.25.1
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-01-18 4:48 bug#68559: [PATCH] Improve Python shell completion Liu Hui
@ 2024-01-18 6:39 ` Eli Zaretskii
2024-01-21 9:34 ` kobarity
2024-02-15 14:43 ` Mattias Engdegård
1 sibling, 1 reply; 68+ messages in thread
From: Eli Zaretskii @ 2024-01-18 6:39 UTC (permalink / raw)
To: Liu Hui, kobarity; +Cc: 68559
> From: Liu Hui <liuhui1610@gmail.com>
> Date: Thu, 18 Jan 2024 12:48:37 +0800
>
> Currently python-shell-completion-at-point doesn't respect the
> delimiter setting of readline completer and always split the text,
> resulting in poor completions when completers that don't need word
> splitting, e.g. jedi[1], is used. This patch fixes the problem.
>
> Meanwhile, this patch adds an option 'python-shell-readline-completer'
> to make Python shell provide better completion experience (e.g.
> completing function parameters, dictionary keys) by default if users
> are using vanilla Python interpreter with jedi or using IPython, where
> the completion support for IPython has been enhanced (e.g. showing
> type annotations and function signatures).
>
> BTW, I think it may make sense to move Python snippets to a separate
> file to make them easier to be maintained.
Thanks. Let's see what others think about this (I don't use Python
enough to have an opinion that matters).
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-01-18 6:39 ` Eli Zaretskii
@ 2024-01-21 9:34 ` kobarity
2024-01-23 11:31 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: kobarity @ 2024-01-21 9:34 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, 68559
Eli Zaretskii wrote:
>
> > From: Liu Hui <liuhui1610@gmail.com>
> > Date: Thu, 18 Jan 2024 12:48:37 +0800
> >
> > Currently python-shell-completion-at-point doesn't respect the
> > delimiter setting of readline completer and always split the text,
> > resulting in poor completions when completers that don't need word
> > splitting, e.g. jedi[1], is used. This patch fixes the problem.
> >
> > Meanwhile, this patch adds an option 'python-shell-readline-completer'
> > to make Python shell provide better completion experience (e.g.
> > completing function parameters, dictionary keys) by default if users
> > are using vanilla Python interpreter with jedi or using IPython, where
> > the completion support for IPython has been enhanced (e.g. showing
> > type annotations and function signatures).
> >
> > BTW, I think it may make sense to move Python snippets to a separate
> > file to make them easier to be maintained.
>
> Thanks. Let's see what others think about this (I don't use Python
> enough to have an opinion that matters).
Thanks for the patch. I have tried it and found it to be a very
powerful enhancement.
Is it possible to allow completion of keyword arguments in a
multi-line function call like the following?
#+begin_src python
re.split(
'b',
'abc',
maxs
#+end_src
I am not sure if Python snippets should be separated. Do other
language supports keep them separate?
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-01-21 9:34 ` kobarity
@ 2024-01-23 11:31 ` Liu Hui
2024-01-23 14:15 ` kobarity
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-01-23 11:31 UTC (permalink / raw)
To: kobarity; +Cc: Eli Zaretskii, 68559
[-- Attachment #1: Type: text/plain, Size: 754 bytes --]
kobarity <kobarity@gmail.com> 于2024年1月21日周日 17:34写道：
> Is it possible to allow completion of keyword arguments in a
> multi-line function call like the following?
>
> #+begin_src python
> re.split(
> 'b',
> 'abc',
> maxs
> #+end_src
I have added experimental support in the attached patch, can you test
it please?
> I am not sure if Python snippets should be separated. Do other
> language supports keep them separate?
I think it allows to reduce code duplication (e.g. the IPython support
code existing in both python-shell-completion-setup-code and
python-shell-readline-ipython-setup-code) and make code more simple.
It seems other languages do not contain as many code snippets as
python.el.
[-- Attachment #2: 0001-Improve-Python-shell-completion.patch --]
[-- Type: text/x-patch, Size: 24720 bytes --]
From 17be2968b6c53dd9fe323b8c28c3ce45efc63318 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Thu, 18 Jan 2024 12:00:00 +0800
Subject: [PATCH] Improve Python shell completion
* lisp/progmodes/python.el (python-shell-completion-setup-code): Fix
the completion code of IPython. Change the return value to JSON
string and ...
(python-shell-completion-get-completions): ... simplify parsing.
(inferior-python-mode): Update docstring.
(python-shell-readline-completer): New option.
(python-shell-readline-completer-delims)
(python-shell-readline-jedi-setup-code)
(python-shell-readline-ipython-setup-code): New variables.
(python-shell-completion-native-setup): Setup a suitable readline
completer and set the completer delimiter.
(python-shell-completion-native-get-completions): Convert output
string to completions properly.
(python-shell--get-multiline-input):
(python-shell--extra-completion-context): New functions.
(python-shell-completion-at-point): Send text beginning from the line
start if the completion backend does not need word splitting. Remove
the detection of import statement because it is not needed anymore.
Create proper completion table based on completions returned from
different backends.
* test/lisp/progmodes/python-tests.el (python-shell-completion-module):
(python-shell-completion-parameters):
(python-shell-completion-multi-line-function-call): New functions.
(python-shell-completion-at-point-jedi-completer):
(python-shell-completion-at-point-ipython): New tests.
* etc/NEWS: Announce the change.
---
etc/NEWS | 6 +
lisp/progmodes/python.el | 292 +++++++++++++++++++++++-----
test/lisp/progmodes/python-tests.el | 78 ++++++++
3 files changed, 324 insertions(+), 52 deletions(-)
diff --git a/etc/NEWS b/etc/NEWS
index 03b8c3b517a..6fd337727c5 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1029,6 +1029,12 @@ instead of:
This allows the user to specify command line arguments to the non
interactive Python interpreter specified by 'python-interpreter'.
+*** New user option 'python-shell-readline-completer'.
+This allows the user to specify the readline completer used for Python
+shell completion. The default is 'auto', which means a suitable
+completer will be configured automatically according to the Python
+interpreter.
+
** use-package
+++
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index ff799e1e662..f54b1b29104 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -128,9 +128,9 @@
;; receiving escape sequences (with some limitations, i.e. completion
;; in blocks does not work). The code executed for the "fallback"
;; completion can be found in `python-shell-completion-setup-code' and
-;; `python-shell-completion-string-code' variables. Their default
-;; values enable completion for both CPython and IPython, and probably
-;; any readline based shell (it's known to work with PyPy). If your
+;; `python-shell-completion-get-completions'. Their default values
+;; enable completion for both CPython and IPython, and probably any
+;; readline based shell (it's known to work with PyPy). If your
;; Python installation lacks readline (like CPython for Windows),
;; installing pyreadline (URL `https://ipython.org/pyreadline.html')
;; should suffice. To troubleshoot why you are not getting any
@@ -3601,7 +3601,6 @@ inferior-python-mode
`python-shell-prompt-block-regexp',
`python-shell-font-lock-enable',
`python-shell-completion-setup-code',
-`python-shell-completion-string-code',
`python-eldoc-setup-code',
`python-ffap-setup-code' can
customize this mode for different Python interpreters.
@@ -4319,8 +4318,9 @@ python-shell-completion-setup-code
completions = []
completer = None
+ import json
try:
- import readline
+ import readline, re
try:
import __builtin__
@@ -4331,16 +4331,29 @@ python-shell-completion-setup-code
is_ipython = ('__IPYTHON__' in builtins or
'__IPYTHON__active' in builtins)
- splits = text.split()
- is_module = splits and splits[0] in ('from', 'import')
-
- if is_ipython and is_module:
- from IPython.core.completerlib import module_completion
- completions = module_completion(text.strip())
- elif is_ipython and '__IP' in builtins:
- completions = __IP.complete(text)
- elif is_ipython and 'get_ipython' in builtins:
- completions = get_ipython().Completer.all_completions(text)
+
+ if is_ipython and 'get_ipython' in builtins:
+ def filter_c(prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ import IPython
+ try:
+ if IPython.version_info[0] >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ completions = [
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if filter_c(text, c.text)]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ completions = [text + m[len(part):] for m in matches if filter_c(text, m)]
+ except:
+ pass
else:
# Try to reuse current completer.
completer = readline.get_completer()
@@ -4363,7 +4376,7 @@ python-shell-completion-setup-code
finally:
if getattr(completer, 'PYTHON_EL_WRAPPED', False):
completer.print_mode = True
- return completions"
+ return json.dumps(completions)"
"Code used to setup completion in inferior Python processes."
:type 'string)
@@ -4404,6 +4417,79 @@ python-shell-completion-native-try-output-timeout
:version "25.1"
:type 'float)
+(defcustom python-shell-readline-completer 'auto
+ "The readline completer used for Python shell completion.
+If the value is non-nil, Python shell will setup the readline
+completer unless it has been set elsewhere (e.g. in the
+PYTHONSTARTUP file). Below are possible values:
+- `auto': the completer is determined according to the
+interpreter. Specifically, the IPython completer, defined in
+`python-shell-readline-ipython-setup-code', is used when the
+interpreter is ipython, otherwise the Jedi completer is used.
+- a string: Python code to setup the readline. It should define
+the function `__PYTHON_EL_setup_readline_completer'. See
+`python-shell-readline-jedi-setup-code' for reference.
+- `nil': Python shell will do nothing.
+
+In any case, if the completer is not set successfully in the end,
+fallback to the built-in rlcompleter."
+ :type '(choice (const :tag "Automatic" auto)
+ (const :tag "No configuration" nil)
+ (string :tag "Python setup code"))
+ :version "30.1")
+
+(defvar python-shell-readline-completer-delims nil
+ "Word delimiters used by the readline completer.
+It is automatically set by Python shell.")
+
+(defconst python-shell-readline-jedi-setup-code
+ "
+def __PYTHON_EL_setup_readline_completer():
+ from jedi.utils import setup_readline
+ setup_readline()"
+ "Code used to setup readline completer with Jedi.")
+
+(defconst python-shell-readline-ipython-setup-code
+ "
+def __PYTHON_EL_setup_readline_completer():
+ import readline, re, json, IPython
+
+ class __ipython_RL:
+ def __init__(self, v):
+ self.version = v
+
+ def filter(self, prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ def complete(self, text, state):
+ if state == 0:
+ try:
+ if self.version >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ self.matches = [json.dumps([
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if self.filter(text, c.text)])]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ self.matches = [text + m[len(part):] for m in matches
+ if self.filter(text, m)]
+ except Exception:
+ pass
+ try:
+ return self.matches[state]
+ except IndexError:
+ return None
+
+ readline.set_completer(__ipython_RL(IPython.version_info[0]).complete)
+ readline.set_completer_delims('')"
+ "Code used to setup readline completer for IPython.")
+
(defvar python-shell-completion-native-redirect-buffer
" *Python completions redirect*"
"Buffer to be used to redirect output of readline commands.")
@@ -4427,7 +4513,20 @@ python-shell-completion-native-try
(defun python-shell-completion-native-setup ()
"Try to setup native completion, return non-nil on success."
(let* ((process (python-shell-get-process))
- (output (python-shell-send-string-no-output "
+ (completer (pcase python-shell-readline-completer
+ ('auto
+ (if (string-match-p "ipython[23]?\\'" python-shell-interpreter)
+ python-shell-readline-ipython-setup-code
+ python-shell-readline-jedi-setup-code))
+ ((pred stringp) python-shell-readline-completer)
+ (_ "")))
+ (output (python-shell-send-string-no-output
+ (concat "
+try:
+ del __PYTHON_EL_setup_readline_completer
+except:
+ pass
+" completer "
def __PYTHON_EL_native_completion_setup():
try:
import readline
@@ -4500,11 +4599,23 @@ python-shell-completion-native-setup
else:
return completion
+ def is_rlcompleter(completer):
+ try:
+ if completer.__self__.__module__ == 'rlcompleter':
+ return True
+ else:
+ return False
+ except Exception:
+ return False
+
completer = readline.get_completer()
- if not completer:
- # Used as last resort to avoid breaking customizations.
- import rlcompleter
+ if not completer or is_rlcompleter(completer):
+ try:
+ __PYTHON_EL_setup_readline_completer()
+ except:
+ # Used as last resort to avoid breaking customizations.
+ import rlcompleter
completer = readline.get_completer()
if completer and not getattr(completer, 'PYTHON_EL_WRAPPED', False):
@@ -4539,9 +4650,13 @@ python-shell-completion-native-setup
print ('python.el: native completion setup failed, %s: %s'
% sys.exc_info()[:2])
-__PYTHON_EL_native_completion_setup()" process)))
+__PYTHON_EL_native_completion_setup()") process)))
(when (string-match-p "python\\.el: native completion setup loaded"
output)
+ (setq-local python-shell-readline-completer-delims
+ (string-trim-right
+ (python-shell-send-string-no-output
+ "import readline; print(readline.get_completer_delims())")))
(python-shell-completion-native-try))))
(defun python-shell-completion-native-turn-off (&optional msg)
@@ -4609,6 +4724,8 @@ python-shell-completion-native-get-completions
(let* ((original-filter-fn (process-filter process))
(redirect-buffer (get-buffer-create
python-shell-completion-native-redirect-buffer))
+ (sep (if (string= python-shell-readline-completer-delims "")
+ "[\n\r]+" "[ \f\t\n\r\v()]+"))
(trigger "\t")
(new-input (concat input trigger))
(input-length
@@ -4651,32 +4768,63 @@ python-shell-completion-native-get-completions
process python-shell-completion-native-output-timeout
comint-redirect-finished-regexp)
(re-search-backward "0__dummy_completion__" nil t)
- (cl-remove-duplicates
- (split-string
- (buffer-substring-no-properties
- (line-beginning-position) (point-min))
- "[ \f\t\n\r\v()]+" t)
- :test #'string=))))
+ (let ((str (buffer-substring-no-properties
+ (line-beginning-position) (point-min))))
+ (if (string= "[" (substring str 0 1))
+ (condition-case nil
+ (python--parse-json-array str)
+ (t (cl-remove-duplicates (split-string str sep t)
+ :test #'string=)))
+ (cl-remove-duplicates (split-string str sep t)
+ :test #'string=))))))
(set-process-filter process original-filter-fn)))))
(defun python-shell-completion-get-completions (process input)
"Get completions of INPUT using PROCESS."
(with-current-buffer (process-buffer process)
- (let ((completions
- (python-util-strip-string
- (python-shell-send-string-no-output
- (format
- "%s\nprint(';'.join(__PYTHON_EL_get_completions(%s)))"
+ (python--parse-json-array
+ (python-shell-send-string-no-output
+ (format "%s\nprint(__PYTHON_EL_get_completions(%s))"
python-shell-completion-setup-code
(python-shell--encode-string input))
- process))))
- (when (> (length completions) 2)
- (split-string completions
- "^'\\|^\"\\|;\\|'$\\|\"$" t)))))
+ process))))
(defvar-local python-shell--capf-cache nil
"Variable to store cached completions and invalidation keys.")
+(defun python-shell--get-multiline-input ()
+ "Return lines at a multi-line input in Python shell."
+ (save-excursion
+ (let ((p (point)) lines)
+ (when (progn
+ (beginning-of-line)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) p) lines)
+ (while (progn (comint-previous-prompt 1)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ lines)))
+
+(defun python-shell--extra-completion-context ()
+ "Return extra completion context of current line in Python shell.
+It currently supports multi-line function call."
+ (let ((lines (python-shell--get-multiline-input)) beg)
+ (when (not (zerop (length lines)))
+ (with-temp-buffer
+ (delay-mode-hooks
+ (insert (string-join lines "\n"))
+ (python-mode)
+ (setq beg (pos-bol))
+ (python-nav-up-list -1)
+ (when (< (point) beg)
+ (while (re-search-backward
+ (python-rx symbol-name) (pos-bol) t))
+ (string-replace
+ "\n" "" (buffer-substring-no-properties
+ (point)
+ (progn (goto-char (point-max)) (pos-eol 0))))))))))
+
(defun python-shell-completion-at-point (&optional process)
"Function for `completion-at-point-functions' in `inferior-python-mode'.
Optional argument PROCESS forces completions to be retrieved
@@ -4687,21 +4835,21 @@ python-shell-completion-at-point
;; Working on a shell buffer: use prompt end.
(cdr (python-util-comint-last-prompt))
(line-beginning-position)))
- (import-statement
- (when (string-match-p
- (rx (* space) word-start (or "from" "import") word-end space)
- (buffer-substring-no-properties line-start (point)))
- (buffer-substring-no-properties line-start (point))))
+ (no-delims (with-current-buffer (process-buffer process)
+ (if python-shell-completion-native-enable
+ (string= python-shell-readline-completer-delims "")
+ (string-match-p "ipython[23]?\\'" python-shell-interpreter))))
(start
(if (< (point) line-start)
(point)
(save-excursion
- (if (not (re-search-backward
- (python-rx
- (or whitespace open-paren close-paren
- string-delimiter simple-operator))
- line-start
- t 1))
+ (if (or no-delims
+ (not (re-search-backward
+ (python-rx
+ (or whitespace open-paren close-paren
+ string-delimiter simple-operator))
+ line-start
+ t 1)))
line-start
(forward-char (length (match-string-no-properties 0)))
(point)))))
@@ -4741,18 +4889,58 @@ python-shell-completion-at-point
(t #'python-shell-completion-native-get-completions))))
(prev-prompt (car python-shell--capf-cache))
(re (or (cadr python-shell--capf-cache) regexp-unmatchable))
- (prefix (buffer-substring-no-properties start end)))
+ (prefix (buffer-substring-no-properties start end))
+ (prefix-offset 0)
+ ;; Send extra context for cases like completing function
+ ;; parameters for multi-line function call.
+ (extra-context (and no-delims
+ (python-shell--extra-completion-context)))
+ (extra-offset (length extra-context)))
+ (unless (zerop extra-offset)
+ (setq prefix (concat extra-context prefix)))
;; To invalidate the cache, we check if the prompt position or the
;; completion prefix changed.
(unless (and (equal prev-prompt (car prompt-boundaries))
- (string-match re prefix))
+ (string-match re prefix)
+ (setq prefix-offset (- (length prefix) (match-end 1))))
(setq python-shell--capf-cache
`(,(car prompt-boundaries)
,(if (string-empty-p prefix)
regexp-unmatchable
- (concat "\\`" (regexp-quote prefix) "\\(?:\\sw\\|\\s_\\)*\\'"))
- ,@(funcall completion-fn process (or import-statement prefix)))))
- (list start end (cddr python-shell--capf-cache))))
+ (concat "\\`\\(" (regexp-quote prefix) "\\)\\(?:\\sw\\|\\s_\\)*\\'"))
+ ,@(funcall completion-fn process prefix))))
+ (let ((cands (cddr python-shell--capf-cache)))
+ (cond
+ ((stringp (car cands))
+ (if no-delims
+ ;; Reduce completion candidates due to long prefix.
+ (if-let ((Lp (length prefix))
+ ((string-match "\\(\\sw\\|\\s_\\)+\\'" prefix))
+ (L (match-beginning 0)))
+ ;; If extra-offset is not zero:
+ ;; start end
+ ;; o------------------o---------o-------o
+ ;; |<- extra-offset ->|
+ ;; |<----------- L ------------>|
+ ;; new-start
+ (list (+ start L (- extra-offset)) end
+ (mapcar (lambda (s) (substring s L)) cands))
+ (list end end (mapcar (lambda (s) (substring s Lp)) cands)))
+ (list start end cands)))
+ ;; python-shell-completion(-native)-get-completions may produce
+ ;; a list of (text start end type signature) for completion.
+ ;; See `python-shell-readline-ipython-setup-code' and
+ ;; `python-shell-completion-setup-code'.
+ ((consp (car cands))
+ (list (+ start (nth 1 (car cands)) (- extra-offset))
+ ;; Candidates may be cached, so the end position should
+ ;; be adjusted according to current completion prefix.
+ (+ start (nth 2 (car cands)) (- extra-offset) prefix-offset)
+ cands
+ :annotation-function
+ (lambda (c) (concat " " (nth 3 (assoc c cands))))
+ :company-docsig
+ (lambda (c) (nth 4 (assoc c cands)))))))))
(define-obsolete-function-alias
'python-shell-completion-complete-at-point
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 97ffd5fe20f..f6926c421fe 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4787,6 +4787,84 @@ python-shell-completion-at-point-native-1
(end-of-line 0)
(should-not (nth 2 (python-shell-completion-at-point))))))
+(defun python-shell-completion-module ()
+ "Check if modules can be completed in Python shell."
+ (insert "import datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "import datetime"))
+ (kill-line)
+ (insert "from datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime"))
+ (end-of-line)
+ (insert " import timed")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime import timedelta"))
+ (kill-line))
+
+(defun python-shell-completion-parameters ()
+ "Check if parameters can be completed in Python shell."
+ (insert "import re")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "re.split('b', 'abc', maxs")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit="
+ (buffer-substring (line-beginning-position) (point))))
+ (insert "0, ")
+ (should (python-shell-completion-at-point))
+ ;; Test if cache is used.
+ (cl-letf (((symbol-function 'python-shell-completion-get-completions)
+ 'ignore)
+ ((symbol-function 'python-shell-completion-native-get-completions)
+ 'ignore))
+ (insert "fla")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit=0, flags="
+ (buffer-substring (line-beginning-position) (point)))))
+ (beginning-of-line)
+ (kill-line))
+
+(defun python-shell-completion-multi-line-function-call ()
+ "Check if parameters can be completed in multi-line function call."
+ (insert "re.split('b', 'abc',")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "maxs")
+ (completion-at-point)
+ (should (string= "maxsplit="
+ (buffer-substring (line-beginning-position) (point)))))
+
+(ert-deftest python-shell-completion-at-point-jedi-completer ()
+ "Check if Python shell completion works with Jedi."
+ (skip-unless (executable-find python-tests-shell-interpreter))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-shell-completion-module)
+ (python-shell-completion-parameters)
+ (python-shell-completion-multi-line-function-call))))
+
+(ert-deftest python-shell-completion-at-point-ipython ()
+ "Check if Python shell completion works for IPython."
+ (let ((python-shell-interpreter "ipython")
+ (python-shell-interpreter-args "-i --simple-prompt"))
+ (skip-unless (executable-find python-shell-interpreter))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-off)
+ (python-shell-completion-module)
+ (python-shell-completion-parameters)
+ (python-shell-completion-native-turn-on)
+ (python-shell-completion-module)
+ (python-shell-completion-parameters)
+ (python-shell-completion-multi-line-function-call)))))
\f
;;; PDB Track integration
--
2.25.1
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-01-23 11:31 ` Liu Hui
@ 2024-01-23 14:15 ` kobarity
2024-01-24 10:07 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: kobarity @ 2024-01-23 14:15 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, 68559
Liu Hui wrote:
> kobarity <kobarity@gmail.com> 于2024年1月21日周日 17:34写道：
>
> > Is it possible to allow completion of keyword arguments in a
> > multi-line function call like the following?
> >
> > #+begin_src python
> > re.split(
> > 'b',
> > 'abc',
> > maxs
> > #+end_src
>
> I have added experimental support in the attached patch, can you test
> it please?
Thank you. I tried the new patch and confirmed that the Python Shell
buffer allows keyword argument completion on multi-line function
calls. However, my expectation is that it can be done in Python
buffer as well (after calling `python-shell-send-buffer'). Is this
also possible?
> > I am not sure if Python snippets should be separated. Do other
> > language supports keep them separate?
>
> I think it allows to reduce code duplication (e.g. the IPython support
> code existing in both python-shell-completion-setup-code and
> python-shell-readline-ipython-setup-code) and make code more simple.
> It seems other languages do not contain as many code snippets as
> python.el.
Sorry, I misunderstood that snippets meant skeletons. I agree that
python.el has long Python codes, which is not easy to maintain. Are
you proposing to separate the snippets into .py file? If it's
acceptable from the Emacs' source code management perspective, I think
that is a good approach.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-01-23 14:15 ` kobarity
@ 2024-01-24 10:07 ` Liu Hui
2024-01-25 15:38 ` kobarity
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-01-24 10:07 UTC (permalink / raw)
To: kobarity; +Cc: Eli Zaretskii, 68559
[-- Attachment #1: Type: text/plain, Size: 1438 bytes --]
kobarity <kobarity@gmail.com> 于2024年1月23日周二 22:15写道：
> Thank you. I tried the new patch and confirmed that the Python Shell
> buffer allows keyword argument completion on multi-line function
> calls. However, my expectation is that it can be done in Python
> buffer as well (after calling `python-shell-send-buffer'). Is this
> also possible?
Thank you for the feedback. I have updated the patch to add support in
Python buffer.
> > > I am not sure if Python snippets should be separated. Do other
> > > language supports keep them separate?
> >
> > I think it allows to reduce code duplication (e.g. the IPython support
> > code existing in both python-shell-completion-setup-code and
> > python-shell-readline-ipython-setup-code) and make code more simple.
> > It seems other languages do not contain as many code snippets as
> > python.el.
>
> Sorry, I misunderstood that snippets meant skeletons. I agree that
> python.el has long Python codes, which is not easy to maintain. Are
> you proposing to separate the snippets into .py file? If it's
> acceptable from the Emacs' source code management perspective, I think
> that is a good approach.
Yes, I'd like to at least put some flags and functions shared by
python-shell-completion-native-setup and python-*-setup-code in a
separate .py file. On the other hand, they are not changed frequently,
so it is not in urgent need.
[-- Attachment #2: 0001-Improve-Python-shell-completion-bug-68559.patch --]
[-- Type: text/x-patch, Size: 24987 bytes --]
From abefec11d52875a76703c25c0998b5d775442a7a Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Thu, 18 Jan 2024 12:00:00 +0800
Subject: [PATCH] Improve Python shell completion (bug#68559)
* lisp/progmodes/python.el (python-shell-completion-setup-code): Fix
the completion code of IPython. Change the return value to JSON
string and ...
(python-shell-completion-get-completions): ... simplify parsing.
(inferior-python-mode): Update docstring.
(python-shell-readline-completer): New option.
(python-shell-readline-completer-delims)
(python-shell--readline-jedi-setup-code)
(python-shell--readline-ipython-setup-code): New variables.
(python-shell-completion-native-setup): Setup a suitable readline
completer and set the completer delimiter.
(python-shell-completion-native-get-completions): Convert output
string to completions properly.
(python-shell--get-multiline-input)
(python-shell--extra-completion-context)
(python-shell-completion-extra-context): New functions.
(python-shell-completion-at-point): Send text beginning from the line
start if the completion backend does not need word splitting. Remove
the detection of import statement because it is not needed anymore.
Create proper completion table based on completions returned from
different backends.
* test/lisp/progmodes/python-tests.el (python-tests--completion-module):
(python-tests--completion-parameters):
(python-tests--completion-extra-context): New helper functions.
(python-shell-completion-at-point-jedi-completer):
(python-shell-completion-at-point-ipython): New tests.
* etc/NEWS: Announce the change.
---
etc/NEWS | 6 +
lisp/progmodes/python.el | 304 +++++++++++++++++++++++-----
test/lisp/progmodes/python-tests.el | 78 +++++++
3 files changed, 336 insertions(+), 52 deletions(-)
diff --git a/etc/NEWS b/etc/NEWS
index 03b8c3b517a..6fd337727c5 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1029,6 +1029,12 @@ instead of:
This allows the user to specify command line arguments to the non
interactive Python interpreter specified by 'python-interpreter'.
+*** New user option 'python-shell-readline-completer'.
+This allows the user to specify the readline completer used for Python
+shell completion. The default is 'auto', which means a suitable
+completer will be configured automatically according to the Python
+interpreter.
+
** use-package
+++
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index ff799e1e662..63445ab0e50 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -128,9 +128,9 @@
;; receiving escape sequences (with some limitations, i.e. completion
;; in blocks does not work). The code executed for the "fallback"
;; completion can be found in `python-shell-completion-setup-code' and
-;; `python-shell-completion-string-code' variables. Their default
-;; values enable completion for both CPython and IPython, and probably
-;; any readline based shell (it's known to work with PyPy). If your
+;; `python-shell-completion-get-completions'. Their default values
+;; enable completion for both CPython and IPython, and probably any
+;; readline based shell (it's known to work with PyPy). If your
;; Python installation lacks readline (like CPython for Windows),
;; installing pyreadline (URL `https://ipython.org/pyreadline.html')
;; should suffice. To troubleshoot why you are not getting any
@@ -3601,7 +3601,6 @@ inferior-python-mode
`python-shell-prompt-block-regexp',
`python-shell-font-lock-enable',
`python-shell-completion-setup-code',
-`python-shell-completion-string-code',
`python-eldoc-setup-code',
`python-ffap-setup-code' can
customize this mode for different Python interpreters.
@@ -4319,8 +4318,9 @@ python-shell-completion-setup-code
completions = []
completer = None
+ import json
try:
- import readline
+ import readline, re
try:
import __builtin__
@@ -4331,16 +4331,29 @@ python-shell-completion-setup-code
is_ipython = ('__IPYTHON__' in builtins or
'__IPYTHON__active' in builtins)
- splits = text.split()
- is_module = splits and splits[0] in ('from', 'import')
-
- if is_ipython and is_module:
- from IPython.core.completerlib import module_completion
- completions = module_completion(text.strip())
- elif is_ipython and '__IP' in builtins:
- completions = __IP.complete(text)
- elif is_ipython and 'get_ipython' in builtins:
- completions = get_ipython().Completer.all_completions(text)
+
+ if is_ipython and 'get_ipython' in builtins:
+ def filter_c(prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ import IPython
+ try:
+ if IPython.version_info[0] >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ completions = [
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if filter_c(text, c.text)]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ completions = [text + m[len(part):] for m in matches if filter_c(text, m)]
+ except:
+ pass
else:
# Try to reuse current completer.
completer = readline.get_completer()
@@ -4363,7 +4376,7 @@ python-shell-completion-setup-code
finally:
if getattr(completer, 'PYTHON_EL_WRAPPED', False):
completer.print_mode = True
- return completions"
+ return json.dumps(completions)"
"Code used to setup completion in inferior Python processes."
:type 'string)
@@ -4404,6 +4417,78 @@ python-shell-completion-native-try-output-timeout
:version "25.1"
:type 'float)
+(defcustom python-shell-readline-completer 'auto
+ "The readline completer used for Python shell completion.
+If the value is non-nil, Python shell will setup the readline
+completer unless it has been set elsewhere (e.g. in the
+PYTHONSTARTUP file). Below are possible values:
+- `auto': the completer is determined according to the
+interpreter. Specifically, the IPython completer, defined in
+`python-shell--readline-ipython-setup-code', is used when the
+interpreter is ipython, otherwise the Jedi completer is used.
+- a string: Python code to setup the readline. See
+`python-shell--readline-jedi-setup-code' for reference.
+- `nil': Python shell will do nothing.
+
+In any case, if the completer is not set successfully in the end,
+fallback to the built-in rlcompleter."
+ :type '(choice (const :tag "Automatic" auto)
+ (const :tag "No configuration" nil)
+ (string :tag "Python setup code"))
+ :version "30.1")
+
+(defvar python-shell-readline-completer-delims nil
+ "Word delimiters used by the readline completer.
+It is automatically set by Python shell.")
+
+(defconst python-shell--readline-jedi-setup-code
+ "
+def __PYTHON_EL_setup_readline_completer():
+ from jedi.utils import setup_readline
+ setup_readline()"
+ "Code used to setup readline completer with Jedi.")
+
+(defconst python-shell--readline-ipython-setup-code
+ "
+def __PYTHON_EL_setup_readline_completer():
+ import readline, re, json, IPython
+
+ class __ipython_RL:
+ def __init__(self, v):
+ self.version = v
+
+ def filter(self, prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ def complete(self, text, state):
+ if state == 0:
+ try:
+ if self.version >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ self.matches = [json.dumps([
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if self.filter(text, c.text)])]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ self.matches = [text + m[len(part):] for m in matches
+ if self.filter(text, m)]
+ except Exception:
+ pass
+ try:
+ return self.matches[state]
+ except IndexError:
+ return None
+
+ readline.set_completer(__ipython_RL(IPython.version_info[0]).complete)
+ readline.set_completer_delims('')"
+ "Code used to setup readline completer for IPython.")
+
(defvar python-shell-completion-native-redirect-buffer
" *Python completions redirect*"
"Buffer to be used to redirect output of readline commands.")
@@ -4427,7 +4512,20 @@ python-shell-completion-native-try
(defun python-shell-completion-native-setup ()
"Try to setup native completion, return non-nil on success."
(let* ((process (python-shell-get-process))
- (output (python-shell-send-string-no-output "
+ (completer (pcase python-shell-readline-completer
+ ('auto
+ (if (string-match-p "ipython[23]?\\'" python-shell-interpreter)
+ python-shell--readline-ipython-setup-code
+ python-shell--readline-jedi-setup-code))
+ ((pred stringp) python-shell-readline-completer)
+ (_ "")))
+ (output (python-shell-send-string-no-output
+ (concat "
+try:
+ del __PYTHON_EL_setup_readline_completer
+except:
+ pass
+" completer "
def __PYTHON_EL_native_completion_setup():
try:
import readline
@@ -4500,11 +4598,23 @@ python-shell-completion-native-setup
else:
return completion
+ def is_rlcompleter(completer):
+ try:
+ if completer.__self__.__module__ == 'rlcompleter':
+ return True
+ else:
+ return False
+ except Exception:
+ return False
+
completer = readline.get_completer()
- if not completer:
- # Used as last resort to avoid breaking customizations.
- import rlcompleter
+ if not completer or is_rlcompleter(completer):
+ try:
+ __PYTHON_EL_setup_readline_completer()
+ except:
+ # Used as last resort to avoid breaking customizations.
+ import rlcompleter
completer = readline.get_completer()
if completer and not getattr(completer, 'PYTHON_EL_WRAPPED', False):
@@ -4539,9 +4649,13 @@ python-shell-completion-native-setup
print ('python.el: native completion setup failed, %s: %s'
% sys.exc_info()[:2])
-__PYTHON_EL_native_completion_setup()" process)))
+__PYTHON_EL_native_completion_setup()") process)))
(when (string-match-p "python\\.el: native completion setup loaded"
output)
+ (setq-local python-shell-readline-completer-delims
+ (string-trim-right
+ (python-shell-send-string-no-output
+ "import readline; print(readline.get_completer_delims())")))
(python-shell-completion-native-try))))
(defun python-shell-completion-native-turn-off (&optional msg)
@@ -4609,6 +4723,8 @@ python-shell-completion-native-get-completions
(let* ((original-filter-fn (process-filter process))
(redirect-buffer (get-buffer-create
python-shell-completion-native-redirect-buffer))
+ (sep (if (string= python-shell-readline-completer-delims "")
+ "[\n\r]+" "[ \f\t\n\r\v()]+"))
(trigger "\t")
(new-input (concat input trigger))
(input-length
@@ -4651,28 +4767,72 @@ python-shell-completion-native-get-completions
process python-shell-completion-native-output-timeout
comint-redirect-finished-regexp)
(re-search-backward "0__dummy_completion__" nil t)
- (cl-remove-duplicates
- (split-string
- (buffer-substring-no-properties
- (line-beginning-position) (point-min))
- "[ \f\t\n\r\v()]+" t)
- :test #'string=))))
+ (let ((str (buffer-substring-no-properties
+ (line-beginning-position) (point-min))))
+ (if (string= "[" (substring str 0 1))
+ (condition-case nil
+ (python--parse-json-array str)
+ (t (cl-remove-duplicates (split-string str sep t)
+ :test #'string=)))
+ (cl-remove-duplicates (split-string str sep t)
+ :test #'string=))))))
(set-process-filter process original-filter-fn)))))
(defun python-shell-completion-get-completions (process input)
"Get completions of INPUT using PROCESS."
(with-current-buffer (process-buffer process)
- (let ((completions
- (python-util-strip-string
- (python-shell-send-string-no-output
- (format
- "%s\nprint(';'.join(__PYTHON_EL_get_completions(%s)))"
+ (python--parse-json-array
+ (python-shell-send-string-no-output
+ (format "%s\nprint(__PYTHON_EL_get_completions(%s))"
python-shell-completion-setup-code
(python-shell--encode-string input))
- process))))
- (when (> (length completions) 2)
- (split-string completions
- "^'\\|^\"\\|;\\|'$\\|\"$" t)))))
+ process))))
+
+(defun python-shell--get-multiline-input ()
+ "Return lines at a multi-line input in Python shell."
+ (save-excursion
+ (let ((p (point)) lines)
+ (when (progn
+ (beginning-of-line)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) p) lines)
+ (while (progn (comint-previous-prompt 1)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ lines)))
+
+(defun python-shell--extra-completion-context ()
+ "Get extra completion context of current input in Python shell."
+ (let ((lines (python-shell--get-multiline-input))
+ (python-indent-guess-indent-offset nil))
+ (when (not (zerop (length lines)))
+ (with-temp-buffer
+ (delay-mode-hooks
+ (insert (string-join lines "\n"))
+ (python-mode)
+ (python-shell-completion-extra-context))))))
+
+(defun python-shell-completion-extra-context (&optional pos)
+ "Get extra completion context at position POS in Python buffer.
+If optional argument POS is nil, use current position.
+
+Readline completers could use current line as the completion
+context, which may be insufficient. In this function, extra
+context (e.g. multi-line function call) is found and reformatted
+as one line, which is required by native completion."
+ (let (bound p)
+ (save-excursion
+ (and pos (goto-char pos))
+ (setq bound (pos-bol))
+ (python-nav-up-list -1)
+ (when (and (< (point) bound)
+ (looking-back
+ (python-rx (+ (or "." symbol-name))) (pos-bol) t))
+ (setq p (match-beginning 0))))
+ (when p
+ (replace-regexp-in-string
+ "\n[ \t]*" "" (buffer-substring-no-properties p (1- bound))))))
(defvar-local python-shell--capf-cache nil
"Variable to store cached completions and invalidation keys.")
@@ -4687,21 +4847,21 @@ python-shell-completion-at-point
;; Working on a shell buffer: use prompt end.
(cdr (python-util-comint-last-prompt))
(line-beginning-position)))
- (import-statement
- (when (string-match-p
- (rx (* space) word-start (or "from" "import") word-end space)
- (buffer-substring-no-properties line-start (point)))
- (buffer-substring-no-properties line-start (point))))
+ (no-delims (with-current-buffer (process-buffer process)
+ (if python-shell-completion-native-enable
+ (string= python-shell-readline-completer-delims "")
+ (string-match-p "ipython[23]?\\'" python-shell-interpreter))))
(start
(if (< (point) line-start)
(point)
(save-excursion
- (if (not (re-search-backward
- (python-rx
- (or whitespace open-paren close-paren
- string-delimiter simple-operator))
- line-start
- t 1))
+ (if (or no-delims
+ (not (re-search-backward
+ (python-rx
+ (or whitespace open-paren close-paren
+ string-delimiter simple-operator))
+ line-start
+ t 1)))
line-start
(forward-char (length (match-string-no-properties 0)))
(point)))))
@@ -4741,18 +4901,58 @@ python-shell-completion-at-point
(t #'python-shell-completion-native-get-completions))))
(prev-prompt (car python-shell--capf-cache))
(re (or (cadr python-shell--capf-cache) regexp-unmatchable))
- (prefix (buffer-substring-no-properties start end)))
+ (prefix (buffer-substring-no-properties start end))
+ (prefix-offset 0)
+ (extra-context (when no-delims
+ (if is-shell-buffer
+ (python-shell--extra-completion-context)
+ (python-shell-completion-extra-context))))
+ (extra-offset (length extra-context)))
+ (unless (zerop extra-offset)
+ (setq prefix (concat extra-context prefix)))
;; To invalidate the cache, we check if the prompt position or the
;; completion prefix changed.
(unless (and (equal prev-prompt (car prompt-boundaries))
- (string-match re prefix))
+ (string-match re prefix)
+ (setq prefix-offset (- (length prefix) (match-end 1))))
(setq python-shell--capf-cache
`(,(car prompt-boundaries)
,(if (string-empty-p prefix)
regexp-unmatchable
- (concat "\\`" (regexp-quote prefix) "\\(?:\\sw\\|\\s_\\)*\\'"))
- ,@(funcall completion-fn process (or import-statement prefix)))))
- (list start end (cddr python-shell--capf-cache))))
+ (concat "\\`\\(" (regexp-quote prefix) "\\)\\(?:\\sw\\|\\s_\\)*\\'"))
+ ,@(funcall completion-fn process prefix))))
+ (let ((cands (cddr python-shell--capf-cache)))
+ (cond
+ ((stringp (car cands))
+ (if no-delims
+ ;; Reduce completion candidates due to long prefix.
+ (if-let ((Lp (length prefix))
+ ((string-match "\\(\\sw\\|\\s_\\)+\\'" prefix))
+ (L (match-beginning 0)))
+ ;; If extra-offset is not zero:
+ ;; start end
+ ;; o------------------o---------o-------o
+ ;; |<- extra-offset ->|
+ ;; |<----------- L ------------>|
+ ;; new-start
+ (list (+ start L (- extra-offset)) end
+ (mapcar (lambda (s) (substring s L)) cands))
+ (list end end (mapcar (lambda (s) (substring s Lp)) cands)))
+ (list start end cands)))
+ ;; python-shell-completion(-native)-get-completions may produce
+ ;; a list of (text start end type signature) for completion.
+ ;; See `python-shell-readline-ipython-setup-code' and
+ ;; `python-shell-completion-setup-code'.
+ ((consp (car cands))
+ (list (+ start (nth 1 (car cands)) (- extra-offset))
+ ;; Candidates may be cached, so the end position should
+ ;; be adjusted according to current completion prefix.
+ (+ start (nth 2 (car cands)) (- extra-offset) prefix-offset)
+ cands
+ :annotation-function
+ (lambda (c) (concat " " (nth 3 (assoc c cands))))
+ :company-docsig
+ (lambda (c) (nth 4 (assoc c cands)))))))))
(define-obsolete-function-alias
'python-shell-completion-complete-at-point
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 97ffd5fe20f..ae23ff8ebe2 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4787,6 +4787,84 @@ python-shell-completion-at-point-native-1
(end-of-line 0)
(should-not (nth 2 (python-shell-completion-at-point))))))
+(defun python-tests--completion-module ()
+ "Check if modules can be completed in Python shell."
+ (insert "import datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "import datetime"))
+ (kill-line)
+ (insert "from datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime"))
+ (end-of-line)
+ (insert " import timed")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime import timedelta"))
+ (kill-line))
+
+(defun python-tests--completion-parameters ()
+ "Check if parameters can be completed in Python shell."
+ (insert "import re")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "re.split('b', 'abc', maxs")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit="
+ (buffer-substring (line-beginning-position) (point))))
+ (insert "0, ")
+ (should (python-shell-completion-at-point))
+ ;; Test if cache is used.
+ (cl-letf (((symbol-function 'python-shell-completion-get-completions)
+ 'ignore)
+ ((symbol-function 'python-shell-completion-native-get-completions)
+ 'ignore))
+ (insert "fla")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit=0, flags="
+ (buffer-substring (line-beginning-position) (point)))))
+ (beginning-of-line)
+ (kill-line))
+
+(defun python-tests--completion-extra-context ()
+ "Check if extra context is used for completion."
+ (insert "re.split('b', 'abc',")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "maxs")
+ (completion-at-point)
+ (should (string= "maxsplit="
+ (buffer-substring (line-beginning-position) (point)))))
+
+(ert-deftest python-shell-completion-at-point-jedi-completer ()
+ "Check if Python shell completion works with Jedi."
+ (skip-unless (executable-find python-tests-shell-interpreter))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context))))
+
+(ert-deftest python-shell-completion-at-point-ipython ()
+ "Check if Python shell completion works for IPython."
+ (let ((python-shell-interpreter "ipython")
+ (python-shell-interpreter-args "-i --simple-prompt"))
+ (skip-unless (executable-find python-shell-interpreter))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-off)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-shell-completion-native-turn-on)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context)))))
\f
;;; PDB Track integration
--
2.25.1
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-01-24 10:07 ` Liu Hui
@ 2024-01-25 15:38 ` kobarity
2024-01-26 10:12 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: kobarity @ 2024-01-25 15:38 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, 68559
Liu Hui wrote:
> kobarity <kobarity@gmail.com> 于2024年1月23日周二 22:15写道：
> > Thank you. I tried the new patch and confirmed that the Python Shell
> > buffer allows keyword argument completion on multi-line function
> > calls. However, my expectation is that it can be done in Python
> > buffer as well (after calling `python-shell-send-buffer'). Is this
> > also possible?
> Thank you for the feedback. I have updated the patch to add support in
> Python buffer.
Thank you. It's working very nice.
I apologize for reporting in the piecemeal fashion, but I have noticed
the following differences from the current completion.
1. Module names cannot be completed in Python buffer.
2. Completion is not working in comments.
I'm not sure if the completion should work in comments, but some
people might want it because the current python-mode allows it in both
Python buffer and Python Shell buffer.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-01-25 15:38 ` kobarity
@ 2024-01-26 10:12 ` Liu Hui
2024-01-28 13:22 ` kobarity
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-01-26 10:12 UTC (permalink / raw)
To: kobarity; +Cc: Eli Zaretskii, 68559
[-- Attachment #1: Type: text/plain, Size: 1567 bytes --]
kobarity <kobarity@gmail.com> 于2024年1月25日周四 23:38写道：
>
> Liu Hui wrote:
> > kobarity <kobarity@gmail.com> 于2024年1月23日周二 22:15写道：
> > > Thank you. I tried the new patch and confirmed that the Python Shell
> > > buffer allows keyword argument completion on multi-line function
> > > calls. However, my expectation is that it can be done in Python
> > > buffer as well (after calling `python-shell-send-buffer'). Is this
> > > also possible?
> > Thank you for the feedback. I have updated the patch to add support in
> > Python buffer.
>
> Thank you. It's working very nice.
>
> I apologize for reporting in the piecemeal fashion, but I have noticed
> the following differences from the current completion.
No worries. I appreciate your valuable feedback for improving the
quality of the patch.
> 1. Module names cannot be completed in Python buffer.
Can you elaborate? The completion of module names never worked for me
except using IPython with non-native completion mechanism.
Because module names cannot be completed using vanilla python
interpreter in the terminal, they are also not completed even with the
patch, unless jedi is available.
> 2. Completion is not working in comments.
>
> I'm not sure if the completion should work in comments, but some
> people might want it because the current python-mode allows it in both
> Python buffer and Python Shell buffer.
Thanks for pointing it out. The attached patch should now restore
previous basic completion behavior in comments.
[-- Attachment #2: 0001-Improve-Python-shell-completion-bug-68559.patch --]
[-- Type: text/x-patch, Size: 25288 bytes --]
From 770b306fc9ac2406517536e5276649578a18955a Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Thu, 18 Jan 2024 12:00:00 +0800
Subject: [PATCH] Improve Python shell completion (bug#68559)
* lisp/progmodes/python.el (python-shell-completion-setup-code): Fix
the completion code of IPython. Change the return value to JSON
string and ...
(python-shell-completion-get-completions): ... simplify parsing.
(inferior-python-mode): Update docstring.
(python-shell-readline-completer): New option.
(python-shell-readline-completer-delims): New variable indicating the
word delimiters of readline completer.
(python-shell--readline-jedi-setup-code)
(python-shell--readline-ipython-setup-code): New internal variables.
(python-shell-completion-native-setup): Setup a suitable readline
completer and set the completer delimiter.
(python-shell-completion-native-get-completions): Convert output
string to completions properly.
(python-shell--get-multiline-input)
(python-shell--extra-completion-context)
(python-shell-completion-extra-context): New functions.
(python-shell-completion-at-point): Send text beginning from the line
start if the completion backend does not need word splitting. Remove
the detection of import statement because it is not needed anymore.
Create proper completion table based on completions returned from
different backends.
* test/lisp/progmodes/python-tests.el (python-tests--completion-module):
(python-tests--completion-parameters):
(python-tests--completion-extra-context): New helper functions.
(python-shell-completion-at-point-jedi-completer):
(python-shell-completion-at-point-ipython): New tests.
* etc/NEWS: Announce the change.
---
etc/NEWS | 6 +
lisp/progmodes/python.el | 309 +++++++++++++++++++++++-----
test/lisp/progmodes/python-tests.el | 78 +++++++
3 files changed, 341 insertions(+), 52 deletions(-)
diff --git a/etc/NEWS b/etc/NEWS
index a1874313502..9f05bdf4705 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1047,6 +1047,12 @@ instead of:
This allows the user to specify command line arguments to the non
interactive Python interpreter specified by 'python-interpreter'.
+*** New user option 'python-shell-readline-completer'.
+This allows the user to specify the readline completer used for Python
+shell completion. The default is 'auto', which means a suitable
+completer will be configured automatically according to the Python
+interpreter.
+
** use-package
+++
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index ff799e1e662..8e7998ee8f6 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -128,9 +128,9 @@
;; receiving escape sequences (with some limitations, i.e. completion
;; in blocks does not work). The code executed for the "fallback"
;; completion can be found in `python-shell-completion-setup-code' and
-;; `python-shell-completion-string-code' variables. Their default
-;; values enable completion for both CPython and IPython, and probably
-;; any readline based shell (it's known to work with PyPy). If your
+;; `python-shell-completion-get-completions'. Their default values
+;; enable completion for both CPython and IPython, and probably any
+;; readline based shell (it's known to work with PyPy). If your
;; Python installation lacks readline (like CPython for Windows),
;; installing pyreadline (URL `https://ipython.org/pyreadline.html')
;; should suffice. To troubleshoot why you are not getting any
@@ -3601,7 +3601,6 @@ inferior-python-mode
`python-shell-prompt-block-regexp',
`python-shell-font-lock-enable',
`python-shell-completion-setup-code',
-`python-shell-completion-string-code',
`python-eldoc-setup-code',
`python-ffap-setup-code' can
customize this mode for different Python interpreters.
@@ -4319,8 +4318,9 @@ python-shell-completion-setup-code
completions = []
completer = None
+ import json
try:
- import readline
+ import readline, re
try:
import __builtin__
@@ -4331,16 +4331,29 @@ python-shell-completion-setup-code
is_ipython = ('__IPYTHON__' in builtins or
'__IPYTHON__active' in builtins)
- splits = text.split()
- is_module = splits and splits[0] in ('from', 'import')
-
- if is_ipython and is_module:
- from IPython.core.completerlib import module_completion
- completions = module_completion(text.strip())
- elif is_ipython and '__IP' in builtins:
- completions = __IP.complete(text)
- elif is_ipython and 'get_ipython' in builtins:
- completions = get_ipython().Completer.all_completions(text)
+
+ if is_ipython and 'get_ipython' in builtins:
+ def filter_c(prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ import IPython
+ try:
+ if IPython.version_info[0] >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ completions = [
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if filter_c(text, c.text)]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ completions = [text + m[len(part):] for m in matches if filter_c(text, m)]
+ except:
+ pass
else:
# Try to reuse current completer.
completer = readline.get_completer()
@@ -4363,7 +4376,7 @@ python-shell-completion-setup-code
finally:
if getattr(completer, 'PYTHON_EL_WRAPPED', False):
completer.print_mode = True
- return completions"
+ return json.dumps(completions)"
"Code used to setup completion in inferior Python processes."
:type 'string)
@@ -4404,6 +4417,78 @@ python-shell-completion-native-try-output-timeout
:version "25.1"
:type 'float)
+(defcustom python-shell-readline-completer 'auto
+ "The readline completer used for Python shell completion.
+If the value is non-nil, Python shell will setup the readline
+completer unless it has been set elsewhere (e.g. in the
+PYTHONSTARTUP file). Below are possible values:
+- `auto': the completer is determined according to the
+interpreter. Specifically, the IPython completer, defined in
+`python-shell--readline-ipython-setup-code', is used when the
+interpreter is ipython, otherwise the Jedi completer is used.
+- a string: Python code to setup the readline. See
+`python-shell--readline-jedi-setup-code' for reference.
+- `nil': Python shell will do nothing.
+
+In any case, if the completer is not set successfully in the end,
+fallback to the built-in rlcompleter."
+ :type '(choice (const :tag "Automatic" auto)
+ (const :tag "No configuration" nil)
+ (string :tag "Python setup code"))
+ :version "30.1")
+
+(defvar python-shell-readline-completer-delims nil
+ "Word delimiters used by the readline completer.
+It is automatically set by Python shell.")
+
+(defconst python-shell--readline-jedi-setup-code
+ "
+def __PYTHON_EL_setup_readline_completer():
+ from jedi.utils import setup_readline
+ setup_readline()"
+ "Code used to setup readline completer with Jedi.")
+
+(defconst python-shell--readline-ipython-setup-code
+ "
+def __PYTHON_EL_setup_readline_completer():
+ import readline, re, json, IPython
+
+ class __ipython_RL:
+ def __init__(self, v):
+ self.version = v
+
+ def filter(self, prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ def complete(self, text, state):
+ if state == 0:
+ try:
+ if self.version >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ self.matches = [json.dumps([
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if self.filter(text, c.text)])]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ self.matches = [text + m[len(part):] for m in matches
+ if self.filter(text, m)]
+ except Exception:
+ pass
+ try:
+ return self.matches[state]
+ except IndexError:
+ return None
+
+ readline.set_completer(__ipython_RL(IPython.version_info[0]).complete)
+ readline.set_completer_delims('')"
+ "Code used to setup readline completer for IPython.")
+
(defvar python-shell-completion-native-redirect-buffer
" *Python completions redirect*"
"Buffer to be used to redirect output of readline commands.")
@@ -4427,7 +4512,20 @@ python-shell-completion-native-try
(defun python-shell-completion-native-setup ()
"Try to setup native completion, return non-nil on success."
(let* ((process (python-shell-get-process))
- (output (python-shell-send-string-no-output "
+ (completer (pcase python-shell-readline-completer
+ ('auto
+ (if (string-match-p "ipython[23]?\\'" python-shell-interpreter)
+ python-shell--readline-ipython-setup-code
+ python-shell--readline-jedi-setup-code))
+ ((pred stringp) python-shell-readline-completer)
+ (_ "")))
+ (output (python-shell-send-string-no-output
+ (concat "
+try:
+ del __PYTHON_EL_setup_readline_completer
+except:
+ pass
+" completer "
def __PYTHON_EL_native_completion_setup():
try:
import readline
@@ -4500,11 +4598,23 @@ python-shell-completion-native-setup
else:
return completion
+ def is_rlcompleter(completer):
+ try:
+ if completer.__self__.__module__ == 'rlcompleter':
+ return True
+ else:
+ return False
+ except Exception:
+ return False
+
completer = readline.get_completer()
- if not completer:
- # Used as last resort to avoid breaking customizations.
- import rlcompleter
+ if not completer or is_rlcompleter(completer):
+ try:
+ __PYTHON_EL_setup_readline_completer()
+ except:
+ # Used as last resort to avoid breaking customizations.
+ import rlcompleter
completer = readline.get_completer()
if completer and not getattr(completer, 'PYTHON_EL_WRAPPED', False):
@@ -4539,9 +4649,13 @@ python-shell-completion-native-setup
print ('python.el: native completion setup failed, %s: %s'
% sys.exc_info()[:2])
-__PYTHON_EL_native_completion_setup()" process)))
+__PYTHON_EL_native_completion_setup()") process)))
(when (string-match-p "python\\.el: native completion setup loaded"
output)
+ (setq-local python-shell-readline-completer-delims
+ (string-trim-right
+ (python-shell-send-string-no-output
+ "import readline; print(readline.get_completer_delims())")))
(python-shell-completion-native-try))))
(defun python-shell-completion-native-turn-off (&optional msg)
@@ -4609,6 +4723,8 @@ python-shell-completion-native-get-completions
(let* ((original-filter-fn (process-filter process))
(redirect-buffer (get-buffer-create
python-shell-completion-native-redirect-buffer))
+ (sep (if (string= python-shell-readline-completer-delims "")
+ "[\n\r]+" "[ \f\t\n\r\v()]+"))
(trigger "\t")
(new-input (concat input trigger))
(input-length
@@ -4651,28 +4767,72 @@ python-shell-completion-native-get-completions
process python-shell-completion-native-output-timeout
comint-redirect-finished-regexp)
(re-search-backward "0__dummy_completion__" nil t)
- (cl-remove-duplicates
- (split-string
- (buffer-substring-no-properties
- (line-beginning-position) (point-min))
- "[ \f\t\n\r\v()]+" t)
- :test #'string=))))
+ (let ((str (buffer-substring-no-properties
+ (line-beginning-position) (point-min))))
+ (if (string= "[" (substring str 0 1))
+ (condition-case nil
+ (python--parse-json-array str)
+ (t (cl-remove-duplicates (split-string str sep t)
+ :test #'string=)))
+ (cl-remove-duplicates (split-string str sep t)
+ :test #'string=))))))
(set-process-filter process original-filter-fn)))))
(defun python-shell-completion-get-completions (process input)
"Get completions of INPUT using PROCESS."
(with-current-buffer (process-buffer process)
- (let ((completions
- (python-util-strip-string
- (python-shell-send-string-no-output
- (format
- "%s\nprint(';'.join(__PYTHON_EL_get_completions(%s)))"
+ (python--parse-json-array
+ (python-shell-send-string-no-output
+ (format "%s\nprint(__PYTHON_EL_get_completions(%s))"
python-shell-completion-setup-code
(python-shell--encode-string input))
- process))))
- (when (> (length completions) 2)
- (split-string completions
- "^'\\|^\"\\|;\\|'$\\|\"$" t)))))
+ process))))
+
+(defun python-shell--get-multiline-input ()
+ "Return lines at a multi-line input in Python shell."
+ (save-excursion
+ (let ((p (point)) lines)
+ (when (progn
+ (beginning-of-line)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) p) lines)
+ (while (progn (comint-previous-prompt 1)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ lines)))
+
+(defun python-shell--extra-completion-context ()
+ "Get extra completion context of current input in Python shell."
+ (let ((lines (python-shell--get-multiline-input))
+ (python-indent-guess-indent-offset nil))
+ (when (not (zerop (length lines)))
+ (with-temp-buffer
+ (delay-mode-hooks
+ (insert (string-join lines "\n"))
+ (python-mode)
+ (python-shell-completion-extra-context))))))
+
+(defun python-shell-completion-extra-context (&optional pos)
+ "Get extra completion context at position POS in Python buffer.
+If optional argument POS is nil, use current position.
+
+Readline completers could use current line as the completion
+context, which may be insufficient. In this function, extra
+context (e.g. multi-line function call) is found and reformatted
+as one line, which is required by native completion."
+ (let (bound p)
+ (save-excursion
+ (and pos (goto-char pos))
+ (setq bound (pos-bol))
+ (python-nav-up-list -1)
+ (when (and (< (point) bound)
+ (looking-back
+ (python-rx (+ (or "." symbol-name))) (pos-bol) t))
+ (setq p (match-beginning 0))))
+ (when p
+ (replace-regexp-in-string
+ "\n[ \t]*" "" (buffer-substring-no-properties p (1- bound))))))
(defvar-local python-shell--capf-cache nil
"Variable to store cached completions and invalidation keys.")
@@ -4687,21 +4847,26 @@ python-shell-completion-at-point
;; Working on a shell buffer: use prompt end.
(cdr (python-util-comint-last-prompt))
(line-beginning-position)))
- (import-statement
- (when (string-match-p
- (rx (* space) word-start (or "from" "import") word-end space)
- (buffer-substring-no-properties line-start (point)))
- (buffer-substring-no-properties line-start (point))))
+ (no-delims
+ (and (not (if is-shell-buffer
+ (eq 'font-lock-comment-face
+ (get-text-property (1- (point)) 'face))
+ (python-syntax-context 'comment)))
+ (with-current-buffer (process-buffer process)
+ (if python-shell-completion-native-enable
+ (string= python-shell-readline-completer-delims "")
+ (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
(start
(if (< (point) line-start)
(point)
(save-excursion
- (if (not (re-search-backward
- (python-rx
- (or whitespace open-paren close-paren
- string-delimiter simple-operator))
- line-start
- t 1))
+ (if (or no-delims
+ (not (re-search-backward
+ (python-rx
+ (or whitespace open-paren close-paren
+ string-delimiter simple-operator))
+ line-start
+ t 1)))
line-start
(forward-char (length (match-string-no-properties 0)))
(point)))))
@@ -4741,18 +4906,58 @@ python-shell-completion-at-point
(t #'python-shell-completion-native-get-completions))))
(prev-prompt (car python-shell--capf-cache))
(re (or (cadr python-shell--capf-cache) regexp-unmatchable))
- (prefix (buffer-substring-no-properties start end)))
+ (prefix (buffer-substring-no-properties start end))
+ (prefix-offset 0)
+ (extra-context (when no-delims
+ (if is-shell-buffer
+ (python-shell--extra-completion-context)
+ (python-shell-completion-extra-context))))
+ (extra-offset (length extra-context)))
+ (unless (zerop extra-offset)
+ (setq prefix (concat extra-context prefix)))
;; To invalidate the cache, we check if the prompt position or the
;; completion prefix changed.
(unless (and (equal prev-prompt (car prompt-boundaries))
- (string-match re prefix))
+ (string-match re prefix)
+ (setq prefix-offset (- (length prefix) (match-end 1))))
(setq python-shell--capf-cache
`(,(car prompt-boundaries)
,(if (string-empty-p prefix)
regexp-unmatchable
- (concat "\\`" (regexp-quote prefix) "\\(?:\\sw\\|\\s_\\)*\\'"))
- ,@(funcall completion-fn process (or import-statement prefix)))))
- (list start end (cddr python-shell--capf-cache))))
+ (concat "\\`\\(" (regexp-quote prefix) "\\)\\(?:\\sw\\|\\s_\\)*\\'"))
+ ,@(funcall completion-fn process prefix))))
+ (let ((cands (cddr python-shell--capf-cache)))
+ (cond
+ ((stringp (car cands))
+ (if no-delims
+ ;; Reduce completion candidates due to long prefix.
+ (if-let ((Lp (length prefix))
+ ((string-match "\\(\\sw\\|\\s_\\)+\\'" prefix))
+ (L (match-beginning 0)))
+ ;; If extra-offset is not zero:
+ ;; start end
+ ;; o------------------o---------o-------o
+ ;; |<- extra-offset ->|
+ ;; |<----------- L ------------>|
+ ;; new-start
+ (list (+ start L (- extra-offset)) end
+ (mapcar (lambda (s) (substring s L)) cands))
+ (list end end (mapcar (lambda (s) (substring s Lp)) cands)))
+ (list start end cands)))
+ ;; python-shell-completion(-native)-get-completions may produce
+ ;; a list of (text start end type signature) for completion.
+ ;; See `python-shell--readline-ipython-setup-code' and
+ ;; `python-shell-completion-setup-code'.
+ ((consp (car cands))
+ (list (+ start (nth 1 (car cands)) (- extra-offset))
+ ;; Candidates may be cached, so the end position should
+ ;; be adjusted according to current completion prefix.
+ (+ start (nth 2 (car cands)) (- extra-offset) prefix-offset)
+ cands
+ :annotation-function
+ (lambda (c) (concat " " (nth 3 (assoc c cands))))
+ :company-docsig
+ (lambda (c) (nth 4 (assoc c cands)))))))))
(define-obsolete-function-alias
'python-shell-completion-complete-at-point
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 97ffd5fe20f..ae23ff8ebe2 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4787,6 +4787,84 @@ python-shell-completion-at-point-native-1
(end-of-line 0)
(should-not (nth 2 (python-shell-completion-at-point))))))
+(defun python-tests--completion-module ()
+ "Check if modules can be completed in Python shell."
+ (insert "import datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "import datetime"))
+ (kill-line)
+ (insert "from datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime"))
+ (end-of-line)
+ (insert " import timed")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime import timedelta"))
+ (kill-line))
+
+(defun python-tests--completion-parameters ()
+ "Check if parameters can be completed in Python shell."
+ (insert "import re")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "re.split('b', 'abc', maxs")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit="
+ (buffer-substring (line-beginning-position) (point))))
+ (insert "0, ")
+ (should (python-shell-completion-at-point))
+ ;; Test if cache is used.
+ (cl-letf (((symbol-function 'python-shell-completion-get-completions)
+ 'ignore)
+ ((symbol-function 'python-shell-completion-native-get-completions)
+ 'ignore))
+ (insert "fla")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit=0, flags="
+ (buffer-substring (line-beginning-position) (point)))))
+ (beginning-of-line)
+ (kill-line))
+
+(defun python-tests--completion-extra-context ()
+ "Check if extra context is used for completion."
+ (insert "re.split('b', 'abc',")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "maxs")
+ (completion-at-point)
+ (should (string= "maxsplit="
+ (buffer-substring (line-beginning-position) (point)))))
+
+(ert-deftest python-shell-completion-at-point-jedi-completer ()
+ "Check if Python shell completion works with Jedi."
+ (skip-unless (executable-find python-tests-shell-interpreter))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context))))
+
+(ert-deftest python-shell-completion-at-point-ipython ()
+ "Check if Python shell completion works for IPython."
+ (let ((python-shell-interpreter "ipython")
+ (python-shell-interpreter-args "-i --simple-prompt"))
+ (skip-unless (executable-find python-shell-interpreter))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-off)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-shell-completion-native-turn-on)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context)))))
\f
;;; PDB Track integration
--
2.25.1
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-01-26 10:12 ` Liu Hui
@ 2024-01-28 13:22 ` kobarity
2024-01-29 13:15 ` kobarity
2024-02-04 12:09 ` Liu Hui
0 siblings, 2 replies; 68+ messages in thread
From: kobarity @ 2024-01-28 13:22 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, 68559
Liu Hui wrote:
> kobarity <kobarity@gmail.com> 于2024年1月25日周四 23:38写道：
> > 1. Module names cannot be completed in Python buffer.
> Can you elaborate? The completion of module names never worked for me
> except using IPython with non-native completion mechanism.
>
> Because module names cannot be completed using vanilla python
> interpreter in the terminal, they are also not completed even with the
> patch, unless jedi is available.
Hmm, I must have done something wrong. I cannot reproduce the problem
of module name completion even with the previous patch.
As for the current python-mode, I was able to complete the module
names because I enabled Jedi in PYTHONSTARTUP. Sorry for the confusion.
I'm experiencing strange behavior regarding completion of import
statement in a block in Python buffer. If I try to type the following
lines and then try to complete it, it will fail.
#+begin_src python
try:
im
#+end_src
However, when I try to complete at the beginning of the second line:
#+begin_src python
try:
#+end_src
"import" keyword also appears as a candidate. If I cancel the
candidates and type "im" and try to complete it, it will succeed.
This behavior does not occur in Python Shell buffer.
Another thing I noticed is the multi-line import statement. If the
import statement is one-line, each items (IGNORECASE and MULTILINE in
the example below) can be completed.
#+begin_src python
from re import IGNORECASE, MULTILINE
#+end_src
However, they cannot be completed if the import statement spans
multi-line.
#+begin_src python
from re import (
IGN
#+end_src
This happens in both Python buffer and Python Shell buffer. Perhaps
this is a limitation of Jedi completer?
> > 2. Completion is not working in comments.
> >
> > I'm not sure if the completion should work in comments, but some
> > people might want it because the current python-mode allows it in both
> > Python buffer and Python Shell buffer.
>
> Thanks for pointing it out. The attached patch should now restore
> previous basic completion behavior in comments.
Thanks, I confirmed that basic completion (without keyword argument
completion) can be performed within comments.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-01-28 13:22 ` kobarity
@ 2024-01-29 13:15 ` kobarity
2024-02-01 9:52 ` Eli Zaretskii
2024-02-04 12:09 ` Liu Hui
1 sibling, 1 reply; 68+ messages in thread
From: kobarity @ 2024-01-29 13:15 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, 68559
kobarity wrote:
> Liu Hui wrote:
> > Thanks for pointing it out. The attached patch should now restore
> > previous basic completion behavior in comments.
>
> Thanks, I confirmed that basic completion (without keyword argument
> completion) can be performed within comments.
One additional comment regarding ERT. pyenv installs an ipython shim
to the shims directory, if there is at least one virtualenv providing
ipython. It can be found by `executable-find'. However, it is not
usable unless the virtualenv is activated. As a result, the
`python-shell-completion-at-point-ipython' ERT would fail unless a
virtualenv which provides ipython is activated.
This may be something pyenv users should take care of, but it is more
convenient for pyenv users like me to make the following changes.
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index ae23ff8ebe2..9452f136cfb 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4854,7 +4854,10 @@ python-shell-completion-at-point-ipython
"Check if Python shell completion works for IPython."
(let ((python-shell-interpreter "ipython")
(python-shell-interpreter-args "-i --simple-prompt"))
- (skip-unless (executable-find python-shell-interpreter))
+ (skip-unless
+ (and
+ (executable-find python-shell-interpreter)
+ (eql (call-process python-shell-interpreter nil nil nil "--version") 0)))
(python-tests-with-temp-buffer-with-shell
""
(python-shell-with-shell-buffer
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-01-29 13:15 ` kobarity
@ 2024-02-01 9:52 ` Eli Zaretskii
2024-02-01 14:39 ` kobarity
0 siblings, 1 reply; 68+ messages in thread
From: Eli Zaretskii @ 2024-02-01 9:52 UTC (permalink / raw)
To: kobarity; +Cc: liuhui1610, 68559
> Date: Mon, 29 Jan 2024 22:15:20 +0900
> From: kobarity <kobarity@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,
> 68559@debbugs.gnu.org
>
> kobarity wrote:
> > Liu Hui wrote:
> > > Thanks for pointing it out. The attached patch should now restore
> > > previous basic completion behavior in comments.
> >
> > Thanks, I confirmed that basic completion (without keyword argument
> > completion) can be performed within comments.
>
> One additional comment regarding ERT. pyenv installs an ipython shim
> to the shims directory, if there is at least one virtualenv providing
> ipython. It can be found by `executable-find'. However, it is not
> usable unless the virtualenv is activated. As a result, the
> `python-shell-completion-at-point-ipython' ERT would fail unless a
> virtualenv which provides ipython is activated.
>
> This may be something pyenv users should take care of, but it is more
> convenient for pyenv users like me to make the following changes.
Do we have a consensus about this issue? Should I install the
original patch, or are you still discussing it, and an updated patch
will be posted soon?
Thanks.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-01 9:52 ` Eli Zaretskii
@ 2024-02-01 14:39 ` kobarity
2024-02-01 15:02 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: kobarity @ 2024-02-01 14:39 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: liuhui1610, 68559
Eli Zaretskii wrote:
> > Date: Mon, 29 Jan 2024 22:15:20 +0900
> > From: kobarity <kobarity@gmail.com>
> > Cc: Eli Zaretskii <eliz@gnu.org>,
> > 68559@debbugs.gnu.org
> >
> > kobarity wrote:
> > > Liu Hui wrote:
> > > > Thanks for pointing it out. The attached patch should now restore
> > > > previous basic completion behavior in comments.
> > >
> > > Thanks, I confirmed that basic completion (without keyword argument
> > > completion) can be performed within comments.
> >
> > One additional comment regarding ERT. pyenv installs an ipython shim
> > to the shims directory, if there is at least one virtualenv providing
> > ipython. It can be found by `executable-find'. However, it is not
> > usable unless the virtualenv is activated. As a result, the
> > `python-shell-completion-at-point-ipython' ERT would fail unless a
> > virtualenv which provides ipython is activated.
> >
> > This may be something pyenv users should take care of, but it is more
> > convenient for pyenv users like me to make the following changes.
>
> Do we have a consensus about this issue? Should I install the
> original patch, or are you still discussing it, and an updated patch
> will be posted soon?
>
> Thanks.
Although I think there are a few things left unanswered, the latest
patch sent on Jan 26th is very worthwhile and stable enough to be
installed. So, unless Liu is preparing a new patch, I agree that the
latest patch should be installed. That way more people can try it and
find the problem, if any.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-01 14:39 ` kobarity
@ 2024-02-01 15:02 ` Liu Hui
0 siblings, 0 replies; 68+ messages in thread
From: Liu Hui @ 2024-02-01 15:02 UTC (permalink / raw)
To: kobarity; +Cc: Eli Zaretskii, 68559
kobarity <kobarity@gmail.com> 于2024年2月1日周四 22:40写道：
>
> Eli Zaretskii wrote:
> > Do we have a consensus about this issue? Should I install the
> > original patch, or are you still discussing it, and an updated patch
> > will be posted soon?
> >
> > Thanks.
>
> Although I think there are a few things left unanswered, the latest
> patch sent on Jan 26th is very worthwhile and stable enough to be
> installed. So, unless Liu is preparing a new patch, I agree that the
> latest patch should be installed. That way more people can try it and
> find the problem, if any.
Sorry for the late reply. Recent events have occupied all my time. I
plan to update the patch in the next day or two to address remaining
concerns.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-01-28 13:22 ` kobarity
2024-01-29 13:15 ` kobarity
@ 2024-02-04 12:09 ` Liu Hui
2024-02-04 14:35 ` kobarity
1 sibling, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-04 12:09 UTC (permalink / raw)
To: kobarity; +Cc: Eli Zaretskii, 68559
[-- Attachment #1: Type: text/plain, Size: 3845 bytes --]
kobarity <kobarity@gmail.com> writes:
> I'm experiencing strange behavior regarding completion of import
> statement in a block in Python buffer. If I try to type the following
> lines and then try to complete it, it will fail.
>
> #+begin_src python
> try:
> im
> #+end_src
The problem can be reproduced by running python with jedi completer in
a terminal. The reason is that readline completer can only see the
text in the current line, i.e. " im"; in this case, jedi does not
complete the text to " import".
Similarly, the jedi completer cannot complete "except" after ex:
try:
pass
ex|
> However, when I try to complete at the beginning of the second line:
>
> #+begin_src python
> try:
>
> #+end_src
>
>
> "import" keyword also appears as a candidate. If I cancel the
> candidates and type "im" and try to complete it, it will succeed.
It is because jedi produces completions including " import" for
blank string " ". Due to the same prefix between " " and
" im", completion cache is reused for the latter. Then " import"
can be completed.
It is more a limitation of readline completer than a problem with
jedi, as we cannot provide proper completion context for jedi. We may
define a custom completer to combine jedi and rlcompleter, e.g.
(setq python-shell-readline-completer "
def __PYTHON_EL_setup_readline_completer():
import readline, rlcompleter
import re, sys, os, __main__
from jedi import Interpreter
class MyJediRL:
def __init__(self):
self.rlcompleter = rlcompleter.Completer()
self.rldelim = readline.get_completer_delims()
def complete(self, text, state):
if state == 0:
sys.path.insert(0, os.getcwd())
try:
interpreter = Interpreter(text, [__main__.__dict__])
completions = interpreter.complete(fuzzy=False)
self.matches = [
text[:len(text) - c._like_name_length] + c.name_with_symbols
for c in completions
]
# try rlcompleter
sub = re.split('[' + re.escape(self.rldelim) + ']', text)[-1]
i = 0
while True:
completion = self.rlcompleter.complete(sub, i)
if not completion:
break
i += 1
completion = text[:len(text)-len(sub)] + completion.rstrip(' ()')
if completion not in self.matches:
self.matches.append(completion)
except:
raise
finally:
sys.path.pop(0)
try:
return self.matches[state]
except IndexError:
return None
readline.set_completer(MyJediRL().complete)
readline.set_completer_delims('')")
> Another thing I noticed is the multi-line import statement. If the
> import statement is one-line, each items (IGNORECASE and MULTILINE in
> the example below) can be completed.
>
> #+begin_src python
> from re import IGNORECASE, MULTILINE
> #+end_src
>
>
> However, they cannot be completed if the import statement spans
> multi-line.
>
> #+begin_src python
> from re import (
> IGN
> #+end_src
>
> This happens in both Python buffer and Python Shell buffer. Perhaps
> this is a limitation of Jedi completer?
Yes. Because readline completer cannot see cross-line context, I added
the function "python-shell-completion-extra-context" in previous patch
to address the case of multi-line function call. I have updated the
attached patch to handle multi-line import statement.
The change to python-tests.el has been incorporated in the patch.
Thanks!
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Improve-Python-shell-completion-bug-68559.patch --]
[-- Type: text/x-diff, Size: 26123 bytes --]
From a5da22803e967bc7df5f227a49bb81ce44fdf204 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Thu, 18 Jan 2024 12:00:00 +0800
Subject: [PATCH] Improve Python shell completion (bug#68559)
* lisp/progmodes/python.el (python-shell-completion-setup-code): Fix
the completion code of IPython. Change the return value to JSON
string and ...
(python-shell-completion-get-completions): ... simplify parsing.
(inferior-python-mode): Update docstring.
(python-shell-readline-completer): New option.
(python-shell-readline-completer-delims): New variable indicating the
word delimiters of readline completer.
(python-shell--readline-jedi-setup-code)
(python-shell--readline-ipython-setup-code): New internal variables.
(python-shell-completion-native-setup): Setup a suitable readline
completer and set the completer delimiter.
(python-shell-completion-native-get-completions): Convert output
string to completions properly.
(python-shell--get-multiline-input)
(python-shell--extra-completion-context)
(python-shell-completion-extra-context): New functions.
(python-shell-completion-at-point): Send text beginning from the line
start if the completion backend does not need word splitting. Remove
the detection of import statement because it is not needed anymore.
Create proper completion table based on completions returned from
different backends.
* test/lisp/progmodes/python-tests.el (python-tests--completion-module)
(python-tests--completion-parameters)
(python-tests--completion-extra-context): New helper functions.
(python-shell-completion-at-point-jedi-completer)
(python-shell-completion-at-point-ipython): New tests.
* etc/NEWS: Announce the change.
---
etc/NEWS | 6 +
lisp/progmodes/python.el | 313 +++++++++++++++++++++++-----
test/lisp/progmodes/python-tests.el | 91 ++++++++
3 files changed, 358 insertions(+), 52 deletions(-)
diff --git a/etc/NEWS b/etc/NEWS
index 816613de4ec..2c8e70e4a15 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1086,6 +1086,12 @@ instead of:
This allows the user to specify command line arguments to the non
interactive Python interpreter specified by 'python-interpreter'.
+*** New user option 'python-shell-readline-completer'.
+This allows the user to specify the readline completer used for Python
+shell completion. The default is 'auto', which means a suitable
+completer will be configured automatically according to the Python
+interpreter.
+
** use-package
+++
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 9d840efb9da..0291f398a65 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -128,9 +128,9 @@ ;;; Commentary:
;; receiving escape sequences (with some limitations, i.e. completion
;; in blocks does not work). The code executed for the "fallback"
;; completion can be found in `python-shell-completion-setup-code' and
-;; `python-shell-completion-string-code' variables. Their default
-;; values enable completion for both CPython and IPython, and probably
-;; any readline based shell (it's known to work with PyPy). If your
+;; `python-shell-completion-get-completions'. Their default values
+;; enable completion for both CPython and IPython, and probably any
+;; readline based shell (it's known to work with PyPy). If your
;; Python installation lacks readline (like CPython for Windows),
;; installing pyreadline (URL `https://ipython.org/pyreadline.html')
;; should suffice. To troubleshoot why you are not getting any
@@ -3604,7 +3604,6 @@ (define-derived-mode inferior-python-mode comint-mode "Inferior Python"
`python-shell-prompt-block-regexp',
`python-shell-font-lock-enable',
`python-shell-completion-setup-code',
-`python-shell-completion-string-code',
`python-eldoc-setup-code',
`python-ffap-setup-code' can
customize this mode for different Python interpreters.
@@ -4244,8 +4243,9 @@ (defcustom python-shell-completion-setup-code
completions = []
completer = None
+ import json
try:
- import readline
+ import readline, re
try:
import __builtin__
@@ -4256,16 +4256,29 @@ (defcustom python-shell-completion-setup-code
is_ipython = ('__IPYTHON__' in builtins or
'__IPYTHON__active' in builtins)
- splits = text.split()
- is_module = splits and splits[0] in ('from', 'import')
-
- if is_ipython and is_module:
- from IPython.core.completerlib import module_completion
- completions = module_completion(text.strip())
- elif is_ipython and '__IP' in builtins:
- completions = __IP.complete(text)
- elif is_ipython and 'get_ipython' in builtins:
- completions = get_ipython().Completer.all_completions(text)
+
+ if is_ipython and 'get_ipython' in builtins:
+ def filter_c(prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ import IPython
+ try:
+ if IPython.version_info[0] >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ completions = [
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if filter_c(text, c.text)]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ completions = [text + m[len(part):] for m in matches if filter_c(text, m)]
+ except:
+ pass
else:
# Try to reuse current completer.
completer = readline.get_completer()
@@ -4288,7 +4301,7 @@ (defcustom python-shell-completion-setup-code
finally:
if getattr(completer, 'PYTHON_EL_WRAPPED', False):
completer.print_mode = True
- return completions"
+ return json.dumps(completions)"
"Code used to setup completion in inferior Python processes."
:type 'string)
@@ -4329,6 +4342,78 @@ (defcustom python-shell-completion-native-try-output-timeout 1.0
:version "25.1"
:type 'float)
+(defcustom python-shell-readline-completer 'auto
+ "The readline completer used for Python shell completion.
+If the value is non-nil, Python shell will setup the readline
+completer unless it has been set elsewhere (e.g. in the
+PYTHONSTARTUP file). Below are possible values:
+- `auto': the completer is determined according to the
+interpreter. Specifically, the IPython completer, defined in
+`python-shell--readline-ipython-setup-code', is used when the
+interpreter is ipython, otherwise the Jedi completer is used.
+- a string: Python code to setup the readline. See
+`python-shell--readline-jedi-setup-code' for reference.
+- `nil': Python shell will do nothing.
+
+In any case, if the completer is not set successfully in the end,
+fallback to the built-in rlcompleter."
+ :type '(choice (const :tag "Automatic" auto)
+ (const :tag "No configuration" nil)
+ (string :tag "Python setup code"))
+ :version "30.1")
+
+(defvar python-shell-readline-completer-delims nil
+ "Word delimiters used by the readline completer.
+It is automatically set by Python shell.")
+
+(defconst python-shell--readline-jedi-setup-code
+ "
+def __PYTHON_EL_setup_readline_completer():
+ from jedi.utils import setup_readline
+ setup_readline()"
+ "Code used to setup readline completer with Jedi.")
+
+(defconst python-shell--readline-ipython-setup-code
+ "
+def __PYTHON_EL_setup_readline_completer():
+ import readline, re, json, IPython
+
+ class __ipython_RL:
+ def __init__(self, v):
+ self.version = v
+
+ def filter(self, prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ def complete(self, text, state):
+ if state == 0:
+ try:
+ if self.version >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ self.matches = [json.dumps([
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if self.filter(text, c.text)])]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ self.matches = [text + m[len(part):] for m in matches
+ if self.filter(text, m)]
+ except Exception:
+ pass
+ try:
+ return self.matches[state]
+ except IndexError:
+ return None
+
+ readline.set_completer(__ipython_RL(IPython.version_info[0]).complete)
+ readline.set_completer_delims('')"
+ "Code used to setup readline completer for IPython.")
+
(defvar python-shell-completion-native-redirect-buffer
" *Python completions redirect*"
"Buffer to be used to redirect output of readline commands.")
@@ -4352,7 +4437,20 @@ (defun python-shell-completion-native-try ()
(defun python-shell-completion-native-setup ()
"Try to setup native completion, return non-nil on success."
(let* ((process (python-shell-get-process))
- (output (python-shell-send-string-no-output "
+ (completer (pcase python-shell-readline-completer
+ ('auto
+ (if (string-match-p "ipython[23]?\\'" python-shell-interpreter)
+ python-shell--readline-ipython-setup-code
+ python-shell--readline-jedi-setup-code))
+ ((pred stringp) python-shell-readline-completer)
+ (_ "")))
+ (output (python-shell-send-string-no-output
+ (concat "
+try:
+ del __PYTHON_EL_setup_readline_completer
+except:
+ pass
+" completer "
def __PYTHON_EL_native_completion_setup():
try:
import readline
@@ -4425,11 +4523,23 @@ (defun python-shell-completion-native-setup ()
else:
return completion
+ def is_rlcompleter(completer):
+ try:
+ if completer.__self__.__module__ == 'rlcompleter':
+ return True
+ else:
+ return False
+ except Exception:
+ return False
+
completer = readline.get_completer()
- if not completer:
- # Used as last resort to avoid breaking customizations.
- import rlcompleter
+ if not completer or is_rlcompleter(completer):
+ try:
+ __PYTHON_EL_setup_readline_completer()
+ except:
+ # Used as last resort to avoid breaking customizations.
+ import rlcompleter
completer = readline.get_completer()
if completer and not getattr(completer, 'PYTHON_EL_WRAPPED', False):
@@ -4464,9 +4574,13 @@ (defun python-shell-completion-native-setup ()
print ('python.el: native completion setup failed, %s: %s'
% sys.exc_info()[:2])
-__PYTHON_EL_native_completion_setup()" process)))
+__PYTHON_EL_native_completion_setup()") process)))
(when (string-match-p "python\\.el: native completion setup loaded"
output)
+ (setq-local python-shell-readline-completer-delims
+ (string-trim-right
+ (python-shell-send-string-no-output
+ "import readline; print(readline.get_completer_delims())")))
(python-shell-completion-native-try))))
(defun python-shell-completion-native-turn-off (&optional msg)
@@ -4534,6 +4648,8 @@ (defun python-shell-completion-native-get-completions (process input)
(let* ((original-filter-fn (process-filter process))
(redirect-buffer (get-buffer-create
python-shell-completion-native-redirect-buffer))
+ (sep (if (string= python-shell-readline-completer-delims "")
+ "[\n\r]+" "[ \f\t\n\r\v()]+"))
(trigger "\t")
(new-input (concat input trigger))
(input-length
@@ -4576,28 +4692,76 @@ (defun python-shell-completion-native-get-completions (process input)
process python-shell-completion-native-output-timeout
comint-redirect-finished-regexp)
(re-search-backward "0__dummy_completion__" nil t)
- (cl-remove-duplicates
- (split-string
- (buffer-substring-no-properties
- (line-beginning-position) (point-min))
- "[ \f\t\n\r\v()]+" t)
- :test #'string=))))
+ (let ((str (buffer-substring-no-properties
+ (line-beginning-position) (point-min))))
+ (if (string= "[" (substring str 0 1))
+ (condition-case nil
+ (python--parse-json-array str)
+ (t (cl-remove-duplicates (split-string str sep t)
+ :test #'string=)))
+ (cl-remove-duplicates (split-string str sep t)
+ :test #'string=))))))
(set-process-filter process original-filter-fn)))))
(defun python-shell-completion-get-completions (process input)
"Get completions of INPUT using PROCESS."
(with-current-buffer (process-buffer process)
- (let ((completions
- (python-util-strip-string
- (python-shell-send-string-no-output
- (format
- "%s\nprint(';'.join(__PYTHON_EL_get_completions(%s)))"
+ (python--parse-json-array
+ (python-shell-send-string-no-output
+ (format "%s\nprint(__PYTHON_EL_get_completions(%s))"
python-shell-completion-setup-code
(python-shell--encode-string input))
- process))))
- (when (> (length completions) 2)
- (split-string completions
- "^'\\|^\"\\|;\\|'$\\|\"$" t)))))
+ process))))
+
+(defun python-shell--get-multiline-input ()
+ "Return lines at a multi-line input in Python shell."
+ (save-excursion
+ (let ((p (point)) lines)
+ (when (progn
+ (beginning-of-line)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) p) lines)
+ (while (progn (comint-previous-prompt 1)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ lines)))
+
+(defun python-shell--extra-completion-context ()
+ "Get extra completion context of current input in Python shell."
+ (let ((lines (python-shell--get-multiline-input))
+ (python-indent-guess-indent-offset nil))
+ (when (not (zerop (length lines)))
+ (with-temp-buffer
+ (delay-mode-hooks
+ (insert (string-join lines "\n"))
+ (python-mode)
+ (python-shell-completion-extra-context))))))
+
+(defun python-shell-completion-extra-context (&optional pos)
+ "Get extra completion context at position POS in Python buffer.
+If optional argument POS is nil, use current position.
+
+Readline completers could use current line as the completion
+context, which may be insufficient. In this function, extra
+context (e.g. multi-line function call) is found and reformatted
+as one line, which is required by native completion."
+ (let (bound p)
+ (save-excursion
+ (and pos (goto-char pos))
+ (setq bound (pos-bol))
+ (python-nav-up-list -1)
+ (when (and (< (point) bound)
+ (or
+ (looking-back
+ (python-rx (group (+ (or "." symbol-name)))) (pos-bol) t)
+ (progn
+ (forward-line 0)
+ (looking-at "^[ \t]*\\(from \\)"))))
+ (setq p (match-beginning 1))))
+ (when p
+ (replace-regexp-in-string
+ "\n[ \t]*" "" (buffer-substring-no-properties p (1- bound))))))
(defvar-local python-shell--capf-cache nil
"Variable to store cached completions and invalidation keys.")
@@ -4612,21 +4776,26 @@ (defun python-shell-completion-at-point (&optional process)
;; Working on a shell buffer: use prompt end.
(cdr (python-util-comint-last-prompt))
(line-beginning-position)))
- (import-statement
- (when (string-match-p
- (rx (* space) word-start (or "from" "import") word-end space)
- (buffer-substring-no-properties line-start (point)))
- (buffer-substring-no-properties line-start (point))))
+ (no-delims
+ (and (not (if is-shell-buffer
+ (eq 'font-lock-comment-face
+ (get-text-property (1- (point)) 'face))
+ (python-syntax-context 'comment)))
+ (with-current-buffer (process-buffer process)
+ (if python-shell-completion-native-enable
+ (string= python-shell-readline-completer-delims "")
+ (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
(start
(if (< (point) line-start)
(point)
(save-excursion
- (if (not (re-search-backward
- (python-rx
- (or whitespace open-paren close-paren
- string-delimiter simple-operator))
- line-start
- t 1))
+ (if (or no-delims
+ (not (re-search-backward
+ (python-rx
+ (or whitespace open-paren close-paren
+ string-delimiter simple-operator))
+ line-start
+ t 1)))
line-start
(forward-char (length (match-string-no-properties 0)))
(point)))))
@@ -4666,18 +4835,58 @@ (defun python-shell-completion-at-point (&optional process)
(t #'python-shell-completion-native-get-completions))))
(prev-prompt (car python-shell--capf-cache))
(re (or (cadr python-shell--capf-cache) regexp-unmatchable))
- (prefix (buffer-substring-no-properties start end)))
+ (prefix (buffer-substring-no-properties start end))
+ (prefix-offset 0)
+ (extra-context (when no-delims
+ (if is-shell-buffer
+ (python-shell--extra-completion-context)
+ (python-shell-completion-extra-context))))
+ (extra-offset (length extra-context)))
+ (unless (zerop extra-offset)
+ (setq prefix (concat extra-context prefix)))
;; To invalidate the cache, we check if the prompt position or the
;; completion prefix changed.
(unless (and (equal prev-prompt (car prompt-boundaries))
- (string-match re prefix))
+ (string-match re prefix)
+ (setq prefix-offset (- (length prefix) (match-end 1))))
(setq python-shell--capf-cache
`(,(car prompt-boundaries)
,(if (string-empty-p prefix)
regexp-unmatchable
- (concat "\\`" (regexp-quote prefix) "\\(?:\\sw\\|\\s_\\)*\\'"))
- ,@(funcall completion-fn process (or import-statement prefix)))))
- (list start end (cddr python-shell--capf-cache))))
+ (concat "\\`\\(" (regexp-quote prefix) "\\)\\(?:\\sw\\|\\s_\\)*\\'"))
+ ,@(funcall completion-fn process prefix))))
+ (let ((cands (cddr python-shell--capf-cache)))
+ (cond
+ ((stringp (car cands))
+ (if no-delims
+ ;; Reduce completion candidates due to long prefix.
+ (if-let ((Lp (length prefix))
+ ((string-match "\\(\\sw\\|\\s_\\)+\\'" prefix))
+ (L (match-beginning 0)))
+ ;; If extra-offset is not zero:
+ ;; start end
+ ;; o------------------o---------o-------o
+ ;; |<- extra-offset ->|
+ ;; |<----------- L ------------>|
+ ;; new-start
+ (list (+ start L (- extra-offset)) end
+ (mapcar (lambda (s) (substring s L)) cands))
+ (list end end (mapcar (lambda (s) (substring s Lp)) cands)))
+ (list start end cands)))
+ ;; python-shell-completion(-native)-get-completions may produce
+ ;; a list of (text start end type signature) for completion.
+ ;; See `python-shell--readline-ipython-setup-code' and
+ ;; `python-shell-completion-setup-code'.
+ ((consp (car cands))
+ (list (+ start (nth 1 (car cands)) (- extra-offset))
+ ;; Candidates may be cached, so the end position should
+ ;; be adjusted according to current completion prefix.
+ (+ start (nth 2 (car cands)) (- extra-offset) prefix-offset)
+ cands
+ :annotation-function
+ (lambda (c) (concat " " (nth 3 (assoc c cands))))
+ :company-docsig
+ (lambda (c) (nth 4 (assoc c cands)))))))))
(define-obsolete-function-alias
'python-shell-completion-complete-at-point
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 59957ff0712..9f9914896be 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4799,6 +4799,97 @@ (ert-deftest python-shell-completion-at-point-native-1 ()
(end-of-line 0)
(should-not (nth 2 (python-shell-completion-at-point))))))
+(defun python-tests--completion-module ()
+ "Check if modules can be completed in Python shell."
+ (insert "import datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "import datetime"))
+ (kill-line)
+ (insert "from datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime"))
+ (end-of-line)
+ (insert " import timed")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime import timedelta"))
+ (kill-line))
+
+(defun python-tests--completion-parameters ()
+ "Check if parameters can be completed in Python shell."
+ (insert "import re")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "re.split('b', 'abc', maxs")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit="
+ (buffer-substring (line-beginning-position) (point))))
+ (insert "0, ")
+ (should (python-shell-completion-at-point))
+ ;; Test if cache is used.
+ (cl-letf (((symbol-function 'python-shell-completion-get-completions)
+ 'ignore)
+ ((symbol-function 'python-shell-completion-native-get-completions)
+ 'ignore))
+ (insert "fla")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit=0, flags="
+ (buffer-substring (line-beginning-position) (point)))))
+ (beginning-of-line)
+ (kill-line))
+
+(defun python-tests--completion-extra-context ()
+ "Check if extra context is used for completion."
+ (insert "re.split('b', 'abc',")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "maxs")
+ (completion-at-point)
+ (should (string= "maxsplit="
+ (buffer-substring (line-beginning-position) (point))))
+ (insert "0)")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "from re import (")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "IGN")
+ (completion-at-point)
+ (should (string= "IGNORECASE"
+ (buffer-substring (line-beginning-position) (point)))))
+
+(ert-deftest python-shell-completion-at-point-jedi-completer ()
+ "Check if Python shell completion works with Jedi."
+ (skip-unless (executable-find python-tests-shell-interpreter))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context))))
+
+(ert-deftest python-shell-completion-at-point-ipython ()
+ "Check if Python shell completion works for IPython."
+ (let ((python-shell-interpreter "ipython")
+ (python-shell-interpreter-args "-i --simple-prompt"))
+ (skip-unless
+ (and
+ (executable-find python-shell-interpreter)
+ (eql (call-process python-shell-interpreter nil nil nil "--version") 0)))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-off)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-shell-completion-native-turn-on)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context)))))
\f
;;; PDB Track integration
--
2.39.2
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-04 12:09 ` Liu Hui
@ 2024-02-04 14:35 ` kobarity
2024-02-05 15:03 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: kobarity @ 2024-02-04 14:35 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, 68559
Liu Hui wrote:
> kobarity <kobarity@gmail.com> writes:
> > I'm experiencing strange behavior regarding completion of import
> > statement in a block in Python buffer. If I try to type the following
> > lines and then try to complete it, it will fail.
> >
> > #+begin_src python
> > try:
> > im
> > #+end_src
>
> The problem can be reproduced by running python with jedi completer in
> a terminal. The reason is that readline completer can only see the
> text in the current line, i.e. " im"; in this case, jedi does not
> complete the text to " import".
>
> Similarly, the jedi completer cannot complete "except" after ex:
>
> try:
> pass
> ex|
>
> > However, when I try to complete at the beginning of the second line:
> >
> > #+begin_src python
> > try:
> >
> > #+end_src
> >
> >
> > "import" keyword also appears as a candidate. If I cancel the
> > candidates and type "im" and try to complete it, it will succeed.
>
> It is because jedi produces completions including " import" for
> blank string " ". Due to the same prefix between " " and
> " im", completion cache is reused for the latter. Then " import"
> can be completed.
>
> It is more a limitation of readline completer than a problem with
> jedi, as we cannot provide proper completion context for jedi. We may
> define a custom completer to combine jedi and rlcompleter, e.g.
>
> (setq python-shell-readline-completer "
> def __PYTHON_EL_setup_readline_completer():
> import readline, rlcompleter
> import re, sys, os, __main__
> from jedi import Interpreter
>
> class MyJediRL:
> def __init__(self):
> self.rlcompleter = rlcompleter.Completer()
> self.rldelim = readline.get_completer_delims()
>
> def complete(self, text, state):
> if state == 0:
> sys.path.insert(0, os.getcwd())
> try:
> interpreter = Interpreter(text, [__main__.__dict__])
> completions = interpreter.complete(fuzzy=False)
> self.matches = [
> text[:len(text) - c._like_name_length] + c.name_with_symbols
> for c in completions
> ]
>
> # try rlcompleter
> sub = re.split('[' + re.escape(self.rldelim) + ']', text)[-1]
> i = 0
> while True:
> completion = self.rlcompleter.complete(sub, i)
> if not completion:
> break
> i += 1
> completion = text[:len(text)-len(sub)] + completion.rstrip(' ()')
> if completion not in self.matches:
> self.matches.append(completion)
> except:
> raise
> finally:
> sys.path.pop(0)
> try:
> return self.matches[state]
> except IndexError:
> return None
>
> readline.set_completer(MyJediRL().complete)
> readline.set_completer_delims('')")
Thank you for the detailed explanation and the workaround. I
confirmed that the problem is solved by the above workaround. Just to
confirm, are you of the opinion that this workaround should not be the
default?
> > Another thing I noticed is the multi-line import statement. If the
> > import statement is one-line, each items (IGNORECASE and MULTILINE in
> > the example below) can be completed.
> >
> > #+begin_src python
> > from re import IGNORECASE, MULTILINE
> > #+end_src
> >
> >
> > However, they cannot be completed if the import statement spans
> > multi-line.
> >
> > #+begin_src python
> > from re import (
> > IGN
> > #+end_src
> >
> > This happens in both Python buffer and Python Shell buffer. Perhaps
> > this is a limitation of Jedi completer?
>
> Yes. Because readline completer cannot see cross-line context, I added
> the function "python-shell-completion-extra-context" in previous patch
> to address the case of multi-line function call. I have updated the
> attached patch to handle multi-line import statement.
Thank you very much. I confirmed that the new patch allows completion
of multi-line import statements.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-04 14:35 ` kobarity
@ 2024-02-05 15:03 ` Liu Hui
2024-02-06 1:25 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-05 15:03 UTC (permalink / raw)
To: kobarity; +Cc: Eli Zaretskii, 68559
[-- Attachment #1: Type: text/plain, Size: 2949 bytes --]
On Sun, Feb 4, 2024 at 10:35 PM kobarity <kobarity@gmail.com> wrote:
> > It is more a limitation of readline completer than a problem with
> > jedi, as we cannot provide proper completion context for jedi. We may
> > define a custom completer to combine jedi and rlcompleter, e.g.
> >
> > (setq python-shell-readline-completer "
> > def __PYTHON_EL_setup_readline_completer():
> > import readline, rlcompleter
> > import re, sys, os, __main__
> > from jedi import Interpreter
> >
> > class MyJediRL:
> > def __init__(self):
> > self.rlcompleter = rlcompleter.Completer()
> > self.rldelim = readline.get_completer_delims()
> >
> > def complete(self, text, state):
> > if state == 0:
> > sys.path.insert(0, os.getcwd())
> > try:
> > interpreter = Interpreter(text, [__main__.__dict__])
> > completions = interpreter.complete(fuzzy=False)
> > self.matches = [
> > text[:len(text) - c._like_name_length] + c.name_with_symbols
> > for c in completions
> > ]
> >
> > # try rlcompleter
> > sub = re.split('[' + re.escape(self.rldelim) + ']', text)[-1]
> > i = 0
> > while True:
> > completion = self.rlcompleter.complete(sub, i)
> > if not completion:
> > break
> > i += 1
> > completion = text[:len(text)-len(sub)] + completion.rstrip(' ()')
> > if completion not in self.matches:
> > self.matches.append(completion)
> > except:
> > raise
> > finally:
> > sys.path.pop(0)
> > try:
> > return self.matches[state]
> > except IndexError:
> > return None
> >
> > readline.set_completer(MyJediRL().complete)
> > readline.set_completer_delims('')")
>
> Thank you for the detailed explanation and the workaround. I
> confirmed that the problem is solved by the above workaround. Just to
> confirm, are you of the opinion that this workaround should not be the
> default?
I'm not sure if we should add more Python code in the form of strings
to python.el, which increases maintenance burden IMO. Maybe they could
be distributed separately at ELPA/Git forges.
Actually, I'm considering simplifying this patch to mainly fix the bug
that python shell completion doesn't respect the delimiter of readline
completer. The new patch has been attached. It should support
the completer defined in the PYTHONSTARTUP file, e.g., jedi or a
custom completer like the above one. WDYT?
[-- Attachment #2: 0001-Respect-the-delimiter-of-completer-in-Python-shell-c.patch --]
[-- Type: text/x-patch, Size: 20619 bytes --]
From 082596e908ee45d18c50c3b9f0b5020b779adb77 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Thu, 18 Jan 2024 12:00:00 +0800
Subject: [PATCH] Respect the delimiter of completer in Python shell completion
* lisp/progmodes/python.el: (python-shell-completion-setup-code): Fix
the completion code of IPython. Change the return value to JSON string
and ...
(python-shell-completion-get-completions): ... simplify parsing.
(inferior-python-mode): Update docstring.
(python-shell-readline-completer-delims): New variable indicating the
word delimiters of readline completer.
(python-shell-completion-native-setup): Set the completer delimiter.
(python-shell-completion-native-get-completions): Convert output string
to completions properly.
(python-shell--get-multiline-input)
(python-shell--extra-completion-context)
(python-shell-completion-extra-context): New functions.
(python-shell-completion-at-point): Send text beginning from the line
start if the completion backend does not need word splitting. Remove
the detection of import statement because it is not needed anymore.
Create proper completion table based on completions returned from
different backends.
* test/lisp/progmodes/python-tests.el (python-tests--completion-module)
(python-tests--completion-parameters)
(python-tests--completion-extra-context): New helper functions.
(python-shell-completion-at-point-jedi-completer)
(python-shell-completion-at-point-ipython): New tests.
* etc/NEWS: Announce the change. (bug#68559)
---
lisp/progmodes/python.el | 212 ++++++++++++++++++++++------
test/lisp/progmodes/python-tests.el | 92 ++++++++++++
2 files changed, 257 insertions(+), 47 deletions(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 9d840efb9da..45336c8396f 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -128,9 +128,9 @@ ;;; Commentary:
;; receiving escape sequences (with some limitations, i.e. completion
;; in blocks does not work). The code executed for the "fallback"
;; completion can be found in `python-shell-completion-setup-code' and
-;; `python-shell-completion-string-code' variables. Their default
-;; values enable completion for both CPython and IPython, and probably
-;; any readline based shell (it's known to work with PyPy). If your
+;; `python-shell-completion-get-completions'. Their default values
+;; enable completion for both CPython and IPython, and probably any
+;; readline based shell (it's known to work with PyPy). If your
;; Python installation lacks readline (like CPython for Windows),
;; installing pyreadline (URL `https://ipython.org/pyreadline.html')
;; should suffice. To troubleshoot why you are not getting any
@@ -3604,7 +3604,6 @@ (define-derived-mode inferior-python-mode comint-mode "Inferior Python"
`python-shell-prompt-block-regexp',
`python-shell-font-lock-enable',
`python-shell-completion-setup-code',
-`python-shell-completion-string-code',
`python-eldoc-setup-code',
`python-ffap-setup-code' can
customize this mode for different Python interpreters.
@@ -4244,8 +4243,9 @@ (defcustom python-shell-completion-setup-code
completions = []
completer = None
+ import json
try:
- import readline
+ import readline, re
try:
import __builtin__
@@ -4256,16 +4256,29 @@ (defcustom python-shell-completion-setup-code
is_ipython = ('__IPYTHON__' in builtins or
'__IPYTHON__active' in builtins)
- splits = text.split()
- is_module = splits and splits[0] in ('from', 'import')
-
- if is_ipython and is_module:
- from IPython.core.completerlib import module_completion
- completions = module_completion(text.strip())
- elif is_ipython and '__IP' in builtins:
- completions = __IP.complete(text)
- elif is_ipython and 'get_ipython' in builtins:
- completions = get_ipython().Completer.all_completions(text)
+
+ if is_ipython and 'get_ipython' in builtins:
+ def filter_c(prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ import IPython
+ try:
+ if IPython.version_info[0] >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ completions = [
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if filter_c(text, c.text)]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ completions = [text + m[len(part):] for m in matches if filter_c(text, m)]
+ except:
+ pass
else:
# Try to reuse current completer.
completer = readline.get_completer()
@@ -4288,7 +4301,7 @@ (defcustom python-shell-completion-setup-code
finally:
if getattr(completer, 'PYTHON_EL_WRAPPED', False):
completer.print_mode = True
- return completions"
+ return json.dumps(completions)"
"Code used to setup completion in inferior Python processes."
:type 'string)
@@ -4329,6 +4342,10 @@ (defcustom python-shell-completion-native-try-output-timeout 1.0
:version "25.1"
:type 'float)
+(defvar python-shell-readline-completer-delims nil
+ "Word delimiters used by the readline completer.
+It is automatically set by Python shell.")
+
(defvar python-shell-completion-native-redirect-buffer
" *Python completions redirect*"
"Buffer to be used to redirect output of readline commands.")
@@ -4467,6 +4484,10 @@ (defun python-shell-completion-native-setup ()
__PYTHON_EL_native_completion_setup()" process)))
(when (string-match-p "python\\.el: native completion setup loaded"
output)
+ (setq-local python-shell-readline-completer-delims
+ (string-trim-right
+ (python-shell-send-string-no-output
+ "import readline; print(readline.get_completer_delims())")))
(python-shell-completion-native-try))))
(defun python-shell-completion-native-turn-off (&optional msg)
@@ -4534,6 +4555,8 @@ (defun python-shell-completion-native-get-completions (process input)
(let* ((original-filter-fn (process-filter process))
(redirect-buffer (get-buffer-create
python-shell-completion-native-redirect-buffer))
+ (sep (if (string= python-shell-readline-completer-delims "")
+ "[\n\r]+" "[ \f\t\n\r\v()]+"))
(trigger "\t")
(new-input (concat input trigger))
(input-length
@@ -4576,28 +4599,80 @@ (defun python-shell-completion-native-get-completions (process input)
process python-shell-completion-native-output-timeout
comint-redirect-finished-regexp)
(re-search-backward "0__dummy_completion__" nil t)
- (cl-remove-duplicates
- (split-string
- (buffer-substring-no-properties
- (line-beginning-position) (point-min))
- "[ \f\t\n\r\v()]+" t)
- :test #'string=))))
+ (let ((str (buffer-substring-no-properties
+ (line-beginning-position) (point-min))))
+ ;; The readline completer is allowed to return a list
+ ;; of (text start end type signature) as a JSON
+ ;; string. See the return value for IPython in
+ ;; `python-shell-completion-setup-code'.
+ (if (string= "[" (substring str 0 1))
+ (condition-case nil
+ (python--parse-json-array str)
+ (t (cl-remove-duplicates (split-string str sep t)
+ :test #'string=)))
+ (cl-remove-duplicates (split-string str sep t)
+ :test #'string=))))))
(set-process-filter process original-filter-fn)))))
(defun python-shell-completion-get-completions (process input)
"Get completions of INPUT using PROCESS."
(with-current-buffer (process-buffer process)
- (let ((completions
- (python-util-strip-string
- (python-shell-send-string-no-output
- (format
- "%s\nprint(';'.join(__PYTHON_EL_get_completions(%s)))"
+ (python--parse-json-array
+ (python-shell-send-string-no-output
+ (format "%s\nprint(__PYTHON_EL_get_completions(%s))"
python-shell-completion-setup-code
(python-shell--encode-string input))
- process))))
- (when (> (length completions) 2)
- (split-string completions
- "^'\\|^\"\\|;\\|'$\\|\"$" t)))))
+ process))))
+
+(defun python-shell--get-multiline-input ()
+ "Return lines at a multi-line input in Python shell."
+ (save-excursion
+ (let ((p (point)) lines)
+ (when (progn
+ (beginning-of-line)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) p) lines)
+ (while (progn (comint-previous-prompt 1)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ lines)))
+
+(defun python-shell--extra-completion-context ()
+ "Get extra completion context of current input in Python shell."
+ (let ((lines (python-shell--get-multiline-input))
+ (python-indent-guess-indent-offset nil))
+ (when (not (zerop (length lines)))
+ (with-temp-buffer
+ (delay-mode-hooks
+ (insert (string-join lines "\n"))
+ (python-mode)
+ (python-shell-completion-extra-context))))))
+
+(defun python-shell-completion-extra-context (&optional pos)
+ "Get extra completion context at position POS in Python buffer.
+If optional argument POS is nil, use current position.
+
+Readline completers could use current line as the completion
+context, which may be insufficient. In this function, extra
+context (e.g. multi-line function call) is found and reformatted
+as one line, which is required by native completion."
+ (let (bound p)
+ (save-excursion
+ (and pos (goto-char pos))
+ (setq bound (pos-bol))
+ (python-nav-up-list -1)
+ (when (and (< (point) bound)
+ (or
+ (looking-back
+ (python-rx (group (+ (or "." symbol-name)))) (pos-bol) t)
+ (progn
+ (forward-line 0)
+ (looking-at "^[ \t]*\\(from \\)"))))
+ (setq p (match-beginning 1))))
+ (when p
+ (replace-regexp-in-string
+ "\n[ \t]*" "" (buffer-substring-no-properties p (1- bound))))))
(defvar-local python-shell--capf-cache nil
"Variable to store cached completions and invalidation keys.")
@@ -4612,21 +4687,26 @@ (defun python-shell-completion-at-point (&optional process)
;; Working on a shell buffer: use prompt end.
(cdr (python-util-comint-last-prompt))
(line-beginning-position)))
- (import-statement
- (when (string-match-p
- (rx (* space) word-start (or "from" "import") word-end space)
- (buffer-substring-no-properties line-start (point)))
- (buffer-substring-no-properties line-start (point))))
+ (no-delims
+ (and (not (if is-shell-buffer
+ (eq 'font-lock-comment-face
+ (get-text-property (1- (point)) 'face))
+ (python-syntax-context 'comment)))
+ (with-current-buffer (process-buffer process)
+ (if python-shell-completion-native-enable
+ (string= python-shell-readline-completer-delims "")
+ (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
(start
(if (< (point) line-start)
(point)
(save-excursion
- (if (not (re-search-backward
- (python-rx
- (or whitespace open-paren close-paren
- string-delimiter simple-operator))
- line-start
- t 1))
+ (if (or no-delims
+ (not (re-search-backward
+ (python-rx
+ (or whitespace open-paren close-paren
+ string-delimiter simple-operator))
+ line-start
+ t 1)))
line-start
(forward-char (length (match-string-no-properties 0)))
(point)))))
@@ -4666,18 +4746,56 @@ (defun python-shell-completion-at-point (&optional process)
(t #'python-shell-completion-native-get-completions))))
(prev-prompt (car python-shell--capf-cache))
(re (or (cadr python-shell--capf-cache) regexp-unmatchable))
- (prefix (buffer-substring-no-properties start end)))
+ (prefix (buffer-substring-no-properties start end))
+ (prefix-offset 0)
+ (extra-context (when no-delims
+ (if is-shell-buffer
+ (python-shell--extra-completion-context)
+ (python-shell-completion-extra-context))))
+ (extra-offset (length extra-context)))
+ (unless (zerop extra-offset)
+ (setq prefix (concat extra-context prefix)))
;; To invalidate the cache, we check if the prompt position or the
;; completion prefix changed.
(unless (and (equal prev-prompt (car prompt-boundaries))
- (string-match re prefix))
+ (string-match re prefix)
+ (setq prefix-offset (- (length prefix) (match-end 1))))
(setq python-shell--capf-cache
`(,(car prompt-boundaries)
,(if (string-empty-p prefix)
regexp-unmatchable
- (concat "\\`" (regexp-quote prefix) "\\(?:\\sw\\|\\s_\\)*\\'"))
- ,@(funcall completion-fn process (or import-statement prefix)))))
- (list start end (cddr python-shell--capf-cache))))
+ (concat "\\`\\(" (regexp-quote prefix) "\\)\\(?:\\sw\\|\\s_\\)*\\'"))
+ ,@(funcall completion-fn process prefix))))
+ (let ((cands (cddr python-shell--capf-cache)))
+ (cond
+ ((stringp (car cands))
+ (if no-delims
+ ;; Reduce completion candidates due to long prefix.
+ (if-let ((Lp (length prefix))
+ ((string-match "\\(\\sw\\|\\s_\\)+\\'" prefix))
+ (L (match-beginning 0)))
+ ;; If extra-offset is not zero:
+ ;; start end
+ ;; o------------------o---------o-------o
+ ;; |<- extra-offset ->|
+ ;; |<----------- L ------------>|
+ ;; new-start
+ (list (+ start L (- extra-offset)) end
+ (mapcar (lambda (s) (substring s L)) cands))
+ (list end end (mapcar (lambda (s) (substring s Lp)) cands)))
+ (list start end cands)))
+ ;; python-shell-completion(-native)-get-completions may produce a
+ ;; list of (text start end type signature) for completion.
+ ((consp (car cands))
+ (list (+ start (nth 1 (car cands)) (- extra-offset))
+ ;; Candidates may be cached, so the end position should
+ ;; be adjusted according to current completion prefix.
+ (+ start (nth 2 (car cands)) (- extra-offset) prefix-offset)
+ cands
+ :annotation-function
+ (lambda (c) (concat " " (nth 3 (assoc c cands))))
+ :company-docsig
+ (lambda (c) (nth 4 (assoc c cands)))))))))
(define-obsolete-function-alias
'python-shell-completion-complete-at-point
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 59957ff0712..af6c199b5bd 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4799,6 +4799,98 @@ (ert-deftest python-shell-completion-at-point-native-1 ()
(end-of-line 0)
(should-not (nth 2 (python-shell-completion-at-point))))))
+(defun python-tests--completion-module ()
+ "Check if modules can be completed in Python shell."
+ (insert "import datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "import datetime"))
+ (kill-line)
+ (insert "from datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime"))
+ (end-of-line)
+ (insert " import timed")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime import timedelta"))
+ (kill-line))
+
+(defun python-tests--completion-parameters ()
+ "Check if parameters can be completed in Python shell."
+ (insert "import re")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "re.split('b', 'abc', maxs")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit="
+ (buffer-substring (line-beginning-position) (point))))
+ (insert "0, ")
+ (should (python-shell-completion-at-point))
+ ;; Test if cache is used.
+ (cl-letf (((symbol-function 'python-shell-completion-get-completions)
+ 'ignore)
+ ((symbol-function 'python-shell-completion-native-get-completions)
+ 'ignore))
+ (insert "fla")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit=0, flags="
+ (buffer-substring (line-beginning-position) (point)))))
+ (beginning-of-line)
+ (kill-line))
+
+(defun python-tests--completion-extra-context ()
+ "Check if extra context is used for completion."
+ (insert "re.split('b', 'abc',")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "maxs")
+ (completion-at-point)
+ (should (string= "maxsplit="
+ (buffer-substring (line-beginning-position) (point))))
+ (insert "0)")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "from re import (")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "IGN")
+ (completion-at-point)
+ (should (string= "IGNORECASE"
+ (buffer-substring (line-beginning-position) (point)))))
+
+(ert-deftest python-shell-completion-at-point-jedi-completer ()
+ "Check if Python shell completion works when Jedi completer is used."
+ (skip-unless (executable-find python-tests-shell-interpreter))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context))))
+
+(ert-deftest python-shell-completion-at-point-ipython ()
+ "Check if Python shell completion works for IPython."
+ (let ((python-shell-interpreter "ipython")
+ (python-shell-interpreter-args "-i --simple-prompt"))
+ (skip-unless
+ (and
+ (executable-find python-shell-interpreter)
+ (eql (call-process python-shell-interpreter nil nil nil "--version") 0)))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-off)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context)))))
\f
;;; PDB Track integration
--
2.39.2
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-05 15:03 ` Liu Hui
@ 2024-02-06 1:25 ` Liu Hui
2024-02-06 15:12 ` kobarity
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-06 1:25 UTC (permalink / raw)
To: kobarity; +Cc: Eli Zaretskii, 68559
[-- Attachment #1: Type: text/plain, Size: 949 bytes --]
On Mon, Feb 5, 2024 at 11:03 PM Liu Hui <liuhui1610@gmail.com> wrote:
> > Thank you for the detailed explanation and the workaround. I
> > confirmed that the problem is solved by the above workaround. Just to
> > confirm, are you of the opinion that this workaround should not be the
> > default?
>
> I'm not sure if we should add more Python code in the form of strings
> to python.el, which increases maintenance burden IMO. Maybe they could
> be distributed separately at ELPA/Git forges.
>
> Actually, I'm considering simplifying this patch to mainly fix the bug
> that python shell completion doesn't respect the delimiter of readline
> completer. The new patch has been attached. It should support
> the completer defined in the PYTHONSTARTUP file, e.g., jedi or a
> custom completer like the above one. WDYT?
I have updated documentation about readline completer and fixed an
error in commit log in the attach patch.
[-- Attachment #2: 0001-Respect-the-delimiter-of-completer-in-Python-shell-c.patch --]
[-- Type: text/x-patch, Size: 21255 bytes --]
From 529e5f6180197130ec39dbddd39b2aa2b6deff67 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Thu, 18 Jan 2024 12:00:00 +0800
Subject: [PATCH] Respect the delimiter of completer in Python shell completion
* lisp/progmodes/python.el: (python-shell-completion-setup-code): Fix
the completion code of IPython. Change the return value to JSON string
and ...
(python-shell-completion-get-completions): ... simplify parsing.
(inferior-python-mode): Update docstring.
(python-shell-readline-completer-delims): New variable indicating the
word delimiters of readline completer.
(python-shell-completion-native-setup): Set the completer delimiter.
(python-shell-completion-native-get-completions): Convert output string
to completions properly.
(python-shell--get-multiline-input)
(python-shell--extra-completion-context)
(python-shell-completion-extra-context): New functions.
(python-shell-completion-at-point): Send text beginning from the line
start if the completion backend does not need word splitting. Remove
the detection of import statement because it is not needed anymore.
Create proper completion table based on completions returned from
different backends.
* test/lisp/progmodes/python-tests.el (python-tests--completion-module)
(python-tests--completion-parameters)
(python-tests--completion-extra-context): New helper functions.
(python-shell-completion-at-point-jedi-completer)
(python-shell-completion-at-point-ipython): New tests. (bug#68559)
---
lisp/progmodes/python.el | 218 ++++++++++++++++++++++------
test/lisp/progmodes/python-tests.el | 92 ++++++++++++
2 files changed, 263 insertions(+), 47 deletions(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 9d840efb9da..d88da4e2700 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -128,9 +128,9 @@ ;;; Commentary:
;; receiving escape sequences (with some limitations, i.e. completion
;; in blocks does not work). The code executed for the "fallback"
;; completion can be found in `python-shell-completion-setup-code' and
-;; `python-shell-completion-string-code' variables. Their default
-;; values enable completion for both CPython and IPython, and probably
-;; any readline based shell (it's known to work with PyPy). If your
+;; `python-shell-completion-get-completions'. Their default values
+;; enable completion for both CPython and IPython, and probably any
+;; readline based shell (it's known to work with PyPy). If your
;; Python installation lacks readline (like CPython for Windows),
;; installing pyreadline (URL `https://ipython.org/pyreadline.html')
;; should suffice. To troubleshoot why you are not getting any
@@ -141,6 +141,12 @@ ;;; Commentary:
;; If you see an error, then you need to either install pyreadline or
;; setup custom code that avoids that dependency.
+;; By default, the "native" completion uses the built-in rlcompleter.
+;; To use other readline completer (e.g. Jedi) or a custom one, you just
+;; need to set it in the PYTHONSTARTUP file. You can set an
+;; Emacs-specific completer by testing the environment variable
+;; INSIDE_EMACS.
+
;; Shell virtualenv support: The shell also contains support for
;; virtualenvs and other special environment modifications thanks to
;; `python-shell-process-environment' and `python-shell-exec-path'.
@@ -3604,7 +3610,6 @@ (define-derived-mode inferior-python-mode comint-mode "Inferior Python"
`python-shell-prompt-block-regexp',
`python-shell-font-lock-enable',
`python-shell-completion-setup-code',
-`python-shell-completion-string-code',
`python-eldoc-setup-code',
`python-ffap-setup-code' can
customize this mode for different Python interpreters.
@@ -4244,8 +4249,9 @@ (defcustom python-shell-completion-setup-code
completions = []
completer = None
+ import json
try:
- import readline
+ import readline, re
try:
import __builtin__
@@ -4256,16 +4262,29 @@ (defcustom python-shell-completion-setup-code
is_ipython = ('__IPYTHON__' in builtins or
'__IPYTHON__active' in builtins)
- splits = text.split()
- is_module = splits and splits[0] in ('from', 'import')
-
- if is_ipython and is_module:
- from IPython.core.completerlib import module_completion
- completions = module_completion(text.strip())
- elif is_ipython and '__IP' in builtins:
- completions = __IP.complete(text)
- elif is_ipython and 'get_ipython' in builtins:
- completions = get_ipython().Completer.all_completions(text)
+
+ if is_ipython and 'get_ipython' in builtins:
+ def filter_c(prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ import IPython
+ try:
+ if IPython.version_info[0] >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ completions = [
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if filter_c(text, c.text)]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ completions = [text + m[len(part):] for m in matches if filter_c(text, m)]
+ except:
+ pass
else:
# Try to reuse current completer.
completer = readline.get_completer()
@@ -4288,7 +4307,7 @@ (defcustom python-shell-completion-setup-code
finally:
if getattr(completer, 'PYTHON_EL_WRAPPED', False):
completer.print_mode = True
- return completions"
+ return json.dumps(completions)"
"Code used to setup completion in inferior Python processes."
:type 'string)
@@ -4329,6 +4348,10 @@ (defcustom python-shell-completion-native-try-output-timeout 1.0
:version "25.1"
:type 'float)
+(defvar python-shell-readline-completer-delims nil
+ "Word delimiters used by the readline completer.
+It is automatically set by Python shell.")
+
(defvar python-shell-completion-native-redirect-buffer
" *Python completions redirect*"
"Buffer to be used to redirect output of readline commands.")
@@ -4467,6 +4490,10 @@ (defun python-shell-completion-native-setup ()
__PYTHON_EL_native_completion_setup()" process)))
(when (string-match-p "python\\.el: native completion setup loaded"
output)
+ (setq-local python-shell-readline-completer-delims
+ (string-trim-right
+ (python-shell-send-string-no-output
+ "import readline; print(readline.get_completer_delims())")))
(python-shell-completion-native-try))))
(defun python-shell-completion-native-turn-off (&optional msg)
@@ -4534,6 +4561,8 @@ (defun python-shell-completion-native-get-completions (process input)
(let* ((original-filter-fn (process-filter process))
(redirect-buffer (get-buffer-create
python-shell-completion-native-redirect-buffer))
+ (sep (if (string= python-shell-readline-completer-delims "")
+ "[\n\r]+" "[ \f\t\n\r\v()]+"))
(trigger "\t")
(new-input (concat input trigger))
(input-length
@@ -4576,28 +4605,80 @@ (defun python-shell-completion-native-get-completions (process input)
process python-shell-completion-native-output-timeout
comint-redirect-finished-regexp)
(re-search-backward "0__dummy_completion__" nil t)
- (cl-remove-duplicates
- (split-string
- (buffer-substring-no-properties
- (line-beginning-position) (point-min))
- "[ \f\t\n\r\v()]+" t)
- :test #'string=))))
+ (let ((str (buffer-substring-no-properties
+ (line-beginning-position) (point-min))))
+ ;; The readline completer is allowed to return a list
+ ;; of (text start end type signature) as a JSON
+ ;; string. See the return value for IPython in
+ ;; `python-shell-completion-setup-code'.
+ (if (string= "[" (substring str 0 1))
+ (condition-case nil
+ (python--parse-json-array str)
+ (t (cl-remove-duplicates (split-string str sep t)
+ :test #'string=)))
+ (cl-remove-duplicates (split-string str sep t)
+ :test #'string=))))))
(set-process-filter process original-filter-fn)))))
(defun python-shell-completion-get-completions (process input)
"Get completions of INPUT using PROCESS."
(with-current-buffer (process-buffer process)
- (let ((completions
- (python-util-strip-string
- (python-shell-send-string-no-output
- (format
- "%s\nprint(';'.join(__PYTHON_EL_get_completions(%s)))"
+ (python--parse-json-array
+ (python-shell-send-string-no-output
+ (format "%s\nprint(__PYTHON_EL_get_completions(%s))"
python-shell-completion-setup-code
(python-shell--encode-string input))
- process))))
- (when (> (length completions) 2)
- (split-string completions
- "^'\\|^\"\\|;\\|'$\\|\"$" t)))))
+ process))))
+
+(defun python-shell--get-multiline-input ()
+ "Return lines at a multi-line input in Python shell."
+ (save-excursion
+ (let ((p (point)) lines)
+ (when (progn
+ (beginning-of-line)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) p) lines)
+ (while (progn (comint-previous-prompt 1)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ lines)))
+
+(defun python-shell--extra-completion-context ()
+ "Get extra completion context of current input in Python shell."
+ (let ((lines (python-shell--get-multiline-input))
+ (python-indent-guess-indent-offset nil))
+ (when (not (zerop (length lines)))
+ (with-temp-buffer
+ (delay-mode-hooks
+ (insert (string-join lines "\n"))
+ (python-mode)
+ (python-shell-completion-extra-context))))))
+
+(defun python-shell-completion-extra-context (&optional pos)
+ "Get extra completion context at position POS in Python buffer.
+If optional argument POS is nil, use current position.
+
+Readline completers could use current line as the completion
+context, which may be insufficient. In this function, extra
+context (e.g. multi-line function call) is found and reformatted
+as one line, which is required by native completion."
+ (let (bound p)
+ (save-excursion
+ (and pos (goto-char pos))
+ (setq bound (pos-bol))
+ (python-nav-up-list -1)
+ (when (and (< (point) bound)
+ (or
+ (looking-back
+ (python-rx (group (+ (or "." symbol-name)))) (pos-bol) t)
+ (progn
+ (forward-line 0)
+ (looking-at "^[ \t]*\\(from \\)"))))
+ (setq p (match-beginning 1))))
+ (when p
+ (replace-regexp-in-string
+ "\n[ \t]*" "" (buffer-substring-no-properties p (1- bound))))))
(defvar-local python-shell--capf-cache nil
"Variable to store cached completions and invalidation keys.")
@@ -4612,21 +4693,26 @@ (defun python-shell-completion-at-point (&optional process)
;; Working on a shell buffer: use prompt end.
(cdr (python-util-comint-last-prompt))
(line-beginning-position)))
- (import-statement
- (when (string-match-p
- (rx (* space) word-start (or "from" "import") word-end space)
- (buffer-substring-no-properties line-start (point)))
- (buffer-substring-no-properties line-start (point))))
+ (no-delims
+ (and (not (if is-shell-buffer
+ (eq 'font-lock-comment-face
+ (get-text-property (1- (point)) 'face))
+ (python-syntax-context 'comment)))
+ (with-current-buffer (process-buffer process)
+ (if python-shell-completion-native-enable
+ (string= python-shell-readline-completer-delims "")
+ (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
(start
(if (< (point) line-start)
(point)
(save-excursion
- (if (not (re-search-backward
- (python-rx
- (or whitespace open-paren close-paren
- string-delimiter simple-operator))
- line-start
- t 1))
+ (if (or no-delims
+ (not (re-search-backward
+ (python-rx
+ (or whitespace open-paren close-paren
+ string-delimiter simple-operator))
+ line-start
+ t 1)))
line-start
(forward-char (length (match-string-no-properties 0)))
(point)))))
@@ -4666,18 +4752,56 @@ (defun python-shell-completion-at-point (&optional process)
(t #'python-shell-completion-native-get-completions))))
(prev-prompt (car python-shell--capf-cache))
(re (or (cadr python-shell--capf-cache) regexp-unmatchable))
- (prefix (buffer-substring-no-properties start end)))
+ (prefix (buffer-substring-no-properties start end))
+ (prefix-offset 0)
+ (extra-context (when no-delims
+ (if is-shell-buffer
+ (python-shell--extra-completion-context)
+ (python-shell-completion-extra-context))))
+ (extra-offset (length extra-context)))
+ (unless (zerop extra-offset)
+ (setq prefix (concat extra-context prefix)))
;; To invalidate the cache, we check if the prompt position or the
;; completion prefix changed.
(unless (and (equal prev-prompt (car prompt-boundaries))
- (string-match re prefix))
+ (string-match re prefix)
+ (setq prefix-offset (- (length prefix) (match-end 1))))
(setq python-shell--capf-cache
`(,(car prompt-boundaries)
,(if (string-empty-p prefix)
regexp-unmatchable
- (concat "\\`" (regexp-quote prefix) "\\(?:\\sw\\|\\s_\\)*\\'"))
- ,@(funcall completion-fn process (or import-statement prefix)))))
- (list start end (cddr python-shell--capf-cache))))
+ (concat "\\`\\(" (regexp-quote prefix) "\\)\\(?:\\sw\\|\\s_\\)*\\'"))
+ ,@(funcall completion-fn process prefix))))
+ (let ((cands (cddr python-shell--capf-cache)))
+ (cond
+ ((stringp (car cands))
+ (if no-delims
+ ;; Reduce completion candidates due to long prefix.
+ (if-let ((Lp (length prefix))
+ ((string-match "\\(\\sw\\|\\s_\\)+\\'" prefix))
+ (L (match-beginning 0)))
+ ;; If extra-offset is not zero:
+ ;; start end
+ ;; o------------------o---------o-------o
+ ;; |<- extra-offset ->|
+ ;; |<----------- L ------------>|
+ ;; new-start
+ (list (+ start L (- extra-offset)) end
+ (mapcar (lambda (s) (substring s L)) cands))
+ (list end end (mapcar (lambda (s) (substring s Lp)) cands)))
+ (list start end cands)))
+ ;; python-shell-completion(-native)-get-completions may produce a
+ ;; list of (text start end type signature) for completion.
+ ((consp (car cands))
+ (list (+ start (nth 1 (car cands)) (- extra-offset))
+ ;; Candidates may be cached, so the end position should
+ ;; be adjusted according to current completion prefix.
+ (+ start (nth 2 (car cands)) (- extra-offset) prefix-offset)
+ cands
+ :annotation-function
+ (lambda (c) (concat " " (nth 3 (assoc c cands))))
+ :company-docsig
+ (lambda (c) (nth 4 (assoc c cands)))))))))
(define-obsolete-function-alias
'python-shell-completion-complete-at-point
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 59957ff0712..af6c199b5bd 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4799,6 +4799,98 @@ (ert-deftest python-shell-completion-at-point-native-1 ()
(end-of-line 0)
(should-not (nth 2 (python-shell-completion-at-point))))))
+(defun python-tests--completion-module ()
+ "Check if modules can be completed in Python shell."
+ (insert "import datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "import datetime"))
+ (kill-line)
+ (insert "from datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime"))
+ (end-of-line)
+ (insert " import timed")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime import timedelta"))
+ (kill-line))
+
+(defun python-tests--completion-parameters ()
+ "Check if parameters can be completed in Python shell."
+ (insert "import re")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "re.split('b', 'abc', maxs")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit="
+ (buffer-substring (line-beginning-position) (point))))
+ (insert "0, ")
+ (should (python-shell-completion-at-point))
+ ;; Test if cache is used.
+ (cl-letf (((symbol-function 'python-shell-completion-get-completions)
+ 'ignore)
+ ((symbol-function 'python-shell-completion-native-get-completions)
+ 'ignore))
+ (insert "fla")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit=0, flags="
+ (buffer-substring (line-beginning-position) (point)))))
+ (beginning-of-line)
+ (kill-line))
+
+(defun python-tests--completion-extra-context ()
+ "Check if extra context is used for completion."
+ (insert "re.split('b', 'abc',")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "maxs")
+ (completion-at-point)
+ (should (string= "maxsplit="
+ (buffer-substring (line-beginning-position) (point))))
+ (insert "0)")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "from re import (")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "IGN")
+ (completion-at-point)
+ (should (string= "IGNORECASE"
+ (buffer-substring (line-beginning-position) (point)))))
+
+(ert-deftest python-shell-completion-at-point-jedi-completer ()
+ "Check if Python shell completion works when Jedi completer is used."
+ (skip-unless (executable-find python-tests-shell-interpreter))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context))))
+
+(ert-deftest python-shell-completion-at-point-ipython ()
+ "Check if Python shell completion works for IPython."
+ (let ((python-shell-interpreter "ipython")
+ (python-shell-interpreter-args "-i --simple-prompt"))
+ (skip-unless
+ (and
+ (executable-find python-shell-interpreter)
+ (eql (call-process python-shell-interpreter nil nil nil "--version") 0)))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-off)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context)))))
\f
;;; PDB Track integration
--
2.39.2
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-06 1:25 ` Liu Hui
@ 2024-02-06 15:12 ` kobarity
2024-02-07 13:22 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: kobarity @ 2024-02-06 15:12 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, 68559
Liu Hui wrote:
> On Mon, Feb 5, 2024 at 11:03 PM Liu Hui <liuhui1610@gmail.com> wrote:
>
> > > Thank you for the detailed explanation and the workaround. I
> > > confirmed that the problem is solved by the above workaround. Just to
> > > confirm, are you of the opinion that this workaround should not be the
> > > default?
> >
> > I'm not sure if we should add more Python code in the form of strings
> > to python.el, which increases maintenance burden IMO. Maybe they could
> > be distributed separately at ELPA/Git forges.
> >
> > Actually, I'm considering simplifying this patch to mainly fix the bug
> > that python shell completion doesn't respect the delimiter of readline
> > completer. The new patch has been attached. It should support
> > the completer defined in the PYTHONSTARTUP file, e.g., jedi or a
> > custom completer like the above one. WDYT?
>
> I have updated documentation about readline completer and fixed an
> error in commit log in the attach patch.
Thanks, I understood your concern. I tried the new patch and
confirmed that your MyJediRL can be used as a custom completer by
storing it in a file specified in the PYTHONSTARTUP environment
variable. This may be the better way to go.
Sorry, I noticed one more thing. As written in the comments below, it
would be better to use `line-beginning-position' and
`line-end-position' instead of `pos-bol' and `pos-eol'.
;; This is a GNU ELPA :core package. Avoid functionality that is not
;; compatible with the version of Emacs recorded above.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-06 15:12 ` kobarity
@ 2024-02-07 13:22 ` Liu Hui
2024-02-07 15:19 ` kobarity
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-07 13:22 UTC (permalink / raw)
To: kobarity; +Cc: Eli Zaretskii, 68559
[-- Attachment #1: Type: text/plain, Size: 730 bytes --]
On Tue, Feb 6, 2024 at 11:12 PM kobarity <kobarity@gmail.com> wrote:
> Sorry, I noticed one more thing. As written in the comments below, it
> would be better to use `line-beginning-position' and
> `line-end-position' instead of `pos-bol' and `pos-eol'.
>
> ;; This is a GNU ELPA :core package. Avoid functionality that is not
> ;; compatible with the version of Emacs recorded above.
Thanks for pointing it out! `pos-bol' was used because it can ignore
the prompt in `python-shell--get-multiline-input'. Fortunately, I find
`pos-bol' and `pos-eol' are available in the compat package since
29.1.1.0, so it should be enough to upgrade the compat version in
Package-Requires. I have updated the attached patch.
[-- Attachment #2: 0001-Respect-the-delimiter-of-completer-in-Python-shell-c.patch --]
[-- Type: text/x-patch, Size: 21693 bytes --]
From a6da4c0b88326859de5847aaa97816f89fc9782d Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Thu, 18 Jan 2024 12:00:00 +0800
Subject: [PATCH] Respect the delimiter of completer in Python shell completion
* lisp/progmodes/python.el: (python-shell-completion-setup-code): Fix
the completion code of IPython. Change the return value to JSON string
and ...
(python-shell-completion-get-completions): ... simplify parsing.
(inferior-python-mode): Update docstring.
(python-shell-readline-completer-delims): New variable indicating the
word delimiters of readline completer.
(python-shell-completion-native-setup): Set the completer delimiter.
(python-shell-completion-native-get-completions): Convert output string
to completions properly.
(python-shell--get-multiline-input)
(python-shell--extra-completion-context)
(python-shell-completion-extra-context): New functions.
(python-shell-completion-at-point): Send text beginning from the line
start if the completion backend does not need word splitting. Remove
the detection of import statement because it is not needed anymore.
Create proper completion table based on completions returned from
different backends.
* test/lisp/progmodes/python-tests.el (python-tests--completion-module)
(python-tests--completion-parameters)
(python-tests--completion-extra-context): New helper functions.
(python-shell-completion-at-point-jedi-completer)
(python-shell-completion-at-point-ipython): New tests. (bug#68559)
---
lisp/progmodes/python.el | 220 ++++++++++++++++++++++------
test/lisp/progmodes/python-tests.el | 92 ++++++++++++
2 files changed, 264 insertions(+), 48 deletions(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 9d840efb9da..b1654b6a5aa 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -5,7 +5,7 @@ ;;; python.el --- Python's flying circus support for Emacs -*- lexical-binding:
;; Author: Fabián E. Gallina <fgallina@gnu.org>
;; URL: https://github.com/fgallina/python.el
;; Version: 0.28
-;; Package-Requires: ((emacs "24.4") (compat "28.1.2.1") (seq "2.23"))
+;; Package-Requires: ((emacs "24.4") (compat "29.1.1.0") (seq "2.23"))
;; Maintainer: emacs-devel@gnu.org
;; Created: Jul 2010
;; Keywords: languages
@@ -128,9 +128,9 @@ ;;; Commentary:
;; receiving escape sequences (with some limitations, i.e. completion
;; in blocks does not work). The code executed for the "fallback"
;; completion can be found in `python-shell-completion-setup-code' and
-;; `python-shell-completion-string-code' variables. Their default
-;; values enable completion for both CPython and IPython, and probably
-;; any readline based shell (it's known to work with PyPy). If your
+;; `python-shell-completion-get-completions'. Their default values
+;; enable completion for both CPython and IPython, and probably any
+;; readline based shell (it's known to work with PyPy). If your
;; Python installation lacks readline (like CPython for Windows),
;; installing pyreadline (URL `https://ipython.org/pyreadline.html')
;; should suffice. To troubleshoot why you are not getting any
@@ -141,6 +141,12 @@ ;;; Commentary:
;; If you see an error, then you need to either install pyreadline or
;; setup custom code that avoids that dependency.
+;; By default, the "native" completion uses the built-in rlcompleter.
+;; To use other readline completer (e.g. Jedi) or a custom one, you just
+;; need to set it in the PYTHONSTARTUP file. You can set an
+;; Emacs-specific completer by testing the environment variable
+;; INSIDE_EMACS.
+
;; Shell virtualenv support: The shell also contains support for
;; virtualenvs and other special environment modifications thanks to
;; `python-shell-process-environment' and `python-shell-exec-path'.
@@ -3604,7 +3610,6 @@ (define-derived-mode inferior-python-mode comint-mode "Inferior Python"
`python-shell-prompt-block-regexp',
`python-shell-font-lock-enable',
`python-shell-completion-setup-code',
-`python-shell-completion-string-code',
`python-eldoc-setup-code',
`python-ffap-setup-code' can
customize this mode for different Python interpreters.
@@ -4244,8 +4249,9 @@ (defcustom python-shell-completion-setup-code
completions = []
completer = None
+ import json
try:
- import readline
+ import readline, re
try:
import __builtin__
@@ -4256,16 +4262,29 @@ (defcustom python-shell-completion-setup-code
is_ipython = ('__IPYTHON__' in builtins or
'__IPYTHON__active' in builtins)
- splits = text.split()
- is_module = splits and splits[0] in ('from', 'import')
-
- if is_ipython and is_module:
- from IPython.core.completerlib import module_completion
- completions = module_completion(text.strip())
- elif is_ipython and '__IP' in builtins:
- completions = __IP.complete(text)
- elif is_ipython and 'get_ipython' in builtins:
- completions = get_ipython().Completer.all_completions(text)
+
+ if is_ipython and 'get_ipython' in builtins:
+ def filter_c(prefix, c):
+ if re.match('_+(i?[0-9]+)?$', c):
+ return False
+ elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix):
+ return False
+ return True
+
+ import IPython
+ try:
+ if IPython.version_info[0] >= 6:
+ from IPython.core.completer import provisionalcompleter
+ with provisionalcompleter():
+ completions = [
+ [c.text, c.start, c.end, c.type or '?', c.signature or '']
+ for c in get_ipython().Completer.completions(text, len(text))
+ if filter_c(text, c.text)]
+ else:
+ part, matches = get_ipython().Completer.complete(line_buffer=text)
+ completions = [text + m[len(part):] for m in matches if filter_c(text, m)]
+ except:
+ pass
else:
# Try to reuse current completer.
completer = readline.get_completer()
@@ -4288,7 +4307,7 @@ (defcustom python-shell-completion-setup-code
finally:
if getattr(completer, 'PYTHON_EL_WRAPPED', False):
completer.print_mode = True
- return completions"
+ return json.dumps(completions)"
"Code used to setup completion in inferior Python processes."
:type 'string)
@@ -4329,6 +4348,10 @@ (defcustom python-shell-completion-native-try-output-timeout 1.0
:version "25.1"
:type 'float)
+(defvar python-shell-readline-completer-delims nil
+ "Word delimiters used by the readline completer.
+It is automatically set by Python shell.")
+
(defvar python-shell-completion-native-redirect-buffer
" *Python completions redirect*"
"Buffer to be used to redirect output of readline commands.")
@@ -4467,6 +4490,10 @@ (defun python-shell-completion-native-setup ()
__PYTHON_EL_native_completion_setup()" process)))
(when (string-match-p "python\\.el: native completion setup loaded"
output)
+ (setq-local python-shell-readline-completer-delims
+ (string-trim-right
+ (python-shell-send-string-no-output
+ "import readline; print(readline.get_completer_delims())")))
(python-shell-completion-native-try))))
(defun python-shell-completion-native-turn-off (&optional msg)
@@ -4534,6 +4561,8 @@ (defun python-shell-completion-native-get-completions (process input)
(let* ((original-filter-fn (process-filter process))
(redirect-buffer (get-buffer-create
python-shell-completion-native-redirect-buffer))
+ (sep (if (string= python-shell-readline-completer-delims "")
+ "[\n\r]+" "[ \f\t\n\r\v()]+"))
(trigger "\t")
(new-input (concat input trigger))
(input-length
@@ -4576,28 +4605,80 @@ (defun python-shell-completion-native-get-completions (process input)
process python-shell-completion-native-output-timeout
comint-redirect-finished-regexp)
(re-search-backward "0__dummy_completion__" nil t)
- (cl-remove-duplicates
- (split-string
- (buffer-substring-no-properties
- (line-beginning-position) (point-min))
- "[ \f\t\n\r\v()]+" t)
- :test #'string=))))
+ (let ((str (buffer-substring-no-properties
+ (line-beginning-position) (point-min))))
+ ;; The readline completer is allowed to return a list
+ ;; of (text start end type signature) as a JSON
+ ;; string. See the return value for IPython in
+ ;; `python-shell-completion-setup-code'.
+ (if (string= "[" (substring str 0 1))
+ (condition-case nil
+ (python--parse-json-array str)
+ (t (cl-remove-duplicates (split-string str sep t)
+ :test #'string=)))
+ (cl-remove-duplicates (split-string str sep t)
+ :test #'string=))))))
(set-process-filter process original-filter-fn)))))
(defun python-shell-completion-get-completions (process input)
"Get completions of INPUT using PROCESS."
(with-current-buffer (process-buffer process)
- (let ((completions
- (python-util-strip-string
- (python-shell-send-string-no-output
- (format
- "%s\nprint(';'.join(__PYTHON_EL_get_completions(%s)))"
+ (python--parse-json-array
+ (python-shell-send-string-no-output
+ (format "%s\nprint(__PYTHON_EL_get_completions(%s))"
python-shell-completion-setup-code
(python-shell--encode-string input))
- process))))
- (when (> (length completions) 2)
- (split-string completions
- "^'\\|^\"\\|;\\|'$\\|\"$" t)))))
+ process))))
+
+(defun python-shell--get-multiline-input ()
+ "Return lines at a multi-line input in Python shell."
+ (save-excursion
+ (let ((p (point)) lines)
+ (when (progn
+ (beginning-of-line)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) p) lines)
+ (while (progn (comint-previous-prompt 1)
+ (looking-back python-shell-prompt-block-regexp (pos-bol)))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ (push (buffer-substring-no-properties (point) (pos-eol)) lines))
+ lines)))
+
+(defun python-shell--extra-completion-context ()
+ "Get extra completion context of current input in Python shell."
+ (let ((lines (python-shell--get-multiline-input))
+ (python-indent-guess-indent-offset nil))
+ (when (not (zerop (length lines)))
+ (with-temp-buffer
+ (delay-mode-hooks
+ (insert (string-join lines "\n"))
+ (python-mode)
+ (python-shell-completion-extra-context))))))
+
+(defun python-shell-completion-extra-context (&optional pos)
+ "Get extra completion context at position POS in Python buffer.
+If optional argument POS is nil, use current position.
+
+Readline completers could use current line as the completion
+context, which may be insufficient. In this function, extra
+context (e.g. multi-line function call) is found and reformatted
+as one line, which is required by native completion."
+ (let (bound p)
+ (save-excursion
+ (and pos (goto-char pos))
+ (setq bound (pos-bol))
+ (python-nav-up-list -1)
+ (when (and (< (point) bound)
+ (or
+ (looking-back
+ (python-rx (group (+ (or "." symbol-name)))) (pos-bol) t)
+ (progn
+ (forward-line 0)
+ (looking-at "^[ \t]*\\(from \\)"))))
+ (setq p (match-beginning 1))))
+ (when p
+ (replace-regexp-in-string
+ "\n[ \t]*" "" (buffer-substring-no-properties p (1- bound))))))
(defvar-local python-shell--capf-cache nil
"Variable to store cached completions and invalidation keys.")
@@ -4612,21 +4693,26 @@ (defun python-shell-completion-at-point (&optional process)
;; Working on a shell buffer: use prompt end.
(cdr (python-util-comint-last-prompt))
(line-beginning-position)))
- (import-statement
- (when (string-match-p
- (rx (* space) word-start (or "from" "import") word-end space)
- (buffer-substring-no-properties line-start (point)))
- (buffer-substring-no-properties line-start (point))))
+ (no-delims
+ (and (not (if is-shell-buffer
+ (eq 'font-lock-comment-face
+ (get-text-property (1- (point)) 'face))
+ (python-syntax-context 'comment)))
+ (with-current-buffer (process-buffer process)
+ (if python-shell-completion-native-enable
+ (string= python-shell-readline-completer-delims "")
+ (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
(start
(if (< (point) line-start)
(point)
(save-excursion
- (if (not (re-search-backward
- (python-rx
- (or whitespace open-paren close-paren
- string-delimiter simple-operator))
- line-start
- t 1))
+ (if (or no-delims
+ (not (re-search-backward
+ (python-rx
+ (or whitespace open-paren close-paren
+ string-delimiter simple-operator))
+ line-start
+ t 1)))
line-start
(forward-char (length (match-string-no-properties 0)))
(point)))))
@@ -4666,18 +4752,56 @@ (defun python-shell-completion-at-point (&optional process)
(t #'python-shell-completion-native-get-completions))))
(prev-prompt (car python-shell--capf-cache))
(re (or (cadr python-shell--capf-cache) regexp-unmatchable))
- (prefix (buffer-substring-no-properties start end)))
+ (prefix (buffer-substring-no-properties start end))
+ (prefix-offset 0)
+ (extra-context (when no-delims
+ (if is-shell-buffer
+ (python-shell--extra-completion-context)
+ (python-shell-completion-extra-context))))
+ (extra-offset (length extra-context)))
+ (unless (zerop extra-offset)
+ (setq prefix (concat extra-context prefix)))
;; To invalidate the cache, we check if the prompt position or the
;; completion prefix changed.
(unless (and (equal prev-prompt (car prompt-boundaries))
- (string-match re prefix))
+ (string-match re prefix)
+ (setq prefix-offset (- (length prefix) (match-end 1))))
(setq python-shell--capf-cache
`(,(car prompt-boundaries)
,(if (string-empty-p prefix)
regexp-unmatchable
- (concat "\\`" (regexp-quote prefix) "\\(?:\\sw\\|\\s_\\)*\\'"))
- ,@(funcall completion-fn process (or import-statement prefix)))))
- (list start end (cddr python-shell--capf-cache))))
+ (concat "\\`\\(" (regexp-quote prefix) "\\)\\(?:\\sw\\|\\s_\\)*\\'"))
+ ,@(funcall completion-fn process prefix))))
+ (let ((cands (cddr python-shell--capf-cache)))
+ (cond
+ ((stringp (car cands))
+ (if no-delims
+ ;; Reduce completion candidates due to long prefix.
+ (if-let ((Lp (length prefix))
+ ((string-match "\\(\\sw\\|\\s_\\)+\\'" prefix))
+ (L (match-beginning 0)))
+ ;; If extra-offset is not zero:
+ ;; start end
+ ;; o------------------o---------o-------o
+ ;; |<- extra-offset ->|
+ ;; |<----------- L ------------>|
+ ;; new-start
+ (list (+ start L (- extra-offset)) end
+ (mapcar (lambda (s) (substring s L)) cands))
+ (list end end (mapcar (lambda (s) (substring s Lp)) cands)))
+ (list start end cands)))
+ ;; python-shell-completion(-native)-get-completions may produce a
+ ;; list of (text start end type signature) for completion.
+ ((consp (car cands))
+ (list (+ start (nth 1 (car cands)) (- extra-offset))
+ ;; Candidates may be cached, so the end position should
+ ;; be adjusted according to current completion prefix.
+ (+ start (nth 2 (car cands)) (- extra-offset) prefix-offset)
+ cands
+ :annotation-function
+ (lambda (c) (concat " " (nth 3 (assoc c cands))))
+ :company-docsig
+ (lambda (c) (nth 4 (assoc c cands)))))))))
(define-obsolete-function-alias
'python-shell-completion-complete-at-point
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 59957ff0712..af6c199b5bd 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4799,6 +4799,98 @@ (ert-deftest python-shell-completion-at-point-native-1 ()
(end-of-line 0)
(should-not (nth 2 (python-shell-completion-at-point))))))
+(defun python-tests--completion-module ()
+ "Check if modules can be completed in Python shell."
+ (insert "import datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "import datetime"))
+ (kill-line)
+ (insert "from datet")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime"))
+ (end-of-line)
+ (insert " import timed")
+ (completion-at-point)
+ (beginning-of-line)
+ (should (looking-at-p "from datetime import timedelta"))
+ (kill-line))
+
+(defun python-tests--completion-parameters ()
+ "Check if parameters can be completed in Python shell."
+ (insert "import re")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "re.split('b', 'abc', maxs")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit="
+ (buffer-substring (line-beginning-position) (point))))
+ (insert "0, ")
+ (should (python-shell-completion-at-point))
+ ;; Test if cache is used.
+ (cl-letf (((symbol-function 'python-shell-completion-get-completions)
+ 'ignore)
+ ((symbol-function 'python-shell-completion-native-get-completions)
+ 'ignore))
+ (insert "fla")
+ (completion-at-point)
+ (should (string= "re.split('b', 'abc', maxsplit=0, flags="
+ (buffer-substring (line-beginning-position) (point)))))
+ (beginning-of-line)
+ (kill-line))
+
+(defun python-tests--completion-extra-context ()
+ "Check if extra context is used for completion."
+ (insert "re.split('b', 'abc',")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "maxs")
+ (completion-at-point)
+ (should (string= "maxsplit="
+ (buffer-substring (line-beginning-position) (point))))
+ (insert "0)")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "from re import (")
+ (comint-send-input)
+ (python-tests-shell-wait-for-prompt)
+ (insert "IGN")
+ (completion-at-point)
+ (should (string= "IGNORECASE"
+ (buffer-substring (line-beginning-position) (point)))))
+
+(ert-deftest python-shell-completion-at-point-jedi-completer ()
+ "Check if Python shell completion works when Jedi completer is used."
+ (skip-unless (executable-find python-tests-shell-interpreter))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context))))
+
+(ert-deftest python-shell-completion-at-point-ipython ()
+ "Check if Python shell completion works for IPython."
+ (let ((python-shell-interpreter "ipython")
+ (python-shell-interpreter-args "-i --simple-prompt"))
+ (skip-unless
+ (and
+ (executable-find python-shell-interpreter)
+ (eql (call-process python-shell-interpreter nil nil nil "--version") 0)))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-off)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context)))))
\f
;;; PDB Track integration
--
2.39.2
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-07 13:22 ` Liu Hui
@ 2024-02-07 15:19 ` kobarity
2024-02-08 12:13 ` Eli Zaretskii
0 siblings, 1 reply; 68+ messages in thread
From: kobarity @ 2024-02-07 15:19 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, 68559
Liu Hui wrote:
> On Tue, Feb 6, 2024 at 11:12 PM kobarity <kobarity@gmail.com> wrote:
>
> > Sorry, I noticed one more thing. As written in the comments below, it
> > would be better to use `line-beginning-position' and
> > `line-end-position' instead of `pos-bol' and `pos-eol'.
> >
> > ;; This is a GNU ELPA :core package. Avoid functionality that is not
> > ;; compatible with the version of Emacs recorded above.
>
> Thanks for pointing it out! `pos-bol' was used because it can ignore
> the prompt in `python-shell--get-multiline-input'. Fortunately, I find
> `pos-bol' and `pos-eol' are available in the compat package since
> 29.1.1.0, so it should be enough to upgrade the compat version in
> Package-Requires. I have updated the attached patch.
I see. The patch looks good to me. I appreciate your efforts!
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-07 15:19 ` kobarity
@ 2024-02-08 12:13 ` Eli Zaretskii
2024-02-08 13:33 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: Eli Zaretskii @ 2024-02-08 12:13 UTC (permalink / raw)
To: kobarity; +Cc: liuhui1610, 68559
> Date: Thu, 08 Feb 2024 00:19:55 +0900
> From: kobarity <kobarity@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,
> 68559@debbugs.gnu.org
>
> Liu Hui wrote:
> > On Tue, Feb 6, 2024 at 11:12 PM kobarity <kobarity@gmail.com> wrote:
> >
> > > Sorry, I noticed one more thing. As written in the comments below, it
> > > would be better to use `line-beginning-position' and
> > > `line-end-position' instead of `pos-bol' and `pos-eol'.
> > >
> > > ;; This is a GNU ELPA :core package. Avoid functionality that is not
> > > ;; compatible with the version of Emacs recorded above.
> >
> > Thanks for pointing it out! `pos-bol' was used because it can ignore
> > the prompt in `python-shell--get-multiline-input'. Fortunately, I find
> > `pos-bol' and `pos-eol' are available in the compat package since
> > 29.1.1.0, so it should be enough to upgrade the compat version in
> > Package-Requires. I have updated the attached patch.
>
> I see. The patch looks good to me. I appreciate your efforts!
Thanks, I installed it on the master branch.
What about the other patch, one you posted in
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68559#46
Does that one need to be installed as well, or is it included in the
one I just installed?
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-08 12:13 ` Eli Zaretskii
@ 2024-02-08 13:33 ` Liu Hui
2024-02-08 13:46 ` Eli Zaretskii
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-08 13:33 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: kobarity, 68559
On Thu, Feb 8, 2024 at 8:13 PM Eli Zaretskii <eliz@gnu.org> wrote:
> What about the other patch, one you posted in
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68559#46
>
> Does that one need to be installed as well, or is it included in the
> one I just installed?
The installed one is complete. Thanks.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-08 13:33 ` Liu Hui
@ 2024-02-08 13:46 ` Eli Zaretskii
2024-02-08 14:16 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: Eli Zaretskii @ 2024-02-08 13:46 UTC (permalink / raw)
To: Liu Hui; +Cc: kobarity, 68559
> From: Liu Hui <liuhui1610@gmail.com>
> Date: Thu, 8 Feb 2024 21:33:10 +0800
> Cc: kobarity <kobarity@gmail.com>, 68559@debbugs.gnu.org
>
> On Thu, Feb 8, 2024 at 8:13 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > What about the other patch, one you posted in
> >
> > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68559#46
> >
> > Does that one need to be installed as well, or is it included in the
> > one I just installed?
>
> The installed one is complete. Thanks.
Great, thanks. Then this bug can be closed now, right?
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-08 13:46 ` Eli Zaretskii
@ 2024-02-08 14:16 ` Liu Hui
2024-02-08 16:43 ` Eli Zaretskii
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-08 14:16 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: kobarity, 68559
On Thu, Feb 8, 2024 at 9:46 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Liu Hui <liuhui1610@gmail.com>
> > Date: Thu, 8 Feb 2024 21:33:10 +0800
> > Cc: kobarity <kobarity@gmail.com>, 68559@debbugs.gnu.org
> >
> > On Thu, Feb 8, 2024 at 8:13 PM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > What about the other patch, one you posted in
> > >
> > > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68559#46
> > >
> > > Does that one need to be installed as well, or is it included in the
> > > one I just installed?
> >
> > The installed one is complete. Thanks.
>
> Great, thanks. Then this bug can be closed now, right?
Yes, please.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-08 14:16 ` Liu Hui
@ 2024-02-08 16:43 ` Eli Zaretskii
0 siblings, 0 replies; 68+ messages in thread
From: Eli Zaretskii @ 2024-02-08 16:43 UTC (permalink / raw)
To: Liu Hui; +Cc: kobarity, 68559-done
> From: Liu Hui <liuhui1610@gmail.com>
> Date: Thu, 8 Feb 2024 22:16:23 +0800
> Cc: kobarity@gmail.com, 68559@debbugs.gnu.org
>
> On Thu, Feb 8, 2024 at 9:46 PM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > From: Liu Hui <liuhui1610@gmail.com>
> > > Date: Thu, 8 Feb 2024 21:33:10 +0800
> > > Cc: kobarity <kobarity@gmail.com>, 68559@debbugs.gnu.org
> > >
> > > On Thu, Feb 8, 2024 at 8:13 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > >
> > > > What about the other patch, one you posted in
> > > >
> > > > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68559#46
> > > >
> > > > Does that one need to be installed as well, or is it included in the
> > > > one I just installed?
> > >
> > > The installed one is complete. Thanks.
> >
> > Great, thanks. Then this bug can be closed now, right?
>
> Yes, please.
Thanks, closing.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-01-18 4:48 bug#68559: [PATCH] Improve Python shell completion Liu Hui
2024-01-18 6:39 ` Eli Zaretskii
@ 2024-02-15 14:43 ` Mattias Engdegård
2024-02-15 16:37 ` Eli Zaretskii
2024-02-16 3:24 ` Liu Hui
1 sibling, 2 replies; 68+ messages in thread
From: Mattias Engdegård @ 2024-02-15 14:43 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, kobarity, 68559
[-- Attachment #1: Type: text/plain, Size: 319 bytes --]
The recent change on master, 0b9c7148fd, causes several failures in python-tests here (macOS). Log attached.
It seems to have something to do with completion. Maybe the new code is sensitive to details of Python's command line editor? The standard Python interpreter is unlikely to use GNU readline, for example.
[-- Attachment #2: python-tests.log --]
[-- Type: application/octet-stream, Size: 48228 bytes --]
GEN lisp/progmodes/python-tests.log
Running 360 tests (2024-02-15 15:38:05+0100, selector `(not (or (tag :unstable) (tag :nativecomp)))')
passed 1/360 python-auto-fill-docstring (0.001040 sec)
Fontifying *temp*-895436...
Fontifying *temp*-895436... (syntactically...)
Fontifying *temp*-895436... (regexps...)
Fontifying *temp*-895436... (regexps....)
Fontifying *temp*-895436... (regexps.....)
Fontifying *temp*-895436... (regexps......)
Fontifying *temp*-895436... (regexps.......)
Fontifying *temp*-895436... (regexps........)
Fontifying *temp*-895436... (regexps.........)
Fontifying *temp*-895436... (regexps..........)
Fontifying *temp*-895436... (regexps...........)
Fontifying *temp*-895436... (regexps............)
Fontifying *temp*-895436... (regexps.............)
Fontifying *temp*-895436... (regexps..............)
Fontifying *temp*-895436... (regexps...............)
Fontifying *temp*-895436... (regexps................)
passed 2/360 python-bob-infloop-avoid (0.000804 sec)
Test python-completion-at-point-1 backtrace:
json-parse-string("__PYTHON_EL_eval_file(\"/var/folders/qy/zstv16390
python--parse-json-array("__PYTHON_EL_eval_file(\"/var/folders/qy/zs
python-shell-completion-get-completions(#<process Python[ *temp*-713
python-shell-completion-at-point(#<process Python[ *temp*-713815]>)
python-completion-at-point()
completion--capf-wrapper(python-completion-at-point all)
run-hook-wrapped(completion--capf-wrapper python-completion-at-point
completion-at-point()
apply(completion-at-point nil)
(setq value-3676 (apply fn-3674 args-3675))
(unwind-protect (setq value-3676 (apply fn-3674 args-3675)) (setq fo
(if (unwind-protect (setq value-3676 (apply fn-3674 args-3675)) (set
(let (form-description-3678) (if (unwind-protect (setq value-3676 (a
(let ((value-3676 'ert-form-evaluation-aborted-3677)) (let (form-des
(let* ((fn-3674 #'completion-at-point) (args-3675 (condition-case er
(let ((inhibit-message t)) (python-shell-send-buffer) (python-tests-
(progn (run-python nil t) (insert "\nimport abc\n") (goto-char (poin
(unwind-protect (progn (run-python nil t) (insert "\nimport abc\n")
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3669 #'executable-find) (args-3670 (cond
#f(compiled-function () #<bytecode -0x3de848aaf387fce>)()
handler-bind-1(#f(compiled-function () #<bytecode -0x3de848aaf387fce
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-completion-at-point-1 :documen
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-completion-at-point-1 condition:
(json-parse-error "invalid token near '_'" "<string>" 1 1 1)
FAILED 3/360 python-completion-at-point-1 (0.404730 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4903
Test python-completion-at-point-2 backtrace:
json-parse-string("__PYTHON_EL_eval_file(\"/var/folders/qy/zstv16390
python--parse-json-array("__PYTHON_EL_eval_file(\"/var/folders/qy/zs
python-shell-completion-get-completions(#<process Python[ *temp*-102
python-shell-completion-at-point(#<process Python[ *temp*-102788]>)
python-completion-at-point()
completion--capf-wrapper(python-completion-at-point all)
run-hook-wrapped(completion--capf-wrapper python-completion-at-point
completion-at-point()
apply(completion-at-point nil)
(setq value-3691 (apply fn-3689 args-3690))
(unwind-protect (setq value-3691 (apply fn-3689 args-3690)) (setq fo
(if (unwind-protect (setq value-3691 (apply fn-3689 args-3690)) (set
(let (form-description-3693) (if (unwind-protect (setq value-3691 (a
(let ((value-3691 'ert-form-evaluation-aborted-3692)) (let (form-des
(let* ((fn-3689 #'completion-at-point) (args-3690 (condition-case er
(let ((inhibit-message t)) (python-shell-send-buffer) (python-tests-
(progn (run-python nil t) (insert "\nimport abc\n") (goto-char (poin
(unwind-protect (progn (run-python nil t) (insert "\nimport abc\n")
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3684 #'executable-find) (args-3685 (cond
#f(compiled-function () #<bytecode -0x3de848aaf387fce>)()
handler-bind-1(#f(compiled-function () #<bytecode -0x3de848aaf387fce
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-completion-at-point-2 :documen
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-completion-at-point-2 condition:
(json-parse-error "invalid token near '_'" "<string>" 1 1 1)
FAILED 4/360 python-completion-at-point-2 (0.381182 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4918
Test python-completion-at-point-native-1 backtrace:
json-parse-string("__PYTHON_EL_eval_file(\"/var/folders/qy/zstv16390
python--parse-json-array("__PYTHON_EL_eval_file(\"/var/folders/qy/zs
python-shell-completion-get-completions(#<process Python[ *temp*-704
python-shell-completion-at-point(#<process Python[ *temp*-704829]>)
python-completion-at-point()
completion--capf-wrapper(python-completion-at-point all)
run-hook-wrapped(completion--capf-wrapper python-completion-at-point
completion-at-point()
apply(completion-at-point nil)
(setq value-3718 (apply fn-3716 args-3717))
(unwind-protect (setq value-3718 (apply fn-3716 args-3717)) (setq fo
(if (unwind-protect (setq value-3718 (apply fn-3716 args-3717)) (set
(let (form-description-3720) (if (unwind-protect (setq value-3718 (a
(let ((value-3718 'ert-form-evaluation-aborted-3719)) (let (form-des
(let* ((fn-3716 #'completion-at-point) (args-3717 (condition-case er
(let ((inhibit-message t)) (python-shell-completion-native-turn-on)
(progn (run-python nil t) (insert "\nimport abc\n") (goto-char (poin
(unwind-protect (progn (run-python nil t) (insert "\nimport abc\n")
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3711 #'executable-find) (args-3712 (cond
#f(compiled-function () #<bytecode -0x3de848aaf387fce>)()
handler-bind-1(#f(compiled-function () #<bytecode -0x3de848aaf387fce
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-completion-at-point-native-1 :
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-completion-at-point-native-1 condition:
(json-parse-error "invalid token near '_'" "<string>" 1 1 1)
FAILED 5/360 python-completion-at-point-native-1 (0.407805 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4966
Test python-completion-at-point-native-2 backtrace:
json-parse-string("__PYTHON_EL_eval_file(\"/var/folders/qy/zstv16390
python--parse-json-array("__PYTHON_EL_eval_file(\"/var/folders/qy/zs
python-shell-completion-get-completions(#<process Python[ *temp*-734
python-shell-completion-at-point(#<process Python[ *temp*-734580]>)
python-completion-at-point()
completion--capf-wrapper(python-completion-at-point all)
run-hook-wrapped(completion--capf-wrapper python-completion-at-point
completion-at-point()
apply(completion-at-point nil)
(setq value-3733 (apply fn-3731 args-3732))
(unwind-protect (setq value-3733 (apply fn-3731 args-3732)) (setq fo
(if (unwind-protect (setq value-3733 (apply fn-3731 args-3732)) (set
(let (form-description-3735) (if (unwind-protect (setq value-3733 (a
(let ((value-3733 'ert-form-evaluation-aborted-3734)) (let (form-des
(let* ((fn-3731 #'completion-at-point) (args-3732 (condition-case er
(let ((inhibit-message t)) (python-shell-completion-native-turn-on)
(progn (run-python nil t) (insert "\nimport abc\n") (goto-char (poin
(unwind-protect (progn (run-python nil t) (insert "\nimport abc\n")
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3726 #'executable-find) (args-3727 (cond
#f(compiled-function () #<bytecode -0x3de848aaf387fce>)()
handler-bind-1(#f(compiled-function () #<bytecode -0x3de848aaf387fce
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-completion-at-point-native-2 :
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-completion-at-point-native-2 condition:
(json-parse-error "invalid token near '_'" "<string>" 1 1 1)
FAILED 6/360 python-completion-at-point-native-2 (0.399853 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4982
Test python-completion-at-point-native-with-eldoc-1 backtrace:
json-parse-string("__PYTHON_EL_eval_file(\"/var/folders/qy/zstv16390
python--parse-json-array("__PYTHON_EL_eval_file(\"/var/folders/qy/zs
python-shell-completion-get-completions(#<process Python[ *temp*-155
python-shell-completion-at-point(#<process Python[ *temp*-155177]>)
python-completion-at-point()
completion--capf-wrapper(python-completion-at-point all)
run-hook-wrapped(completion--capf-wrapper python-completion-at-point
completion-at-point()
apply(completion-at-point nil)
(setq value-3753 (apply fn-3751 args-3752))
(unwind-protect (setq value-3753 (apply fn-3751 args-3752)) (setq fo
(if (unwind-protect (setq value-3753 (apply fn-3751 args-3752)) (set
(let (form-description-3755) (if (unwind-protect (setq value-3753 (a
(let ((value-3753 'ert-form-evaluation-aborted-3754)) (let (form-des
(let* ((fn-3751 #'completion-at-point) (args-3752 (condition-case er
(let ((inhibit-message t)) (python-shell-completion-native-turn-on)
(progn (run-python nil t) (insert "\nimport abc\n") (goto-char (poin
(unwind-protect (progn (run-python nil t) (insert "\nimport abc\n")
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3746 #'executable-find) (args-3747 (cond
#f(compiled-function () #<bytecode -0x3de848aaf387fce>)()
handler-bind-1(#f(compiled-function () #<bytecode -0x3de848aaf387fce
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-completion-at-point-native-wit
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-completion-at-point-native-with-eldoc-1 condition:
(json-parse-error "invalid token near '_'" "<string>" 1 1 1)
FAILED 7/360 python-completion-at-point-native-with-eldoc-1 (0.451243 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:5015
Test python-completion-at-point-native-with-ffap-1 backtrace:
json-parse-string("__PYTHON_EL_eval_file(\"/var/folders/qy/zstv16390
python--parse-json-array("__PYTHON_EL_eval_file(\"/var/folders/qy/zs
python-shell-completion-get-completions(#<process Python[ *temp*-392
python-shell-completion-at-point(#<process Python[ *temp*-392361]>)
python-completion-at-point()
completion--capf-wrapper(python-completion-at-point all)
run-hook-wrapped(completion--capf-wrapper python-completion-at-point
completion-at-point()
apply(completion-at-point nil)
(setq value-3743 (apply fn-3741 args-3742))
(unwind-protect (setq value-3743 (apply fn-3741 args-3742)) (setq fo
(if (unwind-protect (setq value-3743 (apply fn-3741 args-3742)) (set
(let (form-description-3745) (if (unwind-protect (setq value-3743 (a
(let ((value-3743 'ert-form-evaluation-aborted-3744)) (let (form-des
(let* ((fn-3741 #'completion-at-point) (args-3742 (condition-case er
(let ((inhibit-message t)) (python-shell-completion-native-turn-on)
(progn (run-python nil t) (insert "\nimport abc\n") (goto-char (poin
(unwind-protect (progn (run-python nil t) (insert "\nimport abc\n")
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3736 #'executable-find) (args-3737 (cond
#f(compiled-function () #<bytecode -0x3de848aaf387fce>)()
handler-bind-1(#f(compiled-function () #<bytecode -0x3de848aaf387fce
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-completion-at-point-native-wit
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-completion-at-point-native-with-ffap-1 condition:
(json-parse-error "invalid token near '_'" "<string>" 1 1 1)
FAILED 8/360 python-completion-at-point-native-with-ffap-1 (0.461626 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4999
passed 9/360 python-completion-at-point-pdb-1 (0.403759 sec)
passed 10/360 python-completion-at-point-while-running-1 (0.200969 sec)
passed 11/360 python-eldoc--get-doc-at-point-1 (0.409866 sec)
passed 12/360 python-eldoc--get-doc-at-point-while-running-1 (0.196968 sec)
passed 13/360 python-eldoc--get-symbol-at-point-1 (0.001494 sec)
passed 14/360 python-eldoc--get-symbol-at-point-2 (0.001215 sec)
passed 15/360 python-eldoc--get-symbol-at-point-3 (0.000355 sec)
passed 16/360 python-eldoc--get-symbol-at-point-4 (0.000367 sec)
passed 17/360 python-end-of-defun-1 (0.000406 sec)
skipped 18/360 python-ffap-module-path-1 (0.000411 sec)
passed 19/360 python-ffap-module-path-while-running-1 (0.204620 sec)
passed 20/360 python-fill-docstring (0.001313 sec)
passed 21/360 python-fill-paragraph-single-quoted-string-1 (0.000401 sec)
passed 22/360 python-fill-paragraph-single-quoted-string-2 (0.000313 sec)
passed 23/360 python-fill-paragraph-triple-quoted-string-1 (0.003809 sec)
passed 24/360 python-font-lock-assignment-statement-1 (0.000670 sec)
passed 25/360 python-font-lock-assignment-statement-10 (0.000358 sec)
passed 26/360 python-font-lock-assignment-statement-11 (0.000561 sec)
passed 27/360 python-font-lock-assignment-statement-12 (0.000589 sec)
passed 28/360 python-font-lock-assignment-statement-13 (0.000741 sec)
passed 29/360 python-font-lock-assignment-statement-14 (0.000586 sec)
passed 30/360 python-font-lock-assignment-statement-15 (0.000470 sec)
passed 31/360 python-font-lock-assignment-statement-16 (0.000378 sec)
passed 32/360 python-font-lock-assignment-statement-17 (0.000632 sec)
passed 33/360 python-font-lock-assignment-statement-18 (0.000620 sec)
passed 34/360 python-font-lock-assignment-statement-2 (0.000803 sec)
passed 35/360 python-font-lock-assignment-statement-3 (0.000593 sec)
passed 36/360 python-font-lock-assignment-statement-4 (0.000677 sec)
passed 37/360 python-font-lock-assignment-statement-5 (0.000494 sec)
passed 38/360 python-font-lock-assignment-statement-6 (0.000409 sec)
passed 39/360 python-font-lock-assignment-statement-7 (0.000516 sec)
passed 40/360 python-font-lock-assignment-statement-8 (0.000442 sec)
passed 41/360 python-font-lock-assignment-statement-9 (0.000669 sec)
passed 42/360 python-font-lock-escape-sequence-bytes-newline (0.000449 sec)
passed 43/360 python-font-lock-escape-sequence-hex-octal (0.000797 sec)
passed 44/360 python-font-lock-escape-sequence-multiline-string (0.015089 sec)
passed 45/360 python-font-lock-escape-sequence-string-newline (0.001174 sec)
passed 46/360 python-font-lock-escape-sequence-unicode (0.000670 sec)
passed 47/360 python-font-lock-keywords-level-1-1 (0.000784 sec)
passed 48/360 python-font-lock-keywords-level-1-2 (0.000567 sec)
passed 49/360 python-font-lock-operator-1 (0.000616 sec)
passed 50/360 python-font-lock-operator-2 (0.000685 sec)
passed 51/360 python-font-lock-raw-escape-sequence (0.000931 sec)
passed 52/360 python-font-lock-string-literal-concatenation (0.000505 sec)
Hiding all blocks...
Hiding all blocks...done
passed 53/360 python-hideshow-hide-all-1 (0.000726 sec)
Hiding all blocks...
Hiding all blocks...done
passed 54/360 python-hideshow-hide-all-2 (0.000482 sec)
Hiding all blocks...
Hiding all blocks...done
passed 55/360 python-hideshow-hide-all-3 (0.000423 sec)
passed 56/360 python-hideshow-hide-block-1 (0.000525 sec)
Hiding blocks ...
Hiding blocks ... done
Showing all blocks ...
Showing all blocks ... done
passed 57/360 python-hideshow-hide-levels-1 (0.001158 sec)
Showing all blocks ...
Showing all blocks ... done
passed 58/360 python-hideshow-hide-levels-2 (0.063466 sec)
Hiding blocks ...
Hiding blocks ... done
passed 59/360 python-hideshow-hide-levels-3 (0.001151 sec)
Hiding blocks ...
Hiding blocks ... done
passed 60/360 python-hideshow-hide-levels-4 (0.001123 sec)
passed 61/360 python-imenu-create-flat-index-1 (0.000588 sec)
passed 62/360 python-imenu-create-flat-index-2 (0.000365 sec)
passed 63/360 python-imenu-create-index-1 (0.000564 sec)
passed 64/360 python-imenu-create-index-2 (0.000345 sec)
passed 65/360 python-imenu-create-index-3 (0.000337 sec)
passed 66/360 python-imenu-create-index-4 (0.000354 sec)
passed 67/360 python-indent-after-async-block-1 (0.000366 sec)
passed 68/360 python-indent-after-async-block-2 (0.000356 sec)
passed 69/360 python-indent-after-async-block-3 (0.000366 sec)
passed 70/360 python-indent-after-backslash-1 (0.000755 sec)
passed 71/360 python-indent-after-backslash-2 (0.001117 sec)
passed 72/360 python-indent-after-backslash-3 (0.000536 sec)
passed 73/360 python-indent-after-backslash-4 (0.001198 sec)
passed 74/360 python-indent-after-backslash-5 (0.000804 sec)
passed 75/360 python-indent-after-backslash-6 (0.000594 sec)
passed 76/360 python-indent-after-bare-match (0.000396 sec)
passed 77/360 python-indent-after-block-1 (0.000360 sec)
passed 78/360 python-indent-after-block-2 (0.000345 sec)
passed 79/360 python-indent-after-block-3 (0.000522 sec)
passed 80/360 python-indent-after-case-block (0.000358 sec)
passed 81/360 python-indent-after-comment-1 (0.000815 sec)
passed 82/360 python-indent-after-comment-2 (0.001021 sec)
passed 83/360 python-indent-after-comment-3 (0.000543 sec)
passed 84/360 python-indent-after-match-block (0.000368 sec)
passed 85/360 python-indent-after-re-match (0.000388 sec)
passed 86/360 python-indent-base-case (0.000483 sec)
passed 87/360 python-indent-block-enders-1 (0.000707 sec)
passed 88/360 python-indent-block-enders-2 (0.001406 sec)
passed 89/360 python-indent-block-enders-3 (0.000738 sec)
passed 90/360 python-indent-block-enders-4 (0.000479 sec)
passed 91/360 python-indent-block-enders-5 (0.000496 sec)
passed 92/360 python-indent-dedent-line-backspace-1 (0.000467 sec)
passed 93/360 python-indent-dedent-line-backspace-2 (0.000312 sec)
passed 94/360 python-indent-dedent-line-backspace-3 (0.000580 sec)
passed 95/360 python-indent-dedenters-1 (0.000499 sec)
Closes if hide_details:
Closes except Exception:
Closes if save:
passed 96/360 python-indent-dedenters-2 (0.002075 sec)
Closes try:
passed 97/360 python-indent-dedenters-3 (0.000975 sec)
Closes try:
passed 98/360 python-indent-dedenters-4 (0.000954 sec)
Closes if save:
passed 99/360 python-indent-dedenters-5 (0.001225 sec)
passed 100/360 python-indent-dedenters-6 (0.000430 sec)
passed 101/360 python-indent-dedenters-7 (0.000557 sec)
Closes if (a == 1 or
Closes if (a == 1 or
Closes if (a == 1 or
passed 102/360 python-indent-dedenters-8 (0.054888 sec)
Closes case 1:
passed 103/360 python-indent-dedenters-9 (0.001045 sec)
Closes if hide_details:
Closes except Exception:
Closes if save:
passed 104/360 python-indent-dedenters-comment-else (0.002278 sec)
passed 105/360 python-indent-electric-colon-1 (0.000376 sec)
Closes if do:
passed 106/360 python-indent-electric-colon-2 (0.000483 sec)
Closes if do:
Closes if do:
Closes if do:
passed 107/360 python-indent-electric-colon-3 (0.000718 sec)
Closes if True:
passed 108/360 python-indent-electric-colon-4 (0.000566 sec)
passed 109/360 python-indent-electric-comma-after-multiline-string (0.000423 sec)
passed 110/360 python-indent-electric-comma-inside-multiline-string (0.000405 sec)
passed 111/360 python-indent-hanging-close-paren (0.000351 sec)
passed 112/360 python-indent-inside-paren-1 (0.001338 sec)
passed 113/360 python-indent-inside-paren-2 (0.001057 sec)
passed 114/360 python-indent-inside-paren-3 (0.000878 sec)
passed 115/360 python-indent-inside-paren-4 (0.000625 sec)
passed 116/360 python-indent-inside-paren-5 (0.000799 sec)
passed 117/360 python-indent-inside-paren-6 (0.000444 sec)
passed 118/360 python-indent-inside-paren-7 (0.000303 sec)
passed 119/360 python-indent-inside-paren-8 (0.000416 sec)
passed 120/360 python-indent-inside-paren-9 (0.000911 sec)
passed 121/360 python-indent-inside-paren-block-1 (0.000900 sec)
passed 122/360 python-indent-inside-paren-block-2 (0.000817 sec)
passed 123/360 python-indent-inside-paren-block-3 (0.000588 sec)
passed 124/360 python-indent-inside-paren-block-4 (0.000584 sec)
passed 125/360 python-indent-inside-string-1 (0.000646 sec)
passed 126/360 python-indent-inside-string-2 (0.001953 sec)
passed 127/360 python-indent-inside-string-3 (0.000754 sec)
passed 128/360 python-indent-pep8-1 (0.000450 sec)
passed 129/360 python-indent-pep8-2 (0.000557 sec)
passed 130/360 python-indent-pep8-3 (0.000930 sec)
passed 131/360 python-indent-region-1 (0.000539 sec)
passed 132/360 python-indent-region-2 (0.000637 sec)
passed 133/360 python-indent-region-3 (0.000518 sec)
passed 134/360 python-indent-region-4 (0.000454 sec)
passed 135/360 python-indent-region-5 (0.002005 sec)
passed 136/360 python-info-assignment-continuation-line-p-1 (0.000400 sec)
passed 137/360 python-info-assignment-continuation-line-p-2 (0.000387 sec)
passed 138/360 python-info-assignment-statement-p-1 (0.000417 sec)
passed 139/360 python-info-assignment-statement-p-2 (0.000410 sec)
passed 140/360 python-info-assignment-statement-p-3 (0.000480 sec)
passed 141/360 python-info-beginning-of-backslash-1 (0.000776 sec)
passed 142/360 python-info-beginning-of-block-p-1 (0.000614 sec)
passed 143/360 python-info-beginning-of-block-p-2 (0.000437 sec)
passed 144/360 python-info-beginning-of-statement-p-1 (0.000333 sec)
passed 145/360 python-info-beginning-of-statement-p-2 (0.000726 sec)
passed 146/360 python-info-block-continuation-line-p-1 (0.000608 sec)
passed 147/360 python-info-block-continuation-line-p-2 (0.000561 sec)
passed 148/360 python-info-continuation-line-p-1 (0.000517 sec)
passed 149/360 python-info-current-defun-1 (0.000750 sec)
passed 150/360 python-info-current-defun-2 (0.011937 sec)
passed 151/360 python-info-current-defun-3 (0.008679 sec)
passed 152/360 python-info-current-defun-4 (0.001098 sec)
passed 153/360 python-info-current-line-comment-p-1 (0.000450 sec)
passed 154/360 python-info-current-line-empty-p (0.000382 sec)
passed 155/360 python-info-current-symbol-1 (0.000746 sec)
passed 156/360 python-info-current-symbol-2 (0.000596 sec)
failed 157/360 python-info-current-symbol-3 (0.000335 sec)
passed 158/360 python-info-dedenter-opening-block-message-1 (0.000274 sec)
Closes try:
Closes try:
passed 159/360 python-info-dedenter-opening-block-message-2 (0.000339 sec)
Closes except:
Closes except:
passed 160/360 python-info-dedenter-opening-block-message-3 (0.000832 sec)
Closes else:
Closes else:
passed 161/360 python-info-dedenter-opening-block-message-4 (0.000633 sec)
Closes if a:
Closes if a:
passed 162/360 python-info-dedenter-opening-block-message-5 (0.000618 sec)
passed 163/360 python-info-dedenter-opening-block-position-1 (0.001020 sec)
passed 164/360 python-info-dedenter-opening-block-position-2 (0.000366 sec)
passed 165/360 python-info-dedenter-opening-block-position-3 (0.001109 sec)
passed 166/360 python-info-dedenter-opening-block-positions-1 (0.001203 sec)
passed 167/360 python-info-dedenter-opening-block-positions-2 (0.000448 sec)
passed 168/360 python-info-dedenter-opening-block-positions-3 (0.000665 sec)
passed 169/360 python-info-dedenter-opening-block-positions-4 (0.000411 sec)
passed 170/360 python-info-dedenter-opening-block-positions-5 (0.000499 sec)
passed 171/360 python-info-dedenter-opening-block-positions-6 (0.000833 sec)
passed 172/360 python-info-dedenter-opening-block-positions-7 (0.000914 sec)
passed 173/360 python-info-dedenter-statement-p-1 (0.000607 sec)
passed 174/360 python-info-dedenter-statement-p-2 (0.000506 sec)
passed 175/360 python-info-dedenter-statement-p-3 (0.000331 sec)
passed 176/360 python-info-dedenter-statement-p-4 (0.000340 sec)
passed 177/360 python-info-dedenter-statement-p-5 (0.000324 sec)
passed 178/360 python-info-dedenter-statement-p-6 (0.000377 sec)
passed 179/360 python-info-docstring-p-1 (0.001164 sec)
passed 180/360 python-info-docstring-p-2 (0.000981 sec)
passed 181/360 python-info-docstring-p-3 (0.001510 sec)
passed 182/360 python-info-docstring-p-4 (0.001840 sec)
passed 183/360 python-info-docstring-p-5 (0.002432 sec)
passed 184/360 python-info-docstring-p-6 (0.001988 sec)
passed 185/360 python-info-docstring-p-7 (0.000505 sec)
passed 186/360 python-info-docstring-p-8 (0.000400 sec)
passed 187/360 python-info-encoding-1 (0.000639 sec)
passed 188/360 python-info-encoding-2 (0.000430 sec)
passed 189/360 python-info-encoding-from-cookie-1 (0.000405 sec)
passed 190/360 python-info-encoding-from-cookie-2 (0.000408 sec)
passed 191/360 python-info-encoding-from-cookie-3 (0.000286 sec)
passed 192/360 python-info-encoding-from-cookie-4 (0.000264 sec)
passed 193/360 python-info-encoding-from-cookie-5 (0.000270 sec)
passed 194/360 python-info-encoding-from-cookie-6 (0.000265 sec)
passed 195/360 python-info-encoding-from-cookie-7 (0.000279 sec)
passed 196/360 python-info-end-of-block-p-1 (0.000608 sec)
passed 197/360 python-info-end-of-block-p-2 (0.000722 sec)
passed 198/360 python-info-end-of-statement-p-1 (0.000410 sec)
passed 199/360 python-info-end-of-statement-p-2 (0.000461 sec)
passed 200/360 python-info-line-ends-backslash-p-1 (0.000418 sec)
passed 201/360 python-info-looking-at-beginning-of-block-1 (0.000500 sec)
passed 202/360 python-info-looking-at-beginning-of-defun-1 (0.000797 sec)
passed 203/360 python-info-looking-at-beginning-of-defun-2 (0.000608 sec)
passed 204/360 python-info-looking-at-beginning-of-defun-3 (0.000522 sec)
passed 205/360 python-info-statement-ends-block-p-1 (0.000447 sec)
passed 206/360 python-info-statement-ends-block-p-2 (0.000436 sec)
passed 207/360 python-info-statement-starts-block-p-1 (0.000346 sec)
passed 208/360 python-info-statement-starts-block-p-2 (0.000390 sec)
passed 209/360 python-info-triple-quoted-string-p-1 (0.000493 sec)
passed 210/360 python-info-triple-quoted-string-p-2 (0.000383 sec)
passed 211/360 python-info-triple-quoted-string-p-3 (0.000473 sec)
Mark set
Mark set
passed 212/360 python-mark-defun-1 (0.001020 sec)
Mark set
Mark set
passed 213/360 python-mark-defun-2 (0.001136 sec)
Mark set
Mark set
passed 214/360 python-mark-defun-3 (0.000968 sec)
Mark set
Mark set
passed 215/360 python-mark-defun-4 (0.000974 sec)
Mark set
Mark set
Mark set
Mark set
passed 216/360 python-mark-defun-5 (0.001505 sec)
passed 217/360 python-nav-backward-defun-1 (0.000660 sec)
passed 218/360 python-nav-backward-defun-2 (0.000493 sec)
passed 219/360 python-nav-backward-defun-3 (0.000345 sec)
passed 220/360 python-nav-backward-defun-4 (0.000305 sec)
passed 221/360 python-nav-backward-statement-1 (0.000463 sec)
failed 222/360 python-nav-backward-statement-2 (0.000408 sec)
failed 223/360 python-nav-backward-up-list-1 (0.000457 sec)
passed 224/360 python-nav-beginning-of-block-1 (0.000862 sec)
passed 225/360 python-nav-beginning-of-block-2 (0.000381 sec)
passed 226/360 python-nav-beginning-of-defun-1 (0.001112 sec)
passed 227/360 python-nav-beginning-of-defun-2 (0.000954 sec)
passed 228/360 python-nav-beginning-of-defun-3 (0.000628 sec)
passed 229/360 python-nav-beginning-of-defun-4 (0.000821 sec)
passed 230/360 python-nav-beginning-of-defun-5 (0.000534 sec)
passed 231/360 python-nav-beginning-of-defun-6 (0.000737 sec)
passed 232/360 python-nav-beginning-of-statement-1 (0.000764 sec)
passed 233/360 python-nav-end-of-block-1 (0.001861 sec)
passed 234/360 python-nav-end-of-block-2 (0.000340 sec)
passed 235/360 python-nav-end-of-defun-1 (0.000966 sec)
passed 236/360 python-nav-end-of-defun-2 (0.002294 sec)
passed 237/360 python-nav-end-of-defun-3 (0.000347 sec)
passed 238/360 python-nav-end-of-statement-1 (0.000533 sec)
passed 239/360 python-nav-end-of-statement-2 (0.000482 sec)
passed 240/360 python-nav-end-of-statement-3 (0.000539 sec)
passed 241/360 python-nav-end-of-statement-4 (0.000551 sec)
passed 242/360 python-nav-forward-block-1 (0.000995 sec)
passed 243/360 python-nav-forward-block-2 (0.000373 sec)
passed 244/360 python-nav-forward-defun-1 (0.000442 sec)
passed 245/360 python-nav-forward-defun-2 (0.000595 sec)
passed 246/360 python-nav-forward-defun-3 (0.000470 sec)
passed 247/360 python-nav-forward-defun-4 (0.000805 sec)
passed 248/360 python-nav-forward-sexp-1 (0.001713 sec)
passed 249/360 python-nav-forward-sexp-2 (0.001812 sec)
passed 250/360 python-nav-forward-sexp-3 (0.001452 sec)
passed 251/360 python-nav-forward-sexp-safe-1 (0.000907 sec)
passed 252/360 python-nav-forward-statement-1 (0.000523 sec)
passed 253/360 python-nav-up-list-1 (0.000296 sec)
passed 254/360 python-parens-electric-indent-1 (0.001948 sec)
passed 255/360 python-shell-buffer-substring-1 (0.000728 sec)
passed 256/360 python-shell-buffer-substring-10 (0.000505 sec)
passed 257/360 python-shell-buffer-substring-11 (0.000599 sec)
passed 258/360 python-shell-buffer-substring-12 (0.000640 sec)
passed 259/360 python-shell-buffer-substring-13 (0.000773 sec)
passed 260/360 python-shell-buffer-substring-14 (0.000511 sec)
passed 261/360 python-shell-buffer-substring-15 (0.000452 sec)
passed 262/360 python-shell-buffer-substring-16 (0.000440 sec)
passed 263/360 python-shell-buffer-substring-17 (0.000998 sec)
passed 264/360 python-shell-buffer-substring-18 (0.000789 sec)
passed 265/360 python-shell-buffer-substring-2 (0.000863 sec)
passed 266/360 python-shell-buffer-substring-3 (0.000638 sec)
passed 267/360 python-shell-buffer-substring-4 (0.000699 sec)
passed 268/360 python-shell-buffer-substring-5 (0.000605 sec)
passed 269/360 python-shell-buffer-substring-6 (0.000535 sec)
passed 270/360 python-shell-buffer-substring-7 (0.000522 sec)
passed 271/360 python-shell-buffer-substring-8 (0.039097 sec)
passed 272/360 python-shell-buffer-substring-9 (0.000453 sec)
passed 273/360 python-shell-calculate-exec-path-1 (0.000117 sec)
passed 274/360 python-shell-calculate-exec-path-2 (0.000089 sec)
passed 275/360 python-shell-calculate-exec-path-3 (0.000085 sec)
passed 276/360 python-shell-calculate-exec-path-4 (0.001826 sec)
passed 277/360 python-shell-calculate-exec-path-5 (0.000082 sec)
passed 278/360 python-shell-calculate-exec-path-6 (0.000387 sec)
passed 279/360 python-shell-calculate-process-environment-1 (0.000137 sec)
passed 280/360 python-shell-calculate-process-environment-2 (0.000159 sec)
passed 281/360 python-shell-calculate-process-environment-3 (0.000152 sec)
passed 282/360 python-shell-calculate-process-environment-4 (0.000118 sec)
passed 283/360 python-shell-calculate-process-environment-5 (0.000115 sec)
passed 284/360 python-shell-calculate-process-environment-6 (0.000116 sec)
passed 285/360 python-shell-calculate-process-environment-7 (0.000127 sec)
passed 286/360 python-shell-calculate-process-environment-8 (0.000124 sec)
passed 287/360 python-shell-calculate-pythonpath-1 (0.000108 sec)
passed 288/360 python-shell-calculate-pythonpath-2 (0.000079 sec)
Test python-shell-completion-at-point-1 backtrace:
signal(json-parse-error ("invalid token near '_'" "<string>" 1 1 1))
apply(signal (json-parse-error ("invalid token near '_'" "<string>"
(setq value-3589 (apply fn-3587 args-3588))
(unwind-protect (setq value-3589 (apply fn-3587 args-3588)) (setq fo
(if (unwind-protect (setq value-3589 (apply fn-3587 args-3588)) (set
(let (form-description-3591) (if (unwind-protect (setq value-3589 (a
(let ((value-3589 'ert-form-evaluation-aborted-3590)) (let (form-des
(let* ((fn-3587 #'nth) (args-3588 (condition-case err (list 2 (pytho
(save-current-buffer (set-buffer (process-buffer shell-process)) (in
(let ((shell-process (python-shell-get-process-or-error))) (save-cur
(progn (run-python nil t) (insert "") (goto-char (point-min)) (pytho
(unwind-protect (progn (run-python nil t) (insert "") (goto-char (po
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3582 #'executable-find) (args-3583 (cond
#f(compiled-function () #<bytecode -0x3de848aaf387fce>)()
handler-bind-1(#f(compiled-function () #<bytecode -0x3de848aaf387fce
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-shell-completion-at-point-1 :d
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-shell-completion-at-point-1 condition:
(json-parse-error "invalid token near '_'" "<string>" 1 1 1)
FAILED 289/360 python-shell-completion-at-point-1 (0.394116 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4777
skipped 290/360 python-shell-completion-at-point-ipython (0.000420 sec)
Warning (python): Your `python-shell-interpreter' doesn't seem to support readline, yet `python-shell-completion-native-enable' was t and "python3" is not part of the `python-shell-completion-native-disabled-interpreters' list. Native completions have been disabled locally. Consider installing the python package "readline".
skipped 291/360 python-shell-completion-at-point-jedi-completer (0.280412 sec)
Warning (python): Your `python-shell-interpreter' doesn't seem to support readline, yet `python-shell-completion-native-enable' was t and "python3" is not part of the `python-shell-completion-native-disabled-interpreters' list. Native completions have been disabled locally. Consider installing the python package "readline".
Test python-shell-completion-at-point-native-1 backtrace:
signal(json-parse-error ("invalid token near '_'" "<string>" 1 1 1))
apply(signal (json-parse-error ("invalid token near '_'" "<string>"
(setq value-3604 (apply fn-3602 args-3603))
(unwind-protect (setq value-3604 (apply fn-3602 args-3603)) (setq fo
(if (unwind-protect (setq value-3604 (apply fn-3602 args-3603)) (set
(let (form-description-3606) (if (unwind-protect (setq value-3604 (a
(let ((value-3604 'ert-form-evaluation-aborted-3605)) (let (form-des
(let* ((fn-3602 #'nth) (args-3603 (condition-case err (list 2 (pytho
(save-current-buffer (set-buffer (process-buffer shell-process)) (in
(let ((shell-process (python-shell-get-process-or-error))) (save-cur
(progn (run-python nil t) (insert "") (goto-char (point-min)) (pytho
(unwind-protect (progn (run-python nil t) (insert "") (goto-char (po
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3597 #'executable-find) (args-3598 (cond
#f(compiled-function () #<bytecode -0x3de848aaf387fce>)()
handler-bind-1(#f(compiled-function () #<bytecode -0x3de848aaf387fce
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-shell-completion-at-point-nati
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-shell-completion-at-point-native-1 condition:
(json-parse-error "invalid token near '_'" "<string>" 1 1 1)
FAILED 292/360 python-shell-completion-at-point-native-1 (0.409009 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4790
passed 293/360 python-shell-completion-native-interpreter-disabled-p-1 (0.000129 sec)
Can't guess python-indent-offset, using defaults: 4
passed 294/360 python-shell-get-process-1 (0.119660 sec)
passed 295/360 python-shell-get-process-name-1 (0.000397 sec)
Can't guess python-indent-offset, using defaults: 4
passed 296/360 python-shell-get-process-name-2 (0.007572 sec)
Can't guess python-indent-offset, using defaults: 4
passed 297/360 python-shell-internal-get-or-create-process-1 (0.061770 sec)
passed 298/360 python-shell-internal-get-process-name-1 (0.000318 sec)
Can't guess python-indent-offset, using defaults: 4
passed 299/360 python-shell-internal-get-process-name-2 (0.006945 sec)
passed 300/360 python-shell-make-comint-1 (0.062339 sec)
passed 301/360 python-shell-make-comint-2 (0.051213 sec)
passed 302/360 python-shell-make-comint-3 (0.050301 sec)
passed 303/360 python-shell-make-comint-4 (0.055261 sec)
passed 304/360 python-shell-prompt-detect-1 (0.077066 sec)
passed 305/360 python-shell-prompt-detect-2 (0.078614 sec)
passed 306/360 python-shell-prompt-detect-3 (0.000390 sec)
Warning (python): Python shell prompts cannot be detected.
If your emacs session hangs when starting python shells
recover with `keyboard-quit' and then try fixing the
interactive flag for your interpreter by adjusting the
`python-shell-interpreter-interactive-arg' or add regexps
matching shell prompts in the directory-local friendly vars:
+ `python-shell-prompt-regexp'
+ `python-shell-prompt-block-regexp'
+ `python-shell-prompt-output-regexp'
Or alternatively in:
+ `python-shell-prompt-input-regexps'
+ `python-shell-prompt-output-regexps'
passed 307/360 python-shell-prompt-detect-4 (0.083172 sec)
passed 308/360 python-shell-prompt-detect-5 (0.079290 sec)
passed 309/360 python-shell-prompt-detect-6 (0.001412 sec)
passed 310/360 python-shell-prompt-set-calculated-regexps-1 (0.000211 sec)
passed 311/360 python-shell-prompt-set-calculated-regexps-2 (0.000106 sec)
passed 312/360 python-shell-prompt-set-calculated-regexps-3 (0.000105 sec)
passed 313/360 python-shell-prompt-set-calculated-regexps-4 (0.000103 sec)
passed 314/360 python-shell-prompt-set-calculated-regexps-5 (0.000100 sec)
passed 315/360 python-shell-prompt-set-calculated-regexps-6 (0.083597 sec)
passed 316/360 python-shell-prompt-validate-regexps-1 (0.000161 sec)
passed 317/360 python-shell-prompt-validate-regexps-2 (0.000198 sec)
passed 318/360 python-shell-prompt-validate-regexps-3 (0.000152 sec)
passed 319/360 python-shell-prompt-validate-regexps-4 (0.000123 sec)
passed 320/360 python-shell-prompt-validate-regexps-5 (0.000118 sec)
passed 321/360 python-shell-prompt-validate-regexps-6 (0.000111 sec)
passed 322/360 python-shell-prompt-validate-regexps-7 (0.000087 sec)
passed 323/360 python-shell-with-environment-1 (0.000323 sec)
passed 324/360 python-shell-with-environment-2 (0.000575 sec)
passed 325/360 python-shell-with-environment-3 (0.000584 sec)
passed 326/360 python-syntax-after-python-backspace (0.000540 sec)
passed 327/360 python-syntax-context-1 (0.000579 sec)
passed 328/360 python-tests--fill-long-first-line (0.001301 sec)
passed 329/360 python-tests--flymake-command-output-pattern (0.000159 sec)
passed 330/360 python-tests--run-python-selects-window (0.089348 sec)
passed 331/360 python-tests-look-at-1 (0.000405 sec)
passed 332/360 python-tests-look-at-2 (0.000268 sec)
passed 333/360 python-triple-double-quote-pairing (0.002917 sec)
passed 334/360 python-triple-single-quote-pairing (0.001202 sec)
passed 335/360 python-ts-mode-assignement-face-2 (0.052428 sec)
passed 336/360 python-ts-mode-builtin-call-face (0.019162 sec)
passed 337/360 python-ts-mode-class-patterns-face (0.002167 sec)
passed 338/360 python-ts-mode-compound-keywords-face (0.012803 sec)
passed 339/360 python-ts-mode-disabled-string-interpolation (0.002925 sec)
passed 340/360 python-ts-mode-dotted-decorator-face-1 (0.003119 sec)
passed 341/360 python-ts-mode-dotted-decorator-face-2 (0.002183 sec)
passed 342/360 python-ts-mode-interpolation-doc-string (0.001884 sec)
passed 343/360 python-ts-mode-interpolation-nested-string (0.001666 sec)
passed 344/360 python-ts-mode-isinstance-type-face-1 (0.001781 sec)
passed 345/360 python-ts-mode-isinstance-type-face-2 (0.001595 sec)
passed 346/360 python-ts-mode-isinstance-type-face-3 (0.001949 sec)
passed 347/360 python-ts-mode-level-fontification-wo-interpolation (0.001787 sec)
passed 348/360 python-ts-mode-named-assignement-face-1 (0.001459 sec)
passed 349/360 python-ts-mode-nested-types-face-1 (0.001656 sec)
passed 350/360 python-ts-mode-superclass-type-face (0.001582 sec)
passed 351/360 python-ts-mode-types-face-1 (0.001745 sec)
passed 352/360 python-ts-mode-types-face-2 (0.002995 sec)
passed 353/360 python-ts-mode-types-face-3 (0.003777 sec)
passed 354/360 python-ts-mode-union-types-face-1 (0.002735 sec)
passed 355/360 python-ts-mode-union-types-face-2 (0.001914 sec)
passed 356/360 python-util-clone-local-variables-1 (0.000445 sec)
passed 357/360 python-util-forward-comment-1 (0.001760 sec)
passed 358/360 python-util-goto-line-1 (0.000713 sec)
passed 359/360 python-util-strip-string-1 (0.000253 sec)
passed 360/360 python-util-valid-regexp-p-1 (0.000145 sec)
Ran 360 tests, 349 results as expected, 8 unexpected, 3 skipped (2024-02-15 15:38:14+0100, 8.953487 sec)
3 expected failures
8 unexpected results:
FAILED python-completion-at-point-1
FAILED python-completion-at-point-2
FAILED python-completion-at-point-native-1
FAILED python-completion-at-point-native-2
FAILED python-completion-at-point-native-with-eldoc-1
FAILED python-completion-at-point-native-with-ffap-1
FAILED python-shell-completion-at-point-1
FAILED python-shell-completion-at-point-native-1
3 skipped results:
SKIPPED python-ffap-module-path-1
SKIPPED python-shell-completion-at-point-ipython
SKIPPED python-shell-completion-at-point-jedi-completer
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-15 14:43 ` Mattias Engdegård
@ 2024-02-15 16:37 ` Eli Zaretskii
2024-02-15 16:48 ` Eli Zaretskii
2024-02-16 4:06 ` Liu Hui
2024-02-16 3:24 ` Liu Hui
1 sibling, 2 replies; 68+ messages in thread
From: Eli Zaretskii @ 2024-02-15 16:37 UTC (permalink / raw)
To: Mattias Engdegård; +Cc: liuhui1610, kobarity, 68559
> From: Mattias Engdegård <mattias.engdegard@gmail.com>
> Date: Thu, 15 Feb 2024 15:43:32 +0100
> Cc: Eli Zaretskii <eliz@gnu.org>,
> kobarity@gmail.com,
> 68559@debbugs.gnu.org
> The recent change on master, 0b9c7148fd, causes several failures in python-tests here (macOS). Log attached.
>
> It seems to have something to do with completion. Maybe the new code is sensitive to details of Python's command line editor? The standard Python interpreter is unlikely to use GNU readline, for example.
The python-*-completion-at-point-* tests never worked for me, and I
always assumed it was something specific to MS-Windows. But maybe
not. There was no change in the tests that fail for me before and
after the recent changes related to Python.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-15 16:37 ` Eli Zaretskii
@ 2024-02-15 16:48 ` Eli Zaretskii
2024-02-15 17:21 ` Mattias Engdegård
2024-02-19 13:18 ` Basil L. Contovounesios
2024-02-16 4:06 ` Liu Hui
1 sibling, 2 replies; 68+ messages in thread
From: Eli Zaretskii @ 2024-02-15 16:48 UTC (permalink / raw)
To: mattias.engdegard; +Cc: liuhui1610, kobarity, 68559
> Cc: liuhui1610@gmail.com, kobarity@gmail.com, 68559@debbugs.gnu.org
> Date: Thu, 15 Feb 2024 18:37:13 +0200
> From: Eli Zaretskii <eliz@gnu.org>
>
> > It seems to have something to do with completion. Maybe the new code is sensitive to details of Python's command line editor? The standard Python interpreter is unlikely to use GNU readline, for example.
>
> The python-*-completion-at-point-* tests never worked for me, and I
> always assumed it was something specific to MS-Windows. But maybe
> not. There was no change in the tests that fail for me before and
> after the recent changes related to Python.
And, btw, on this Gnu/Linux system:
$ uname -a
Linux (REDACTED) 5.15.0-94-generic #104+11.0trisquel25 SMP Sat Feb 10 06:24:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
all the Python tests that failed for you still pass, with the current
master. So I wonder what is it that causes those failures on your
system.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-15 16:48 ` Eli Zaretskii
@ 2024-02-15 17:21 ` Mattias Engdegård
2024-02-19 13:18 ` Basil L. Contovounesios
1 sibling, 0 replies; 68+ messages in thread
From: Mattias Engdegård @ 2024-02-15 17:21 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: liuhui1610, kobarity, 68559
15 feb. 2024 kl. 17.48 skrev Eli Zaretskii <eliz@gnu.org>:
> The python-*-completion-at-point-* tests never worked for me, and I
> always assumed it was something specific to MS-Windows. But maybe
> not. There was no change in the tests that fail for me before and
> after the recent changes related to Python.
Right, I don't think the Windows port is obscure enough to deserve test failures either; it's an unnecessary impediment. If it cannot easily be fixed then it's better to mark them to be skipped on Windows so that you don't see it every time you run 'make check'.
Completion in the Python shell worked on macOS before the change but now pressing TAB causes the process to hang, so something was clearly broken by the patch.
> all the Python tests that failed for you still pass, with the current
> master. So I wonder what is it that causes those failures on your
> system.
The standard Python installation on macOS uses a libedit-based readline module and the patch author probably didn't test the changes with that configuration.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-15 14:43 ` Mattias Engdegård
2024-02-15 16:37 ` Eli Zaretskii
@ 2024-02-16 3:24 ` Liu Hui
2024-02-16 9:34 ` kobarity
1 sibling, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-16 3:24 UTC (permalink / raw)
To: Mattias Engdegård; +Cc: Eli Zaretskii, kobarity, 68559
On Thu, Feb 15, 2024 at 10:43 PM Mattias Engdegård
<mattias.engdegard@gmail.com> wrote:
>
> The recent change on master, 0b9c7148fd, causes several failures in python-tests here (macOS). Log attached.
>
> It seems to have something to do with completion. Maybe the new code is sensitive to details of Python's command line editor? The standard Python interpreter is unlikely to use GNU readline, for example.
Hi,
According to the log, the problem may be related to the json library,
which is introduced by the patch to convert the completion results
from Python to elisp. What is the version of Python on macOS? AFAIK
the json library is added to the Standard library since version 2.6.
If the library is not available, could you setup a Python virtual
environment containing the library and run the python tests in it?
Thanks!
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-15 16:37 ` Eli Zaretskii
2024-02-15 16:48 ` Eli Zaretskii
@ 2024-02-16 4:06 ` Liu Hui
2024-02-16 7:41 ` Eli Zaretskii
1 sibling, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-16 4:06 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: Mattias Engdegård, kobarity, 68559
On Fri, Feb 16, 2024 at 12:37 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Mattias Engdegård <mattias.engdegard@gmail.com>
> > Date: Thu, 15 Feb 2024 15:43:32 +0100
> > Cc: Eli Zaretskii <eliz@gnu.org>,
> > kobarity@gmail.com,
> > 68559@debbugs.gnu.org
> > The recent change on master, 0b9c7148fd, causes several failures in python-tests here (macOS). Log attached.
> >
> > It seems to have something to do with completion. Maybe the new code is sensitive to details of Python's command line editor? The standard Python interpreter is unlikely to use GNU readline, for example.
>
> The python-*-completion-at-point-* tests never worked for me, and I
> always assumed it was something specific to MS-Windows. But maybe
> not. There was no change in the tests that fail for me before and
> after the recent changes related to Python.
The Python shell completion relies on the readline module, which is
not available for Python on MS-Windows. According to the instruction
in python.el:
;; readline based shell (it's known to work with PyPy). If your
;; Python installation lacks readline (like CPython for Windows),
;; installing pyreadline (URL `https://ipython.org/pyreadline.html')
;; should suffice. To troubleshoot why you are not getting any
;; completions, you can try the following in your Python shell:
;; >>> import readline, rlcompleter
;; If you see an error, then you need to either install pyreadline or
;; setup custom code that avoids that dependency.
It may be necessary to install pyreadline (for Python 2.7) or
pyreadline3 (for Python 3).
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-16 4:06 ` Liu Hui
@ 2024-02-16 7:41 ` Eli Zaretskii
2024-02-16 12:51 ` Eli Zaretskii
0 siblings, 1 reply; 68+ messages in thread
From: Eli Zaretskii @ 2024-02-16 7:41 UTC (permalink / raw)
To: Liu Hui; +Cc: mattias.engdegard, kobarity, 68559
> From: Liu Hui <liuhui1610@gmail.com>
> Date: Fri, 16 Feb 2024 12:06:38 +0800
> Cc: Mattias Engdegård <mattias.engdegard@gmail.com>,
> kobarity@gmail.com, 68559@debbugs.gnu.org
>
> On Fri, Feb 16, 2024 at 12:37 AM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > The python-*-completion-at-point-* tests never worked for me, and I
> > always assumed it was something specific to MS-Windows. But maybe
> > not. There was no change in the tests that fail for me before and
> > after the recent changes related to Python.
>
> The Python shell completion relies on the readline module, which is
> not available for Python on MS-Windows. According to the instruction
> in python.el:
>
> ;; readline based shell (it's known to work with PyPy). If your
> ;; Python installation lacks readline (like CPython for Windows),
> ;; installing pyreadline (URL `https://ipython.org/pyreadline.html')
> ;; should suffice. To troubleshoot why you are not getting any
> ;; completions, you can try the following in your Python shell:
>
> ;; >>> import readline, rlcompleter
>
> ;; If you see an error, then you need to either install pyreadline or
> ;; setup custom code that avoids that dependency.
I don't know if I have CPython, but the above does show an error
message.
> It may be necessary to install pyreadline (for Python 2.7) or
> pyreadline3 (for Python 3).
I will see if I can do that, thanks.
Regardless, patches to the test suite to skip the tests which rely on
those modules, if they aren't installed, will be welcome.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-16 3:24 ` Liu Hui
@ 2024-02-16 9:34 ` kobarity
2024-02-16 11:45 ` Mattias Engdegård
0 siblings, 1 reply; 68+ messages in thread
From: kobarity @ 2024-02-16 9:34 UTC (permalink / raw)
To: Liu Hui; +Cc: Mattias Engdegård, Eli Zaretskii, 68559
Liu Hui wrote:
> On Thu, Feb 15, 2024 at 10:43 PM Mattias Engdegård
> <mattias.engdegard@gmail.com> wrote:
> >
> > The recent change on master, 0b9c7148fd, causes several failures in python-tests here (macOS). Log attached.
> >
> > It seems to have something to do with completion. Maybe the new code is sensitive to details of Python's command line editor? The standard Python interpreter is unlikely to use GNU readline, for example.
>
> Hi,
>
> According to the log, the problem may be related to the json library,
> which is introduced by the patch to convert the completion results
> from Python to elisp. What is the version of Python on macOS? AFAIK
> the json library is added to the Standard library since version 2.6.
> If the library is not available, could you setup a Python virtual
> environment containing the library and run the python tests in it?
> Thanks!
This is caused by the fact that the input is echoed back on MacOS
Python. This is described in etc/PROBLEMS:
*** In Inferior Python mode, input is echoed and native completion doesn't work.
The following log shows that it tries to parse the echoed back string
as a JSON string.
Test python-completion-at-point-1 backtrace:
json-parse-string("__PYTHON_EL_eval_file(\"/var/folders/qy/zstv16390
python--parse-json-array("__PYTHON_EL_eval_file(\"/var/folders/qy/zs
I think the echoed back string were ignored before the patch is
applied. However, after applying the patch, the returned string is
now parsed as JSON, which I believe is the reason for the error.
So one workaround would be to remove the echoed back string before
parsing as JSON.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-16 9:34 ` kobarity
@ 2024-02-16 11:45 ` Mattias Engdegård
2024-02-16 15:24 ` kobarity
0 siblings, 1 reply; 68+ messages in thread
From: Mattias Engdegård @ 2024-02-16 11:45 UTC (permalink / raw)
To: kobarity; +Cc: Liu Hui, Eli Zaretskii, 68559
16 feb. 2024 kl. 10.34 skrev kobarity <kobarity@gmail.com>:
> This is caused by the fact that the input is echoed back on MacOS
> Python.
Right, that bug needs to be fixed as well, but the echo problem existed prior to the change that broke the tests.
An alternative might be to disable the tty echo altogether. If I do it right after process creation then it has no effect; presumably Python or its readline module turns on echo just a bit later on. Anyway, running
import tty
tty.setraw(0)
in the python shell seems to put it right, with working completion and the annoying echo gone.
> So one workaround would be to remove the echoed back string before
> parsing as JSON.
Yes, either that or turning off echo in the tty.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-16 7:41 ` Eli Zaretskii
@ 2024-02-16 12:51 ` Eli Zaretskii
0 siblings, 0 replies; 68+ messages in thread
From: Eli Zaretskii @ 2024-02-16 12:51 UTC (permalink / raw)
To: liuhui1610, mattias.engdegard, kobarity; +Cc: 68559
> Cc: mattias.engdegard@gmail.com, kobarity@gmail.com, 68559@debbugs.gnu.org
> Date: Fri, 16 Feb 2024 09:41:37 +0200
> From: Eli Zaretskii <eliz@gnu.org>
>
> > It may be necessary to install pyreadline (for Python 2.7) or
> > pyreadline3 (for Python 3).
>
> I will see if I can do that, thanks.
I've now installed pyreadline3, and I can confirm that the completion
tests which were failing before now pass.
Thanks.
> Regardless, patches to the test suite to skip the tests which rely on
> those modules, if they aren't installed, will be welcome.
This is still true.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-16 11:45 ` Mattias Engdegård
@ 2024-02-16 15:24 ` kobarity
2024-02-16 15:52 ` Eli Zaretskii
` (2 more replies)
0 siblings, 3 replies; 68+ messages in thread
From: kobarity @ 2024-02-16 15:24 UTC (permalink / raw)
To: Mattias Engdegård; +Cc: Liu Hui, Eli Zaretskii, 68559
[-- Attachment #1: Type: text/plain, Size: 1532 bytes --]
Mattias Engdegård wrote:
>
> 16 feb. 2024 kl. 10.34 skrev kobarity <kobarity@gmail.com>:
>
> > This is caused by the fact that the input is echoed back on MacOS
> > Python.
>
> Right, that bug needs to be fixed as well, but the echo problem existed prior to the change that broke the tests.
>
> An alternative might be to disable the tty echo altogether. If I do it right after process creation then it has no effect; presumably Python or its readline module turns on echo just a bit later on. Anyway, running
>
> import tty
> tty.setraw(0)
>
> in the python shell seems to put it right, with working completion and the annoying echo gone.
>
> > So one workaround would be to remove the echoed back string before
> > parsing as JSON.
>
> Yes, either that or turning off echo in the tty.
I made prototype patches for each method. I don't use Mac so it would
be helpful if you could try these.
0001-Remove-echoed-back-string-in-python-shell-completion.patch
extracts only the last line to exclude echoed back strings.
0001-Set-tty-mode-to-raw-when-setting-up-Inferior-Python.patch sets
the Inferior Python tty to raw mode. python-ffap-module-path-1 will
no longer need to be skipped on Mac. If it is safe to set tty to raw
mode on all UNIX based systems, I prefer this method.
By the way, is it necessary to send
`python-shell-completion-setup-code' for every completion in
`python-shell-completion-get-completions'? To me it seems sufficient
to send it once at initialization.
[-- Attachment #2: 0001-Remove-echoed-back-string-in-python-shell-completion.patch --]
[-- Type: application/octet-stream, Size: 1480 bytes --]
From 601419eef799d2c68e3789222f5ebe2a8c31af97 Mon Sep 17 00:00:00 2001
From: kobarity <kobarity@gmail.com>
Date: Fri, 16 Feb 2024 22:48:19 +0900
Subject: [PATCH] Remove echoed back string in
python-shell-completion-get-completions
* lisp/progmodes/python.el (python-shell-completion-get-completions):
Remove echoed back string. (Bug#68559)
---
lisp/progmodes/python.el | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index b7e43f3fc68..a842a498113 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -4624,11 +4624,13 @@ python-shell-completion-get-completions
"Get completions of INPUT using PROCESS."
(with-current-buffer (process-buffer process)
(python--parse-json-array
- (python-shell-send-string-no-output
- (format "%s\nprint(__PYTHON_EL_get_completions(%s))"
- python-shell-completion-setup-code
- (python-shell--encode-string input))
- process))))
+ (car (last (split-string
+ (python-shell-send-string-no-output
+ (format "%s\nprint(__PYTHON_EL_get_completions(%s))"
+ python-shell-completion-setup-code
+ (python-shell--encode-string input))
+ process)
+ "[\n\r]+" t))))))
(defun python-shell--get-multiline-input ()
"Return lines at a multi-line input in Python shell."
--
2.34.1
[-- Attachment #3: 0001-Set-tty-mode-to-raw-when-setting-up-Inferior-Python.patch --]
[-- Type: application/octet-stream, Size: 2531 bytes --]
From 31738d481333bf6c258a694cf57bce944bce7778 Mon Sep 17 00:00:00 2001
From: kobarity <kobarity@gmail.com>
Date: Fri, 16 Feb 2024 22:52:06 +0900
Subject: [PATCH] Set tty mode to raw when setting up Inferior Python
* lisp/progmodes/python.el (python-shell-setup-code): New constant.
(python-shell-comint-watch-for-first-prompt-output-filter): Send
`python-shell-setup-code' to the Inferior Python process.
* test/lisp/progmodes/python-tests.el (python-ffap-module-path-1):
Eliminate skipping on Mac. (Bug#68559)
---
lisp/progmodes/python.el | 11 +++++++++++
test/lisp/progmodes/python-tests.el | 5 -----
2 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index b7e43f3fc68..5501926e69d 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -3521,6 +3521,16 @@ python-shell-first-prompt-hook
:version "25.1"
:type 'hook)
+(defconst python-shell-setup-code
+ "\
+try:
+ import tty
+except ImportError:
+ pass
+else:
+ tty.setraw(0)"
+ "Code used to setup the inferior Python processes.")
+
(defconst python-shell-eval-setup-code
"\
def __PYTHON_EL_eval(source, filename):
@@ -3586,6 +3596,7 @@ python-shell-comint-watch-for-first-prompt-output-filter
(format "exec(%s)\n" (python-shell--encode-string string))))))
;; Bootstrap: the normal definition of `python-shell-send-string'
;; depends on the Python code sent here.
+ (python-shell-send-string-no-output python-shell-setup-code)
(python-shell-send-string-no-output python-shell-eval-setup-code)
(python-shell-send-string-no-output python-shell-eval-file-setup-code))
(with-current-buffer (current-buffer)
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index af6c199b5bd..6c6cd9eee2b 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -5037,11 +5037,6 @@ python-completion-at-point-native-with-eldoc-1
(ert-deftest python-ffap-module-path-1 ()
(skip-unless (executable-find python-tests-shell-interpreter))
- ;; Skip the test on macOS, since the standard Python installation uses
- ;; libedit rather than readline which confuses the running of an inferior
- ;; interpreter in this case (see bug#59477 and bug#25753).
- (skip-when (eq system-type 'darwin))
- (trace-function 'python-shell-output-filter)
(python-tests-with-temp-buffer-with-shell
"
import abc
--
2.34.1
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-16 15:24 ` kobarity
@ 2024-02-16 15:52 ` Eli Zaretskii
2024-02-16 20:10 ` Mattias Engdegård
2024-02-17 4:36 ` Liu Hui
2 siblings, 0 replies; 68+ messages in thread
From: Eli Zaretskii @ 2024-02-16 15:52 UTC (permalink / raw)
To: kobarity; +Cc: liuhui1610, mattias.engdegard, 68559
> Date: Sat, 17 Feb 2024 00:24:29 +0900
> From: kobarity <kobarity@gmail.com>
> Cc: Liu Hui <liuhui1610@gmail.com>,
> Eli Zaretskii <eliz@gnu.org>,
> 68559@debbugs.gnu.org
> > Right, that bug needs to be fixed as well, but the echo problem existed prior to the change that broke the tests.
> >
> > An alternative might be to disable the tty echo altogether. If I do it right after process creation then it has no effect; presumably Python or its readline module turns on echo just a bit later on. Anyway, running
> >
> > import tty
> > tty.setraw(0)
> >
> > in the python shell seems to put it right, with working completion and the annoying echo gone.
> >
> > > So one workaround would be to remove the echoed back string before
> > > parsing as JSON.
> >
> > Yes, either that or turning off echo in the tty.
>
> I made prototype patches for each method. I don't use Mac so it would
> be helpful if you could try these.
>
> 0001-Remove-echoed-back-string-in-python-shell-completion.patch
> extracts only the last line to exclude echoed back strings.
>
> 0001-Set-tty-mode-to-raw-when-setting-up-Inferior-Python.patch sets
> the Inferior Python tty to raw mode. python-ffap-module-path-1 will
> no longer need to be skipped on Mac. If it is safe to set tty to raw
> mode on all UNIX based systems, I prefer this method.
Will this work on MS-Windows as well? If you are unsure, would you
please tell me how to test whether this works on Windows, so I could
collect the information for you?
Thanks.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-16 15:24 ` kobarity
2024-02-16 15:52 ` Eli Zaretskii
@ 2024-02-16 20:10 ` Mattias Engdegård
2024-02-17 13:33 ` kobarity
2024-02-17 4:36 ` Liu Hui
2 siblings, 1 reply; 68+ messages in thread
From: Mattias Engdegård @ 2024-02-16 20:10 UTC (permalink / raw)
To: kobarity; +Cc: Liu Hui, Eli Zaretskii, 68559
16 feb. 2024 kl. 16.24 skrev kobarity <kobarity@gmail.com>:
> I made prototype patches for each method. I don't use Mac so it would
> be helpful if you could try these.
Nice, thank you! I can confirm that they both appear to work, at least in the sense that the python-tests pass (except for the ones skipped intentionally), and the python shell behaves reasonable.
Only the set-tty-mode patch eliminates echo in the interactive python shell; in that sense it's preferable.
Both produce the very annoying warning
Warning (python): Your ‘python-shell-interpreter’ doesn’t seem to support readline, yet ‘python-shell-completion-native-enable’ was t and "python3" is not part of the ‘python-shell-completion-native-disabled-interpreters’ list. Native completions have been disabled locally. Consider installing the python package "readline".
which is not even correct since the standard Python does have a working readline module, even if it uses libedit.
> 0001-Set-tty-mode-to-raw-when-setting-up-Inferior-Python.patch sets
> the Inferior Python tty to raw mode. python-ffap-module-path-1 will
> no longer need to be skipped on Mac.
Right, that test runs and passes.
> If it is safe to set tty to raw
> mode on all UNIX based systems, I prefer this method.
Same here. I see no reason why it wouldn't be safe, either.
> By the way, is it necessary to send
> `python-shell-completion-setup-code' for every completion in
> `python-shell-completion-get-completions'? To me it seems sufficient
> to send it once at initialization.
Indeed, it does seem a bit extravagant.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-16 15:24 ` kobarity
2024-02-16 15:52 ` Eli Zaretskii
2024-02-16 20:10 ` Mattias Engdegård
@ 2024-02-17 4:36 ` Liu Hui
2024-02-17 13:20 ` Mattias Engdegård
2 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-17 4:36 UTC (permalink / raw)
To: kobarity; +Cc: Mattias Engdegård, Eli Zaretskii, 68559
kobarity <kobarity@gmail.com> writes:
> 0001-Set-tty-mode-to-raw-when-setting-up-Inferior-Python.patch sets
> the Inferior Python tty to raw mode. python-ffap-module-path-1 will
> no longer need to be skipped on Mac. If it is safe to set tty to raw
> mode on all UNIX based systems, I prefer this method.
How about the following change, which only affects libedit-based
readline? It may enable native completion on mac, but I cannot test
it.
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index b7e43f3fc68..f59bc19367b 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -4286,6 +4286,9 @@ (defcustom python-shell-completion-setup-code
except:
pass
else:
+ if readline.__doc__ and 'libedit' in readline.__doc__:
+ import tty
+ tty.setraw(0)
# Try to reuse current completer.
completer = readline.get_completer()
if not completer:
@@ -4471,8 +4474,8 @@ (defun python-shell-completion-native-setup ()
instance.rlcomplete = new_completer
if readline.__doc__ and 'libedit' in readline.__doc__:
- raise Exception('''libedit based readline is known not to work,
- see etc/PROBLEMS under \"In Inferior Python mode, input is echoed\".''')
+ import tty
+ tty.setraw(0)
readline.parse_and_bind('bind ^I rl_complete')
else:
readline.parse_and_bind('tab: complete')
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-17 4:36 ` Liu Hui
@ 2024-02-17 13:20 ` Mattias Engdegård
0 siblings, 0 replies; 68+ messages in thread
From: Mattias Engdegård @ 2024-02-17 13:20 UTC (permalink / raw)
To: Liu Hui; +Cc: kobarity, Eli Zaretskii, 68559
[-- Attachment #1: Type: text/plain, Size: 312 bytes --]
17 feb. 2024 kl. 05.36 skrev Liu Hui <liuhui1610@gmail.com>:
> How about the following change, which only affects libedit-based
> readline? It may enable native completion on mac, but I cannot test
> it.
Thank you, but that change alone doesn't seem to suffice for python-tests to pass. Log attached.
[-- Attachment #2: python-tests.log --]
[-- Type: application/octet-stream, Size: 44268 bytes --]
ELC ../../emacs/test/lisp/progmodes/python-tests.elc
GEN lisp/progmodes/python-tests.log
Running 360 tests (2024-02-17 14:08:16+0100, selector `(not (or (tag :unstable) (tag :nativecomp)))')
passed 1/360 python-auto-fill-docstring (0.000969 sec)
Fontifying *temp*-195786...
Fontifying *temp*-195786... (syntactically...)
Fontifying *temp*-195786... (regexps...)
Fontifying *temp*-195786... (regexps....)
Fontifying *temp*-195786... (regexps.....)
Fontifying *temp*-195786... (regexps......)
Fontifying *temp*-195786... (regexps.......)
Fontifying *temp*-195786... (regexps........)
Fontifying *temp*-195786... (regexps.........)
Fontifying *temp*-195786... (regexps..........)
Fontifying *temp*-195786... (regexps...........)
Fontifying *temp*-195786... (regexps............)
Fontifying *temp*-195786... (regexps.............)
Fontifying *temp*-195786... (regexps..............)
Fontifying *temp*-195786... (regexps...............)
Fontifying *temp*-195786... (regexps................)
passed 2/360 python-bob-infloop-avoid (0.000757 sec)
Test python-completion-at-point-1 backtrace:
json-parse-string("__PYTHON_EL_eval_file(\"/var/folders/qy/zstv16390
python--parse-json-array("__PYTHON_EL_eval_file(\"/var/folders/qy/zs
python-shell-completion-get-completions(#<process Python[ *temp*-828
python-shell-completion-at-point(#<process Python[ *temp*-828935]>)
python-completion-at-point()
completion--capf-wrapper(python-completion-at-point all)
run-hook-wrapped(completion--capf-wrapper python-completion-at-point
completion-at-point()
apply(completion-at-point nil)
(setq value-3676 (apply fn-3674 args-3675))
(unwind-protect (setq value-3676 (apply fn-3674 args-3675)) (setq fo
(if (unwind-protect (setq value-3676 (apply fn-3674 args-3675)) (set
(let (form-description-3678) (if (unwind-protect (setq value-3676 (a
(let ((value-3676 'ert-form-evaluation-aborted-3677)) (let (form-des
(let* ((fn-3674 #'completion-at-point) (args-3675 (condition-case er
(let ((inhibit-message t)) (python-shell-send-buffer) (python-tests-
(progn (run-python nil t) (insert "\nimport abc\n") (goto-char (poin
(unwind-protect (progn (run-python nil t) (insert "\nimport abc\n")
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3669 #'executable-find) (args-3670 (cond
#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf>)()
handler-bind-1(#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-completion-at-point-1 :documen
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-completion-at-point-1 condition:
(json-parse-error "invalid token near '_'" "<string>" 1 1 1)
FAILED 3/360 python-completion-at-point-1 (0.474468 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4903
Test python-completion-at-point-2 backtrace:
json-parse-string("__PYTHON_EL_eval_file(\"/var/folders/qy/zstv16390
python--parse-json-array("__PYTHON_EL_eval_file(\"/var/folders/qy/zs
python-shell-completion-get-completions(#<process Python[ *temp*-238
python-shell-completion-at-point(#<process Python[ *temp*-238097]>)
python-completion-at-point()
completion--capf-wrapper(python-completion-at-point all)
run-hook-wrapped(completion--capf-wrapper python-completion-at-point
completion-at-point()
apply(completion-at-point nil)
(setq value-3691 (apply fn-3689 args-3690))
(unwind-protect (setq value-3691 (apply fn-3689 args-3690)) (setq fo
(if (unwind-protect (setq value-3691 (apply fn-3689 args-3690)) (set
(let (form-description-3693) (if (unwind-protect (setq value-3691 (a
(let ((value-3691 'ert-form-evaluation-aborted-3692)) (let (form-des
(let* ((fn-3689 #'completion-at-point) (args-3690 (condition-case er
(let ((inhibit-message t)) (python-shell-send-buffer) (python-tests-
(progn (run-python nil t) (insert "\nimport abc\n") (goto-char (poin
(unwind-protect (progn (run-python nil t) (insert "\nimport abc\n")
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3684 #'executable-find) (args-3685 (cond
#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf>)()
handler-bind-1(#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-completion-at-point-2 :documen
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-completion-at-point-2 condition:
(json-parse-error "invalid token near '_'" "<string>" 1 1 1)
FAILED 4/360 python-completion-at-point-2 (0.378594 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4918
Test python-completion-at-point-native-1 backtrace:
signal(ert-test-failed (((should (completion-at-point)) :form (compl
ert-fail(((should (completion-at-point)) :form (completion-at-point)
(if (unwind-protect (setq value-3718 (apply fn-3716 args-3717)) (set
(let (form-description-3720) (if (unwind-protect (setq value-3718 (a
(let ((value-3718 'ert-form-evaluation-aborted-3719)) (let (form-des
(let* ((fn-3716 #'completion-at-point) (args-3717 (condition-case er
(let ((inhibit-message t)) (python-shell-completion-native-turn-on)
(progn (run-python nil t) (insert "\nimport abc\n") (goto-char (poin
(unwind-protect (progn (run-python nil t) (insert "\nimport abc\n")
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3711 #'executable-find) (args-3712 (cond
#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf>)()
handler-bind-1(#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-completion-at-point-native-1 :
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-completion-at-point-native-1 condition:
(ert-test-failed
((should (completion-at-point)) :form (completion-at-point) :value
nil))
FAILED 5/360 python-completion-at-point-native-1 (3.420235 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4966
Test python-completion-at-point-native-2 backtrace:
signal(ert-test-failed (((should (completion-at-point)) :form (compl
ert-fail(((should (completion-at-point)) :form (completion-at-point)
(if (unwind-protect (setq value-3733 (apply fn-3731 args-3732)) (set
(let (form-description-3735) (if (unwind-protect (setq value-3733 (a
(let ((value-3733 'ert-form-evaluation-aborted-3734)) (let (form-des
(let* ((fn-3731 #'completion-at-point) (args-3732 (condition-case er
(let ((inhibit-message t)) (python-shell-completion-native-turn-on)
(progn (run-python nil t) (insert "\nimport abc\n") (goto-char (poin
(unwind-protect (progn (run-python nil t) (insert "\nimport abc\n")
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3726 #'executable-find) (args-3727 (cond
#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf>)()
handler-bind-1(#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-completion-at-point-native-2 :
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-completion-at-point-native-2 condition:
(ert-test-failed
((should (completion-at-point)) :form (completion-at-point) :value
nil))
FAILED 6/360 python-completion-at-point-native-2 (3.411212 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4982
Test python-completion-at-point-native-with-eldoc-1 backtrace:
signal(ert-test-failed (((should (completion-at-point)) :form (compl
ert-fail(((should (completion-at-point)) :form (completion-at-point)
(if (unwind-protect (setq value-3753 (apply fn-3751 args-3752)) (set
(let (form-description-3755) (if (unwind-protect (setq value-3753 (a
(let ((value-3753 'ert-form-evaluation-aborted-3754)) (let (form-des
(let* ((fn-3751 #'completion-at-point) (args-3752 (condition-case er
(let ((inhibit-message t)) (python-shell-completion-native-turn-on)
(progn (run-python nil t) (insert "\nimport abc\n") (goto-char (poin
(unwind-protect (progn (run-python nil t) (insert "\nimport abc\n")
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3746 #'executable-find) (args-3747 (cond
#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf>)()
handler-bind-1(#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-completion-at-point-native-wit
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-completion-at-point-native-with-eldoc-1 condition:
(ert-test-failed
((should (completion-at-point)) :form (completion-at-point) :value
nil))
FAILED 7/360 python-completion-at-point-native-with-eldoc-1 (3.472809 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:5015
Test python-completion-at-point-native-with-ffap-1 backtrace:
signal(ert-test-failed (((should (completion-at-point)) :form (compl
ert-fail(((should (completion-at-point)) :form (completion-at-point)
(if (unwind-protect (setq value-3743 (apply fn-3741 args-3742)) (set
(let (form-description-3745) (if (unwind-protect (setq value-3743 (a
(let ((value-3743 'ert-form-evaluation-aborted-3744)) (let (form-des
(let* ((fn-3741 #'completion-at-point) (args-3742 (condition-case er
(let ((inhibit-message t)) (python-shell-completion-native-turn-on)
(progn (run-python nil t) (insert "\nimport abc\n") (goto-char (poin
(unwind-protect (progn (run-python nil t) (insert "\nimport abc\n")
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3736 #'executable-find) (args-3737 (cond
#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf>)()
handler-bind-1(#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-completion-at-point-native-wit
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-completion-at-point-native-with-ffap-1 condition:
(ert-test-failed
((should (completion-at-point)) :form (completion-at-point) :value
nil))
FAILED 8/360 python-completion-at-point-native-with-ffap-1 (3.473754 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4999
passed 9/360 python-completion-at-point-pdb-1 (0.514344 sec)
passed 10/360 python-completion-at-point-while-running-1 (0.203331 sec)
passed 11/360 python-eldoc--get-doc-at-point-1 (0.416836 sec)
passed 12/360 python-eldoc--get-doc-at-point-while-running-1 (0.201219 sec)
passed 13/360 python-eldoc--get-symbol-at-point-1 (0.001408 sec)
passed 14/360 python-eldoc--get-symbol-at-point-2 (0.001300 sec)
passed 15/360 python-eldoc--get-symbol-at-point-3 (0.000353 sec)
passed 16/360 python-eldoc--get-symbol-at-point-4 (0.000365 sec)
passed 17/360 python-end-of-defun-1 (0.000403 sec)
skipped 18/360 python-ffap-module-path-1 (0.000507 sec)
passed 19/360 python-ffap-module-path-while-running-1 (0.204248 sec)
passed 20/360 python-fill-docstring (0.001434 sec)
passed 21/360 python-fill-paragraph-single-quoted-string-1 (0.000909 sec)
passed 22/360 python-fill-paragraph-single-quoted-string-2 (0.000369 sec)
passed 23/360 python-fill-paragraph-triple-quoted-string-1 (0.003542 sec)
passed 24/360 python-font-lock-assignment-statement-1 (0.000618 sec)
passed 25/360 python-font-lock-assignment-statement-10 (0.000487 sec)
passed 26/360 python-font-lock-assignment-statement-11 (0.000953 sec)
passed 27/360 python-font-lock-assignment-statement-12 (0.000504 sec)
passed 28/360 python-font-lock-assignment-statement-13 (0.000774 sec)
passed 29/360 python-font-lock-assignment-statement-14 (0.000396 sec)
passed 30/360 python-font-lock-assignment-statement-15 (0.000363 sec)
passed 31/360 python-font-lock-assignment-statement-16 (0.000368 sec)
passed 32/360 python-font-lock-assignment-statement-17 (0.000348 sec)
passed 33/360 python-font-lock-assignment-statement-18 (0.000592 sec)
passed 34/360 python-font-lock-assignment-statement-2 (0.000334 sec)
passed 35/360 python-font-lock-assignment-statement-3 (0.000320 sec)
passed 36/360 python-font-lock-assignment-statement-4 (0.000805 sec)
passed 37/360 python-font-lock-assignment-statement-5 (0.000691 sec)
passed 38/360 python-font-lock-assignment-statement-6 (0.000625 sec)
passed 39/360 python-font-lock-assignment-statement-7 (0.000499 sec)
passed 40/360 python-font-lock-assignment-statement-8 (0.000449 sec)
passed 41/360 python-font-lock-assignment-statement-9 (0.000551 sec)
passed 42/360 python-font-lock-escape-sequence-bytes-newline (0.000447 sec)
passed 43/360 python-font-lock-escape-sequence-hex-octal (0.000793 sec)
passed 44/360 python-font-lock-escape-sequence-multiline-string (0.013924 sec)
passed 45/360 python-font-lock-escape-sequence-string-newline (0.001196 sec)
passed 46/360 python-font-lock-escape-sequence-unicode (0.000698 sec)
passed 47/360 python-font-lock-keywords-level-1-1 (0.000317 sec)
passed 48/360 python-font-lock-keywords-level-1-2 (0.000300 sec)
passed 49/360 python-font-lock-operator-1 (0.000813 sec)
passed 50/360 python-font-lock-operator-2 (0.000577 sec)
passed 51/360 python-font-lock-raw-escape-sequence (0.001108 sec)
passed 52/360 python-font-lock-string-literal-concatenation (0.000499 sec)
Hiding all blocks...
Hiding all blocks...done
passed 53/360 python-hideshow-hide-all-1 (0.000739 sec)
Hiding all blocks...
Hiding all blocks...done
passed 54/360 python-hideshow-hide-all-2 (0.000487 sec)
Hiding all blocks...
Hiding all blocks...done
passed 55/360 python-hideshow-hide-all-3 (0.000418 sec)
passed 56/360 python-hideshow-hide-block-1 (0.000531 sec)
Hiding blocks ...
Hiding blocks ... done
Showing all blocks ...
Showing all blocks ... done
passed 57/360 python-hideshow-hide-levels-1 (0.001552 sec)
Showing all blocks ...
Showing all blocks ... done
passed 58/360 python-hideshow-hide-levels-2 (0.000666 sec)
Hiding blocks ...
Hiding blocks ... done
passed 59/360 python-hideshow-hide-levels-3 (0.001000 sec)
Hiding blocks ...
Hiding blocks ... done
passed 60/360 python-hideshow-hide-levels-4 (0.001441 sec)
passed 61/360 python-imenu-create-flat-index-1 (0.000892 sec)
passed 62/360 python-imenu-create-flat-index-2 (0.000358 sec)
passed 63/360 python-imenu-create-index-1 (0.000555 sec)
passed 64/360 python-imenu-create-index-2 (0.000344 sec)
passed 65/360 python-imenu-create-index-3 (0.000331 sec)
passed 66/360 python-imenu-create-index-4 (0.000350 sec)
passed 67/360 python-indent-after-async-block-1 (0.000515 sec)
passed 68/360 python-indent-after-async-block-2 (0.000386 sec)
passed 69/360 python-indent-after-async-block-3 (0.000364 sec)
passed 70/360 python-indent-after-backslash-1 (0.000792 sec)
passed 71/360 python-indent-after-backslash-2 (0.001414 sec)
passed 72/360 python-indent-after-backslash-3 (0.000629 sec)
passed 73/360 python-indent-after-backslash-4 (0.000913 sec)
passed 74/360 python-indent-after-backslash-5 (0.000501 sec)
passed 75/360 python-indent-after-backslash-6 (0.000897 sec)
passed 76/360 python-indent-after-bare-match (0.000608 sec)
passed 77/360 python-indent-after-block-1 (0.000547 sec)
passed 78/360 python-indent-after-block-2 (0.000419 sec)
passed 79/360 python-indent-after-block-3 (0.000526 sec)
passed 80/360 python-indent-after-case-block (0.000359 sec)
passed 81/360 python-indent-after-comment-1 (0.000820 sec)
passed 82/360 python-indent-after-comment-2 (0.001027 sec)
passed 83/360 python-indent-after-comment-3 (0.000379 sec)
passed 84/360 python-indent-after-match-block (0.000493 sec)
passed 85/360 python-indent-after-re-match (0.000623 sec)
passed 86/360 python-indent-base-case (0.000440 sec)
passed 87/360 python-indent-block-enders-1 (0.000678 sec)
passed 88/360 python-indent-block-enders-2 (0.000862 sec)
passed 89/360 python-indent-block-enders-3 (0.000453 sec)
passed 90/360 python-indent-block-enders-4 (0.000454 sec)
passed 91/360 python-indent-block-enders-5 (0.000614 sec)
passed 92/360 python-indent-dedent-line-backspace-1 (0.000428 sec)
passed 93/360 python-indent-dedent-line-backspace-2 (0.000259 sec)
passed 94/360 python-indent-dedent-line-backspace-3 (0.000464 sec)
passed 95/360 python-indent-dedenters-1 (0.000417 sec)
Closes if hide_details:
Closes except Exception:
Closes if save:
passed 96/360 python-indent-dedenters-2 (0.001706 sec)
Closes try:
passed 97/360 python-indent-dedenters-3 (0.000751 sec)
Closes try:
passed 98/360 python-indent-dedenters-4 (0.000825 sec)
Closes if save:
passed 99/360 python-indent-dedenters-5 (0.001327 sec)
passed 100/360 python-indent-dedenters-6 (0.001020 sec)
passed 101/360 python-indent-dedenters-7 (0.000662 sec)
Closes if (a == 1 or
Closes if (a == 1 or
Closes if (a == 1 or
passed 102/360 python-indent-dedenters-8 (0.001472 sec)
Closes case 1:
passed 103/360 python-indent-dedenters-9 (0.000582 sec)
Closes if hide_details:
Closes except Exception:
Closes if save:
passed 104/360 python-indent-dedenters-comment-else (0.003784 sec)
passed 105/360 python-indent-electric-colon-1 (0.000420 sec)
Closes if do:
passed 106/360 python-indent-electric-colon-2 (0.000480 sec)
Closes if do:
Closes if do:
Closes if do:
passed 107/360 python-indent-electric-colon-3 (0.000724 sec)
Closes if True:
passed 108/360 python-indent-electric-colon-4 (0.000562 sec)
passed 109/360 python-indent-electric-comma-after-multiline-string (0.000467 sec)
passed 110/360 python-indent-electric-comma-inside-multiline-string (0.000702 sec)
passed 111/360 python-indent-hanging-close-paren (0.000450 sec)
passed 112/360 python-indent-inside-paren-1 (0.001556 sec)
passed 113/360 python-indent-inside-paren-2 (0.000875 sec)
passed 114/360 python-indent-inside-paren-3 (0.000436 sec)
passed 115/360 python-indent-inside-paren-4 (0.000375 sec)
passed 116/360 python-indent-inside-paren-5 (0.000495 sec)
passed 117/360 python-indent-inside-paren-6 (0.001114 sec)
passed 118/360 python-indent-inside-paren-7 (0.000481 sec)
passed 119/360 python-indent-inside-paren-8 (0.000563 sec)
passed 120/360 python-indent-inside-paren-9 (0.000931 sec)
passed 121/360 python-indent-inside-paren-block-1 (0.000826 sec)
passed 122/360 python-indent-inside-paren-block-2 (0.000831 sec)
passed 123/360 python-indent-inside-paren-block-3 (0.000598 sec)
passed 124/360 python-indent-inside-paren-block-4 (0.000804 sec)
passed 125/360 python-indent-inside-string-1 (0.000824 sec)
passed 126/360 python-indent-inside-string-2 (0.002007 sec)
passed 127/360 python-indent-inside-string-3 (0.001548 sec)
passed 128/360 python-indent-pep8-1 (0.000829 sec)
passed 129/360 python-indent-pep8-2 (0.000796 sec)
passed 130/360 python-indent-pep8-3 (0.000469 sec)
passed 131/360 python-indent-region-1 (0.000342 sec)
passed 132/360 python-indent-region-2 (0.000414 sec)
passed 133/360 python-indent-region-3 (0.000983 sec)
passed 134/360 python-indent-region-4 (0.000713 sec)
passed 135/360 python-indent-region-5 (0.002212 sec)
passed 136/360 python-info-assignment-continuation-line-p-1 (0.000397 sec)
passed 137/360 python-info-assignment-continuation-line-p-2 (0.000438 sec)
passed 138/360 python-info-assignment-statement-p-1 (0.000405 sec)
passed 139/360 python-info-assignment-statement-p-2 (0.000421 sec)
passed 140/360 python-info-assignment-statement-p-3 (0.000556 sec)
passed 141/360 python-info-beginning-of-backslash-1 (0.000660 sec)
passed 142/360 python-info-beginning-of-block-p-1 (0.000477 sec)
passed 143/360 python-info-beginning-of-block-p-2 (0.000649 sec)
passed 144/360 python-info-beginning-of-statement-p-1 (0.000376 sec)
passed 145/360 python-info-beginning-of-statement-p-2 (0.000333 sec)
passed 146/360 python-info-block-continuation-line-p-1 (0.000364 sec)
passed 147/360 python-info-block-continuation-line-p-2 (0.000758 sec)
passed 148/360 python-info-continuation-line-p-1 (0.000763 sec)
passed 149/360 python-info-current-defun-1 (0.001010 sec)
passed 150/360 python-info-current-defun-2 (0.012160 sec)
passed 151/360 python-info-current-defun-3 (0.009767 sec)
passed 152/360 python-info-current-defun-4 (0.000514 sec)
passed 153/360 python-info-current-line-comment-p-1 (0.000302 sec)
passed 154/360 python-info-current-line-empty-p (0.000253 sec)
passed 155/360 python-info-current-symbol-1 (0.000486 sec)
passed 156/360 python-info-current-symbol-2 (0.000581 sec)
failed 157/360 python-info-current-symbol-3 (0.000336 sec)
passed 158/360 python-info-dedenter-opening-block-message-1 (0.000273 sec)
Closes try:
Closes try:
passed 159/360 python-info-dedenter-opening-block-message-2 (0.000333 sec)
Closes except:
Closes except:
passed 160/360 python-info-dedenter-opening-block-message-3 (0.000334 sec)
Closes else:
Closes else:
passed 161/360 python-info-dedenter-opening-block-message-4 (0.000800 sec)
Closes if a:
Closes if a:
passed 162/360 python-info-dedenter-opening-block-message-5 (0.000618 sec)
passed 163/360 python-info-dedenter-opening-block-position-1 (0.001092 sec)
passed 164/360 python-info-dedenter-opening-block-position-2 (0.000352 sec)
passed 165/360 python-info-dedenter-opening-block-position-3 (0.001099 sec)
passed 166/360 python-info-dedenter-opening-block-positions-1 (0.001124 sec)
passed 167/360 python-info-dedenter-opening-block-positions-2 (0.000468 sec)
passed 168/360 python-info-dedenter-opening-block-positions-3 (0.000941 sec)
passed 169/360 python-info-dedenter-opening-block-positions-4 (0.000519 sec)
passed 170/360 python-info-dedenter-opening-block-positions-5 (0.000737 sec)
passed 171/360 python-info-dedenter-opening-block-positions-6 (0.000408 sec)
passed 172/360 python-info-dedenter-opening-block-positions-7 (0.000446 sec)
passed 173/360 python-info-dedenter-statement-p-1 (0.000661 sec)
passed 174/360 python-info-dedenter-statement-p-2 (0.000527 sec)
passed 175/360 python-info-dedenter-statement-p-3 (0.000499 sec)
passed 176/360 python-info-dedenter-statement-p-4 (0.000499 sec)
passed 177/360 python-info-dedenter-statement-p-5 (0.000326 sec)
passed 178/360 python-info-dedenter-statement-p-6 (0.000385 sec)
passed 179/360 python-info-docstring-p-1 (0.001139 sec)
passed 180/360 python-info-docstring-p-2 (0.001632 sec)
passed 181/360 python-info-docstring-p-3 (0.001860 sec)
passed 182/360 python-info-docstring-p-4 (0.001932 sec)
passed 183/360 python-info-docstring-p-5 (0.002134 sec)
passed 184/360 python-info-docstring-p-6 (0.001738 sec)
passed 185/360 python-info-docstring-p-7 (0.000527 sec)
passed 186/360 python-info-docstring-p-8 (0.000430 sec)
passed 187/360 python-info-encoding-1 (0.000300 sec)
passed 188/360 python-info-encoding-2 (0.000258 sec)
passed 189/360 python-info-encoding-from-cookie-1 (0.000565 sec)
passed 190/360 python-info-encoding-from-cookie-2 (0.000425 sec)
passed 191/360 python-info-encoding-from-cookie-3 (0.000398 sec)
passed 192/360 python-info-encoding-from-cookie-4 (0.000409 sec)
passed 193/360 python-info-encoding-from-cookie-5 (0.000387 sec)
passed 194/360 python-info-encoding-from-cookie-6 (0.000267 sec)
passed 195/360 python-info-encoding-from-cookie-7 (0.000261 sec)
passed 196/360 python-info-end-of-block-p-1 (0.000599 sec)
passed 197/360 python-info-end-of-block-p-2 (0.000709 sec)
passed 198/360 python-info-end-of-statement-p-1 (0.000414 sec)
passed 199/360 python-info-end-of-statement-p-2 (0.000647 sec)
passed 200/360 python-info-line-ends-backslash-p-1 (0.000586 sec)
passed 201/360 python-info-looking-at-beginning-of-block-1 (0.000623 sec)
passed 202/360 python-info-looking-at-beginning-of-defun-1 (0.000662 sec)
passed 203/360 python-info-looking-at-beginning-of-defun-2 (0.000400 sec)
passed 204/360 python-info-looking-at-beginning-of-defun-3 (0.000700 sec)
passed 205/360 python-info-statement-ends-block-p-1 (0.039753 sec)
passed 206/360 python-info-statement-ends-block-p-2 (0.000398 sec)
passed 207/360 python-info-statement-starts-block-p-1 (0.000300 sec)
passed 208/360 python-info-statement-starts-block-p-2 (0.000300 sec)
passed 209/360 python-info-triple-quoted-string-p-1 (0.000406 sec)
passed 210/360 python-info-triple-quoted-string-p-2 (0.000312 sec)
passed 211/360 python-info-triple-quoted-string-p-3 (0.000388 sec)
Mark set
Mark set
passed 212/360 python-mark-defun-1 (0.000850 sec)
Mark set
Mark set
passed 213/360 python-mark-defun-2 (0.000827 sec)
Mark set
Mark set
passed 214/360 python-mark-defun-3 (0.000485 sec)
Mark set
Mark set
passed 215/360 python-mark-defun-4 (0.000526 sec)
Mark set
Mark set
Mark set
Mark set
passed 216/360 python-mark-defun-5 (0.000672 sec)
passed 217/360 python-nav-backward-defun-1 (0.000751 sec)
passed 218/360 python-nav-backward-defun-2 (0.000590 sec)
passed 219/360 python-nav-backward-defun-3 (0.000499 sec)
passed 220/360 python-nav-backward-defun-4 (0.000459 sec)
passed 221/360 python-nav-backward-statement-1 (0.000446 sec)
failed 222/360 python-nav-backward-statement-2 (0.000393 sec)
failed 223/360 python-nav-backward-up-list-1 (0.000436 sec)
passed 224/360 python-nav-beginning-of-block-1 (0.000835 sec)
passed 225/360 python-nav-beginning-of-block-2 (0.000391 sec)
passed 226/360 python-nav-beginning-of-defun-1 (0.001373 sec)
passed 227/360 python-nav-beginning-of-defun-2 (0.001348 sec)
passed 228/360 python-nav-beginning-of-defun-3 (0.000396 sec)
passed 229/360 python-nav-beginning-of-defun-4 (0.000531 sec)
passed 230/360 python-nav-beginning-of-defun-5 (0.000403 sec)
passed 231/360 python-nav-beginning-of-defun-6 (0.000319 sec)
passed 232/360 python-nav-beginning-of-statement-1 (0.000428 sec)
passed 233/360 python-nav-end-of-block-1 (0.002600 sec)
passed 234/360 python-nav-end-of-block-2 (0.000368 sec)
passed 235/360 python-nav-end-of-defun-1 (0.001172 sec)
passed 236/360 python-nav-end-of-defun-2 (0.003453 sec)
passed 237/360 python-nav-end-of-defun-3 (0.000532 sec)
passed 238/360 python-nav-end-of-statement-1 (0.000500 sec)
passed 239/360 python-nav-end-of-statement-2 (0.000266 sec)
passed 240/360 python-nav-end-of-statement-3 (0.000272 sec)
passed 241/360 python-nav-end-of-statement-4 (0.000298 sec)
passed 242/360 python-nav-forward-block-1 (0.000777 sec)
passed 243/360 python-nav-forward-block-2 (0.000327 sec)
passed 244/360 python-nav-forward-defun-1 (0.000395 sec)
passed 245/360 python-nav-forward-defun-2 (0.000358 sec)
passed 246/360 python-nav-forward-defun-3 (0.000304 sec)
passed 247/360 python-nav-forward-defun-4 (0.000280 sec)
passed 248/360 python-nav-forward-sexp-1 (0.000837 sec)
passed 249/360 python-nav-forward-sexp-2 (0.001896 sec)
passed 250/360 python-nav-forward-sexp-3 (0.001493 sec)
passed 251/360 python-nav-forward-sexp-safe-1 (0.001036 sec)
passed 252/360 python-nav-forward-statement-1 (0.000857 sec)
passed 253/360 python-nav-up-list-1 (0.000392 sec)
passed 254/360 python-parens-electric-indent-1 (0.001728 sec)
passed 255/360 python-shell-buffer-substring-1 (0.000598 sec)
passed 256/360 python-shell-buffer-substring-10 (0.000495 sec)
passed 257/360 python-shell-buffer-substring-11 (0.000924 sec)
passed 258/360 python-shell-buffer-substring-12 (0.000810 sec)
passed 259/360 python-shell-buffer-substring-13 (0.000809 sec)
passed 260/360 python-shell-buffer-substring-14 (0.000738 sec)
passed 261/360 python-shell-buffer-substring-15 (0.000437 sec)
passed 262/360 python-shell-buffer-substring-16 (0.000421 sec)
passed 263/360 python-shell-buffer-substring-17 (0.000428 sec)
passed 264/360 python-shell-buffer-substring-18 (0.000412 sec)
passed 265/360 python-shell-buffer-substring-2 (0.001147 sec)
passed 266/360 python-shell-buffer-substring-3 (0.000990 sec)
passed 267/360 python-shell-buffer-substring-4 (0.000597 sec)
passed 268/360 python-shell-buffer-substring-5 (0.000584 sec)
passed 269/360 python-shell-buffer-substring-6 (0.000504 sec)
passed 270/360 python-shell-buffer-substring-7 (0.000501 sec)
passed 271/360 python-shell-buffer-substring-8 (0.000493 sec)
passed 272/360 python-shell-buffer-substring-9 (0.000786 sec)
passed 273/360 python-shell-calculate-exec-path-1 (0.000152 sec)
passed 274/360 python-shell-calculate-exec-path-2 (0.000104 sec)
passed 275/360 python-shell-calculate-exec-path-3 (0.000099 sec)
passed 276/360 python-shell-calculate-exec-path-4 (0.002369 sec)
passed 277/360 python-shell-calculate-exec-path-5 (0.000091 sec)
passed 278/360 python-shell-calculate-exec-path-6 (0.000151 sec)
passed 279/360 python-shell-calculate-process-environment-1 (0.000071 sec)
passed 280/360 python-shell-calculate-process-environment-2 (0.000241 sec)
passed 281/360 python-shell-calculate-process-environment-3 (0.000165 sec)
passed 282/360 python-shell-calculate-process-environment-4 (0.000124 sec)
passed 283/360 python-shell-calculate-process-environment-5 (0.000116 sec)
passed 284/360 python-shell-calculate-process-environment-6 (0.000116 sec)
passed 285/360 python-shell-calculate-process-environment-7 (0.000129 sec)
passed 286/360 python-shell-calculate-process-environment-8 (0.000124 sec)
passed 287/360 python-shell-calculate-pythonpath-1 (0.000114 sec)
passed 288/360 python-shell-calculate-pythonpath-2 (0.000117 sec)
Test python-shell-completion-at-point-1 backtrace:
signal(json-parse-error ("invalid token near '_'" "<string>" 1 1 1))
apply(signal (json-parse-error ("invalid token near '_'" "<string>"
(setq value-3589 (apply fn-3587 args-3588))
(unwind-protect (setq value-3589 (apply fn-3587 args-3588)) (setq fo
(if (unwind-protect (setq value-3589 (apply fn-3587 args-3588)) (set
(let (form-description-3591) (if (unwind-protect (setq value-3589 (a
(let ((value-3589 'ert-form-evaluation-aborted-3590)) (let (form-des
(let* ((fn-3587 #'nth) (args-3588 (condition-case err (list 2 (pytho
(save-current-buffer (set-buffer (process-buffer shell-process)) (in
(let ((shell-process (python-shell-get-process-or-error))) (save-cur
(progn (run-python nil t) (insert "") (goto-char (point-min)) (pytho
(unwind-protect (progn (run-python nil t) (insert "") (goto-char (po
(let ((python-indent-guess-indent-offset nil) (python-shell-completi
(progn (let ((python-indent-guess-indent-offset nil) (python-shell-c
(unwind-protect (progn (let ((python-indent-guess-indent-offset nil)
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current
(closure (t) nil (let* ((fn-3582 #'executable-find) (args-3583 (cond
#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf>)()
handler-bind-1(#f(compiled-function () #<bytecode 0x1c50d36b54ca6edf
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name python-shell-completion-at-point-1 :d
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../emacs/test" "-l" "ert" "-l" "lisp/progm
command-line()
normal-top-level()
Test python-shell-completion-at-point-1 condition:
(json-parse-error "invalid token near '_'" "<string>" 1 1 1)
FAILED 289/360 python-shell-completion-at-point-1 (0.389104 sec) at ../../emacs/test/lisp/progmodes/python-tests.el:4777
skipped 290/360 python-shell-completion-at-point-ipython (0.000458 sec)
Warning (python): Your `python-shell-interpreter' doesn't seem to support readline, yet `python-shell-completion-native-enable' was t and "python3" is not part of the `python-shell-completion-native-disabled-interpreters' list. Native completions have been disabled locally. Consider installing the python package "readline".
skipped 291/360 python-shell-completion-at-point-jedi-completer (1.283608 sec)
Warning (python): Your `python-shell-interpreter' doesn't seem to support readline, yet `python-shell-completion-native-enable' was t and "python3" is not part of the `python-shell-completion-native-disabled-interpreters' list. Native completions have been disabled locally. Consider installing the python package "readline".
passed 292/360 python-shell-completion-at-point-native-1 (1.418923 sec)
passed 293/360 python-shell-completion-native-interpreter-disabled-p-1 (0.000124 sec)
Can't guess python-indent-offset, using defaults: 4
passed 294/360 python-shell-get-process-1 (0.215481 sec)
passed 295/360 python-shell-get-process-name-1 (0.000597 sec)
Can't guess python-indent-offset, using defaults: 4
passed 296/360 python-shell-get-process-name-2 (0.011209 sec)
Can't guess python-indent-offset, using defaults: 4
passed 297/360 python-shell-internal-get-or-create-process-1 (0.063736 sec)
passed 298/360 python-shell-internal-get-process-name-1 (0.000444 sec)
Can't guess python-indent-offset, using defaults: 4
passed 299/360 python-shell-internal-get-process-name-2 (0.011409 sec)
passed 300/360 python-shell-make-comint-1 (0.052506 sec)
passed 301/360 python-shell-make-comint-2 (0.053577 sec)
passed 302/360 python-shell-make-comint-3 (0.050405 sec)
passed 303/360 python-shell-make-comint-4 (0.050541 sec)
passed 304/360 python-shell-prompt-detect-1 (0.077438 sec)
passed 305/360 python-shell-prompt-detect-2 (0.075888 sec)
passed 306/360 python-shell-prompt-detect-3 (0.000279 sec)
Warning (python): Python shell prompts cannot be detected.
If your emacs session hangs when starting python shells
recover with `keyboard-quit' and then try fixing the
interactive flag for your interpreter by adjusting the
`python-shell-interpreter-interactive-arg' or add regexps
matching shell prompts in the directory-local friendly vars:
+ `python-shell-prompt-regexp'
+ `python-shell-prompt-block-regexp'
+ `python-shell-prompt-output-regexp'
Or alternatively in:
+ `python-shell-prompt-input-regexps'
+ `python-shell-prompt-output-regexps'
passed 307/360 python-shell-prompt-detect-4 (0.082467 sec)
passed 308/360 python-shell-prompt-detect-5 (0.081498 sec)
passed 309/360 python-shell-prompt-detect-6 (0.001290 sec)
passed 310/360 python-shell-prompt-set-calculated-regexps-1 (0.000174 sec)
passed 311/360 python-shell-prompt-set-calculated-regexps-2 (0.000122 sec)
passed 312/360 python-shell-prompt-set-calculated-regexps-3 (0.000201 sec)
passed 313/360 python-shell-prompt-set-calculated-regexps-4 (0.000169 sec)
passed 314/360 python-shell-prompt-set-calculated-regexps-5 (0.000183 sec)
passed 315/360 python-shell-prompt-set-calculated-regexps-6 (0.116206 sec)
passed 316/360 python-shell-prompt-validate-regexps-1 (0.000154 sec)
passed 317/360 python-shell-prompt-validate-regexps-2 (0.000154 sec)
passed 318/360 python-shell-prompt-validate-regexps-3 (0.000152 sec)
passed 319/360 python-shell-prompt-validate-regexps-4 (0.000115 sec)
passed 320/360 python-shell-prompt-validate-regexps-5 (0.000103 sec)
passed 321/360 python-shell-prompt-validate-regexps-6 (0.000104 sec)
passed 322/360 python-shell-prompt-validate-regexps-7 (0.000063 sec)
passed 323/360 python-shell-with-environment-1 (0.000364 sec)
passed 324/360 python-shell-with-environment-2 (0.000558 sec)
passed 325/360 python-shell-with-environment-3 (0.000565 sec)
passed 326/360 python-syntax-after-python-backspace (0.000334 sec)
passed 327/360 python-syntax-context-1 (0.000354 sec)
passed 328/360 python-tests--fill-long-first-line (0.000998 sec)
passed 329/360 python-tests--flymake-command-output-pattern (0.000234 sec)
passed 330/360 python-tests--run-python-selects-window (0.096141 sec)
passed 331/360 python-tests-look-at-1 (0.000453 sec)
passed 332/360 python-tests-look-at-2 (0.000298 sec)
passed 333/360 python-triple-double-quote-pairing (0.004338 sec)
passed 334/360 python-triple-single-quote-pairing (0.001394 sec)
passed 335/360 python-ts-mode-assignement-face-2 (0.051551 sec)
passed 336/360 python-ts-mode-builtin-call-face (0.018900 sec)
passed 337/360 python-ts-mode-class-patterns-face (0.002075 sec)
passed 338/360 python-ts-mode-compound-keywords-face (0.004958 sec)
passed 339/360 python-ts-mode-disabled-string-interpolation (0.002239 sec)
passed 340/360 python-ts-mode-dotted-decorator-face-1 (0.002431 sec)
passed 341/360 python-ts-mode-dotted-decorator-face-2 (0.002075 sec)
passed 342/360 python-ts-mode-interpolation-doc-string (0.001603 sec)
passed 343/360 python-ts-mode-interpolation-nested-string (0.001576 sec)
passed 344/360 python-ts-mode-isinstance-type-face-1 (0.001774 sec)
passed 345/360 python-ts-mode-isinstance-type-face-2 (0.003366 sec)
passed 346/360 python-ts-mode-isinstance-type-face-3 (0.002596 sec)
passed 347/360 python-ts-mode-level-fontification-wo-interpolation (0.001768 sec)
passed 348/360 python-ts-mode-named-assignement-face-1 (0.001438 sec)
passed 349/360 python-ts-mode-nested-types-face-1 (0.001659 sec)
passed 350/360 python-ts-mode-superclass-type-face (0.001565 sec)
passed 351/360 python-ts-mode-types-face-1 (0.001577 sec)
passed 352/360 python-ts-mode-types-face-2 (0.003078 sec)
passed 353/360 python-ts-mode-types-face-3 (0.001907 sec)
passed 354/360 python-ts-mode-union-types-face-1 (0.002662 sec)
passed 355/360 python-ts-mode-union-types-face-2 (0.002332 sec)
passed 356/360 python-util-clone-local-variables-1 (0.000391 sec)
passed 357/360 python-util-forward-comment-1 (0.001119 sec)
passed 358/360 python-util-goto-line-1 (0.000271 sec)
passed 359/360 python-util-strip-string-1 (0.000134 sec)
passed 360/360 python-util-valid-regexp-p-1 (0.000088 sec)
Ran 360 tests, 350 results as expected, 7 unexpected, 3 skipped (2024-02-17 14:08:39+0100, 23.058396 sec)
3 expected failures
7 unexpected results:
FAILED python-completion-at-point-1
FAILED python-completion-at-point-2
FAILED python-completion-at-point-native-1
FAILED python-completion-at-point-native-2
FAILED python-completion-at-point-native-with-eldoc-1
FAILED python-completion-at-point-native-with-ffap-1
FAILED python-shell-completion-at-point-1
3 skipped results:
SKIPPED python-ffap-module-path-1
SKIPPED python-shell-completion-at-point-ipython
SKIPPED python-shell-completion-at-point-jedi-completer
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-16 20:10 ` Mattias Engdegård
@ 2024-02-17 13:33 ` kobarity
2024-02-20 10:16 ` Mattias Engdegård
0 siblings, 1 reply; 68+ messages in thread
From: kobarity @ 2024-02-17 13:33 UTC (permalink / raw)
To: Liu Hui; +Cc: Mattias Engdegård, Eli Zaretskii, 68559
Eli Zaretskii wrote:
> > 0001-Remove-echoed-back-string-in-python-shell-completion.patch
> > extracts only the last line to exclude echoed back strings.
> >
> > 0001-Set-tty-mode-to-raw-when-setting-up-Inferior-Python.patch sets
> > the Inferior Python tty to raw mode. python-ffap-module-path-1 will
> > no longer need to be skipped on Mac. If it is safe to set tty to raw
> > mode on all UNIX based systems, I prefer this method.
>
> Will this work on MS-Windows as well? If you are unsure, would you
> please tell me how to test whether this works on Windows, so I could
> collect the information for you?
It does not affect MS-Windows because it sets only when the tty
library can be imported. Although the tty library exists in
MS-Windows, importing the tty library will result in an error because
there is no underlying termios library.
Mattias Engdegård wrote:
>
> 16 feb. 2024 kl. 16.24 skrev kobarity <kobarity@gmail.com>:
>
> > I made prototype patches for each method. I don't use Mac so it would
> > be helpful if you could try these.
>
> Nice, thank you! I can confirm that they both appear to work, at least in the sense that the python-tests pass (except for the ones skipped intentionally), and the python shell behaves reasonable.
>
> Only the set-tty-mode patch eliminates echo in the interactive python shell; in that sense it's preferable.
Thank you for testing my patches.
> Both produce the very annoying warning
>
> Warning (python): Your ‘python-shell-interpreter’ doesn’t seem to support readline, yet ‘python-shell-completion-native-enable’ was t and "python3" is not part of the ‘python-shell-completion-native-disabled-interpreters’ list. Native completions have been disabled locally. Consider installing the python package "readline".
>
> which is not even correct since the standard Python does have a working readline module, even if it uses libedit.
You are right. Maybe the package name should be fixed to "gnureadline"
as described in etc/PROBLEMS.
On Mac, it might be better to set the default value of
`python-shell-completion-native-enable' to nil.
> > 0001-Set-tty-mode-to-raw-when-setting-up-Inferior-Python.patch sets
> > the Inferior Python tty to raw mode. python-ffap-module-path-1 will
> > no longer need to be skipped on Mac.
>
> Right, that test runs and passes.
>
> > If it is safe to set tty to raw
> > mode on all UNIX based systems, I prefer this method.
>
> Same here. I see no reason why it wouldn't be safe, either.
Thanks.
> > By the way, is it necessary to send
> > `python-shell-completion-setup-code' for every completion in
> > `python-shell-completion-get-completions'? To me it seems sufficient
> > to send it once at initialization.
>
> Indeed, it does seem a bit extravagant.
This would be one of the items for future improvement.
Liu Hui wrote:
>
> kobarity <kobarity@gmail.com> writes:
>
> > 0001-Set-tty-mode-to-raw-when-setting-up-Inferior-Python.patch sets
> > the Inferior Python tty to raw mode. python-ffap-module-path-1 will
> > no longer need to be skipped on Mac. If it is safe to set tty to raw
> > mode on all UNIX based systems, I prefer this method.
>
> How about the following change, which only affects libedit-based
> readline? It may enable native completion on mac, but I cannot test
> it.
>
> diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
> index b7e43f3fc68..f59bc19367b 100644
> --- a/lisp/progmodes/python.el
> +++ b/lisp/progmodes/python.el
> @@ -4286,6 +4286,9 @@ (defcustom python-shell-completion-setup-code
> except:
> pass
> else:
> + if readline.__doc__ and 'libedit' in readline.__doc__:
> + import tty
> + tty.setraw(0)
> # Try to reuse current completer.
> completer = readline.get_completer()
> if not completer:
> @@ -4471,8 +4474,8 @@ (defun python-shell-completion-native-setup ()
> instance.rlcomplete = new_completer
>
> if readline.__doc__ and 'libedit' in readline.__doc__:
> - raise Exception('''libedit based readline is known not to work,
> - see etc/PROBLEMS under \"In Inferior Python mode, input is echoed\".''')
> + import tty
> + tty.setraw(0)
> readline.parse_and_bind('bind ^I rl_complete')
> else:
> readline.parse_and_bind('tab: complete')
Disabling echo back may not be sufficient to enable native completions
on Mac. I have not tried raw mode, but have tried
readline.parse_and_bind('setty -echo') and
readline.parse_and_bind('edit on'). Native completions could be
enabled, but it was unstable.
I have no objection if Mac users check the above patch and if it is
OK.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-15 16:48 ` Eli Zaretskii
2024-02-15 17:21 ` Mattias Engdegård
@ 2024-02-19 13:18 ` Basil L. Contovounesios
2024-02-20 4:46 ` Liu Hui
1 sibling, 1 reply; 68+ messages in thread
From: Basil L. Contovounesios @ 2024-02-19 13:18 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: liuhui1610, mattias.engdegard, kobarity, 68559
[-- Attachment #1: Type: text/plain, Size: 1075 bytes --]
Eli Zaretskii [2024-02-15 18:48 +0200] wrote:
>> Cc: liuhui1610@gmail.com, kobarity@gmail.com, 68559@debbugs.gnu.org
>> Date: Thu, 15 Feb 2024 18:37:13 +0200
>> From: Eli Zaretskii <eliz@gnu.org>
>>
>> > It seems to have something to do with completion. Maybe the new code is
>> sensitive to details of Python's command line editor? The standard Python
>> interpreter is unlikely to use GNU readline, for example.
>>
>> The python-*-completion-at-point-* tests never worked for me, and I
>> always assumed it was something specific to MS-Windows. But maybe
>> not. There was no change in the tests that fail for me before and
>> after the recent changes related to Python.
>
> And, btw, on this Gnu/Linux system:
>
> $ uname -a
> Linux (REDACTED) 5.15.0-94-generic #104+11.0trisquel25 SMP Sat Feb 10 06:24:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
>
> all the Python tests that failed for you still pass, with the current
> master. So I wonder what is it that causes those failures on your
> system.
BTW, I have been seeing an IPython test failure on GNU/Linux:
[-- Attachment #2: ipython.txt.gz --]
[-- Type: application/gzip, Size: 6592 bytes --]
[-- Attachment #3: Type: text/plain, Size: 3322 bytes --]
Any pointers?
Thanks,
--
Basil
$ ipython --version
8.21.0
$ python --version
Python 3.12.2
In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
version 1.18.0, Xaw3d scroll bars) of 2024-02-19 built on tia
Repository revision: 8f260bb93f534b24d9a93d3315804ffe0c1fec4f
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101011
System Description: Debian GNU/Linux trixie/sid
Configured using:
'configure 'CFLAGS=-Og -ggdb3' -C --prefix=/home/blc/.local
--enable-checking=structs --without-native-compilation
--with-file-notification=yes --with-x-toolkit=lucid --with-x'
Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XAW3D XDBE XIM XINPUT2 XPM
LUCID ZLIB
Important settings:
value of $LANG: en_IE.UTF-8
value of $XMODIFIERS: @im=ibus
locale-coding-system: utf-8-unix
Major mode: Lisp Interaction
Minor modes in effect:
tooltip-mode: t
global-eldoc-mode: t
eldoc-mode: t
show-paren-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
minibuffer-regexp-mode: t
line-number-mode: t
indent-tabs-mode: t
transient-mark-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
Load-path shadows:
None found.
Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo x-toolkit
xinput2 x multi-tty move-toolbar make-network-process emacs)
Memory information:
((conses 16 39117 8464) (symbols 48 5217 0) (strings 32 13299 1612)
(string-bytes 1 311978) (vectors 16 9245)
(vector-slots 8 110970 8413) (floats 8 23 25) (intervals 56 266 0)
(buffers 984 10))
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-19 13:18 ` Basil L. Contovounesios
@ 2024-02-20 4:46 ` Liu Hui
2024-02-20 13:15 ` Basil L. Contovounesios
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-20 4:46 UTC (permalink / raw)
To: Basil L. Contovounesios; +Cc: Eli Zaretskii, kobarity, mattias.engdegard, 68559
On Mon, Feb 19, 2024 at 9:18 PM Basil L. Contovounesios
<basil@contovou.net> wrote:
>
> BTW, I have been seeing an IPython test failure on GNU/Linux:
>
>
> Any pointers?
>
> Thanks,
Hi,
I cannot reproduce the problem with Python 3.12 and IPython 8.21. Can
you check the results with the following steps? Thanks.
1. start Python shell with IPython interpreter, i.e.
(setq python-shell-interpreter "ipython")
(setq python-shell-interpreter-args "-i --simple-prompt")
M-x run-python
2. M-x python-shell-completion-native-turn-off
3. eval code in the Python shell:
import re
from IPython.core.completer import provisionalcompleter
with provisionalcompleter():
print(list(get_ipython().Completer.completions('re.split("a",
"abc", maxs', 25)))
expected output: [<Completion start=21 end=25 text='maxsplit='
type='param', signature='?',>]
__PYTHON_EL_get_completions('re.split("a", "abc", maxs')
expected output: '[["maxsplit=", 21, 25, "param", ""]]'
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-17 13:33 ` kobarity
@ 2024-02-20 10:16 ` Mattias Engdegård
2024-02-21 13:13 ` kobarity
0 siblings, 1 reply; 68+ messages in thread
From: Mattias Engdegård @ 2024-02-20 10:16 UTC (permalink / raw)
To: kobarity; +Cc: Liu Hui, Eli Zaretskii, 68559
17 feb. 2024 kl. 14.33 skrev kobarity <kobarity@gmail.com>:
> On Mac, it might be better to set the default value of
> `python-shell-completion-native-enable' to nil.
Not sure why the inferior Python process doesn't act on a TAB being sent to it. Is the tty somehow in a state that disables readline/libedit? Something that Emacs does when setting up the pty?
Of course from a software engineering point of view, it's silly to send what essentially are edit keystrokes to Python and then screen-scrape the output. A proper interaction protocol would be the way to go, and would work equally well on any platform including Windows.
> Disabling echo back may not be sufficient to enable native completions
> on Mac. I have not tried raw mode, but have tried
> readline.parse_and_bind('setty -echo') and
> readline.parse_and_bind('edit on'). Native completions could be
> enabled, but it was unstable.
>
> I have no objection if Mac users check the above patch and if it is
> OK.
Afraid it wasn't.
Thanks for your patches. I suggest we apply your set-tty-raw patch on master now since it cures the test failures without breaking anything else (on Mac; I'm assuming no regression elsewhere).
Would you like me to do that for you?
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-20 4:46 ` Liu Hui
@ 2024-02-20 13:15 ` Basil L. Contovounesios
2024-02-21 10:00 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: Basil L. Contovounesios @ 2024-02-20 13:15 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, kobarity, mattias.engdegard, 68559
[-- Attachment #1: Type: text/plain, Size: 817 bytes --]
Liu Hui [2024-02-20 12:46 +0800] wrote:
> On Mon, Feb 19, 2024 at 9:18 PM Basil L. Contovounesios
> <basil@contovou.net> wrote:
>>
>> BTW, I have been seeing an IPython test failure on GNU/Linux:
>> Any pointers?
>
> I cannot reproduce the problem with Python 3.12 and IPython 8.21. Can
> you check the results with the following steps? Thanks.
Thanks. These all work as expected in 'emacs -Q', which made me realise
what may be the problem:
- the Emacs test suite runs under HOME=/nonexistent
- but PATH is unchanged
- I have the latest IPython installed locally using pipx:
$ ls -l $(which ipython)
lrwxrwxrwx 1 blc blc 53 Feb 20 11:46
/home/blc/.local/bin/ipython ->
/home/blc/.local/share/pipx/venvs/ipython/bin/ipython
This is confirmed by the following experiment:
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: prynt.diff --]
[-- Type: text/x-diff, Size: 569 bytes --]
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index af6c199b5bd..3393f93542b 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4883,6 +4883,7 @@ python-shell-completion-at-point-ipython
(python-tests-with-temp-buffer-with-shell
""
(python-shell-with-shell-buffer
+ (make-temp-file "my-py-out-" nil ".txt" (buffer-string))
(python-shell-completion-native-turn-off)
(python-tests--completion-module)
(python-tests--completion-parameters)
[-- Attachment #3: Type: text/plain, Size: 93 bytes --]
Running 'make TEST_LOAD_EL=no test/python-tests' now results in the
following output file:
[-- Attachment #4: my-py-out-kFh7He.txt --]
[-- Type: text/plain, Size: 442 bytes --]
/home/blc/.local/share/pipx/venvs/ipython/lib/python3.12/site-packages/IPython/paths.py:69: UserWarning: IPython parent '/nonexistent' is not a writable location, using a temp directory.
warn("IPython parent '{0}' is not a writable location,"
Python 3.12.2 (main, Feb 18 2024, 16:59:56) [GCC 13.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.21.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]:
[-- Attachment #5: Type: text/plain, Size: 282 bytes --]
In this case list(get_ipython().Completer.completions(...)) is empty.
I'm not sure what the best way to detect or work around this is.
Any ideas?
[ It's a problem shared, to an extent, with Eglot tests, since LSP servers are
often installed in one's HOME. ]
Thanks,
--
Basil
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-20 13:15 ` Basil L. Contovounesios
@ 2024-02-21 10:00 ` Liu Hui
2024-02-21 14:55 ` Basil L. Contovounesios
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-21 10:00 UTC (permalink / raw)
To: Basil L. Contovounesios; +Cc: Eli Zaretskii, kobarity, mattias.engdegard, 68559
[-- Attachment #1: Type: text/plain, Size: 1782 bytes --]
On Tue, Feb 20, 2024 at 9:15 PM Basil L. Contovounesios
<basil@contovou.net> wrote:
>
> Liu Hui [2024-02-20 12:46 +0800] wrote:
>
> > On Mon, Feb 19, 2024 at 9:18 PM Basil L. Contovounesios
> > <basil@contovou.net> wrote:
> >>
> >> BTW, I have been seeing an IPython test failure on GNU/Linux:
> >> Any pointers?
> >
> > I cannot reproduce the problem with Python 3.12 and IPython 8.21. Can
> > you check the results with the following steps? Thanks.
>
> Thanks. These all work as expected in 'emacs -Q', which made me realise
> what may be the problem:
>
> - the Emacs test suite runs under HOME=/nonexistent
> - but PATH is unchanged
> - I have the latest IPython installed locally using pipx:
> $ ls -l $(which ipython)
> lrwxrwxrwx 1 blc blc 53 Feb 20 11:46
> /home/blc/.local/bin/ipython ->
> /home/blc/.local/share/pipx/venvs/ipython/bin/ipython
>
> This is confirmed by the following experiment:
>
>
> Running 'make TEST_LOAD_EL=no test/python-tests' now results in the
> following output file:
>
>
> In this case list(get_ipython().Completer.completions(...)) is empty.
Thank you for the investigation! I didn't realize the problem is
related to HOME=/nonexistent in 'make test'. Now I can also reproduce
it. In fact, I found that the test failure was caused by Jedi, as it
attempts to write cache to a non-existent directory.
When Jedi is directly used as the completion backend, more test
failures will be triggered:
PYTHONSTARTUP="$(python -m jedi repl)" make TEST_LOAD_EL=no test/python-tests
If the cache directory is writable, all tests will pass:
PYTHONSTARTUP="$(python -m jedi repl)" XDG_CACHE_HOME=~/.cache make
TEST_LOAD_EL=no test/python-tests
The attached patch should fix the problem.
[-- Attachment #2: 0001-Fix-Python-shell-completion-test-failures.patch --]
[-- Type: text/x-patch, Size: 1760 bytes --]
From a6773258cf6fe1acc5520f56ef948ba6975cbe32 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Wed, 21 Feb 2024 12:40:06 +0800
Subject: [PATCH] Fix Python shell completion test failures
* test/lisp/progmodes/python-tests.el
(python-tests-with-temp-buffer-with-shell): Set XDG_CACHE_HOME
to a temporary directory. (bug#68559)
---
test/lisp/progmodes/python-tests.el | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index af6c199b5bd..6454cfcc2b7 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -60,12 +60,17 @@ python-tests-with-temp-buffer-with-shell
(python-shell-completion-native-enable nil))
(python-mode)
(unwind-protect
- (progn
- (run-python nil t)
- (insert ,contents)
- (goto-char (point-min))
- (python-tests-shell-wait-for-prompt)
- ,@body)
+ ;; Prevent test failures when Jedi is used as a completion
+ ;; backend, either directly or indirectly (e.g., via
+ ;; IPython). Jedi needs to store cache, but the
+ ;; "/nonexistent" HOME directory is not writable.
+ (ert-with-temp-directory cache-dir
+ (with-environment-variables (("XDG_CACHE_HOME" cache-dir))
+ (run-python nil t)
+ (insert ,contents)
+ (goto-char (point-min))
+ (python-tests-shell-wait-for-prompt)
+ ,@body))
(when (python-shell-get-buffer)
(python-shell-with-shell-buffer
(let (kill-buffer-hook kill-buffer-query-functions)
--
2.25.1
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-20 10:16 ` Mattias Engdegård
@ 2024-02-21 13:13 ` kobarity
2024-02-21 18:20 ` Mattias Engdegård
0 siblings, 1 reply; 68+ messages in thread
From: kobarity @ 2024-02-21 13:13 UTC (permalink / raw)
To: Mattias Engdegård; +Cc: Liu Hui, Eli Zaretskii, 68559
Mattias Engdegård wrote:
> 17 feb. 2024 kl. 14.33 skrev kobarity <kobarity@gmail.com>:
> > On Mac, it might be better to set the default value of
> > `python-shell-completion-native-enable' to nil.
> Not sure why the inferior Python process doesn't act on a TAB being sent to it. Is the tty somehow in a state that disables readline/libedit? Something that Emacs does when setting up the pty?
On Mac, there are two independent problems. One problem, which
existed before Liu's patch was applied, is that native completions
cannot be enabled. The other problem is that the test now fails after
applying Liu's patch. Suppressing echo back solves the latter, but
not the former. Because of the former, leaving
`python-shell-completion-native-enable' at the default setting of t
will result in the warning.
There are two completions methods, native and non-native. If native
completions cannot be used, it falls back to non-native completions.
Native completions is triggered by sending TAB character, while
non-native completions is triggered by sending and evaluating
__PYTHON_EL_get_completions().
As far as I have tried, native completions cannot be enabled for
libedit-based readline package, even on Linux.
So, there are two ways for Mac users to avoid the warning. One is to
use gnureadline instead of libedit, and the other is to give up native
completions and set `python-shell-completion-native-enable' to nil.
This remains correct even with echo back disabled.
> Of course from a software engineering point of view, it's silly to send what essentially are edit keystrokes to Python and then screen-scrape the output. A proper interaction protocol would be the way to go, and would work equally well on any platform including Windows.
I think the protocol between python.el and inferior Python process is
already platform independent. Protocol violations are echo back.
> Thanks for your patches. I suggest we apply your set-tty-raw patch on master now since it cures the test failures without breaking anything else (on Mac; I'm assuming no regression elsewhere).
>
> Would you like me to do that for you?
Yes, please. Thank you. I have no commit right.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-21 10:00 ` Liu Hui
@ 2024-02-21 14:55 ` Basil L. Contovounesios
2024-02-22 10:31 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: Basil L. Contovounesios @ 2024-02-21 14:55 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, kobarity, mattias.engdegard, 68559
Liu Hui [2024-02-21 18:00 +0800] wrote:
> The attached patch should fix the problem.
Thanks! The patch fixes the error, but that's because
python-shell-completion-at-point-ipython is now skipped:
in particular, python-shell-readline-completer-delims evaluates to
"\s\t\n`~!@#$%^&*()-=+[{]}\\|;:'\",<>/?" rather than the empty string.
Any idea why that happens?
> - (progn
> - (run-python nil t)
> - (insert ,contents)
> - (goto-char (point-min))
> - (python-tests-shell-wait-for-prompt)
> - ,@body)
> + ;; Prevent test failures when Jedi is used as a completion
> + ;; backend, either directly or indirectly (e.g., via
> + ;; IPython). Jedi needs to store cache, but the
> + ;; "/nonexistent" HOME directory is not writable.
> + (ert-with-temp-directory cache-dir
^^^^^^^^^
Should this be an uninterned symbol instead?
> + (with-environment-variables (("XDG_CACHE_HOME" cache-dir))
> + (run-python nil t)
> + (insert ,contents)
> + (goto-char (point-min))
> + (python-tests-shell-wait-for-prompt)
> + ,@body))
> (when (python-shell-get-buffer)
> (python-shell-with-shell-buffer
> (let (kill-buffer-hook kill-buffer-query-functions)
Thanks,
--
Basil
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-21 13:13 ` kobarity
@ 2024-02-21 18:20 ` Mattias Engdegård
2024-02-22 16:15 ` kobarity
0 siblings, 1 reply; 68+ messages in thread
From: Mattias Engdegård @ 2024-02-21 18:20 UTC (permalink / raw)
To: kobarity; +Cc: Liu Hui, Eli Zaretskii, 68559
[-- Attachment #1: Type: text/plain, Size: 958 bytes --]
21 feb. 2024 kl. 14.13 skrev kobarity <kobarity@gmail.com>:
> As far as I have tried, native completions cannot be enabled for
> libedit-based readline package, even on Linux.
Did you find out why? Python responds nicely to TAB if run from a terminal, so what is it that we do in Emacs that prevents it from doing so? A TTY setting, or an environment variable (the TERM=dumb)?
> So, there are two ways for Mac users to avoid the warning. One is to
> use gnureadline instead of libedit, and the other is to give up native
> completions and set `python-shell-completion-native-enable' to nil.
Preferably neither. Emacs should adapt to the system environment, and in particular not warn about the default environment. A warning is an indication of a possible risk or other problem, and there is none here.
At most, python-mode could show a calm notice on the echo line but even that is a stretch. What do you think about this rough patch?
[-- Attachment #2: nowarn.diff --]
[-- Type: application/octet-stream, Size: 1402 bytes --]
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 5501926e69d..bedc61408ef 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -4536,18 +4536,11 @@ python-shell-completion-native-turn-on-maybe
((python-shell-completion-native-setup)
(when msg
(message "Shell native completion is enabled.")))
- (t (lwarn
- '(python python-shell-completion-native-turn-on-maybe)
- :warning
- (concat
- "Your `python-shell-interpreter' doesn't seem to "
- "support readline, yet `python-shell-completion-native-enable' "
- (format "was t and %S is not part of the "
- (file-name-nondirectory python-shell-interpreter))
- "`python-shell-completion-native-disabled-interpreters' "
- "list. Native completions have been disabled locally. "
- "Consider installing the python package \"readline\". "))
- (python-shell-completion-native-turn-off msg))))))
+ (t
+ (when msg
+ (message (concat "Python does not use GNU readline;"
+ " no completion in multi-line commands.")))
+ (python-shell-completion-native-turn-off nil))))))
(defun python-shell-completion-native-turn-on-maybe-with-msg ()
"Like `python-shell-completion-native-turn-on-maybe' but force messages."
[-- Attachment #3: Type: text/plain, Size: 1007 bytes --]
> I think the protocol between python.el and inferior Python process is
> already platform independent. Protocol violations are echo back.
No, I meant a protocol that allows Emacs to act as a first-class Python front-end, not simulate a terminal, send keystrokes, use heuristics to determine what in the output stream is a prompt, REPL value, error, completion etc.
For example, it's a bit silly to input multi-line code in the REPL as a sequence of individual single-line commands, when we actually are inside a text editor that can edit multi-line Python code without a problem.
(I'm not suggesting that you or anybody in particular should do this; just that it's feasible. It would clearly be quite some work!)
>> Thanks for your patches. I suggest we apply your set-tty-raw patch on master now since it cures the test failures without breaking anything else (on Mac; I'm assuming no regression elsewhere).
>>
>> Would you like me to do that for you?
>
> Yes, please.
Done!
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-21 14:55 ` Basil L. Contovounesios
@ 2024-02-22 10:31 ` Liu Hui
2024-02-22 13:56 ` Basil L. Contovounesios
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-22 10:31 UTC (permalink / raw)
To: Basil L. Contovounesios; +Cc: Eli Zaretskii, kobarity, mattias.engdegard, 68559
[-- Attachment #1: Type: text/plain, Size: 1675 bytes --]
On Wed, Feb 21, 2024 at 10:55 PM Basil L. Contovounesios
<basil@contovou.net> wrote:
>
> Liu Hui [2024-02-21 18:00 +0800] wrote:
>
> > The attached patch should fix the problem.
>
> Thanks! The patch fixes the error, but that's because
> python-shell-completion-at-point-ipython is now skipped:
Only the native completion part is skipped for the reason below.
> in particular, python-shell-readline-completer-delims evaluates to
> "\s\t\n`~!@#$%^&*()-=+[{]}\\|;:'\",<>/?" rather than the empty string.
"\s\t\n`..." is the delimiter used by rlcompleter, which is the
default completer used by the readline. rlcompleter cannot complete
module names or parameters, so in this case the native completion part
is skipped.
The test is intended to be used with Jedi as the completion backend,
e.g. setting PYTHONSTARTUP="$(python -m jedi repl)", or with a custom
IPython completer defined in the PYTHONSTARTUP file. I have updated the
patch to make the test use Jedi when possible.
> > - (progn
> > - (run-python nil t)
> > - (insert ,contents)
> > - (goto-char (point-min))
> > - (python-tests-shell-wait-for-prompt)
> > - ,@body)
> > + ;; Prevent test failures when Jedi is used as a completion
> > + ;; backend, either directly or indirectly (e.g., via
> > + ;; IPython). Jedi needs to store cache, but the
> > + ;; "/nonexistent" HOME directory is not writable.
> > + (ert-with-temp-directory cache-dir
> ^^^^^^^^^
> Should this be an uninterned symbol instead?
Fixed.
[-- Attachment #2: 0001-Fix-Python-shell-completion-test-failures.patch --]
[-- Type: text/x-patch, Size: 5782 bytes --]
From 8cd85404f5627ffe4b41c19f1d466425eafe31b7 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Wed, 21 Feb 2024 12:40:06 +0800
Subject: [PATCH] Fix Python shell completion test failures
* test/lisp/progmodes/python-tests.el
(python-tests-with-temp-buffer-with-shell): Set XDG_CACHE_HOME
to a temporary directory.
(python-tests--pythonstartup-file): New function.
(python-shell-completion-at-point-jedi-completer)
(python-shell-completion-at-point-ipython): Use Jedi as the
native completion backend when possible. (bug#68559)
---
test/lisp/progmodes/python-tests.el | 87 ++++++++++++++++++-----------
1 file changed, 53 insertions(+), 34 deletions(-)
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index af6c199b5bd..c5b1ab93144 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -55,21 +55,27 @@ python-tests-with-temp-buffer-with-shell
always located at the beginning of buffer. Native completion is
turned off. Shell buffer will be killed on exit."
(declare (indent 1) (debug t))
- `(with-temp-buffer
- (let ((python-indent-guess-indent-offset nil)
- (python-shell-completion-native-enable nil))
- (python-mode)
- (unwind-protect
- (progn
- (run-python nil t)
- (insert ,contents)
- (goto-char (point-min))
- (python-tests-shell-wait-for-prompt)
- ,@body)
- (when (python-shell-get-buffer)
- (python-shell-with-shell-buffer
- (let (kill-buffer-hook kill-buffer-query-functions)
- (kill-buffer))))))))
+ (let ((dir (make-symbol "dir")))
+ `(with-temp-buffer
+ (let ((python-indent-guess-indent-offset nil)
+ (python-shell-completion-native-enable nil))
+ (python-mode)
+ (unwind-protect
+ ;; Prevent test failures when Jedi is used as a completion
+ ;; backend, either directly or indirectly (e.g., via
+ ;; IPython). Jedi needs to store cache, but the
+ ;; "/nonexistent" HOME directory is not writable.
+ (ert-with-temp-directory ,dir
+ (with-environment-variables (("XDG_CACHE_HOME" ,dir))
+ (run-python nil t)
+ (insert ,contents)
+ (goto-char (point-min))
+ (python-tests-shell-wait-for-prompt)
+ ,@body))
+ (when (python-shell-get-buffer)
+ (python-shell-with-shell-buffer
+ (let (kill-buffer-hook kill-buffer-query-functions)
+ (kill-buffer)))))))))
(defmacro python-tests-with-temp-file (contents &rest body)
"Create a `python-mode' enabled file with CONTENTS.
@@ -4860,17 +4866,28 @@ python-tests--completion-extra-context
(should (string= "IGNORECASE"
(buffer-substring (line-beginning-position) (point)))))
+(defun python-tests--pythonstartup-file ()
+ "Return Jedi readline setup file if PYTHONSTARTUP is not set."
+ (or (getenv "PYTHONSTARTUP")
+ (with-temp-buffer
+ (if (eql 0 (call-process python-tests-shell-interpreter
+ nil t nil "-m" "jedi" "repl"))
+ (string-trim (buffer-string))
+ ""))))
+
(ert-deftest python-shell-completion-at-point-jedi-completer ()
"Check if Python shell completion works when Jedi completer is used."
(skip-unless (executable-find python-tests-shell-interpreter))
- (python-tests-with-temp-buffer-with-shell
- ""
- (python-shell-with-shell-buffer
- (python-shell-completion-native-turn-on)
- (skip-unless (string= python-shell-readline-completer-delims ""))
- (python-tests--completion-module)
- (python-tests--completion-parameters)
- (python-tests--completion-extra-context))))
+ (with-environment-variables
+ (("PYTHONSTARTUP" (python-tests--pythonstartup-file)))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context)))))
(ert-deftest python-shell-completion-at-point-ipython ()
"Check if Python shell completion works for IPython."
@@ -4880,17 +4897,19 @@ python-shell-completion-at-point-ipython
(and
(executable-find python-shell-interpreter)
(eql (call-process python-shell-interpreter nil nil nil "--version") 0)))
- (python-tests-with-temp-buffer-with-shell
- ""
- (python-shell-with-shell-buffer
- (python-shell-completion-native-turn-off)
- (python-tests--completion-module)
- (python-tests--completion-parameters)
- (python-shell-completion-native-turn-on)
- (skip-unless (string= python-shell-readline-completer-delims ""))
- (python-tests--completion-module)
- (python-tests--completion-parameters)
- (python-tests--completion-extra-context)))))
+ (with-environment-variables
+ (("PYTHONSTARTUP" (python-tests--pythonstartup-file)))
+ (python-tests-with-temp-buffer-with-shell
+ ""
+ (python-shell-with-shell-buffer
+ (python-shell-completion-native-turn-off)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-shell-completion-native-turn-on)
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context))))))
\f
;;; PDB Track integration
--
2.25.1
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-22 10:31 ` Liu Hui
@ 2024-02-22 13:56 ` Basil L. Contovounesios
2024-02-23 13:07 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: Basil L. Contovounesios @ 2024-02-22 13:56 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, kobarity, mattias.engdegard, 68559
Liu Hui [2024-02-22 18:31 +0800] wrote:
> On Wed, Feb 21, 2024 at 10:55 PM Basil L. Contovounesios
> <basil@contovou.net> wrote:
>>
>> Liu Hui [2024-02-21 18:00 +0800] wrote:
>>
>> > The attached patch should fix the problem.
>>
>> Thanks! The patch fixes the error, but that's because
>> python-shell-completion-at-point-ipython is now skipped:
>
> Only the native completion part is skipped for the reason below.
>
>> in particular, python-shell-readline-completer-delims evaluates to
>> "\s\t\n`~!@#$%^&*()-=+[{]}\\|;:'\",<>/?" rather than the empty string.
>
> "\s\t\n`..." is the delimiter used by rlcompleter, which is the
> default completer used by the readline. rlcompleter cannot complete
> module names or parameters, so in this case the native completion part
> is skipped.
>
> The test is intended to be used with Jedi as the completion backend,
> e.g. setting PYTHONSTARTUP="$(python -m jedi repl)", or with a custom
> IPython completer defined in the PYTHONSTARTUP file. I have updated the
> patch to make the test use Jedi when possible.
Thanks, looks fine to me and runs without issue.
The python-shell-completion-at-point-ipython test is still skipped, with
or without specifying PYTHONSTARTUP="$(python -m jedi repl)", but like
you suggest that's not necessarily a problem.
I'm guessing you don't have write access to emacs.git, but have signed
the CA? If so, and if there are no other comments/objections in a few
days, I'll apply the patch in your name.
Thanks,
--
Basil
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-21 18:20 ` Mattias Engdegård
@ 2024-02-22 16:15 ` kobarity
2024-02-23 11:00 ` Mattias Engdegård
0 siblings, 1 reply; 68+ messages in thread
From: kobarity @ 2024-02-22 16:15 UTC (permalink / raw)
To: Mattias Engdegård; +Cc: Liu Hui, Eli Zaretskii, 68559
Mattias Engdegård wrote:
> 21 feb. 2024 kl. 14.13 skrev kobarity <kobarity@gmail.com>:
> > As far as I have tried, native completions cannot be enabled for
> > libedit-based readline package, even on Linux.
> Did you find out why? Python responds nicely to TAB if run from a terminal, so what is it that we do in Emacs that prevents it from doing so? A TTY setting, or an environment variable (the TERM=dumb)?
No, not exactly, but I don't think it is related to terminal settings.
python.el does not parse the completion candidates shown on the
terminal. Instead, it expects the candidates in a particular format,
which I call the protocol between python.el and inferior Python
process.
__PYTHON_EL_native_completion_setup() defined in
`python-shell-completion-native-setup' sets up the completer to do so.
We can test it by removing empty lines and pasting into Python REPL
outside Emacs. An example of typing "ra" and TAB using readline would
look like this.
>>> raraise
range
0__dummy_completion__ 1__dummy_completion__
>>> ra
However, when I use libedit-based readline and remove the raise line
in __PYTHON_EL_native_completion_setup(), it behaves like this.
>>> raraise
range
Please note that there is no dummy completions, and the new prompt is
not shown. When I hit TAB again, it would be like this.
>>> raraise
range
raise
range
0__dummy_completion__ 0__dummy_completion__ 0__dummy_completion__ 1__dummy_completion__
>>> ra
I think this difference in behavior is the reason why Native
completions does not work with libedit.
> > So, there are two ways for Mac users to avoid the warning. One is to
> > use gnureadline instead of libedit, and the other is to give up native
> > completions and set `python-shell-completion-native-enable' to nil.
>
> Preferably neither. Emacs should adapt to the system environment, and in particular not warn about the default environment. A warning is an indication of a possible risk or other problem, and there is none here.
>
> At most, python-mode could show a calm notice on the echo line but even that is a stretch. What do you think about this rough patch?
Personally, I think this patch would be fine.
I think we can also improve Non-native completions. Current
implementation sends the definition of __PYTHON_EL_get_completions()
every time. However, sending it once during initialization should be
sufficient. Worse, if the number of characters sent exceeds
`comint-max-line-length', it will be sent via file. This is happening
in your environment. Here is the log you presented.
Test python-completion-at-point-1 backtrace:
json-parse-string("__PYTHON_EL_eval_file(\"/var/folders/qy/zstv16390
Thanks to the echo back, we can see __PYTHON_EL_eval_file() was used.
> > I think the protocol between python.el and inferior Python process is
> > already platform independent. Protocol violations are echo back.
>
> No, I meant a protocol that allows Emacs to act as a first-class Python front-end, not simulate a terminal, send keystrokes, use heuristics to determine what in the output stream is a prompt, REPL value, error, completion etc.
>
> For example, it's a bit silly to input multi-line code in the REPL as a sequence of individual single-line commands, when we actually are inside a text editor that can edit multi-line Python code without a problem.
>
> (I'm not suggesting that you or anybody in particular should do this; just that it's feasible. It would clearly be quite some work!)
I am not opposed to this approach, but as you wrote, it is very
different from the current inferior Python implementation.
Jupyter-emacs's approach may be similar to some extent. It uses zmq
as the communication channel.
https://github.com/emacs-jupyter/jupyter
> >> Thanks for your patches. I suggest we apply your set-tty-raw patch on master now since it cures the test failures without breaking anything else (on Mac; I'm assuming no regression elsewhere).
> >> Would you like me to do that for you?
> > Yes, please.
> Done!
Thank you!
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-22 16:15 ` kobarity
@ 2024-02-23 11:00 ` Mattias Engdegård
2024-02-23 14:39 ` kobarity
0 siblings, 1 reply; 68+ messages in thread
From: Mattias Engdegård @ 2024-02-23 11:00 UTC (permalink / raw)
To: kobarity; +Cc: Liu Hui, Eli Zaretskii, 68559
22 feb. 2024 kl. 17.15 skrev kobarity <kobarity@gmail.com>:
>> Did you find out why? Python responds nicely to TAB if run from a terminal, so what is it that we do in Emacs that prevents it from doing so? A TTY setting, or an environment variable (the TERM=dumb)?
>
> No, not exactly, but I don't think it is related to terminal settings.
Well it must be something. I'll see if I can make sense of it.
> I think this difference in behavior is the reason why Native
> completions does not work with libedit.
Thanks for explaining.
> Personally, I think this patch would be fine.
Thank you, now pushed to master.
> I think we can also improve Non-native completions. Current
> implementation sends the definition of __PYTHON_EL_get_completions()
> every time. However, sending it once during initialization should be
> sufficient. Worse, if the number of characters sent exceeds
> `comint-max-line-length', it will be sent via file. This is happening
> in your environment. Here is the log you presented.
>
> Test python-completion-at-point-1 backtrace:
> json-parse-string("__PYTHON_EL_eval_file(\"/var/folders/qy/zstv16390
>
> Thanks to the echo back, we can see __PYTHON_EL_eval_file() was used.
Right. (The need for a file is just an artefact of Comint limitations, isn't it?)
> I am not opposed to this approach, but as you wrote, it is very
> different from the current inferior Python implementation.
> Jupyter-emacs's approach may be similar to some extent. It uses zmq
> as the communication channel.
Thank you, yes, in a sense it's a (small) step towards a more notebook-like interaction model, but Comint already has some of that. This problem is not unique to Python: other REPLs have to solve the problem of multi-line input as well, and it would serve the user if they all worked in the same way (as far as the language differences allow).
In Python's case, it would liberate us from the constraints of the standard terminal REPL.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-22 13:56 ` Basil L. Contovounesios
@ 2024-02-23 13:07 ` Liu Hui
2024-02-28 14:47 ` Basil L. Contovounesios
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-02-23 13:07 UTC (permalink / raw)
To: Basil L. Contovounesios; +Cc: Eli Zaretskii, kobarity, mattias.engdegard, 68559
On Thu, Feb 22, 2024 at 9:56 PM Basil L. Contovounesios
<basil@contovou.net> wrote:
>
> Liu Hui [2024-02-22 18:31 +0800] wrote:
>
> > On Wed, Feb 21, 2024 at 10:55 PM Basil L. Contovounesios
> > <basil@contovou.net> wrote:
> >>
> >> Liu Hui [2024-02-21 18:00 +0800] wrote:
> >>
> >> > The attached patch should fix the problem.
> >>
> >> Thanks! The patch fixes the error, but that's because
> >> python-shell-completion-at-point-ipython is now skipped:
> >
> > Only the native completion part is skipped for the reason below.
> >
> >> in particular, python-shell-readline-completer-delims evaluates to
> >> "\s\t\n`~!@#$%^&*()-=+[{]}\\|;:'\",<>/?" rather than the empty string.
> >
> > "\s\t\n`..." is the delimiter used by rlcompleter, which is the
> > default completer used by the readline. rlcompleter cannot complete
> > module names or parameters, so in this case the native completion part
> > is skipped.
> >
> > The test is intended to be used with Jedi as the completion backend,
> > e.g. setting PYTHONSTARTUP="$(python -m jedi repl)", or with a custom
> > IPython completer defined in the PYTHONSTARTUP file. I have updated the
> > patch to make the test use Jedi when possible.
>
> Thanks, looks fine to me and runs without issue.
>
> The python-shell-completion-at-point-ipython test is still skipped, with
> or without specifying PYTHONSTARTUP="$(python -m jedi repl)", but like
> you suggest that's not necessarily a problem.
I have no idea why the test is skipped. You may still check if native
completion works for IPython with following steps:
1. PYTHONSTARTUP="$(python -m jedi repl)" emacs -Q
2. start Python shell with IPython interpreter, i.e.
(setq python-shell-interpreter "ipython")
(setq python-shell-interpreter-args "-i --simple-prompt")
M-x run-python
There should be text "REPL completion using Jedi xxx" before the
first prompt in the Python shell buffer, and a message "Shell
native completion is enabled.".
3. type "import ab"/"open(enc" and press TAB
expected result: "import abc"/"open(encoding="
> I'm guessing you don't have write access to emacs.git, but have signed
> the CA? If so, and if there are no other comments/objections in a few
> days, I'll apply the patch in your name.
Yes, I've signed the CA. Thanks.
> Is your GitHub username ilupin by any chance?
>
> I ask because there is one commit in emacs.git from someone with the
> same name as you, but with the email address
> ilupin@users.noreply.github.com (I'm guessing the commit was imported
> from eglot.git).
>
> If that was from you, do you have any objection to me mapping
> ilupin@users.noreply.github.com to liuhui1610@gmail.com in the .mailmap
> file in emacs.git? Then all commits will show up under a single name
> and email address.
Yes. I have no objection, thanks.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-23 11:00 ` Mattias Engdegård
@ 2024-02-23 14:39 ` kobarity
2024-02-26 11:06 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: kobarity @ 2024-02-23 14:39 UTC (permalink / raw)
To: Liu Hui, Mattias Engdegård; +Cc: Eli Zaretskii, 68559
Hi Liu,
I noticed that with Non-native completions, Jedi completion is not
enabled even if PYTHONSTARTUP is set and "REPL completion using Jedi
0.18.2" is shown in inferior Python. Is this expected behavior? Jedi
completion is enabled regardless of PYTHONSTARTUP setting when I use
IPython with Non-native completions.
Mattias Engdegård wrote:
> > I think we can also improve Non-native completions. Current
> > implementation sends the definition of __PYTHON_EL_get_completions()
> > every time. However, sending it once during initialization should be
> > sufficient. Worse, if the number of characters sent exceeds
> > `comint-max-line-length', it will be sent via file. This is happening
> > in your environment. Here is the log you presented.
> >
> > Test python-completion-at-point-1 backtrace:
> > json-parse-string("__PYTHON_EL_eval_file(\"/var/folders/qy/zstv16390
> >
> > Thanks to the echo back, we can see __PYTHON_EL_eval_file() was used.
>
> Right. (The need for a file is just an artefact of Comint limitations, isn't it?)
Yes, I think it is the limitation of comint. However, if the
definition is omitted, it is unlikely to reach that limit, since all
that is required is to send the string like
'__PYTHON_EL_get_completions("subproc")'.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-23 14:39 ` kobarity
@ 2024-02-26 11:06 ` Liu Hui
2024-02-26 12:16 ` Mattias Engdegård
2024-02-28 14:49 ` Basil L. Contovounesios
0 siblings, 2 replies; 68+ messages in thread
From: Liu Hui @ 2024-02-26 11:06 UTC (permalink / raw)
To: kobarity; +Cc: Mattias Engdegård, Eli Zaretskii, 68559
[-- Attachment #1: Type: text/plain, Size: 1731 bytes --]
On Fri, Feb 23, 2024 at 10:39 PM kobarity <kobarity@gmail.com> wrote:
>
> Hi Liu,
>
> I noticed that with Non-native completions, Jedi completion is not
> enabled even if PYTHONSTARTUP is set and "REPL completion using Jedi
> 0.18.2" is shown in inferior Python. Is this expected behavior? Jedi
> completion is enabled regardless of PYTHONSTARTUP setting when I use
> IPython with Non-native completions.
Thanks for pointing out the problem! The attached patch should enable
Jedi completion for the non-native case.
On Fri, Feb 16, 2024 at 3:41 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > The Python shell completion relies on the readline module, which is
> > not available for Python on MS-Windows. According to the instruction
> > in python.el:
> >
> > ;; readline based shell (it's known to work with PyPy). If your
> > ;; Python installation lacks readline (like CPython for Windows),
> > ;; installing pyreadline (URL `https://ipython.org/pyreadline.html')
> > ;; should suffice. To troubleshoot why you are not getting any
> > ;; completions, you can try the following in your Python shell:
> >
> > ;; >>> import readline, rlcompleter
> >
> > ;; If you see an error, then you need to either install pyreadline or
> > ;; setup custom code that avoids that dependency.
>
> I don't know if I have CPython, but the above does show an error
> message.
>
> > It may be necessary to install pyreadline (for Python 2.7) or
> > pyreadline3 (for Python 3).
>
> I will see if I can do that, thanks.
>
> Regardless, patches to the test suite to skip the tests which rely on
> those modules, if they aren't installed, will be welcome.
Such tests should be skipped now with this patch.
[-- Attachment #2: 0001-Detect-the-readline-support-for-Python-shell-complet.patch --]
[-- Type: text/x-patch, Size: 8213 bytes --]
From e02e55192e68b6272dea2a23096f90cdee517e74 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Mon, 26 Feb 2024 18:46:36 +0800
Subject: [PATCH] Detect the readline support for Python shell completion
* lisp/progmodes/python.el
(python-shell-readline-completer-delims): Update docstring.
(python-shell-completion-native-setup): Move the detection of
readline to ...
(python-shell-readline-detect): ... new function.
(python-shell-completion-native-turn-on-maybe): Skip if Python
has no readline support.
(python-shell-completion-at-point): Respect the delimiter of
readline completer in non-native completion.
* test/lisp/progmodes/python-tests.el
(python-shell-completion-at-point-1)
(python-shell-completion-at-point-native-1)
(python-completion-at-point-1, python-completion-at-point-2)
(python-completion-at-point-pdb-1)
(python-completion-at-point-while-running-1)
(python-completion-at-point-native-1)
(python-completion-at-point-native-2)
(python-completion-at-point-native-with-ffap-1)
(python-completion-at-point-native-with-eldoc-1): Skip tests if
Python has no readline support.
(python-shell-completion-at-point-jedi-completer): Add test for
non-native Python shell completion. (bug#68559)
---
lisp/progmodes/python.el | 29 ++++++++++++++++++++++-------
test/lisp/progmodes/python-tests.el | 23 ++++++++++++++++++++++-
2 files changed, 44 insertions(+), 8 deletions(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 587d0b36304..6c73ceba4ef 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -4439,7 +4439,24 @@ python-shell-completion-native-try-output-timeout
(defvar python-shell-readline-completer-delims nil
"Word delimiters used by the readline completer.
-It is automatically set by Python shell.")
+It is automatically set by Python shell. A value of nil means the
+Python shell has no readline support.")
+
+(defun python-shell-readline-detect ()
+ "Detect the readline support for Python shell completion."
+ (let* ((process (python-shell-get-process))
+ (output (python-shell-send-string-no-output "
+try:
+ import readline
+ print(readline.get_completer_delims())
+except:
+ print('No readline support')" process)))
+ (unless (string-match-p "No readline support" output)
+ (setq-local python-shell-readline-completer-delims
+ (string-trim-right output)))))
+
+(add-hook 'python-shell-first-prompt-hook
+ 'python-shell-readline-detect -90)
(defvar python-shell-completion-native-redirect-buffer
" *Python completions redirect*"
@@ -4579,10 +4596,6 @@ python-shell-completion-native-setup
__PYTHON_EL_native_completion_setup()" process)))
(when (string-match-p "python\\.el: native completion setup loaded"
output)
- (setq-local python-shell-readline-completer-delims
- (string-trim-right
- (python-shell-send-string-no-output
- "import readline; print(readline.get_completer_delims())")))
(python-shell-completion-native-try))))
(defun python-shell-completion-native-turn-off (&optional msg)
@@ -4611,7 +4624,8 @@ python-shell-completion-native-turn-on-maybe
(cond
((python-shell-completion-native-interpreter-disabled-p)
(python-shell-completion-native-turn-off msg))
- ((python-shell-completion-native-setup)
+ ((and python-shell-readline-completer-delims
+ (python-shell-completion-native-setup))
(when msg
(message "Shell native completion is enabled.")))
(t
@@ -4783,7 +4797,8 @@ python-shell-completion-at-point
(with-current-buffer (process-buffer process)
(if python-shell-completion-native-enable
(string= python-shell-readline-completer-delims "")
- (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
+ (or (string-match-p "ipython[23]?\\'" python-shell-interpreter)
+ (string= python-shell-readline-completer-delims ""))))))
(start
(if (< (point) line-start)
(point)
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 6c6cd9eee2b..e0584f78f82 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4777,6 +4777,7 @@ python-shell-completion-at-point-1
(python-tests-with-temp-buffer-with-shell
""
(python-shell-with-shell-buffer
+ (skip-when (null python-shell-readline-completer-delims))
(insert "import abc")
(comint-send-input)
(python-tests-shell-wait-for-prompt)
@@ -4791,6 +4792,7 @@ python-shell-completion-at-point-native-1
""
(python-shell-completion-native-turn-on)
(python-shell-with-shell-buffer
+ (skip-when (null python-shell-readline-completer-delims))
(insert "import abc")
(comint-send-input)
(python-tests-shell-wait-for-prompt)
@@ -4866,8 +4868,11 @@ python-shell-completion-at-point-jedi-completer
(python-tests-with-temp-buffer-with-shell
""
(python-shell-with-shell-buffer
- (python-shell-completion-native-turn-on)
(skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-shell-completion-native-turn-off)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-shell-completion-native-turn-on)
(python-tests--completion-module)
(python-tests--completion-parameters)
(python-tests--completion-extra-context))))
@@ -4905,6 +4910,8 @@ python-completion-at-point-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-when (null python-shell-readline-completer-delims)))
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
(goto-char (point-max))
@@ -4921,6 +4928,8 @@ python-completion-at-point-2
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-when (null python-shell-readline-completer-delims)))
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
(python-shell-with-shell-buffer
@@ -4940,6 +4949,8 @@ python-completion-at-point-pdb-1
print('Hello')
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-when (null python-shell-readline-completer-delims)))
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
(goto-char (point-max))
@@ -4956,6 +4967,8 @@ python-completion-at-point-while-running-1
time.sleep(3)
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-when (null python-shell-readline-completer-delims)))
(python-shell-send-buffer)
(goto-char (point-max))
(insert "time.")
@@ -4968,6 +4981,8 @@ python-completion-at-point-native-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-when (null python-shell-readline-completer-delims)))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
@@ -4985,6 +5000,8 @@ python-completion-at-point-native-2
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-when (null python-shell-readline-completer-delims)))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
@@ -5001,6 +5018,8 @@ python-completion-at-point-native-with-ffap-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-when (null python-shell-readline-completer-delims)))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
@@ -5017,6 +5036,8 @@ python-completion-at-point-native-with-eldoc-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-when (null python-shell-readline-completer-delims)))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
--
2.25.1
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-26 11:06 ` Liu Hui
@ 2024-02-26 12:16 ` Mattias Engdegård
2024-02-26 15:08 ` kobarity
2024-02-28 14:49 ` Basil L. Contovounesios
1 sibling, 1 reply; 68+ messages in thread
From: Mattias Engdegård @ 2024-02-26 12:16 UTC (permalink / raw)
To: Liu Hui; +Cc: kobarity, Eli Zaretskii, 68559
26 feb. 2024 kl. 12.06 skrev Liu Hui <liuhui1610@gmail.com>:
> Thanks for pointing out the problem! The attached patch should enable
> Jedi completion for the non-native case.
This patch does not break anything further on macOS using the standard system Python.
I just thought you would be happy to know.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-26 12:16 ` Mattias Engdegård
@ 2024-02-26 15:08 ` kobarity
0 siblings, 0 replies; 68+ messages in thread
From: kobarity @ 2024-02-26 15:08 UTC (permalink / raw)
To: Liu Hui, Mattias Engdegård; +Cc: Eli Zaretskii, 68559
Mattias Engdegård wrote:
>
> 26 feb. 2024 kl. 12.06 skrev Liu Hui <liuhui1610@gmail.com>:
>
> > Thanks for pointing out the problem! The attached patch should enable
> > Jedi completion for the non-native case.
>
> This patch does not break anything further on macOS using the standard system Python.
> I just thought you would be happy to know.
Thanks, I confirmed that the patch enables Jedi completion for the
non-native mode.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-23 13:07 ` Liu Hui
@ 2024-02-28 14:47 ` Basil L. Contovounesios
0 siblings, 0 replies; 68+ messages in thread
From: Basil L. Contovounesios @ 2024-02-28 14:47 UTC (permalink / raw)
To: Liu Hui; +Cc: Eli Zaretskii, kobarity, mattias.engdegard, 68559
Liu Hui [2024-02-23 21:07 +0800] wrote:
> On Thu, Feb 22, 2024 at 9:56 PM Basil L. Contovounesios
> <basil@contovou.net> wrote:
>>
>> The python-shell-completion-at-point-ipython test is still skipped, with
>> or without specifying PYTHONSTARTUP="$(python -m jedi repl)", but like
>> you suggest that's not necessarily a problem.
>
> I have no idea why the test is skipped. You may still check if native
> completion works for IPython with following steps:
>
> 1. PYTHONSTARTUP="$(python -m jedi repl)" emacs -Q
Oops:
$ PYTHONSTARTUP="$(python -m jedi repl)" ./src/emacs -Q
/home/blc/.pyenv/versions/3.12.2/bin/python: No module named jedi
Once I 'pip install jedi' the rest works as expected, and
python-shell-completion-at-point-ipython is no longer skipped.
>> I'm guessing you don't have write access to emacs.git, but have signed
>> the CA? If so, and if there are no other comments/objections in a few
>> days, I'll apply the patch in your name.
>
> Yes, I've signed the CA. Thanks.
Done:
Fix Python shell completion test failures
8a2d013be37 2024-02-28 15:25:56 +0100
https://git.sv.gnu.org/cgit/emacs.git/commit/?id=8a2d013be37
>> If that was from you, do you have any objection to me mapping
>> ilupin@users.noreply.github.com to liuhui1610@gmail.com in the .mailmap
>> file in emacs.git? Then all commits will show up under a single name
>> and email address.
>
> Yes. I have no objection, thanks.
Done:
; * .mailmap: Fix GitHub address (bug#68559#170).
1ddd9c8e29f 2024-02-28 15:30:41 +0100
https://git.sv.gnu.org/cgit/emacs.git/commit/?id=1ddd9c8e29f
Thanks,
--
Basil
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-26 11:06 ` Liu Hui
2024-02-26 12:16 ` Mattias Engdegård
@ 2024-02-28 14:49 ` Basil L. Contovounesios
2024-03-06 10:14 ` Liu Hui
1 sibling, 1 reply; 68+ messages in thread
From: Basil L. Contovounesios @ 2024-02-28 14:49 UTC (permalink / raw)
To: Liu Hui; +Cc: kobarity, Eli Zaretskii, Mattias Engdegård, 68559
Liu Hui [2024-02-26 19:06 +0800] wrote:
> +(add-hook 'python-shell-first-prompt-hook
> + 'python-shell-readline-detect -90)
This depth currently prepends the function to the hook, but in Emacs 24
it instead appends to the hook. Is that a problem?
> + (skip-when (null python-shell-readline-completer-delims))
skip-unless here and below?
Thanks,
--
Basil
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-02-28 14:49 ` Basil L. Contovounesios
@ 2024-03-06 10:14 ` Liu Hui
2024-03-08 15:44 ` Basil L. Contovounesios
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-03-06 10:14 UTC (permalink / raw)
To: Basil L. Contovounesios
Cc: kobarity, Eli Zaretskii, Mattias Engdegård, 68559
[-- Attachment #1: Type: text/plain, Size: 723 bytes --]
On Wed, Feb 28, 2024 at 10:50 PM Basil L. Contovounesios
<basil@contovou.net> wrote:
>
> Liu Hui [2024-02-26 19:06 +0800] wrote:
>
> > +(add-hook 'python-shell-first-prompt-hook
> > + 'python-shell-readline-detect -90)
>
> This depth currently prepends the function to the hook, but in Emacs 24
> it instead appends to the hook. Is that a problem?
Thanks for pointing it out! It is a problem because
python-shell-readline-detect is expected to be executed before
python-shell-completion-native-turn-on-maybe-with-msg, i.e. another
function in the hook. I have updated the patch.
> > + (skip-when (null python-shell-readline-completer-delims))
>
> skip-unless here and below?
Done.
[-- Attachment #2: 0001-Detect-the-readline-support-for-Python-shell-complet.patch --]
[-- Type: text/x-patch, Size: 8894 bytes --]
From 1a24ac775d8577da0990de89016afed5bfef37fa Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Mon, 26 Feb 2024 18:46:36 +0800
Subject: [PATCH] Detect the readline support for Python shell completion
* lisp/progmodes/python.el
(python-shell-comint-watch-for-first-prompt-output-filter):
Detect the readline support.
(python-shell-readline-completer-delims): Update docstring.
(python-shell-completion-native-setup): Move the readline
detection code to ...
(python-shell-readline-detect): ... new function.
(python-shell-completion-native-turn-on-maybe): Skip if Python
has no readline support.
(python-shell-completion-at-point): Respect the delimiter of
readline completer in non-native completion.
* test/lisp/progmodes/python-tests.el
(python-shell-completion-at-point-1)
(python-shell-completion-at-point-native-1)
(python-completion-at-point-1, python-completion-at-point-2)
(python-completion-at-point-pdb-1)
(python-completion-at-point-while-running-1)
(python-completion-at-point-native-1)
(python-completion-at-point-native-2)
(python-completion-at-point-native-with-ffap-1)
(python-completion-at-point-native-with-eldoc-1): Skip tests if
Python has no readline support.
(python-shell-completion-at-point-jedi-completer): Add test for
non-native Python shell completion. (bug#68559)
---
lisp/progmodes/python.el | 27 ++++++++++++++++++-------
test/lisp/progmodes/python-tests.el | 31 ++++++++++++++++++++++++-----
2 files changed, 46 insertions(+), 12 deletions(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 587d0b36304..9ac462be416 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -3601,6 +3601,7 @@ python-shell-comint-watch-for-first-prompt-output-filter
(python-shell-send-string-no-output python-shell-eval-file-setup-code))
(with-current-buffer (current-buffer)
(let ((inhibit-quit nil))
+ (python-shell-readline-detect)
(run-hooks 'python-shell-first-prompt-hook))))))
output)
@@ -4439,7 +4440,21 @@ python-shell-completion-native-try-output-timeout
(defvar python-shell-readline-completer-delims nil
"Word delimiters used by the readline completer.
-It is automatically set by Python shell.")
+It is automatically set by Python shell. A value of nil means the
+Python shell has no readline support.")
+
+(defun python-shell-readline-detect ()
+ "Detect the readline support for Python shell completion."
+ (let* ((process (python-shell-get-process))
+ (output (python-shell-send-string-no-output "
+try:
+ import readline
+ print(readline.get_completer_delims())
+except:
+ print('No readline support')" process)))
+ (setq-local python-shell-readline-completer-delims
+ (unless (string-match-p "No readline support" output)
+ (string-trim-right output)))))
(defvar python-shell-completion-native-redirect-buffer
" *Python completions redirect*"
@@ -4579,10 +4594,6 @@ python-shell-completion-native-setup
__PYTHON_EL_native_completion_setup()" process)))
(when (string-match-p "python\\.el: native completion setup loaded"
output)
- (setq-local python-shell-readline-completer-delims
- (string-trim-right
- (python-shell-send-string-no-output
- "import readline; print(readline.get_completer_delims())")))
(python-shell-completion-native-try))))
(defun python-shell-completion-native-turn-off (&optional msg)
@@ -4611,7 +4622,8 @@ python-shell-completion-native-turn-on-maybe
(cond
((python-shell-completion-native-interpreter-disabled-p)
(python-shell-completion-native-turn-off msg))
- ((python-shell-completion-native-setup)
+ ((and python-shell-readline-completer-delims
+ (python-shell-completion-native-setup))
(when msg
(message "Shell native completion is enabled.")))
(t
@@ -4783,7 +4795,8 @@ python-shell-completion-at-point
(with-current-buffer (process-buffer process)
(if python-shell-completion-native-enable
(string= python-shell-readline-completer-delims "")
- (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
+ (or (string-match-p "ipython[23]?\\'" python-shell-interpreter)
+ (string= python-shell-readline-completer-delims ""))))))
(start
(if (< (point) line-start)
(point)
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 1ceee690cfb..e11440cdb5b 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4783,6 +4783,7 @@ python-shell-completion-at-point-1
(python-tests-with-temp-buffer-with-shell
""
(python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims)
(insert "import abc")
(comint-send-input)
(python-tests-shell-wait-for-prompt)
@@ -4797,6 +4798,7 @@ python-shell-completion-at-point-native-1
""
(python-shell-completion-native-turn-on)
(python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims)
(insert "import abc")
(comint-send-input)
(python-tests-shell-wait-for-prompt)
@@ -4883,11 +4885,14 @@ python-shell-completion-at-point-jedi-completer
(python-tests-with-temp-buffer-with-shell
""
(python-shell-with-shell-buffer
- (python-shell-completion-native-turn-on)
- (skip-unless (string= python-shell-readline-completer-delims ""))
- (python-tests--completion-module)
- (python-tests--completion-parameters)
- (python-tests--completion-extra-context)))))
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-shell-completion-native-turn-off)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-shell-completion-native-turn-on)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context)))))
(ert-deftest python-shell-completion-at-point-ipython ()
"Check if Python shell completion works for IPython."
@@ -4924,6 +4929,8 @@ python-completion-at-point-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
(goto-char (point-max))
@@ -4940,6 +4947,8 @@ python-completion-at-point-2
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
(python-shell-with-shell-buffer
@@ -4959,6 +4968,8 @@ python-completion-at-point-pdb-1
print('Hello')
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
(goto-char (point-max))
@@ -4975,6 +4986,8 @@ python-completion-at-point-while-running-1
time.sleep(3)
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-send-buffer)
(goto-char (point-max))
(insert "time.")
@@ -4987,6 +5000,8 @@ python-completion-at-point-native-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
@@ -5004,6 +5019,8 @@ python-completion-at-point-native-2
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
@@ -5020,6 +5037,8 @@ python-completion-at-point-native-with-ffap-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
@@ -5036,6 +5055,8 @@ python-completion-at-point-native-with-eldoc-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
--
2.25.1
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-03-06 10:14 ` Liu Hui
@ 2024-03-08 15:44 ` Basil L. Contovounesios
2024-03-11 11:35 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: Basil L. Contovounesios @ 2024-03-08 15:44 UTC (permalink / raw)
To: Liu Hui; +Cc: kobarity, Eli Zaretskii, Mattias Engdegård, 68559
Liu Hui [2024-03-06 18:14 +0800] wrote:
> I have updated the patch.
Thanks! The build and tests succeed here, without skipped tests.
> + (unless (string-match-p "No readline support" output)
Nit: why not plain 'string-search' instead of a regexp search?
> - (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
> + (or (string-match-p "ipython[23]?\\'" python-shell-interpreter)
> + (string= python-shell-readline-completer-delims ""))))))
Just curious: what does the empty string signify?
[ If it's not just a dumb question, perhaps the meaning could be added
to the variable's docstring/commentary. ]
--
Basil
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-03-08 15:44 ` Basil L. Contovounesios
@ 2024-03-11 11:35 ` Liu Hui
2024-03-11 16:02 ` Basil L. Contovounesios
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-03-11 11:35 UTC (permalink / raw)
To: Basil L. Contovounesios
Cc: kobarity, Eli Zaretskii, Mattias Engdegård, 68559
[-- Attachment #1: Type: text/plain, Size: 1115 bytes --]
On Fri, Mar 8, 2024 at 11:44 PM Basil L. Contovounesios
<basil@contovou.net> wrote:
>
> Liu Hui [2024-03-06 18:14 +0800] wrote:
>
> > I have updated the patch.
>
> Thanks! The build and tests succeed here, without skipped tests.
>
> > + (unless (string-match-p "No readline support" output)
>
> Nit: why not plain 'string-search' instead of a regexp search?
Done.
> > - (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
> > + (or (string-match-p "ipython[23]?\\'" python-shell-interpreter)
> > + (string= python-shell-readline-completer-delims ""))))))
>
> Just curious: what does the empty string signify?
>
> [ If it's not just a dumb question, perhaps the meaning could be added
> to the variable's docstring/commentary. ]
The empty string means no characters are considered delimiters and the
readline completion could consider the entire line of input without
breaking it into parts based on typical delimiters like spaces or
punctuation. The docstring is updated in the attached patch.
[-- Attachment #2: 0001-Detect-the-readline-support-for-Python-shell-complet.patch --]
[-- Type: text/x-patch, Size: 9024 bytes --]
From bb1f8f92d5cd3c19d404e023ab33315bfa8a535a Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Mon, 26 Feb 2024 18:46:36 +0800
Subject: [PATCH] Detect the readline support for Python shell completion
* lisp/progmodes/python.el
(python-shell-comint-watch-for-first-prompt-output-filter):
Detect the readline support.
(python-shell-readline-completer-delims): Update docstring.
(python-shell-completion-native-setup): Move the readline
detection code to ...
(python-shell-readline-detect): ... new function.
(python-shell-completion-native-turn-on-maybe): Skip if Python
has no readline support.
(python-shell-completion-at-point): Respect the delimiter of
readline completer in non-native completion.
* test/lisp/progmodes/python-tests.el
(python-shell-completion-at-point-1)
(python-shell-completion-at-point-native-1)
(python-completion-at-point-1, python-completion-at-point-2)
(python-completion-at-point-pdb-1)
(python-completion-at-point-while-running-1)
(python-completion-at-point-native-1)
(python-completion-at-point-native-2)
(python-completion-at-point-native-with-ffap-1)
(python-completion-at-point-native-with-eldoc-1): Skip tests if
Python has no readline support.
(python-shell-completion-at-point-jedi-completer): Add test for
non-native Python shell completion. (bug#68559)
---
lisp/progmodes/python.el | 29 ++++++++++++++++++++-------
test/lisp/progmodes/python-tests.el | 31 ++++++++++++++++++++++++-----
2 files changed, 48 insertions(+), 12 deletions(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 587d0b36304..0ea0b918872 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -3601,6 +3601,7 @@ python-shell-comint-watch-for-first-prompt-output-filter
(python-shell-send-string-no-output python-shell-eval-file-setup-code))
(with-current-buffer (current-buffer)
(let ((inhibit-quit nil))
+ (python-shell-readline-detect)
(run-hooks 'python-shell-first-prompt-hook))))))
output)
@@ -4439,7 +4440,23 @@ python-shell-completion-native-try-output-timeout
(defvar python-shell-readline-completer-delims nil
"Word delimiters used by the readline completer.
-It is automatically set by Python shell.")
+It is automatically set by Python shell. An empty string means no
+characters are considered delimiters and the readline completion
+considers the entire line of input. A value of nil means the Python
+shell has no readline support.")
+
+(defun python-shell-readline-detect ()
+ "Detect the readline support for Python shell completion."
+ (let* ((process (python-shell-get-process))
+ (output (python-shell-send-string-no-output "
+try:
+ import readline
+ print(readline.get_completer_delims())
+except:
+ print('No readline support')" process)))
+ (setq-local python-shell-readline-completer-delims
+ (unless (string-search "No readline support" output)
+ (string-trim-right output)))))
(defvar python-shell-completion-native-redirect-buffer
" *Python completions redirect*"
@@ -4579,10 +4596,6 @@ python-shell-completion-native-setup
__PYTHON_EL_native_completion_setup()" process)))
(when (string-match-p "python\\.el: native completion setup loaded"
output)
- (setq-local python-shell-readline-completer-delims
- (string-trim-right
- (python-shell-send-string-no-output
- "import readline; print(readline.get_completer_delims())")))
(python-shell-completion-native-try))))
(defun python-shell-completion-native-turn-off (&optional msg)
@@ -4611,7 +4624,8 @@ python-shell-completion-native-turn-on-maybe
(cond
((python-shell-completion-native-interpreter-disabled-p)
(python-shell-completion-native-turn-off msg))
- ((python-shell-completion-native-setup)
+ ((and python-shell-readline-completer-delims
+ (python-shell-completion-native-setup))
(when msg
(message "Shell native completion is enabled.")))
(t
@@ -4783,7 +4797,8 @@ python-shell-completion-at-point
(with-current-buffer (process-buffer process)
(if python-shell-completion-native-enable
(string= python-shell-readline-completer-delims "")
- (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
+ (or (string-match-p "ipython[23]?\\'" python-shell-interpreter)
+ (string= python-shell-readline-completer-delims ""))))))
(start
(if (< (point) line-start)
(point)
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 1ceee690cfb..e11440cdb5b 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4783,6 +4783,7 @@ python-shell-completion-at-point-1
(python-tests-with-temp-buffer-with-shell
""
(python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims)
(insert "import abc")
(comint-send-input)
(python-tests-shell-wait-for-prompt)
@@ -4797,6 +4798,7 @@ python-shell-completion-at-point-native-1
""
(python-shell-completion-native-turn-on)
(python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims)
(insert "import abc")
(comint-send-input)
(python-tests-shell-wait-for-prompt)
@@ -4883,11 +4885,14 @@ python-shell-completion-at-point-jedi-completer
(python-tests-with-temp-buffer-with-shell
""
(python-shell-with-shell-buffer
- (python-shell-completion-native-turn-on)
- (skip-unless (string= python-shell-readline-completer-delims ""))
- (python-tests--completion-module)
- (python-tests--completion-parameters)
- (python-tests--completion-extra-context)))))
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-shell-completion-native-turn-off)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-shell-completion-native-turn-on)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context)))))
(ert-deftest python-shell-completion-at-point-ipython ()
"Check if Python shell completion works for IPython."
@@ -4924,6 +4929,8 @@ python-completion-at-point-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
(goto-char (point-max))
@@ -4940,6 +4947,8 @@ python-completion-at-point-2
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
(python-shell-with-shell-buffer
@@ -4959,6 +4968,8 @@ python-completion-at-point-pdb-1
print('Hello')
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
(goto-char (point-max))
@@ -4975,6 +4986,8 @@ python-completion-at-point-while-running-1
time.sleep(3)
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-send-buffer)
(goto-char (point-max))
(insert "time.")
@@ -4987,6 +5000,8 @@ python-completion-at-point-native-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
@@ -5004,6 +5019,8 @@ python-completion-at-point-native-2
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
@@ -5020,6 +5037,8 @@ python-completion-at-point-native-with-ffap-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
@@ -5036,6 +5055,8 @@ python-completion-at-point-native-with-eldoc-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
--
2.25.1
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-03-11 11:35 ` Liu Hui
@ 2024-03-11 16:02 ` Basil L. Contovounesios
2024-03-13 10:21 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: Basil L. Contovounesios @ 2024-03-11 16:02 UTC (permalink / raw)
To: Liu Hui; +Cc: kobarity, Eli Zaretskii, Mattias Engdegård, 68559
Liu Hui [2024-03-11 19:35 +0800] wrote:
> On Fri, Mar 8, 2024 at 11:44 PM Basil L. Contovounesios
> <basil@contovou.net> wrote:
>>
>> Liu Hui [2024-03-06 18:14 +0800] wrote:
>>
>> > - (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
>> > + (or (string-match-p "ipython[23]?\\'" python-shell-interpreter)
>> > + (string= python-shell-readline-completer-delims ""))))))
>>
>> Just curious: what does the empty string signify?
>>
>> [ If it's not just a dumb question, perhaps the meaning could be added
>> to the variable's docstring/commentary. ]
>
> The empty string means no characters are considered delimiters and the
> readline completion could consider the entire line of input without
> breaking it into parts based on typical delimiters like spaces or
> punctuation. The docstring is updated in the attached patch.
Thanks! But that makes me wonder: in the cases where we check
(string= python-shell-readline-completer-delims "")
is there a possibility that the variable's value will be nil?
(In which case we should compare with equal instead of string=.)
--
Basil
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-03-11 16:02 ` Basil L. Contovounesios
@ 2024-03-13 10:21 ` Liu Hui
2024-03-14 14:24 ` Basil L. Contovounesios
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-03-13 10:21 UTC (permalink / raw)
To: Basil L. Contovounesios
Cc: kobarity, Eli Zaretskii, Mattias Engdegård, 68559
[-- Attachment #1: Type: text/plain, Size: 1440 bytes --]
On Tue, Mar 12, 2024 at 12:02 AM Basil L. Contovounesios
<basil@contovou.net> wrote:
>
> Liu Hui [2024-03-11 19:35 +0800] wrote:
>
> > On Fri, Mar 8, 2024 at 11:44 PM Basil L. Contovounesios
> > <basil@contovou.net> wrote:
> >>
> >> Liu Hui [2024-03-06 18:14 +0800] wrote:
> >>
> >> > - (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
> >> > + (or (string-match-p "ipython[23]?\\'" python-shell-interpreter)
> >> > + (string= python-shell-readline-completer-delims ""))))))
> >>
> >> Just curious: what does the empty string signify?
> >>
> >> [ If it's not just a dumb question, perhaps the meaning could be added
> >> to the variable's docstring/commentary. ]
> >
> > The empty string means no characters are considered delimiters and the
> > readline completion could consider the entire line of input without
> > breaking it into parts based on typical delimiters like spaces or
> > punctuation. The docstring is updated in the attached patch.
>
> Thanks! But that makes me wonder: in the cases where we check
>
> (string= python-shell-readline-completer-delims "")
>
> is there a possibility that the variable's value will be nil?
> (In which case we should compare with equal instead of string=.)
Yes, it is a string with native completion and may be nil with
non-native completion. I have updated the patch. Thanks.
[-- Attachment #2: 0001-Detect-the-readline-support-for-Python-shell-complet.patch --]
[-- Type: text/x-patch, Size: 9022 bytes --]
From d204d5ae2439dc76662e4e13491ff103216a2d12 Mon Sep 17 00:00:00 2001
From: Liu Hui <liuhui1610@gmail.com>
Date: Mon, 26 Feb 2024 18:46:36 +0800
Subject: [PATCH] Detect the readline support for Python shell completion
* lisp/progmodes/python.el
(python-shell-comint-watch-for-first-prompt-output-filter):
Detect the readline support.
(python-shell-readline-completer-delims): Update docstring.
(python-shell-completion-native-setup): Move the readline
detection code to ...
(python-shell-readline-detect): ... new function.
(python-shell-completion-native-turn-on-maybe): Skip if Python
has no readline support.
(python-shell-completion-at-point): Respect the delimiter of
readline completer in non-native completion.
* test/lisp/progmodes/python-tests.el
(python-shell-completion-at-point-1)
(python-shell-completion-at-point-native-1)
(python-completion-at-point-1, python-completion-at-point-2)
(python-completion-at-point-pdb-1)
(python-completion-at-point-while-running-1)
(python-completion-at-point-native-1)
(python-completion-at-point-native-2)
(python-completion-at-point-native-with-ffap-1)
(python-completion-at-point-native-with-eldoc-1): Skip tests if
Python has no readline support.
(python-shell-completion-at-point-jedi-completer): Add test for
non-native Python shell completion. (bug#68559)
---
lisp/progmodes/python.el | 29 ++++++++++++++++++++-------
test/lisp/progmodes/python-tests.el | 31 ++++++++++++++++++++++++-----
2 files changed, 48 insertions(+), 12 deletions(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 587d0b36304..c94bb4fc550 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -3601,6 +3601,7 @@ python-shell-comint-watch-for-first-prompt-output-filter
(python-shell-send-string-no-output python-shell-eval-file-setup-code))
(with-current-buffer (current-buffer)
(let ((inhibit-quit nil))
+ (python-shell-readline-detect)
(run-hooks 'python-shell-first-prompt-hook))))))
output)
@@ -4439,7 +4440,23 @@ python-shell-completion-native-try-output-timeout
(defvar python-shell-readline-completer-delims nil
"Word delimiters used by the readline completer.
-It is automatically set by Python shell.")
+It is automatically set by Python shell. An empty string means no
+characters are considered delimiters and the readline completion
+considers the entire line of input. A value of nil means the Python
+shell has no readline support.")
+
+(defun python-shell-readline-detect ()
+ "Detect the readline support for Python shell completion."
+ (let* ((process (python-shell-get-process))
+ (output (python-shell-send-string-no-output "
+try:
+ import readline
+ print(readline.get_completer_delims())
+except:
+ print('No readline support')" process)))
+ (setq-local python-shell-readline-completer-delims
+ (unless (string-search "No readline support" output)
+ (string-trim-right output)))))
(defvar python-shell-completion-native-redirect-buffer
" *Python completions redirect*"
@@ -4579,10 +4596,6 @@ python-shell-completion-native-setup
__PYTHON_EL_native_completion_setup()" process)))
(when (string-match-p "python\\.el: native completion setup loaded"
output)
- (setq-local python-shell-readline-completer-delims
- (string-trim-right
- (python-shell-send-string-no-output
- "import readline; print(readline.get_completer_delims())")))
(python-shell-completion-native-try))))
(defun python-shell-completion-native-turn-off (&optional msg)
@@ -4611,7 +4624,8 @@ python-shell-completion-native-turn-on-maybe
(cond
((python-shell-completion-native-interpreter-disabled-p)
(python-shell-completion-native-turn-off msg))
- ((python-shell-completion-native-setup)
+ ((and python-shell-readline-completer-delims
+ (python-shell-completion-native-setup))
(when msg
(message "Shell native completion is enabled.")))
(t
@@ -4783,7 +4797,8 @@ python-shell-completion-at-point
(with-current-buffer (process-buffer process)
(if python-shell-completion-native-enable
(string= python-shell-readline-completer-delims "")
- (string-match-p "ipython[23]?\\'" python-shell-interpreter)))))
+ (or (string-match-p "ipython[23]?\\'" python-shell-interpreter)
+ (equal python-shell-readline-completer-delims ""))))))
(start
(if (< (point) line-start)
(point)
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 1ceee690cfb..e11440cdb5b 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -4783,6 +4783,7 @@ python-shell-completion-at-point-1
(python-tests-with-temp-buffer-with-shell
""
(python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims)
(insert "import abc")
(comint-send-input)
(python-tests-shell-wait-for-prompt)
@@ -4797,6 +4798,7 @@ python-shell-completion-at-point-native-1
""
(python-shell-completion-native-turn-on)
(python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims)
(insert "import abc")
(comint-send-input)
(python-tests-shell-wait-for-prompt)
@@ -4883,11 +4885,14 @@ python-shell-completion-at-point-jedi-completer
(python-tests-with-temp-buffer-with-shell
""
(python-shell-with-shell-buffer
- (python-shell-completion-native-turn-on)
- (skip-unless (string= python-shell-readline-completer-delims ""))
- (python-tests--completion-module)
- (python-tests--completion-parameters)
- (python-tests--completion-extra-context)))))
+ (skip-unless (string= python-shell-readline-completer-delims ""))
+ (python-shell-completion-native-turn-off)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-shell-completion-native-turn-on)
+ (python-tests--completion-module)
+ (python-tests--completion-parameters)
+ (python-tests--completion-extra-context)))))
(ert-deftest python-shell-completion-at-point-ipython ()
"Check if Python shell completion works for IPython."
@@ -4924,6 +4929,8 @@ python-completion-at-point-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
(goto-char (point-max))
@@ -4940,6 +4947,8 @@ python-completion-at-point-2
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
(python-shell-with-shell-buffer
@@ -4959,6 +4968,8 @@ python-completion-at-point-pdb-1
print('Hello')
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
(goto-char (point-max))
@@ -4975,6 +4986,8 @@ python-completion-at-point-while-running-1
time.sleep(3)
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-send-buffer)
(goto-char (point-max))
(insert "time.")
@@ -4987,6 +5000,8 @@ python-completion-at-point-native-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
@@ -5004,6 +5019,8 @@ python-completion-at-point-native-2
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
@@ -5020,6 +5037,8 @@ python-completion-at-point-native-with-ffap-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
@@ -5036,6 +5055,8 @@ python-completion-at-point-native-with-eldoc-1
import abc
"
(let ((inhibit-message t))
+ (python-shell-with-shell-buffer
+ (skip-unless python-shell-readline-completer-delims))
(python-shell-completion-native-turn-on)
(python-shell-send-buffer)
(python-tests-shell-wait-for-prompt)
--
2.25.1
^ permalink raw reply related [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-03-13 10:21 ` Liu Hui
@ 2024-03-14 14:24 ` Basil L. Contovounesios
2024-03-16 6:49 ` Liu Hui
0 siblings, 1 reply; 68+ messages in thread
From: Basil L. Contovounesios @ 2024-03-14 14:24 UTC (permalink / raw)
To: Liu Hui; +Cc: kobarity, Eli Zaretskii, Mattias Engdegård, 68559
Liu Hui [2024-03-13 18:21 +0800] wrote:
> On Tue, Mar 12, 2024 at 12:02 AM Basil L. Contovounesios
> <basil@contovou.net> wrote:
>>
>> Thanks! But that makes me wonder: in the cases where we check
>>
>> (string= python-shell-readline-completer-delims "")
>>
>> is there a possibility that the variable's value will be nil?
>> (In which case we should compare with equal instead of string=.)
>
> Yes, it is a string with native completion and may be nil with
> non-native completion.
Sorry, I completely forgot that string= works on symbols too. I was
worried that (string= nil "") would signal an error, but it actually
does what we want.
> I have updated the patch.
Thanks, pushed:
Detect the readline support for Python shell completion
a7057745f5e 2024-03-14 15:09:56 +0100
https://git.sv.gnu.org/cgit/emacs.git/commit/?id=a7057745f5e
Is there more left to do as part of this bug report?
--
Basil
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-03-14 14:24 ` Basil L. Contovounesios
@ 2024-03-16 6:49 ` Liu Hui
2024-03-16 8:27 ` Eli Zaretskii
0 siblings, 1 reply; 68+ messages in thread
From: Liu Hui @ 2024-03-16 6:49 UTC (permalink / raw)
To: Basil L. Contovounesios
Cc: kobarity, Eli Zaretskii, Mattias Engdegård, 68559
On Thu, Mar 14, 2024 at 10:24 PM Basil L. Contovounesios
<basil@contovou.net> wrote:
>
> Liu Hui [2024-03-13 18:21 +0800] wrote:
>
> > On Tue, Mar 12, 2024 at 12:02 AM Basil L. Contovounesios
> > <basil@contovou.net> wrote:
> >>
> >> Thanks! But that makes me wonder: in the cases where we check
> >>
> >> (string= python-shell-readline-completer-delims "")
> >>
> >> is there a possibility that the variable's value will be nil?
> >> (In which case we should compare with equal instead of string=.)
> >
> > Yes, it is a string with native completion and may be nil with
> > non-native completion.
>
> Sorry, I completely forgot that string= works on symbols too. I was
> worried that (string= nil "") would signal an error, but it actually
> does what we want.
>
> > I have updated the patch.
>
> Thanks, pushed:
>
> Detect the readline support for Python shell completion
> a7057745f5e 2024-03-14 15:09:56 +0100
> https://git.sv.gnu.org/cgit/emacs.git/commit/?id=a7057745f5e
>
> Is there more left to do as part of this bug report?
I think there is none.
^ permalink raw reply [flat|nested] 68+ messages in thread
* bug#68559: [PATCH] Improve Python shell completion
2024-03-16 6:49 ` Liu Hui
@ 2024-03-16 8:27 ` Eli Zaretskii
0 siblings, 0 replies; 68+ messages in thread
From: Eli Zaretskii @ 2024-03-16 8:27 UTC (permalink / raw)
To: Liu Hui; +Cc: kobarity, 68559-done, mattias.engdegard, basil
> From: Liu Hui <liuhui1610@gmail.com>
> Date: Sat, 16 Mar 2024 14:49:20 +0800
> Cc: kobarity <kobarity@gmail.com>, Mattias Engdegård <mattias.engdegard@gmail.com>,
> Eli Zaretskii <eliz@gnu.org>, 68559@debbugs.gnu.org
>
> On Thu, Mar 14, 2024 at 10:24 PM Basil L. Contovounesios
> <basil@contovou.net> wrote:
> >
> > Thanks, pushed:
> >
> > Detect the readline support for Python shell completion
> > a7057745f5e 2024-03-14 15:09:56 +0100
> > https://git.sv.gnu.org/cgit/emacs.git/commit/?id=a7057745f5e
> >
> > Is there more left to do as part of this bug report?
>
> I think there is none.
Thanks, closing.
^ permalink raw reply [flat|nested] 68+ messages in thread
end of thread, other threads:[~2024-03-16 8:27 UTC | newest]
Thread overview: 68+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-01-18 4:48 bug#68559: [PATCH] Improve Python shell completion Liu Hui
2024-01-18 6:39 ` Eli Zaretskii
2024-01-21 9:34 ` kobarity
2024-01-23 11:31 ` Liu Hui
2024-01-23 14:15 ` kobarity
2024-01-24 10:07 ` Liu Hui
2024-01-25 15:38 ` kobarity
2024-01-26 10:12 ` Liu Hui
2024-01-28 13:22 ` kobarity
2024-01-29 13:15 ` kobarity
2024-02-01 9:52 ` Eli Zaretskii
2024-02-01 14:39 ` kobarity
2024-02-01 15:02 ` Liu Hui
2024-02-04 12:09 ` Liu Hui
2024-02-04 14:35 ` kobarity
2024-02-05 15:03 ` Liu Hui
2024-02-06 1:25 ` Liu Hui
2024-02-06 15:12 ` kobarity
2024-02-07 13:22 ` Liu Hui
2024-02-07 15:19 ` kobarity
2024-02-08 12:13 ` Eli Zaretskii
2024-02-08 13:33 ` Liu Hui
2024-02-08 13:46 ` Eli Zaretskii
2024-02-08 14:16 ` Liu Hui
2024-02-08 16:43 ` Eli Zaretskii
2024-02-15 14:43 ` Mattias Engdegård
2024-02-15 16:37 ` Eli Zaretskii
2024-02-15 16:48 ` Eli Zaretskii
2024-02-15 17:21 ` Mattias Engdegård
2024-02-19 13:18 ` Basil L. Contovounesios
2024-02-20 4:46 ` Liu Hui
2024-02-20 13:15 ` Basil L. Contovounesios
2024-02-21 10:00 ` Liu Hui
2024-02-21 14:55 ` Basil L. Contovounesios
2024-02-22 10:31 ` Liu Hui
2024-02-22 13:56 ` Basil L. Contovounesios
2024-02-23 13:07 ` Liu Hui
2024-02-28 14:47 ` Basil L. Contovounesios
2024-02-16 4:06 ` Liu Hui
2024-02-16 7:41 ` Eli Zaretskii
2024-02-16 12:51 ` Eli Zaretskii
2024-02-16 3:24 ` Liu Hui
2024-02-16 9:34 ` kobarity
2024-02-16 11:45 ` Mattias Engdegård
2024-02-16 15:24 ` kobarity
2024-02-16 15:52 ` Eli Zaretskii
2024-02-16 20:10 ` Mattias Engdegård
2024-02-17 13:33 ` kobarity
2024-02-20 10:16 ` Mattias Engdegård
2024-02-21 13:13 ` kobarity
2024-02-21 18:20 ` Mattias Engdegård
2024-02-22 16:15 ` kobarity
2024-02-23 11:00 ` Mattias Engdegård
2024-02-23 14:39 ` kobarity
2024-02-26 11:06 ` Liu Hui
2024-02-26 12:16 ` Mattias Engdegård
2024-02-26 15:08 ` kobarity
2024-02-28 14:49 ` Basil L. Contovounesios
2024-03-06 10:14 ` Liu Hui
2024-03-08 15:44 ` Basil L. Contovounesios
2024-03-11 11:35 ` Liu Hui
2024-03-11 16:02 ` Basil L. Contovounesios
2024-03-13 10:21 ` Liu Hui
2024-03-14 14:24 ` Basil L. Contovounesios
2024-03-16 6:49 ` Liu Hui
2024-03-16 8:27 ` Eli Zaretskii
2024-02-17 4:36 ` Liu Hui
2024-02-17 13:20 ` Mattias Engdegård
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.