From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Liu Hui Newsgroups: gmane.emacs.bugs Subject: bug#68559: [PATCH] Improve Python shell completion Date: Sun, 04 Feb 2024 20:09:42 +0800 Message-ID: <87a5og8m7t.fsf@gmail.com> References: <83zfx39mcy.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="10590"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , 68559@debbugs.gnu.org To: kobarity Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Feb 04 13:11:25 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rWbLI-0002Xy-A7 for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 04 Feb 2024 13:11:24 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rWbKx-0001Wc-2f; Sun, 04 Feb 2024 07:11:03 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rWbKl-0001W2-M0 for bug-gnu-emacs@gnu.org; Sun, 04 Feb 2024 07:10:56 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rWbKk-0001Cp-R7 for bug-gnu-emacs@gnu.org; Sun, 04 Feb 2024 07:10:51 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rWbKw-0003PL-8l for bug-gnu-emacs@gnu.org; Sun, 04 Feb 2024 07:11:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Liu Hui Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 04 Feb 2024 12:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 68559 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 68559-submit@debbugs.gnu.org id=B68559.170704861513042 (code B ref 68559); Sun, 04 Feb 2024 12:11:02 +0000 Original-Received: (at 68559) by debbugs.gnu.org; 4 Feb 2024 12:10:15 +0000 Original-Received: from localhost ([127.0.0.1]:48269 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWbK9-0003OG-K9 for submit@debbugs.gnu.org; Sun, 04 Feb 2024 07:10:15 -0500 Original-Received: from mail-pg1-x533.google.com ([2607:f8b0:4864:20::533]:54685) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWbK5-0003Nx-MJ for 68559@debbugs.gnu.org; Sun, 04 Feb 2024 07:10:11 -0500 Original-Received: by mail-pg1-x533.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so3503149a12.1 for <68559@debbugs.gnu.org>; Sun, 04 Feb 2024 04:09:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707048592; x=1707653392; darn=debbugs.gnu.org; h=mime-version:message-id:date:in-reply-to:subject:cc:to:from :references:from:to:cc:subject:date:message-id:reply-to; bh=hKGCWl1espj2PlrlgPWszEgn2NKLDU6YKMrBzd3U89Y=; b=WH9r43hbWd7AQhYrdiTS/XHRNEEmvUddE1BSK8uWEs9KM4pOy4pTndnvnnwGgG3nJs 3PQuTrPja8uIiuhrRJkRo7G4VkR+vy7+4NNp68dnHqdVNSPLRETGkbX7kAkh2lJDGWkB 5js5Hhrq3mFXjjj8hrHEjZzGKPbxqmn/w0NlFXAlace6GX5ddlvjO5LkbKf1ouI9RC8y E7yUL20J6xkWvlrDe2jYTCi2/yE9rpd4jJ0ha8SOQkkuOsAKvx7AVMVBAV9B24+/69hP /nWko2En8WBcJg0RJ0jl+et/7U/vz+7TJyteKd8ufSrS0/BtEa+6Qcv7qitNtmX9O2Xi O7mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707048592; x=1707653392; h=mime-version:message-id:date:in-reply-to:subject:cc:to:from :references:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=hKGCWl1espj2PlrlgPWszEgn2NKLDU6YKMrBzd3U89Y=; b=ecRBuWOZRvbj0/Xio1x/HS6u6rcyKznPaHkDQwWmpv7nKvgkvy82/3MA0JVvJ830HY oXfMROW80as8chubxnVC3Z4IvyFphilXdJPRWf+B/l9DgUVTVRtWNojTS1WbhDGpFHdf aH4wPsqH6+rXKD98ZGJvn4UqurI2NsqBja8A4ud1mCwip3c5XmrFMN5CqXnqegA9TTgc 1n90dS21BA5ghZ8kH63k9Mmb+rChRTrtcF3RUs4+P6C6SIH5lCJqE2l3SKfdv0QtlbKX FeS6p4yqMIiw0NkBsiME0T/Sg7c8lbWtyx8iTbtl0iUTL0O1kATTbBtafm2Er2tgAIr6 kp5g== X-Gm-Message-State: AOJu0Yz+gnpAZIk+6PFeRijDaAlxwkP9EAn9q12c5Rvhm5QsuQBEiyCb kqIDykpOwiTeW6e8r8SJSrMMK5rAVW6r1Ps/VJR8SeUU8Ph5UEYm7OGCuLhi19s= X-Google-Smtp-Source: AGHT+IFoPhsfzfS6lONDUsSlRHCErF+uDeOaMOk28865pkkYpNdR0OIYjJbsKriBv7fKYnVjyWzLVQ== X-Received: by 2002:a05:6a20:9c8d:b0:19e:4a68:46d0 with SMTP id mj13-20020a056a209c8d00b0019e4a6846d0mr8538728pzb.60.1707048591664; Sun, 04 Feb 2024 04:09:51 -0800 (PST) X-Forwarded-Encrypted: i=0; AJvYcCVhpeW+ZBFJN2WuWioF+dLqO1JqAX92H3SIVYH3UmRipaHAzAQQuWIUcSmmh9CTN6TKCFNGIm74HfXij4V3ecqyimh4zoU= Original-Received: from laptop ([112.36.174.104]) by smtp.gmail.com with ESMTPSA id ei29-20020a056a0080dd00b006dfebfd0481sm4745974pfb.7.2024.02.04.04.09.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 04 Feb 2024 04:09:51 -0800 (PST) In-reply-to: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:279402 Archived-At: --=-=-= Content-Type: text/plain kobarity writes: > I'm experiencing strange behavior regarding completion of import > statement in a block in Python buffer. If I try to type the following > lines and then try to complete it, it will fail. > > #+begin_src python > try: > im > #+end_src The problem can be reproduced by running python with jedi completer in a terminal. The reason is that readline completer can only see the text in the current line, i.e. " im"; in this case, jedi does not complete the text to " import". Similarly, the jedi completer cannot complete "except" after ex: try: pass ex| > However, when I try to complete at the beginning of the second line: > > #+begin_src python > try: > > #+end_src > > > "import" keyword also appears as a candidate. If I cancel the > candidates and type "im" and try to complete it, it will succeed. It is because jedi produces completions including " import" for blank string " ". Due to the same prefix between " " and " im", completion cache is reused for the latter. Then " import" can be completed. It is more a limitation of readline completer than a problem with jedi, as we cannot provide proper completion context for jedi. We may define a custom completer to combine jedi and rlcompleter, e.g. (setq python-shell-readline-completer " def __PYTHON_EL_setup_readline_completer(): import readline, rlcompleter import re, sys, os, __main__ from jedi import Interpreter class MyJediRL: def __init__(self): self.rlcompleter = rlcompleter.Completer() self.rldelim = readline.get_completer_delims() def complete(self, text, state): if state == 0: sys.path.insert(0, os.getcwd()) try: interpreter = Interpreter(text, [__main__.__dict__]) completions = interpreter.complete(fuzzy=False) self.matches = [ text[:len(text) - c._like_name_length] + c.name_with_symbols for c in completions ] # try rlcompleter sub = re.split('[' + re.escape(self.rldelim) + ']', text)[-1] i = 0 while True: completion = self.rlcompleter.complete(sub, i) if not completion: break i += 1 completion = text[:len(text)-len(sub)] + completion.rstrip(' ()') if completion not in self.matches: self.matches.append(completion) except: raise finally: sys.path.pop(0) try: return self.matches[state] except IndexError: return None readline.set_completer(MyJediRL().complete) readline.set_completer_delims('')") > Another thing I noticed is the multi-line import statement. If the > import statement is one-line, each items (IGNORECASE and MULTILINE in > the example below) can be completed. > > #+begin_src python > from re import IGNORECASE, MULTILINE > #+end_src > > > However, they cannot be completed if the import statement spans > multi-line. > > #+begin_src python > from re import ( > IGN > #+end_src > > This happens in both Python buffer and Python Shell buffer. Perhaps > this is a limitation of Jedi completer? Yes. Because readline completer cannot see cross-line context, I added the function "python-shell-completion-extra-context" in previous patch to address the case of multi-line function call. I have updated the attached patch to handle multi-line import statement. The change to python-tests.el has been incorporated in the patch. Thanks! --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Improve-Python-shell-completion-bug-68559.patch >From a5da22803e967bc7df5f227a49bb81ce44fdf204 Mon Sep 17 00:00:00 2001 From: Liu Hui Date: Thu, 18 Jan 2024 12:00:00 +0800 Subject: [PATCH] Improve Python shell completion (bug#68559) * lisp/progmodes/python.el (python-shell-completion-setup-code): Fix the completion code of IPython. Change the return value to JSON string and ... (python-shell-completion-get-completions): ... simplify parsing. (inferior-python-mode): Update docstring. (python-shell-readline-completer): New option. (python-shell-readline-completer-delims): New variable indicating the word delimiters of readline completer. (python-shell--readline-jedi-setup-code) (python-shell--readline-ipython-setup-code): New internal variables. (python-shell-completion-native-setup): Setup a suitable readline completer and set the completer delimiter. (python-shell-completion-native-get-completions): Convert output string to completions properly. (python-shell--get-multiline-input) (python-shell--extra-completion-context) (python-shell-completion-extra-context): New functions. (python-shell-completion-at-point): Send text beginning from the line start if the completion backend does not need word splitting. Remove the detection of import statement because it is not needed anymore. Create proper completion table based on completions returned from different backends. * test/lisp/progmodes/python-tests.el (python-tests--completion-module) (python-tests--completion-parameters) (python-tests--completion-extra-context): New helper functions. (python-shell-completion-at-point-jedi-completer) (python-shell-completion-at-point-ipython): New tests. * etc/NEWS: Announce the change. --- etc/NEWS | 6 + lisp/progmodes/python.el | 313 +++++++++++++++++++++++----- test/lisp/progmodes/python-tests.el | 91 ++++++++ 3 files changed, 358 insertions(+), 52 deletions(-) diff --git a/etc/NEWS b/etc/NEWS index 816613de4ec..2c8e70e4a15 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -1086,6 +1086,12 @@ instead of: This allows the user to specify command line arguments to the non interactive Python interpreter specified by 'python-interpreter'. +*** New user option 'python-shell-readline-completer'. +This allows the user to specify the readline completer used for Python +shell completion. The default is 'auto', which means a suitable +completer will be configured automatically according to the Python +interpreter. + ** use-package +++ diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el index 9d840efb9da..0291f398a65 100644 --- a/lisp/progmodes/python.el +++ b/lisp/progmodes/python.el @@ -128,9 +128,9 @@ ;;; Commentary: ;; receiving escape sequences (with some limitations, i.e. completion ;; in blocks does not work). The code executed for the "fallback" ;; completion can be found in `python-shell-completion-setup-code' and -;; `python-shell-completion-string-code' variables. Their default -;; values enable completion for both CPython and IPython, and probably -;; any readline based shell (it's known to work with PyPy). If your +;; `python-shell-completion-get-completions'. Their default values +;; enable completion for both CPython and IPython, and probably any +;; readline based shell (it's known to work with PyPy). If your ;; Python installation lacks readline (like CPython for Windows), ;; installing pyreadline (URL `https://ipython.org/pyreadline.html') ;; should suffice. To troubleshoot why you are not getting any @@ -3604,7 +3604,6 @@ (define-derived-mode inferior-python-mode comint-mode "Inferior Python" `python-shell-prompt-block-regexp', `python-shell-font-lock-enable', `python-shell-completion-setup-code', -`python-shell-completion-string-code', `python-eldoc-setup-code', `python-ffap-setup-code' can customize this mode for different Python interpreters. @@ -4244,8 +4243,9 @@ (defcustom python-shell-completion-setup-code completions = [] completer = None + import json try: - import readline + import readline, re try: import __builtin__ @@ -4256,16 +4256,29 @@ (defcustom python-shell-completion-setup-code is_ipython = ('__IPYTHON__' in builtins or '__IPYTHON__active' in builtins) - splits = text.split() - is_module = splits and splits[0] in ('from', 'import') - - if is_ipython and is_module: - from IPython.core.completerlib import module_completion - completions = module_completion(text.strip()) - elif is_ipython and '__IP' in builtins: - completions = __IP.complete(text) - elif is_ipython and 'get_ipython' in builtins: - completions = get_ipython().Completer.all_completions(text) + + if is_ipython and 'get_ipython' in builtins: + def filter_c(prefix, c): + if re.match('_+(i?[0-9]+)?$', c): + return False + elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix): + return False + return True + + import IPython + try: + if IPython.version_info[0] >= 6: + from IPython.core.completer import provisionalcompleter + with provisionalcompleter(): + completions = [ + [c.text, c.start, c.end, c.type or '?', c.signature or ''] + for c in get_ipython().Completer.completions(text, len(text)) + if filter_c(text, c.text)] + else: + part, matches = get_ipython().Completer.complete(line_buffer=text) + completions = [text + m[len(part):] for m in matches if filter_c(text, m)] + except: + pass else: # Try to reuse current completer. completer = readline.get_completer() @@ -4288,7 +4301,7 @@ (defcustom python-shell-completion-setup-code finally: if getattr(completer, 'PYTHON_EL_WRAPPED', False): completer.print_mode = True - return completions" + return json.dumps(completions)" "Code used to setup completion in inferior Python processes." :type 'string) @@ -4329,6 +4342,78 @@ (defcustom python-shell-completion-native-try-output-timeout 1.0 :version "25.1" :type 'float) +(defcustom python-shell-readline-completer 'auto + "The readline completer used for Python shell completion. +If the value is non-nil, Python shell will setup the readline +completer unless it has been set elsewhere (e.g. in the +PYTHONSTARTUP file). Below are possible values: +- `auto': the completer is determined according to the +interpreter. Specifically, the IPython completer, defined in +`python-shell--readline-ipython-setup-code', is used when the +interpreter is ipython, otherwise the Jedi completer is used. +- a string: Python code to setup the readline. See +`python-shell--readline-jedi-setup-code' for reference. +- `nil': Python shell will do nothing. + +In any case, if the completer is not set successfully in the end, +fallback to the built-in rlcompleter." + :type '(choice (const :tag "Automatic" auto) + (const :tag "No configuration" nil) + (string :tag "Python setup code")) + :version "30.1") + +(defvar python-shell-readline-completer-delims nil + "Word delimiters used by the readline completer. +It is automatically set by Python shell.") + +(defconst python-shell--readline-jedi-setup-code + " +def __PYTHON_EL_setup_readline_completer(): + from jedi.utils import setup_readline + setup_readline()" + "Code used to setup readline completer with Jedi.") + +(defconst python-shell--readline-ipython-setup-code + " +def __PYTHON_EL_setup_readline_completer(): + import readline, re, json, IPython + + class __ipython_RL: + def __init__(self, v): + self.version = v + + def filter(self, prefix, c): + if re.match('_+(i?[0-9]+)?$', c): + return False + elif c[0] == '%' and not re.match('[%a-zA-Z]+$', prefix): + return False + return True + + def complete(self, text, state): + if state == 0: + try: + if self.version >= 6: + from IPython.core.completer import provisionalcompleter + with provisionalcompleter(): + self.matches = [json.dumps([ + [c.text, c.start, c.end, c.type or '?', c.signature or ''] + for c in get_ipython().Completer.completions(text, len(text)) + if self.filter(text, c.text)])] + else: + part, matches = get_ipython().Completer.complete(line_buffer=text) + self.matches = [text + m[len(part):] for m in matches + if self.filter(text, m)] + except Exception: + pass + try: + return self.matches[state] + except IndexError: + return None + + readline.set_completer(__ipython_RL(IPython.version_info[0]).complete) + readline.set_completer_delims('')" + "Code used to setup readline completer for IPython.") + (defvar python-shell-completion-native-redirect-buffer " *Python completions redirect*" "Buffer to be used to redirect output of readline commands.") @@ -4352,7 +4437,20 @@ (defun python-shell-completion-native-try () (defun python-shell-completion-native-setup () "Try to setup native completion, return non-nil on success." (let* ((process (python-shell-get-process)) - (output (python-shell-send-string-no-output " + (completer (pcase python-shell-readline-completer + ('auto + (if (string-match-p "ipython[23]?\\'" python-shell-interpreter) + python-shell--readline-ipython-setup-code + python-shell--readline-jedi-setup-code)) + ((pred stringp) python-shell-readline-completer) + (_ ""))) + (output (python-shell-send-string-no-output + (concat " +try: + del __PYTHON_EL_setup_readline_completer +except: + pass +" completer " def __PYTHON_EL_native_completion_setup(): try: import readline @@ -4425,11 +4523,23 @@ (defun python-shell-completion-native-setup () else: return completion + def is_rlcompleter(completer): + try: + if completer.__self__.__module__ == 'rlcompleter': + return True + else: + return False + except Exception: + return False + completer = readline.get_completer() - if not completer: - # Used as last resort to avoid breaking customizations. - import rlcompleter + if not completer or is_rlcompleter(completer): + try: + __PYTHON_EL_setup_readline_completer() + except: + # Used as last resort to avoid breaking customizations. + import rlcompleter completer = readline.get_completer() if completer and not getattr(completer, 'PYTHON_EL_WRAPPED', False): @@ -4464,9 +4574,13 @@ (defun python-shell-completion-native-setup () print ('python.el: native completion setup failed, %s: %s' % sys.exc_info()[:2]) -__PYTHON_EL_native_completion_setup()" process))) +__PYTHON_EL_native_completion_setup()") process))) (when (string-match-p "python\\.el: native completion setup loaded" output) + (setq-local python-shell-readline-completer-delims + (string-trim-right + (python-shell-send-string-no-output + "import readline; print(readline.get_completer_delims())"))) (python-shell-completion-native-try)))) (defun python-shell-completion-native-turn-off (&optional msg) @@ -4534,6 +4648,8 @@ (defun python-shell-completion-native-get-completions (process input) (let* ((original-filter-fn (process-filter process)) (redirect-buffer (get-buffer-create python-shell-completion-native-redirect-buffer)) + (sep (if (string= python-shell-readline-completer-delims "") + "[\n\r]+" "[ \f\t\n\r\v()]+")) (trigger "\t") (new-input (concat input trigger)) (input-length @@ -4576,28 +4692,76 @@ (defun python-shell-completion-native-get-completions (process input) process python-shell-completion-native-output-timeout comint-redirect-finished-regexp) (re-search-backward "0__dummy_completion__" nil t) - (cl-remove-duplicates - (split-string - (buffer-substring-no-properties - (line-beginning-position) (point-min)) - "[ \f\t\n\r\v()]+" t) - :test #'string=)))) + (let ((str (buffer-substring-no-properties + (line-beginning-position) (point-min)))) + (if (string= "[" (substring str 0 1)) + (condition-case nil + (python--parse-json-array str) + (t (cl-remove-duplicates (split-string str sep t) + :test #'string=))) + (cl-remove-duplicates (split-string str sep t) + :test #'string=)))))) (set-process-filter process original-filter-fn))))) (defun python-shell-completion-get-completions (process input) "Get completions of INPUT using PROCESS." (with-current-buffer (process-buffer process) - (let ((completions - (python-util-strip-string - (python-shell-send-string-no-output - (format - "%s\nprint(';'.join(__PYTHON_EL_get_completions(%s)))" + (python--parse-json-array + (python-shell-send-string-no-output + (format "%s\nprint(__PYTHON_EL_get_completions(%s))" python-shell-completion-setup-code (python-shell--encode-string input)) - process)))) - (when (> (length completions) 2) - (split-string completions - "^'\\|^\"\\|;\\|'$\\|\"$" t))))) + process)))) + +(defun python-shell--get-multiline-input () + "Return lines at a multi-line input in Python shell." + (save-excursion + (let ((p (point)) lines) + (when (progn + (beginning-of-line) + (looking-back python-shell-prompt-block-regexp (pos-bol))) + (push (buffer-substring-no-properties (point) p) lines) + (while (progn (comint-previous-prompt 1) + (looking-back python-shell-prompt-block-regexp (pos-bol))) + (push (buffer-substring-no-properties (point) (pos-eol)) lines)) + (push (buffer-substring-no-properties (point) (pos-eol)) lines)) + lines))) + +(defun python-shell--extra-completion-context () + "Get extra completion context of current input in Python shell." + (let ((lines (python-shell--get-multiline-input)) + (python-indent-guess-indent-offset nil)) + (when (not (zerop (length lines))) + (with-temp-buffer + (delay-mode-hooks + (insert (string-join lines "\n")) + (python-mode) + (python-shell-completion-extra-context)))))) + +(defun python-shell-completion-extra-context (&optional pos) + "Get extra completion context at position POS in Python buffer. +If optional argument POS is nil, use current position. + +Readline completers could use current line as the completion +context, which may be insufficient. In this function, extra +context (e.g. multi-line function call) is found and reformatted +as one line, which is required by native completion." + (let (bound p) + (save-excursion + (and pos (goto-char pos)) + (setq bound (pos-bol)) + (python-nav-up-list -1) + (when (and (< (point) bound) + (or + (looking-back + (python-rx (group (+ (or "." symbol-name)))) (pos-bol) t) + (progn + (forward-line 0) + (looking-at "^[ \t]*\\(from \\)")))) + (setq p (match-beginning 1)))) + (when p + (replace-regexp-in-string + "\n[ \t]*" "" (buffer-substring-no-properties p (1- bound)))))) (defvar-local python-shell--capf-cache nil "Variable to store cached completions and invalidation keys.") @@ -4612,21 +4776,26 @@ (defun python-shell-completion-at-point (&optional process) ;; Working on a shell buffer: use prompt end. (cdr (python-util-comint-last-prompt)) (line-beginning-position))) - (import-statement - (when (string-match-p - (rx (* space) word-start (or "from" "import") word-end space) - (buffer-substring-no-properties line-start (point))) - (buffer-substring-no-properties line-start (point)))) + (no-delims + (and (not (if is-shell-buffer + (eq 'font-lock-comment-face + (get-text-property (1- (point)) 'face)) + (python-syntax-context 'comment))) + (with-current-buffer (process-buffer process) + (if python-shell-completion-native-enable + (string= python-shell-readline-completer-delims "") + (string-match-p "ipython[23]?\\'" python-shell-interpreter))))) (start (if (< (point) line-start) (point) (save-excursion - (if (not (re-search-backward - (python-rx - (or whitespace open-paren close-paren - string-delimiter simple-operator)) - line-start - t 1)) + (if (or no-delims + (not (re-search-backward + (python-rx + (or whitespace open-paren close-paren + string-delimiter simple-operator)) + line-start + t 1))) line-start (forward-char (length (match-string-no-properties 0))) (point))))) @@ -4666,18 +4835,58 @@ (defun python-shell-completion-at-point (&optional process) (t #'python-shell-completion-native-get-completions)))) (prev-prompt (car python-shell--capf-cache)) (re (or (cadr python-shell--capf-cache) regexp-unmatchable)) - (prefix (buffer-substring-no-properties start end))) + (prefix (buffer-substring-no-properties start end)) + (prefix-offset 0) + (extra-context (when no-delims + (if is-shell-buffer + (python-shell--extra-completion-context) + (python-shell-completion-extra-context)))) + (extra-offset (length extra-context))) + (unless (zerop extra-offset) + (setq prefix (concat extra-context prefix))) ;; To invalidate the cache, we check if the prompt position or the ;; completion prefix changed. (unless (and (equal prev-prompt (car prompt-boundaries)) - (string-match re prefix)) + (string-match re prefix) + (setq prefix-offset (- (length prefix) (match-end 1)))) (setq python-shell--capf-cache `(,(car prompt-boundaries) ,(if (string-empty-p prefix) regexp-unmatchable - (concat "\\`" (regexp-quote prefix) "\\(?:\\sw\\|\\s_\\)*\\'")) - ,@(funcall completion-fn process (or import-statement prefix))))) - (list start end (cddr python-shell--capf-cache)))) + (concat "\\`\\(" (regexp-quote prefix) "\\)\\(?:\\sw\\|\\s_\\)*\\'")) + ,@(funcall completion-fn process prefix)))) + (let ((cands (cddr python-shell--capf-cache))) + (cond + ((stringp (car cands)) + (if no-delims + ;; Reduce completion candidates due to long prefix. + (if-let ((Lp (length prefix)) + ((string-match "\\(\\sw\\|\\s_\\)+\\'" prefix)) + (L (match-beginning 0))) + ;; If extra-offset is not zero: + ;; start end + ;; o------------------o---------o-------o + ;; |<- extra-offset ->| + ;; |<----------- L ------------>| + ;; new-start + (list (+ start L (- extra-offset)) end + (mapcar (lambda (s) (substring s L)) cands)) + (list end end (mapcar (lambda (s) (substring s Lp)) cands))) + (list start end cands))) + ;; python-shell-completion(-native)-get-completions may produce + ;; a list of (text start end type signature) for completion. + ;; See `python-shell--readline-ipython-setup-code' and + ;; `python-shell-completion-setup-code'. + ((consp (car cands)) + (list (+ start (nth 1 (car cands)) (- extra-offset)) + ;; Candidates may be cached, so the end position should + ;; be adjusted according to current completion prefix. + (+ start (nth 2 (car cands)) (- extra-offset) prefix-offset) + cands + :annotation-function + (lambda (c) (concat " " (nth 3 (assoc c cands)))) + :company-docsig + (lambda (c) (nth 4 (assoc c cands))))))))) (define-obsolete-function-alias 'python-shell-completion-complete-at-point diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el index 59957ff0712..9f9914896be 100644 --- a/test/lisp/progmodes/python-tests.el +++ b/test/lisp/progmodes/python-tests.el @@ -4799,6 +4799,97 @@ (ert-deftest python-shell-completion-at-point-native-1 () (end-of-line 0) (should-not (nth 2 (python-shell-completion-at-point)))))) +(defun python-tests--completion-module () + "Check if modules can be completed in Python shell." + (insert "import datet") + (completion-at-point) + (beginning-of-line) + (should (looking-at-p "import datetime")) + (kill-line) + (insert "from datet") + (completion-at-point) + (beginning-of-line) + (should (looking-at-p "from datetime")) + (end-of-line) + (insert " import timed") + (completion-at-point) + (beginning-of-line) + (should (looking-at-p "from datetime import timedelta")) + (kill-line)) + +(defun python-tests--completion-parameters () + "Check if parameters can be completed in Python shell." + (insert "import re") + (comint-send-input) + (python-tests-shell-wait-for-prompt) + (insert "re.split('b', 'abc', maxs") + (completion-at-point) + (should (string= "re.split('b', 'abc', maxsplit=" + (buffer-substring (line-beginning-position) (point)))) + (insert "0, ") + (should (python-shell-completion-at-point)) + ;; Test if cache is used. + (cl-letf (((symbol-function 'python-shell-completion-get-completions) + 'ignore) + ((symbol-function 'python-shell-completion-native-get-completions) + 'ignore)) + (insert "fla") + (completion-at-point) + (should (string= "re.split('b', 'abc', maxsplit=0, flags=" + (buffer-substring (line-beginning-position) (point))))) + (beginning-of-line) + (kill-line)) + +(defun python-tests--completion-extra-context () + "Check if extra context is used for completion." + (insert "re.split('b', 'abc',") + (comint-send-input) + (python-tests-shell-wait-for-prompt) + (insert "maxs") + (completion-at-point) + (should (string= "maxsplit=" + (buffer-substring (line-beginning-position) (point)))) + (insert "0)") + (comint-send-input) + (python-tests-shell-wait-for-prompt) + (insert "from re import (") + (comint-send-input) + (python-tests-shell-wait-for-prompt) + (insert "IGN") + (completion-at-point) + (should (string= "IGNORECASE" + (buffer-substring (line-beginning-position) (point))))) + +(ert-deftest python-shell-completion-at-point-jedi-completer () + "Check if Python shell completion works with Jedi." + (skip-unless (executable-find python-tests-shell-interpreter)) + (python-tests-with-temp-buffer-with-shell + "" + (python-shell-with-shell-buffer + (python-shell-completion-native-turn-on) + (skip-unless (string= python-shell-readline-completer-delims "")) + (python-tests--completion-module) + (python-tests--completion-parameters) + (python-tests--completion-extra-context)))) + +(ert-deftest python-shell-completion-at-point-ipython () + "Check if Python shell completion works for IPython." + (let ((python-shell-interpreter "ipython") + (python-shell-interpreter-args "-i --simple-prompt")) + (skip-unless + (and + (executable-find python-shell-interpreter) + (eql (call-process python-shell-interpreter nil nil nil "--version") 0))) + (python-tests-with-temp-buffer-with-shell + "" + (python-shell-with-shell-buffer + (python-shell-completion-native-turn-off) + (python-tests--completion-module) + (python-tests--completion-parameters) + (python-shell-completion-native-turn-on) + (python-tests--completion-module) + (python-tests--completion-parameters) + (python-tests--completion-extra-context))))) ;;; PDB Track integration -- 2.39.2 --=-=-=--