From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Augusto Stoffel Newsgroups: gmane.emacs.bugs Subject: bug#50459: 28.0.50; [PATCH] Python shell completion is incompatible with flex, orderless, etc. Date: Fri, 10 Sep 2021 21:08:12 +0200 Message-ID: <877dfo45b7.fsf@gmail.com> References: <87wnnsl1d1.fsf@gmail.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="4638"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: 50459@debbugs.gnu.org, Gregory Heytings , Michael Albinus To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Sep 10 21:09:16 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mOltk-00011g-Ep for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 10 Sep 2021 21:09:16 +0200 Original-Received: from localhost ([::1]:43922 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mOltj-00059W-GL for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 10 Sep 2021 15:09:15 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:48456) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mOltY-00057C-W5 for bug-gnu-emacs@gnu.org; Fri, 10 Sep 2021 15:09:05 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:55886) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mOltW-0007iZ-7x for bug-gnu-emacs@gnu.org; Fri, 10 Sep 2021 15:09:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mOltV-0001wd-St for bug-gnu-emacs@gnu.org; Fri, 10 Sep 2021 15:09:01 -0400 X-Loop: help-debbugs@gnu.org In-Reply-To: <87wnnsl1d1.fsf@gmail.com> Resent-From: Augusto Stoffel Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 10 Sep 2021 19:09:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50459 X-GNU-PR-Package: emacs Original-Received: via spool by 50459-submit@debbugs.gnu.org id=B50459.16313009037417 (code B ref 50459); Fri, 10 Sep 2021 19:09:01 +0000 Original-Received: (at 50459) by debbugs.gnu.org; 10 Sep 2021 19:08:23 +0000 Original-Received: from localhost ([127.0.0.1]:39199 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mOlss-0001vY-NU for submit@debbugs.gnu.org; Fri, 10 Sep 2021 15:08:23 -0400 Original-Received: from mail-wm1-f44.google.com ([209.85.128.44]:55082) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mOlsq-0001vI-HW for 50459@debbugs.gnu.org; Fri, 10 Sep 2021 15:08:21 -0400 Original-Received: by mail-wm1-f44.google.com with SMTP id s24so1907308wmh.4 for <50459@debbugs.gnu.org>; Fri, 10 Sep 2021 12:08:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:user-agent:mime-version; bh=Quen7fIx7fGvi1IzZugY5gerdZC4POprgZSGGpZtQSk=; b=XQpujMXbWBd8X7N1qR53E30aIAsyLnB9Tzzb2tLgvp4xc3TxZqeqa4Lr25VA/mzhuI X86uRQpfFSk2HP4EsX+J3ymDQjlWvt+Y4blEk1+zfJJkNjPpgMKY+gzm3+UNJiBmXKDo buyoSQrBWVAPw52DUfUDjHnrGYYyoaHoSuku6DFg4bd9VeLdKmR+qzYxWDBxWz441Ptb ZdTbg134IsyZzcfJ8cwlqCUlLZX2YoDJTYp5lwXwtci0NzdWTPaXDhhFqbX3kw4xLJXM vAWiiJxYufIXoS9yX4V3j4JS8W1qew9whTRMw9hO6u8/yYqYUY86Ofs/69wCuNoJncCb xnLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:user-agent :mime-version; bh=Quen7fIx7fGvi1IzZugY5gerdZC4POprgZSGGpZtQSk=; b=Ct7BsTJBa037SJ+yifAPSTUNltymad4FJiZgL2ZdkDQZ2s7uSKZhvGhJhTSmexJRXx iYIAbZSMpcR13njN4Cpkm3jz9SjhfI61a1CFJOteKBSNjsmjJ6dx+R5vXpMyzHAvnTy/ pvzJFZYGNHpGrwDTGJGKTl8m3i6n2Q1RHPzChvMFqdVQYDDrWO6tEaWpwPFUPmMyh/II htqAu0teqHZA6LlwAn7n0QAZwsL/6C7j99vw/AKsc+d4fu5LlPrDx0Hkt8TCXbQTAKoG xrAx/6PueksS+6/a6gpSRr5bJ2CZ8pe1ZW+LqfF1S7Yup1AjUyUR6TAM9jZMAfV8i0yj d7Ng== X-Gm-Message-State: AOAM532jbDRtWWZypx0t1Isd7G1VVcoap39cCNnecdWKs0FUiS0+yKjb Xt7OYIN5bWLzQwabpLEYA3M= X-Google-Smtp-Source: ABdhPJxppNDP0A3zzy/kYER29ruhz0ct+Gii/fMZ7X5ZTtid62YU6/y5lTMyTKKN7wdnERDZ0Ku9bg== X-Received: by 2002:a05:600c:2295:: with SMTP id 21mr9695993wmf.84.1631300894597; Fri, 10 Sep 2021 12:08:14 -0700 (PDT) Original-Received: from ars3 ([2a02:8109:8ac0:56d0::b1d]) by smtp.gmail.com with ESMTPSA id z5sm5149805wmp.26.2021.09.10.12.08.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 12:08:13 -0700 (PDT) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:214036 Archived-At: --=-=-= Content-Type: text/plain Okay, I've attached a patch which seems to work fine. On Fri, 10 Sep 2021 at 16:32, Dmitry Gutov wrote: > On 09.09.2021 19:46, Augusto Stoffel wrote: >> To alleviate this, the completion-at-point function could implement some >> sort of caching. The difficult question is when to invalidate the >> cache. I've attached one possiblility as a draft patch. If the >> approach seems reasonable, then I'll format it properly. > > It's much simpler than what we talked about, but given that > python-shell-completion-at-point doesn't really look past the current > line, your approach should work fine. Yes, and you're right --- I've did it that way because this comint-based completion basically only works for globals anyway. > > I'm not very familiar with the code, so I cannot approve the exact > patch, though, sorry. > > As a bonus, though, maybe add the position of prompt on the > shell-buffer to the invalidation key? Like, if a user imports some new > lib in there, that can bring in new completions. Good idea. I've added this to the patch. It still won't show the new completions until the user evaluates the import, but this is the best you can get by querying an interpreter for the completions. I've also changed a bit the way the "native completion" setup code is sent, so that it doesn't print a message directly in the shell buffer (there is still an echo area message). --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Implement-caching-for-python-shell-completion-at-poi.patch >From 3f27a63fe5be884464a6a68407e0b6990186c4a6 Mon Sep 17 00:00:00 2001 From: Augusto Stoffel Date: Fri, 10 Sep 2021 20:44:10 +0200 Subject: [PATCH] Implement caching for 'python-shell-completion-at-point' * lisp/progmodes/python.el (python-shell-completion-at-point): cache results, since computing them involves talking with the inferior process and, potentially, network communications (python-shell--capf-cache): new variable, for cache (python-shell-completion-get-completions, python-shell-completion-native-get-completions): 'import' argument is not needed anymore. (python-shell-completion-native-setup, python-shell-completion-native-try): pass the setup code synchronously, to avoid printing a message in the shell. --- lisp/progmodes/python.el | 59 ++++++++++++++++++++-------------------- 1 file changed, 30 insertions(+), 29 deletions(-) diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el index 2eef52de0c..e71a8102df 100644 --- a/lisp/progmodes/python.el +++ b/lisp/progmodes/python.el @@ -3577,13 +3577,12 @@ python-shell-completion-native-try python-shell-completion-native-try-output-timeout)) (python-shell-completion-native-get-completions (get-buffer-process (current-buffer)) - nil "_"))) + "_"))) (defun python-shell-completion-native-setup () "Try to setup native completion, return non-nil on success." - (let ((process (python-shell-get-process))) - (with-current-buffer (process-buffer process) - (python-shell-send-string " + (let* ((process (python-shell-get-process)) + (output (python-shell-send-string-no-output " def __PYTHON_EL_native_completion_setup(): try: import readline @@ -3693,14 +3692,10 @@ python-shell-completion-native-setup print ('python.el: native completion setup failed, %s: %s' % sys.exc_info()[:2]) -__PYTHON_EL_native_completion_setup()" process) - (when (and - (python-shell-accept-process-output - process python-shell-completion-native-try-output-timeout) - (save-excursion - (re-search-backward - (regexp-quote "python.el: native completion setup loaded") nil t 1))) - (python-shell-completion-native-try))))) +__PYTHON_EL_native_completion_setup()" process))) + (when (string-match-p "python\\.el: native completion setup loaded" + output) + (python-shell-completion-native-try)))) (defun python-shell-completion-native-turn-off (&optional msg) "Turn off shell native completions. @@ -3760,13 +3755,10 @@ python-shell-completion-native-toggle (python-shell-completion-native-turn-on msg)) python-shell-completion-native-enable)) -(defun python-shell-completion-native-get-completions (process import input) - "Get completions using native readline for PROCESS. -When IMPORT is non-nil takes precedence over INPUT for -completion." +(defun python-shell-completion-native-get-completions (process input) + "Get completions of INPUT using native readline for PROCESS." (with-current-buffer (process-buffer process) - (let* ((input (or import input)) - (original-filter-fn (process-filter process)) + (let* ((original-filter-fn (process-filter process)) (redirect-buffer (get-buffer-create python-shell-completion-native-redirect-buffer)) (trigger "\t") @@ -3818,11 +3810,8 @@ python-shell-completion-native-get-completions :test #'string=)))) (set-process-filter process original-filter-fn))))) -(defun python-shell-completion-get-completions (process import input) - "Do completion at point using PROCESS for IMPORT or INPUT. -When IMPORT is non-nil takes precedence over INPUT for -completion." - (setq input (or import input)) +(defun python-shell-completion-get-completions (process input) + "Get completions of INPUT using PROCESS." (with-current-buffer (process-buffer process) (let ((completions (python-util-strip-string @@ -3836,6 +3825,9 @@ python-shell-completion-get-completions (split-string completions "^'\\|^\"\\|;\\|'$\\|\"$" t))))) +(defvar-local python-shell--capf-cache nil + "Variable to store cached completions and invalidation keys.") + (defun python-shell-completion-at-point (&optional process) "Function for `completion-at-point-functions' in `inferior-python-mode'. Optional argument PROCESS forces completions to be retrieved @@ -3889,12 +3881,21 @@ python-shell-completion-at-point ;; it during a multiline statement (Bug#28051). #'ignore #'python-shell-completion-get-completions)) - (t #'python-shell-completion-native-get-completions))))) - (list start end - (completion-table-dynamic - (apply-partially - completion-fn - process import-statement))))) + (t #'python-shell-completion-native-get-completions)))) + (prev-prompt (car python-shell--capf-cache)) + (re (or (cadr python-shell--capf-cache) regexp-unmatchable)) + (prefix (buffer-substring-no-properties start end))) + ;; To invalidate the cache, we check if the prompt position or the + ;; completion prefix changed. + (unless (and (equal prev-prompt (car prompt-boundaries)) + (string-match re prefix)) + (setq python-shell--capf-cache + `(,(car prompt-boundaries) + ,(if (string-empty-p prefix) + regexp-unmatchable + (concat "\\`" (regexp-quote prefix) "\\(?:\\sw\\|\\s_\\)*\\'")) + ,@(funcall completion-fn process (or import-statement prefix))))) + (list start end (cddr python-shell--capf-cache)))) (define-obsolete-function-alias 'python-shell-completion-complete-at-point -- 2.31.1 --=-=-=--