bug#50459: 28.0.50; Python shell completion is incompatible with flex, orderless, etc.

[Augusto Stoffel <arstoffel@gmail.com>]

On Fri, 10 Sep 2021 at 13:37, Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Augusto Stoffel <arstoffel@gmail.com> writes:
>
>> To alleviate this, the completion-at-point function could implement some
>> sort of caching.  The difficult question is when to invalidate the
>> cache.  I've attached one possiblility as a draft patch.  If the
>> approach seems reasonable, then I'll format it properly.
>
> Would it be possible to do caching at a lower level instead of in
> python-mode?

I'm not an expert in this either, but I think the caching mechanism
would be pretty particular to the circumstances of each completion
table, so it indeed belongs here.  Maybe João can say more?

Since this comint-based completion is not super smart (it's not
context-dependent at all), the rather naive caching invalidation used
here seems sufficient to me.

>
>> As a side effect, the patch also solves the original issue described in
>> this ticket.
>
> [...]
>
>> -    (list start end
>> -          (completion-table-dynamic
>> -           (apply-partially
>> -            completion-fn
>> -            process import-statement)))))
>> +    (let  ((re (or (car python-shell--capf-cache) regexp-unmatchable))
>> +           (prefix (buffer-substring-no-properties start end)))
>> +      (unless (string-match re prefix)
>> +        (setq python-shell--capf-cache
>> + (cons (concat "\\`" (regexp-quote prefix)
>> "\\(?:\\sw\\|\\s_\\)*\\'")
>> + (funcall completion-fn process import-statement prefix)))))
>> +    (list start end (cdr python-shell--capf-cache))))
>
> I'm not sure I understand this patch -- it's not using
> `completion-table-dynamic' at all now?  (But my understanding of the
> completion functions in Emacs is pretty lacking.)

That's true: the logic here is that the completions are computed
eagerly, and then cached until still valid.  So if you type

  fo<tab>o.bar.baz

the inferior process is contacted 3 times: after the <tab>, and after
each dot.  Before, a lazy table was returned, but the inferior would be
contacted after each character anyway (if using Company or similar).