bug#47711: bug#48841: bug#47711: bug#48841: bug#47711: [PATCH VERSION 2] Add new `completion-filter-completions` API and deferred highlighting

[Dmitry Gutov <dmitry@gutov.dev>]

On 25/10/2023 20:52, João Távora wrote:
> And make sure to put 300 000 symbols in the obarray.  The symbols are
> prefixed "yoyo" deliberately.
> 
>      (cl-loop repeat 300000 do (intern (symbol-name (gensym "yoyo"))))
> 
> First a micro-benchmark:
> 
>     ;; Daniel's patch worked by Dmitry (v3)
>     (benchmark-run 50
>      (let ((completion-styles '(flex)))
>        (completion-filter-completions "" obarray 'fboundp 0 nil)
>        (completion-filter-completions "yo" obarray 'fboundp 0 nil)
>        (completion-filter-completions "yoo" obarray 'fboundp 0 nil)
>        ));; => (12.192422429999999 3 0.107881004)
> 
> 
>    ;; lazy-hilit v4 patch attached in this email
>    (benchmark-run 50
>      (let ((completion-styles '(flex))
>            (completion-lazy-hilit (cl-gensym)))
>        (completion-all-completions "" obarray 'fboundp 0 nil)
>        (completion-all-completions "yo" obarray 'fboundp 0 nil)
>        (completion-all-completions "yoo" obarray 'fboundp 0 nil)
>        ));; => (12.267915333 4 0.14799709099999991)

Note on this particular test:

The loop on the first line only creates the symbols in the obarray, but 
not functions. As a result, all the completion-all-completions calls 
return nil because of the 'fboundp' predicate. When you change the 
predicate argument to nil, these timings change considerably (so it's 
wiser to reduce the number of repetitions to 1 or 5 at most), with 
completion-filter-completions being ~2.5x faster than the other.

It is slower in the sorting step, though: mostly due to the extra 
consing created with the alist to-be-sorted, I guess, but also because 
of the repeated string-match call (which, while fast and much faster 
than the match-data call, is still not free).

That's how when compared in practice using fido-vertical-mode the 
results were about the same.