bug#47711: [PATCH VERSION 2] Add new `completion-filter-completions` API and deferred highlighting

["João Távora" <joaotavora@gmail.com>]

João Távora <joaotavora@gmail.com> writes:

> in the absence of any controlled benchmarks I did some of
> my own, using the most controlled environment I could devise.  I start
> Emacs like so:
>
>    ~/Source/Emacs/emacs/src/emacs -Q -f fido-mode -f fido-vertical-mode -l ~/tmp/benchmark.el ~/tmp/benchmark.el

I have know tweaked the benchmark slightly to make it easier to evaluate
speed qualitatively.  Here's what I've been using.

   (require 'cl-lib)
    
   (fido-mode 1)
   (fido-vertical-mode 1)
    
   ;; Introduce 150 000 new functions to really slow things down.
   ;; Probably more than most non-Spacemancs people will have :-)
   (defmacro lotsoffunctions ()
     `(progn
        ,@(cl-loop repeat 150000
                  collect `(defun ,(intern (symbol-name (gensym "heyhey"))) () 42))))
    
   (lotsoffunctions)
    
   (when nil
     ;; Press C-u C-x C-e C-m quickly to produce a quantitative sample
     (benchmark-run (completing-read "" obarray))
    
     ;; Or just press C-h f to experience how fast/slow completion is.
     )

The results are the same as the ones I reported in the previous email.

I've also cleaned up my previous patch of the
scratch/icomplete-lazy-highlight-attempt-2 branch slightly.  It is now
fully opt-in for frontends and completion-styles, so the backward
compatibility problems which I speculated seem to have been exaggerated.

I'm still studying it for flaws, but anyone can have a look.  And, of
course, there are many different ways to realize the "opt-in" for
frontends/styles.  I just chose the one that seemed the simplest given
the current completion framework.

The performance is still very good, it reduces the usual waiting time in
long lists of completions to about half of what it currently is.

João