bug#64735: 29.0.92; find invocations are ~15x slower because of ignores

[Ihor Radchenko <yantar92@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> I am not sure if this specific issue is important.
>> If we want to use find from Emacs, we would need to create Emacs
>> string/strings when reading the output of find anyway.
>
> So how do you explain that using Find is faster than using
> find-lisp.el?
>
> I think the answer is that using Find as a subprocess conses less.

No. It uses less excessive regexp matching Emacs is trying to do in
file-name-handler-alist.

(ignore (let ((gc-cons-threshold most-positive-fixnum)) (benchmark-progn (find-lisp-find-files "/home/yantar92/.data" ""))))
;; Elapsed time: 2.982393s
(ignore (let ((gc-cons-threshold most-positive-fixnum) file-name-handler-alist) (benchmark-progn (find-lisp-find-files "/home/yantar92/.data" ""))))
;; Elapsed time: 0.784461s


    22.83%  emacs         emacs                            [.] Fnconc
    10.01%  emacs         emacs                            [.] Fexpand_file_name
     9.22%  emacs         emacs                            [.] eval_sub
     3.47%  emacs         emacs                            [.] assq_no_quit
     2.68%  emacs         emacs                            [.] getenv_internal_1
     2.50%  emacs         emacs                            [.] mem_insert.isra.0
     2.24%  emacs         emacs                            [.] Fassq
     2.02%  emacs         emacs                            [.] set_buffer_internal_2


(ignore (let ((gc-cons-threshold most-positive-fixnum) file-name-handler-alist) (benchmark-progn (find-lisp-find-files "/home/yantar92/.data" ""))))
;; Elapsed time: 0.624987s

    12.39%  emacs         emacs                                    [.] eval_sub
    12.07%  emacs         emacs                                    [.] Fexpand_file_name
     4.97%  emacs         emacs                                    [.] assq_no_quit
     4.11%  emacs         emacs                                    [.] getenv_internal_1
     2.77%  emacs         emacs                                    [.] set_buffer_internal_2
     2.61%  emacs         emacs                                    [.] mem_insert.isra.0
     2.47%  emacs         emacs                                    [.] make_clear_multibyte_string.part.0

Non-recursive version of `find-lisp-find-files-internal' is below,
though it provides limited improvement.

(defun find-lisp-find-files-internal (directory file-predicate
						directory-predicate)
  "Find files under DIRECTORY which satisfy FILE-PREDICATE.
FILE-PREDICATE is a function which takes two arguments: the file and its
directory.

DIRECTORY-PREDICATE is used to decide whether to descend into directories.
It is a function which takes two arguments, the directory and its parent."
  (setq directory (file-name-as-directory directory))
  (let (results fullname (dirs (list (expand-file-name directory))))
    (while dirs
      (setq directory (pop dirs))
      (dolist (file (directory-files directory nil nil t))
	(setq fullname (concat directory file))
	(when (file-readable-p fullname)
	  ;; If a directory, check it we should descend into it
	  (and (file-directory-p fullname)
               (setq fullname (concat fullname "/"))
               (funcall directory-predicate file directory)
               (push fullname dirs))
	  ;; For all files and directories, call the file predicate
	  (and (funcall file-predicate file directory)
               (push fullname results)))))
    results))

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>