From: Dmitry Gutov <dgutov@yandex.ru>
To: Juri Linkov <juri@linkov.net>
Cc: Stefan Monnier <monnier@IRO.UMontreal.CA>, emacs-devel@gnu.org
Subject: Re: [Emacs-diffs] master 55ec674: * lisp/multifile.el: New file, extracted from etags.el
Date: Thu, 3 Jan 2019 02:02:45 +0300 [thread overview]
Message-ID: <6ff6f811-29eb-6e08-e95c-46bdfee08993@yandex.ru> (raw)
In-Reply-To: <87zhsi7mlt.fsf@mail.linkov.net>
On 03.01.2019 0:53, Juri Linkov wrote:
> On the very old computer from the year 2010, but the most interesting are relative times:
>
> (benchmark 10 '(project-find-regexp "xyz1"))
> => 7s
> (benchmark 10 '(project-files-pipe-grep "xyz1"))
> => 17s
This is too bad. But did you use project-files-pipe-grep from
446bcaed37b66ec112aaec7a7960e20b969c8012 or from
c708231803712bd37154c140afdfd8468cac603e?
It would be helpful to test both implementations.
> (benchmark 10 '(project-files (project-current t)))
> => 11s
This is weird. I can understand that listing all files can be slower on
an old, HDD-based computer. But both project-find-regexp and
project-files use 'find ... -path], and the former even adds Grep on top
of it. Why is the "simpler" operation slower?
Is it about piping the long list of files to Emacs? Why is the 'git
ls-files' example so fast, then? It returns the same long list.
> (benchmark 10 '(shell-command-to-string "find ... \\( -path ... \\) -prune -o -type f -print0"))
> => 11s
>
> (benchmark 10 '(shell-command-to-string "git ls-files"))
> => 0.07s
Could you try making a full project-files implementation on top of it? I
wonder how much slower it will be.
At least test (split-string (shell-command-to-string "git ls-files -z")
"\0" t)
> IMHO, everything is clear: “find” with “-path” filters is slow,
> whereas “git ls-files” is fast.
We're all aware that 'git ls-files' is fast.
But not every project backend is going to be using 'git ls-files' (or a
Git repository). So we should make sure that project-find-regexp does
not noticeably regress when using the fallback implementation of
project-files (based on 'find') if we're going to change its
implementation to be based on project-files. Or regresses as little as
possible.
And then we can implement a faster project-files for the built-in
project backend (based on VC), but only, again, when used with a VCS
that supports fast fetching of the list of files.
And to put it in a different perspective: the difference in speed that
you see between project-find-regexp and project-files-pipe-grep is from
some overhead somewhere. And the same overhead is likely to manifest
even if project-files is based on 'git ls-files'.
next prev parent reply other threads:[~2019-01-02 23:02 UTC|newest]
Thread overview: 103+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20180922154639.23195.66360@vcs0.savannah.gnu.org>
[not found] ` <20180922154640.9D58220310@vcs0.savannah.gnu.org>
2018-12-26 3:34 ` [Emacs-diffs] master 55ec674: * lisp/multifile.el: New file, extracted from etags.el Dmitry Gutov
2018-12-26 20:13 ` Stefan Monnier
2018-12-27 1:49 ` Dmitry Gutov
2018-12-27 14:39 ` Stefan Monnier
2018-12-28 3:45 ` Dmitry Gutov
2018-12-31 11:42 ` Dmitry Gutov
2018-12-31 15:12 ` Eli Zaretskii
2019-01-02 23:47 ` Dmitry Gutov
2019-01-02 1:49 ` Stefan Monnier
2019-01-03 0:41 ` Dmitry Gutov
2019-01-02 21:53 ` Juri Linkov
2019-01-02 23:02 ` Dmitry Gutov [this message]
2019-01-03 0:37 ` Juri Linkov
2019-01-03 11:45 ` Dmitry Gutov
2019-01-03 20:53 ` Juri Linkov
2019-01-06 1:22 ` Dmitry Gutov
2019-01-07 23:22 ` Dmitry Gutov
2019-01-07 23:27 ` Dmitry Gutov
2019-01-08 14:21 ` Michael Albinus
2019-01-08 23:06 ` Dmitry Gutov
2019-01-09 8:10 ` Michael Albinus
2019-01-09 15:24 ` Dmitry Gutov
2019-01-09 14:57 ` Dmitry Gutov
2019-01-09 23:15 ` Juri Linkov
2019-01-10 10:20 ` Dmitry Gutov
2019-01-10 21:41 ` Juri Linkov
2019-01-12 1:48 ` Dmitry Gutov
2019-01-18 3:52 ` Dmitry Gutov
2019-01-18 12:49 ` Stefan Monnier
2019-01-18 19:28 ` Dmitry Gutov
2019-01-18 21:11 ` Stefan Monnier
2019-01-18 22:53 ` Dmitry Gutov
2018-12-29 0:27 ` Dmitry Gutov
2018-12-29 17:09 ` Dmitry Gutov
2018-12-29 21:54 ` Juri Linkov
2018-12-30 23:06 ` Dmitry Gutov
2019-01-02 1:48 ` Stefan Monnier
2019-01-02 22:05 ` Juri Linkov
2019-01-03 3:44 ` Stefan Monnier
2019-01-03 20:45 ` Juri Linkov
2019-01-12 1:10 ` Making project-files the "canonical" generic, was: " Dmitry Gutov
2019-01-12 18:53 ` Making project-files the "canonical" generic Stephen Leake
2019-01-13 0:54 ` Dmitry Gutov
2019-01-15 1:14 ` Stephen Leake
2019-01-16 16:38 ` Stefan Monnier
2019-01-17 2:23 ` Dmitry Gutov
2019-01-17 13:25 ` Stefan Monnier
2019-01-18 1:00 ` Dmitry Gutov
2019-01-16 19:02 ` project--completing-read-strict breaks ada-mode project completion table Stephen Leake
2019-01-16 22:02 ` Stephen Leake
2019-01-17 23:17 ` Stephen Leake
2019-01-18 2:04 ` Dmitry Gutov
2019-01-19 3:35 ` Stephen Leake
2019-01-19 22:05 ` Dmitry Gutov
2019-01-20 19:34 ` Stephen Leake
2019-01-17 2:21 ` Dmitry Gutov
2019-01-17 13:55 ` Stefan Monnier
2019-01-17 21:35 ` John Yates
2019-01-18 2:19 ` Dmitry Gutov
2019-01-18 3:05 ` Stefan Monnier
2019-01-19 0:26 ` Dmitry Gutov
2019-01-21 19:32 ` Stephen Leake
2019-01-22 0:09 ` Dmitry Gutov
2019-02-07 1:20 ` Stephen Leake
2019-02-11 21:50 ` Stefan Monnier
2019-02-12 1:31 ` Stephen Leake
2019-02-15 15:50 ` Stephen Leake
2019-02-15 22:47 ` Stephen Leake
2019-02-15 23:38 ` Stephen Leake
2019-04-19 17:49 ` Stephen Leake
2019-05-03 0:48 ` Dmitry Gutov
2019-05-04 10:39 ` Stephen Leake
2019-05-07 18:02 ` Stephen Leake
2019-05-07 22:35 ` Dmitry Gutov
2019-05-08 1:53 ` Stefan Monnier
2019-05-14 2:14 ` Dmitry Gutov
2019-05-14 2:13 ` Dmitry Gutov
2019-02-19 17:45 ` Stefan Monnier
2019-02-20 19:58 ` Stephen Leake
2019-02-21 2:00 ` Stefan Monnier
2019-01-21 19:36 ` Stephen Leake
2019-01-22 0:20 ` Dmitry Gutov
2019-01-17 3:04 ` Making project-files the "canonical" generic Dmitry Gutov
2018-12-27 20:33 ` [Emacs-diffs] master 55ec674: * lisp/multifile.el: New file, extracted from etags.el Juri Linkov
2018-12-27 23:31 ` Dmitry Gutov
2018-12-27 23:45 ` Juri Linkov
2018-12-28 6:04 ` Dmitry Gutov
2018-12-28 18:07 ` Stefan Monnier
2018-12-29 0:31 ` Dmitry Gutov
2018-12-29 22:02 ` Juri Linkov
2018-12-30 23:13 ` Dmitry Gutov
2019-01-02 22:11 ` Juri Linkov
2019-01-02 23:23 ` Dmitry Gutov
2019-01-03 0:44 ` Juri Linkov
2019-01-03 11:52 ` Dmitry Gutov
2019-01-03 15:35 ` Stefan Monnier
2019-01-03 23:06 ` Dmitry Gutov
2019-02-07 12:23 ` Dmitry Gutov
2019-02-07 13:05 ` Stefan Monnier
2019-02-14 1:11 ` Dmitry Gutov
2019-01-03 20:57 ` Juri Linkov
2019-01-03 23:21 ` Dmitry Gutov
2019-01-05 22:12 ` Juri Linkov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6ff6f811-29eb-6e08-e95c-46bdfee08993@yandex.ru \
--to=dgutov@yandex.ru \
--cc=emacs-devel@gnu.org \
--cc=juri@linkov.net \
--cc=monnier@IRO.UMontreal.CA \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.