From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.devel Subject: Re: [Emacs-diffs] master 55ec674: * lisp/multifile.el: New file, extracted from etags.el Date: Thu, 3 Jan 2019 02:02:45 +0300 Message-ID: <6ff6f811-29eb-6e08-e95c-46bdfee08993@yandex.ru> References: <20180922154639.23195.66360@vcs0.savannah.gnu.org> <20180922154640.9D58220310@vcs0.savannah.gnu.org> <54108dbc-9d12-06ff-3f1d-151118e9b234@yandex.ru> <385f6543-2214-101c-30b8-a8115a8dbede@yandex.ru> <87zhsi7mlt.fsf@mail.linkov.net> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Trace: blaine.gmane.org 1546470116 26823 195.159.176.226 (2 Jan 2019 23:01:56 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 2 Jan 2019 23:01:56 +0000 (UTC) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:64.0) Gecko/20100101 Thunderbird/64.0 Cc: Stefan Monnier , emacs-devel@gnu.org To: Juri Linkov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Jan 03 00:01:52 2019 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from listsout.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gepWI-0006r7-T2 for ged-emacs-devel@m.gmane.org; Thu, 03 Jan 2019 00:01:51 +0100 Original-Received: from localhost ([127.0.0.1]:47880 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gepYP-0005KX-Ga for ged-emacs-devel@m.gmane.org; Wed, 02 Jan 2019 18:04:01 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:39930) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gepXh-0005KA-Tf for emacs-devel@gnu.org; Wed, 02 Jan 2019 18:03:21 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gepXO-0000K5-IG for emacs-devel@gnu.org; Wed, 02 Jan 2019 18:03:13 -0500 Original-Received: from mail-lf1-x142.google.com ([2a00:1450:4864:20::142]:43009) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gepXN-0000CS-Eu for emacs-devel@gnu.org; Wed, 02 Jan 2019 18:02:58 -0500 Original-Received: by mail-lf1-x142.google.com with SMTP id u18so22023166lff.10 for ; Wed, 02 Jan 2019 15:02:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=vZK0ecx8tTXmDSSuMxzJIY3Azw1psvl/hFpwBL3JDIY=; b=Z31tFW5map9yyp3cqU9Cpw/T7UV5xGu2J7U5drndnWMu2SeEsszPPF3xjuqKfrIFGI DPWzUidCJvGTSv1zSFw768UZxlJJ6GQiNNYOpAALskhjaqCJzPiXIOiTAyUtWRh9SaRq wkTLhoGkePZVnM3DLykV1v2gPvQKM0koWVhtWowprjeOzmXo9owyd9ey50AwOE+71SRD Zstiu7sZ8LlIGR8BJo2YX8B/TNF4vtkSKSZIJAu+G7QyS3RD+aXkN8iqfYyKr7sqBvdE HfsLV+F3BpUfrGefucIR6mh42QwVziwdAcSboNfSKJJiaoP7R7dN745qJV2q7oCORLfX vOvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=vZK0ecx8tTXmDSSuMxzJIY3Azw1psvl/hFpwBL3JDIY=; b=YZ7GY6MyuBFaZVyzvDGBd6iWGHuyngPuCafCr2BsGgVsUIVpFmjZWgs0DduQdN09Q3 JgtLfxq0qyITeOQbQHij3yWb0vsl7+yHnNHOO/82Abi/Q6dnVzV8VVfNC4QckaM48abr RZlqIzyyPAv8jps2fJIC7rQdAOZ0yZH1PM8Joux86RRoOyRM5Z1uuwrm4KzrsfkMGZxg qSZ5wYKDW9OIfI+IwfIM6qk7qtnbLrHTbSWU8QYnlOh/sOUvPmea3q/IxxRwB4fZstqq eodRMQpZBX5ffdlOPQdzBcdQzOGSzstIpVNHUXwC9gUi8luJeqBsgBAYx3lyIAHterMx xxPg== X-Gm-Message-State: AA+aEWZoeKwF7CN13zXnOeIHCQgKh/ScqEEHZhLfaDZdPoI4eKP5ekvb hGBUwiyh5c6W8Yyr9tFhCELtI/Zb X-Google-Smtp-Source: AFSGD/WYJkvOZR5c0NUmPBYo7mAcLlHJz7f65MNTtryH4qwBwGrqTQHT/xo1DvTjALtBrEGyGNIAxA== X-Received: by 2002:a19:ced3:: with SMTP id e202mr22034257lfg.13.1546470168256; Wed, 02 Jan 2019 15:02:48 -0800 (PST) Original-Received: from [192.168.0.108] ([79.175.3.73]) by smtp.googlemail.com with ESMTPSA id l21-v6sm11544130ljj.48.2019.01.02.15.02.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 02 Jan 2019 15:02:47 -0800 (PST) In-Reply-To: <87zhsi7mlt.fsf@mail.linkov.net> Content-Language: en-US X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::142 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:232117 Archived-At: On 03.01.2019 0:53, Juri Linkov wrote: > On the very old computer from the year 2010, but the most interesting are relative times: > > (benchmark 10 '(project-find-regexp "xyz1")) > => 7s > (benchmark 10 '(project-files-pipe-grep "xyz1")) > => 17s This is too bad. But did you use project-files-pipe-grep from 446bcaed37b66ec112aaec7a7960e20b969c8012 or from c708231803712bd37154c140afdfd8468cac603e? It would be helpful to test both implementations. > (benchmark 10 '(project-files (project-current t))) > => 11s This is weird. I can understand that listing all files can be slower on an old, HDD-based computer. But both project-find-regexp and project-files use 'find ... -path], and the former even adds Grep on top of it. Why is the "simpler" operation slower? Is it about piping the long list of files to Emacs? Why is the 'git ls-files' example so fast, then? It returns the same long list. > (benchmark 10 '(shell-command-to-string "find ... \\( -path ... \\) -prune -o -type f -print0")) > => 11s > > (benchmark 10 '(shell-command-to-string "git ls-files")) > => 0.07s Could you try making a full project-files implementation on top of it? I wonder how much slower it will be. At least test (split-string (shell-command-to-string "git ls-files -z") "\0" t) > IMHO, everything is clear: “find” with “-path” filters is slow, > whereas “git ls-files” is fast. We're all aware that 'git ls-files' is fast. But not every project backend is going to be using 'git ls-files' (or a Git repository). So we should make sure that project-find-regexp does not noticeably regress when using the fallback implementation of project-files (based on 'find') if we're going to change its implementation to be based on project-files. Or regresses as little as possible. And then we can implement a faster project-files for the built-in project backend (based on VC), but only, again, when used with a VCS that supports fast fetching of the list of files. And to put it in a different perspective: the difference in speed that you see between project-find-regexp and project-files-pipe-grep is from some overhead somewhere. And the same overhead is likely to manifest even if project-files is based on 'git ls-files'.