From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#73484: 31.0.50; Abolishing etags-regen-file-extensions Date: Sun, 6 Oct 2024 22:14:46 +0300 Message-ID: <8b6560a9-e2d6-42ae-ac1d-014700f21804@gutov.dev> References: <87tteaznog.fsf@zephyr.silentflame.com> <8734lrrj4e.fsf@zephyr.silentflame.com> <87o74c1ce1.fsf@zephyr.silentflame.com> <87jzezzg87.fsf_-_@zephyr.silentflame.com> <37e4b3cd-6363-4f55-9921-92a1182679dc@gutov.dev> <86ttdy50ja.fsf@gnu.org> <75fe4289-da41-454d-ba92-22a92ea7002f@gutov.dev> <86frpe2186.fsf@gnu.org> <8e305b6d-8ca8-4437-990f-183ebc007d18@gutov.dev> <865xqa1ggi.fsf@gnu.org> <86ttdtzoof.fsf@gnu.org> <8d7dc133-9828-4023-821f-e4403f899f81@gutov.dev> <86ttdsxt6x.fsf@gnu.org> <52cb1caa-9e7e-45df-b328-d60948d397f6@gutov.dev> <864j5rxca1.fsf@gnu.org> <87a5fiijy9.fsf@tucano.isti.cnr.it> <86jzelvjh4.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="6225"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla Thunderbird Cc: pot@gnu.org, 73484@debbugs.gnu.org, spwhitton@spwhitton.name To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Oct 06 21:16:40 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sxWkB-0001Om-N3 for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 06 Oct 2024 21:16:40 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sxWjc-0007Ox-Fu; Sun, 06 Oct 2024 15:16:04 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sxWjW-0007NK-92 for bug-gnu-emacs@gnu.org; Sun, 06 Oct 2024 15:15:58 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sxWjU-0006XY-6H for bug-gnu-emacs@gnu.org; Sun, 06 Oct 2024 15:15:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=In-Reply-To:From:References:MIME-Version:Date:To:Subject; bh=xSDvjzv2R3JJ3V72JlWgfldCkz/v6Gp4cpwBYZseS0o=; b=sr0lhD7Wz5uyNKJQ1NkqfjjnHTOI0pJ5GLQ2/N56MfzC6czPnc6EA5Q8QVfLfsW68Fm0neDDbzWTeqzWaIy0hIN+enPKeckxgpJHE1Sq39yYEbHs1rJVZtNfb5qUTfy8uLrJGE/eQ4RfeVbFWXrKpHISWOiHA6TWw7idBQE+9tKklQolSZC1SVe6en1JmE+iJtd92yec+30czum+ZRkPCXtGBy6VDBrhlOpslj0nRcsoLiZnkaZFpYNW3ahfJ5ruD9t2Wu8rdAfVEsS4e+4yZvYmHCpB5pDr9Vwb38X5QraR+hS1gc3KBI+fOOicx5L292hTmRCuxlI1OI6yMAOaqA==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1sxWja-0003H3-9C for bug-gnu-emacs@gnu.org; Sun, 06 Oct 2024 15:16:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 06 Oct 2024 19:16:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 73484 X-GNU-PR-Package: emacs Original-Received: via spool by 73484-submit@debbugs.gnu.org id=B73484.172824210511998 (code B ref 73484); Sun, 06 Oct 2024 19:16:02 +0000 Original-Received: (at 73484) by debbugs.gnu.org; 6 Oct 2024 19:15:05 +0000 Original-Received: from localhost ([127.0.0.1]:42312 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sxWie-00037S-Qn for submit@debbugs.gnu.org; Sun, 06 Oct 2024 15:15:05 -0400 Original-Received: from fout-a1-smtp.messagingengine.com ([103.168.172.144]:57063) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sxWib-00036i-Qr for 73484@debbugs.gnu.org; Sun, 06 Oct 2024 15:15:03 -0400 Original-Received: from phl-compute-01.internal (phl-compute-01.phl.internal [10.202.2.41]) by mailfout.phl.internal (Postfix) with ESMTP id 491A513800ED; Sun, 6 Oct 2024 15:14:50 -0400 (EDT) Original-Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-01.internal (MEProxy); Sun, 06 Oct 2024 15:14:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1728242090; x=1728328490; bh=xSDvjzv2R3JJ3V72JlWgfldCkz/v6Gp4cpwBYZseS0o=; b= AcX25BOc4B8xc/imj3VgFe84BrXrXsiw6q77rZMIrfO9CFSi4vZUApJ9k6r0lB08 G9EVNkMcx0HCYOXxAYnUgvWpTnjRhnrBRvvdCeon7tHSVce4XBPPRhPcFqdjff3w YgZIoSLREsO2ytwVmQJCXHPvSHN010sJOPtGPBhjOSH/uxKZIxbofjkAWqcwM6SE XQG1QFPZPOmTAJfnUFkF1FDkz0tv/knKRCKqRSN+9WZhtyCq6aXKTRF5fPvJi6QJ 6OR64Kfzjs6Hdlm8fZcL0bKovRaO8mA6I9GYo+99AcKYXRiovXE90yTw0k2U96X1 jqYUaW3XMX+Lcjoe5BO2iQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1728242090; x= 1728328490; bh=xSDvjzv2R3JJ3V72JlWgfldCkz/v6Gp4cpwBYZseS0o=; b=b MokaAhIM8SbkE9AWDIPlGRz4gRdTc507b++dW2sf1B3QZZzDIkyz9A9wLX62//xk EfE17GefW78fSjGgtl7VuqHGMFcU+iFX7X/jUPEGVbSEyqgZTYS7DEPO0zXX2vea afTd428Y6sOsY8pxB2j0IxokYn2iVZsFuzLlNN5YFkT1xBEdSgVhyIZWWk1Th3CL P6dE15EwRrW1XWtjABdvlLo8K2m8UG1IQHiWm4mN6ggzWr5bHjWwm2VKdvw2hHIV fiBCRSFoQUSR0XjNrR43C4wVestQ7UsGF1vTizXoeaNIUvHS3wAi+EMGrZ7b4KNI 2cHcEdqonaoxuTc8HODVQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrvddvjedgudefiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefkffggfgfuvfevfhfhjggtgfesthejredttddv jeenucfhrhhomhepffhmihhtrhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrd guvghvqeenucggtffrrghtthgvrhhnpeefkeehueetieegveeltdejfeehfeehheejheek uefhfeefleevffelheefhfdvveenucffohhmrghinhepghhithhhuhgsrdgtohhmnecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepughmihhtrhih sehguhhtohhvrdguvghvpdhnsggprhgtphhtthhopeegpdhmohguvgepshhmthhpohhuth dprhgtphhtthhopegvlhhiiiesghhnuhdrohhrghdprhgtphhtthhopehpohhtsehgnhhu rdhorhhgpdhrtghpthhtohepshhpfihhihhtthhonhesshhpfihhihhtthhonhdrnhgrmh gvpdhrtghpthhtohepjeefgeekgeesuggvsggsuhhgshdrghhnuhdrohhrgh X-ME-Proxy: Feedback-ID: i07de48aa:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 6 Oct 2024 15:14:48 -0400 (EDT) Content-Language: en-US In-Reply-To: <86jzelvjh4.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:293099 Archived-At: On 06/10/2024 09:22, Eli Zaretskii wrote: >> Then, the total time increased a lot: from 30 s to 30-40 min. > > I don't understand why. How many files with no extensions are in that > tree, and what was the etags command line in both cases? Sorry, I have to add a correction: it's about 15 min either way. Seems like the first time I either messed up the start time, or the directory was in "cold" cache, or the used etags some much older version. So to reiterate: the current etags-regen scans in around 30s, and the simple switch scans the directory in 15 minutes. Retesting the change from previous email, it doesn't really help. And the 'find-tag' scan did become slower - i.e. from 400 ms to 1200 ms. Not clear about the mechanics (the size of TAGS only went up from 65 to 88 MB). >> But parsing HTML files seems to remain the slowest part. There are a lot >> of them in that project (many test cases), but maybe 3x the number of >> code files, not 60x their number. And they're pretty small, on average. >> If somebody wants to test that locally, here's the repository: >> https://github.com/mozilla/gecko-dev > > If HTML files is what explains the slowdown, then why this change > triggered it? HTML files are supposed to have extensions that tell > etags they are HTML. Okay, I've commented out the most obvious suspects (html, asm, makefile) - all their entries in 'lang_names' - but the scan still takes too long. Maybe it's some other file type, which I haven't found yet. But what is see when monitoring the running scan with 'tail -f TAGS', is the output stops sometimes for like 20 seconds, in the middle of outputting tags of some common code file (like .cpp or .py, a common type), and then resumes, with files of the same type around this one. > And if they don't have extensions, the code you > removed would have caused etags to scan these files anyway, looking > for Fortran or C tags. So how come the change slowed down etags so > much? What am I missing? I think it would also concern "unknown" extensions, right? Like .txt, .png and so on. Anyway, the difference is either due to the different set of files (all project files, rather than files in the specified list of extensions), or due to all file names being printed. Not sure how to verify, yet.