From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Arthur Miller Newsgroups: gmane.emacs.devel Subject: Re: 28.0.50; Proposal: slightly more efficient package-quickstart.el Date: Fri, 23 Jul 2021 00:38:36 +0200 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="19128"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Jul 23 00:39:32 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m6hLo-0004jq-9u for ged-emacs-devel@m.gmane-mx.org; Fri, 23 Jul 2021 00:39:32 +0200 Original-Received: from localhost ([::1]:60028 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m6hLm-0003LD-9h for ged-emacs-devel@m.gmane-mx.org; Thu, 22 Jul 2021 18:39:30 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:46344) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m6hL1-0002gJ-Uq for emacs-devel@gnu.org; Thu, 22 Jul 2021 18:38:43 -0400 Original-Received: from mail-oln040092075109.outbound.protection.outlook.com ([40.92.75.109]:28176 helo=EUR04-VI1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m6hKy-0007z4-Of for emacs-devel@gnu.org; Thu, 22 Jul 2021 18:38:43 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cIufyX8YWFl+kqLeKE+JFBpnpNExRLqisVTexWa5iq0sxU1N5pVoQoSpm0bnn/WQp9wZ5wHDTnaK+DINXQJfEfkQFxjXffgw9tL7g4wvnkZO1DVuOxrQyrQeWKpJCVkc6CbnEykJK9waf65Optx3hzfPazM9EURZW6QTheG/vPo9oMS+5DhmKutnMretESWzcYKo6Vw994fu2XQbXBg8d6Z4PeLQbbhBhACnLgAdTonJp6SFz07nmutWnlYnx25oTTgAIlaVDQ5iSuNyVJ9G2DWXJKRgpD5n1NaKBs+d/pcHv9VTqHotlZV+8MCxwgVpqG0kOg5ORXwMijuEgUCP9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i9Pd3Yq+T4sfUHTkz+uN2CvymM3EddMSfsEKWvyyqkc=; b=nIEBejQPK2nO7uChM3lDhr/K2T1DayrnHM1zid9y3ksof8A3Zh6zyCumnSBIptfINikony944qMXjCkqKNxykzmu6X/M4ReDCuqhoRoIGdafar+XdEKU4eWeFYfdYHUKSvdStzXF/zfbBMmgrR//ENFiyBcQNht8pg5JgtSIA08j7eFm1GwjzCHcRuoxm4lnxLyzRyKg0wBx11KiS0FEEG6bcCuixXgLa+3it0OASzxgEEQxo/EzEN8MeydbF/suC1zIE4wMyoyppM+jIrHTuLShvdRreKrw9Fo+6uy0hGELaO0bAbRsF1v7D8ej02H/TcjKpBZWtE7la4kMXbSnng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=live.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i9Pd3Yq+T4sfUHTkz+uN2CvymM3EddMSfsEKWvyyqkc=; b=rgi3EbCpanZuda58g0rxSoKOoEh7P7XYmoEBRX7aNGnDI8o8jB7/QkebDpi+mxloqT4pOwfCboZQRvCKHy7jQ03DomtGpy23N+YW6+91WfIrfcwf0IgDzctN9v9XokTGeKUSavjJ7shBKfHvqLvMZPWI1IeuBuf3umJZzbwGOkMZiEFnFNqzbWtKUiMlm8URBGK11LdgNt0ttaygyndUzM6U7UVW1SqqxP3PBriMjawv6bDdnSKQVNV+hdQmmOp3jemC4xeUvu1X+CC5uBrgkpFjfEqYBDVUxlv/jCFCFpvpF7AKq6SDB5fsAKzYxU+oim+YTK6mge41wMz/k+2NEw== Original-Received: from VI1EUR04FT049.eop-eur04.prod.protection.outlook.com (2a01:111:e400:7e0e::51) by VI1EUR04HT095.eop-eur04.prod.protection.outlook.com (2a01:111:e400:7e0e::408) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.24; Thu, 22 Jul 2021 22:38:37 +0000 Original-Received: from AM9PR09MB4977.eurprd09.prod.outlook.com (2a01:111:e400:7e0e::44) by VI1EUR04FT049.mail.protection.outlook.com (2a01:111:e400:7e0e::177) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.24 via Frontend Transport; Thu, 22 Jul 2021 22:38:37 +0000 X-IncomingTopHeaderMarker: OriginalChecksum:44EB07E93CB61B3E102F0C7DEBF7BCA7A2F82890A72EDAB504E390E7E4F1BFD7; UpperCasedChecksum:94FC7A3FD7434F19BE1E76981175A7D6F9368CCE1A643072B61BE115B12BE8AC; SizeAsReceived:8801; Count:46 Original-Received: from AM9PR09MB4977.eurprd09.prod.outlook.com ([fe80::e47b:760e:fa35:f28b]) by AM9PR09MB4977.eurprd09.prod.outlook.com ([fe80::e47b:760e:fa35:f28b%7]) with mapi id 15.20.4352.026; Thu, 22 Jul 2021 22:38:37 +0000 In-Reply-To: (Stefan Monnier's message of "Tue, 20 Jul 2021 11:49:32 -0400") X-TMN: [QDGOak7nyirpLFQCpIMWUcsoB3c43Zm6] X-ClientProxiedBy: AM6PR08CA0041.eurprd08.prod.outlook.com (2603:10a6:20b:c0::29) To AM9PR09MB4977.eurprd09.prod.outlook.com (2603:10a6:20b:304::20) X-Microsoft-Original-Message-ID: <871r7qt1n7.fsf@live.com> X-MS-Exchange-MessageSentRepresentingType: 1 Original-Received: from pascal.homepc (81.232.177.30) by AM6PR08CA0041.eurprd08.prod.outlook.com (2603:10a6:20b:c0::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.25 via Frontend Transport; Thu, 22 Jul 2021 22:38:36 +0000 X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 46 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: 4d77275d-823b-4452-b81c-08d94d617212 X-MS-TrafficTypeDiagnostic: VI1EUR04HT095: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: X0kTagHuUnG9P5Okz6aCrTpnwurxyFchrYSTjeAaqcH2OkFmH2Ov25J3K4FKZ7AEgyv07tM5BkUdLzsiS3t1C7BNJlqlAuTm23avFCX1IX7ciLSEgJQWGqkceNOjivqZKeUyK07XjTvNPrlN29clgIyAP62l/BLhugsDPglehBkNjc4PvOWOtEoKzxcDPinrQ9KdMom6k1mVmz25lZmE3u4MSHHpiflngFXQjR7S7QkBTrBh617T4g9w2+fN/90hCCtyVePWCKXtVJ6R/YZHmH9s2KRIAilIWjlryK4+O41XRPfjp9L8Fnlf9P79G9YWnU2oC5rimh0iFDxFhsXSIjjYwdv73LUFb3W90v2mImay0WPBwplONgvnnTv2pWwFhxeGN+qXudR9gjiKeiwBHPhUO+OIHB60KXtogMnXt0E9ZrgEDabGY8xxsKp69WbW X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 7gNZdLafZ+vCWofb8YAXgn8TfTmp5Q2P2iATkWItx+vn3ZP8f456jkF6CclKdomWkPUh8YfzKYmJEZBTeWrF5+TmTRgZ2VBo/eiDWvlAAFpAvEvMkNtk9SPHVfFDiZJBikmTn31EMno6S4HRy2AA7w== X-OriginatorOrg: live.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4d77275d-823b-4452-b81c-08d94d617212 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jul 2021 22:38:37.6247 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-AuthSource: VI1EUR04FT049.eop-eur04.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1EUR04HT095 Received-SPF: pass client-ip=40.92.75.109; envelope-from=arthur.miller@live.com; helo=EUR04-VI1-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:271482 Archived-At: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Stefan Monnier writes: >> Not so much, but it is not so much about noticable difference, more >> about not performing unnecessary computation. I agree that simplicity and code clarity is important, on many levels. But maybe we can have the cake and it it too, as you said=20 for wdired? > But Emacs is more often limited by manpower than CPU, so the impact on > code complexity should not be discounted. > >> As the loop progresses, for every dir N, the loop does N-1 path >> comparisons. These are unnecessary comparisons since when we know for >> sure that the string is not added yet. Trend in Emacs seems to go >> towards having more and more packages installed. This removes >> N(N-1)) comparisons. > > I know. But you have to get to a list of at least 2000 elements > before this N=C2=B2 starts making a noticable (0.2s) difference [tested o= n > an old 1GHz Cortex-A8. ] Of course. I agree that number of packages has to be big to see the noticable difference. I don't know though how high it shold be. I do a lot of testing in my own init file, but I can't say anything for sure, since I do some weird stuff that are not representable of "normal" Emacs usage. > By the time the user has 2000 packages installed, there are several > other performance problems that will be more noticeable, I'm afraid. > E.g. the length of `load-path` itself will be a source of slowdown. > > I know 200 packages is fairly common nowadays, so 1000 or more in not an > outlandish idea, and there are indeed performance issues that come up > when we have many packages, but I suspect that we'll need more thorough > changes to tackle this, and in any case it should be driven by concrete > measurements, otherwise we'll waste our time optimizing details that > don't actually matter. Last weekend I tested actually myself to restructure how my packages are loaded. I noticed that init time increased after I added ~100 packages, just for test, and I didn't required anything of that into Emacs. So I tested the idea to put all .elc file into a single place and skipp in entirety this monstrosity of load-path that results after 200 packages are loaded. I got it to work to a degree, it least I got running Emacs, native compiler not complaining and most packages loaded, but I also got some cyclic dependency, notably for dired and semantic of all things, that actually rendered entire session unusable for the most part. I'll leave that for another day when I have some more time. >>> Beside the hypothetical risk that the regexp matches an `add-to-list` t= o >>> an unrelated variable, I think this risks introducing real bugs for >>> packages which use (add-to-list 'load-path ) to add >>> *sub*directories. >> Yes, I was aware of it, but I am not sure how to write the regex, to >> avoid that risk. > > I don't think you can avoid that problem by fixing the regexp, but > rather by `read`ing the sexp and looking at the directory it adds to > make sure it is indeed one of the ones you're adding "by hand". The problem is that .* matches everything but new line as I learned form Emacs Wiki. After fixing regexp to match the newline, it blows the regex stack as warned on the wiki page. I have encountered another problem, I hope you can explain it, because I don't understand what is going on. When I run this piece which actually tries to match adding to load path:=20 (when (re-search-forward rx-path-beg nil t) (goto-char (line-beginning-position)) (setq temp-point (point)) (forward-sexp) (when (search-backward file nil t 1) (goto-char temp-point) (kill-sexp))) In search backward, regardless of which bound I give to re-search-backward or just search-backward, the search failes. My original thought was to use (line-beginning-position) and I also tested this temp-point. Interesting is that same code works fine from M-:. If I setq those variables I need to run that little piece from M-: and position point in a spot before a statement to remove, it works fine, but in the defun it always fails so I had to run it with nil for the bound. Is it something I don't understand there or a bug?=20 (when (re-search-forward rx-path-beg nil t) (goto-char (line-beginning-position)) (setq temp-point (point)) (forward-sexp) (when (search-backward file (line-beginning-position) t 1) (goto-char temp-point) (kill-sexp))) > Or maybe a better approach is to do something like what we do with > `Info-directory-list`, tho it might also be tricky to "get it right". > >>> Of course, we can fix those problems, but unless there's a compelling >>> performance argument, I think it's a waste of time. >> It's quite simple to do, > > It's not if you want to handle correctly the corner cases, as this email > discussion shows. > >> I don't know, at least in theory :). > > Maybe there's a theoretical gain. But there's a very real practical > loss in time spent getting the code to work correctly and maintaining it > in the future. I understand what you mean, it is up to you to decide. The patch does add few lines, but it is not some tricky, advanced code hard to understand. I could also refactor some code out of package-quickstart-refresh into a smaller support defun if that would=20 make things easier to maintain. By the way, I haven't even touched on those empty let closures, that really just produce work for garbage collector in most cases. I think I have seen 2 packages that used some of those two vars that get declared in those let statements. :) --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=package.patch --- ../emacs/lisp/emacs-lisp/package.el 2021-07-11 12:20:59.257657537 +0200 +++ ./package.el 2021-07-22 23:37:15.167925681 +0200 @@ -4137,7 +4137,9 @@ (package-activated-list ()) ;; Make sure we can load this file without load-source-file-function. (coding-system-for-write 'emacs-internal) - (Info-directory-list '(""))) + (Info-directory-list '("")) + (rx-path-beg "^(add-to-list 'load-path (directory-file-name") + path insertion-point temp-point) (dolist (elt package-alist) (condition-case err (package-activate (car elt)) @@ -4155,9 +4157,24 @@ (let ((load-suffixes '(".el" ".elc"))) (locate-library (package--autoloads-file-name pkg)))) (pfile (prin1-to-string file))) + ;; prepare list of all dirs to add to load-path + (push (package-desc-dir pkg) path) (insert "(let ((load-true-file-name " pfile ")\ (load-file-name " pfile "))\n") (insert-file-contents file) + (setq insertion-point (point)) + ;; since we are precomputing and pasting list of package directories + ;; to add to load-path, remove statements to add individual package from + ;; autoloads file + (goto-char insertion-point) + (when (re-search-forward rx-path-beg nil t) + (goto-char (line-beginning-position)) + (setq temp-point (point)) + (forward-sexp) + (when (search-backward file nil t 1) + (goto-char temp-point) + (kill-sexp))) + (goto-char insertion-point) ;; Fixup the special #$ reader form and throw away comments. (while (re-search-forward "#\\$\\|^;\\(.*\n\\)" nil 'move) (unless (nth 8 (syntax-ppss)) @@ -4175,6 +4192,10 @@ (setq Info-directory-list (append ',info-dirs Info-directory-list))) (current-buffer)))) + (goto-char (point-min)) + (forward-line 3) + (insert (concat "\n(setq load-path (nconc '" (prin1-to-string path) " load-path))\n")) + (goto-char (point-max)) ;; Use `\s' instead of a space character, so this code chunk is not ;; mistaken for an actual file-local section of package.el. (insert " --=-=-=--