From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Loading a package applies automatically to future sessions? Date: Tue, 30 Jan 2018 22:47:00 -0500 Message-ID: References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1517370341 2868 195.159.176.226 (31 Jan 2018 03:45:41 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 31 Jan 2018 03:45:41 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Cc: emacs-devel@gnu.org To: George Plymale II Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Jan 31 04:45:36 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1egjKh-0007KR-Tj for ged-emacs-devel@m.gmane.org; Wed, 31 Jan 2018 04:45:12 +0100 Original-Received: from localhost ([::1]:58635 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1egjMi-0001Oj-Le for ged-emacs-devel@m.gmane.org; Tue, 30 Jan 2018 22:47:16 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:40891) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1egjMa-0001Ng-Mc for emacs-devel@gnu.org; Tue, 30 Jan 2018 22:47:10 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1egjMW-0003w1-O7 for emacs-devel@gnu.org; Tue, 30 Jan 2018 22:47:08 -0500 Original-Received: from pruche.dit.umontreal.ca ([132.204.246.22]:38316) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1egjMW-0003uT-Hw for emacs-devel@gnu.org; Tue, 30 Jan 2018 22:47:04 -0500 Original-Received: from pastel.home (lechon.iro.umontreal.ca [132.204.27.242]) by pruche.dit.umontreal.ca (8.14.7/8.14.1) with ESMTP id w0V3l1Yn021932; Tue, 30 Jan 2018 22:47:01 -0500 Original-Received: by pastel.home (Postfix, from userid 20848) id D9DD060529; Tue, 30 Jan 2018 22:47:00 -0500 (EST) In-Reply-To: (George Plymale, II's message of "Tue, 30 Jan 2018 17:33:25 -0500") X-NAI-Spam-Flag: NO X-NAI-Spam-Threshold: 5 X-NAI-Spam-Score: 0 X-NAI-Spam-Rules: 2 Rules triggered EDT_SA_DN_PASS=0, RV6211=0 X-NAI-Spam-Version: 2.3.0.9418 : core <6211> : inlines <6348> : streams <1777586> : uri <2583223> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 132.204.246.22 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:222326 Archived-At: > (setq package-enable-at-startup nil) > (load "~/.emacs.d/package-fastpath.el") > (setq package--initialized t) I think setting package--initialized to t is wrong, here (IIRC package--initialized means that we've filled the tables keeping track of which packages are available and such). Not a problem when you're just trying to test the speed of my proof-of-concept patch, of course. > Unfortunately, I couldn't byte-compile my package-fastpath.el file due > to some packages which already byte-compile their autoload files, such > as SLIME. Hmm... are you saying that my code ends up fetching code for slime-autoloads.elc instead of slime-autoloads.el? Or that slime-autoloads.el contains (pre)compiled code? Also, I'm not 100% surprised that byte-compiling byte-compiled code would fail, but I can't see any immediate reason why it should fail, so it's quite possible to it'd be easy to make it work. > Now, I do see some improvement in my startup time, but to reiterate, it > is not huge. When I run Emacs with my startup commented out, > `emacs-init-time' yields 1.2 seconds. Not quite sure what "my startup commented out" means. I'll assume it means a more or less empty ~/.emacs, or maybe the use of -Q ? In any case I guess it means that 1.2s is the "speed of light", i.e. the holy grail of package.el on your machine. > When I run it with my startup intact, it yields 1.3 or 1.4 > seconds. This contrasts with my previous `emacs-init-time' which was > 1.6 or 1.7 seconds, but as you can see, it's not a really > big difference. If you subtract the 1.2s taken by "other things", it means that your config used to take 0.4-0.5s of startup time, and that the use of a big precomputed autoloads file reduced that to 0.1-0.2s, so sped it up by factor between 2.5 and 4, which I find a bit disappointing but still respectable (and it depends which proportion was taken by package-initialize vs which proportion was taken by your particular customizations). > It is possible that this sub-par time is due to something on my own > system, but I'm not sure nor convinced of that. Moreover, Stefan's patch > certainly can use some improvements and some notes on how to use his > changes, as it is a bit vexing to figure it out by reading the diff ;) Yes, it needs work. It's just a quick experiment to see the potential gains. > I did notice a few bugs with Stefan's changes. One bug was that > `package-installed-p' no longer yields correct results on installed > packages. As it stands, my patch only pre-sets package-activated-list, whereas package-installed-p requires other things set up by package-initialize. If you don't set package--initialized to t, you'll see that package-installed-p will rightfully complain that you haven't called package-initialize. I guess my patch could also add a setting for `package-alist` to the package-fastpath.el, but I'm not sure I like the idea (it increases the consequences of having an out-of-date package-fastpath.el). > I believe that this is because `package-desc-p' is also broken > by these changes. This broke some code in my startup which I used to > check whether I need to install new packages. Wouldn't it be better for your code to assume that if there's a package-fastpath.el, then those checks have already been performed (and to re-execute those checks in package-fastpath-refresh instead)? Or maybe package-installed-p should be changed such that when it's called with a symbol argument, it should first check if it's in package-activated-list, and if it's not, then it should call package-initialize before checking package-alist. > Another bug was that some packages were placed in bad order in > package-fastpath.el. In other words, if a dependency's autoloads are > written to this file after its dependent package, the dependent package > will err, saying that it couldn't require one of its dependencies. Hmm... I thought this can't happen because the files are concatenated in the same order that they are normally loaded by `package-initialize`. I guess something doesn't work the way I thought it does. Can you investigate to see when or even why it happens? > As a side note and a bit of an opinion, Radon Rosborough made an > interesting remark in one of his messages. He mentioned pip and how > things are done in Python, which really struck me. You never really > think about using a package in a language like Python or Ruby. You just > `require' or `import' it and that's that. It's really simple and the > amount of packages that you have never hurts the startup of the main > program. I know that Emacs is a bit more complicated since it's more of > a text editor than a language and we have somewhat more intricacies to > worry about. But I think that this kind of a model is the kind we need > to be headed for. The fact that it's a text editor shouldn't make any difference in this respect. I'm not sure exactly what you mean by "the amount of packages that you have never hurts the startup of the main program", I guess you mean that program A is not slowed down when you install extra packages. In Emacs the corresponding behavior happens if you install a package by hand and don't activate/use it. `package.el` by default tries to auto-activate all your packages, so indeed it's slowed down by the mere presence of extra packages. I think a key element that lets Python and Ruby work that way is the namespacing and its tight binding to the filesystem. Currently, while conventions are "similar" (e.g. function toto-titi is likely found in file toto*.el in package toto), they're just vague conventions. Making them more strict would indeed make it possible to change `require` so that a (require 'toto-tata) will automatically look for a file toto-tata.el in a package toto without having to "activate" that package beforehand and it would let us have a "missing function handler" which would automatically try to find the function toto-titi in one of the files of the titi package (e.g. by activating this package). This might make it possible to activate some packages lazily, but There'd still be aspects of auto-activation that would go missing, tho: e.g. registering jgraph-mode as a handler for *.jgr files. So some packages would still need some form of auto-activation. > A user should be able to have a million packages and not have to worry > about the subsequent startup time. Heck, in my own Ruby installation I > have 436 gems and I've never even thought about startup time till now. The downside is that you have to explicitly `import` those gems when you need them. In Emacs you could do the same: call (package-initialize t) in your ~/.emacs and then call `package-activate` when you need a particular package. > In any case, I think Stefan's ideas and proposed changes are a good idea > and I am in much the same boat as John Wiegley in that I restart Emacs > often enough that startup time gets on my nerves. IIUC there are 1.2s of your startup time which are spent without even running a single line of package.el code, so speeding up package.el might not be good enough (at least it seems to me that if 1.6s is too slow, then 1.2s is likely to also be too slow). The usual answer in Emacs for those problem is "don't do that" (i.e. use emacsclient instead). I understand it's not always an option. Stefan