From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Radon Rosborough Newsgroups: gmane.emacs.devel Subject: Re: Interoperation between package managers Date: Sun, 13 Aug 2017 17:29:15 -0700 Message-ID: References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Trace: blaine.gmane.org 1502670660 16066 195.159.176.226 (14 Aug 2017 00:31:00 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 14 Aug 2017 00:31:00 +0000 (UTC) Cc: emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Aug 14 02:30:55 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dh3HP-0003Sz-6F for ged-emacs-devel@m.gmane.org; Mon, 14 Aug 2017 02:30:51 +0200 Original-Received: from localhost ([::1]:52689 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dh3HQ-00062s-Sn for ged-emacs-devel@m.gmane.org; Sun, 13 Aug 2017 20:30:52 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:36753) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dh3GZ-00061b-4o for emacs-devel@gnu.org; Sun, 13 Aug 2017 20:30:00 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dh3GX-0005vb-K8 for emacs-devel@gnu.org; Sun, 13 Aug 2017 20:29:59 -0400 Original-Received: from mail-lf0-x22a.google.com ([2a00:1450:4010:c07::22a]:36064) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dh3GX-0005ub-86 for emacs-devel@gnu.org; Sun, 13 Aug 2017 20:29:57 -0400 Original-Received: by mail-lf0-x22a.google.com with SMTP id o85so33684666lff.3 for ; Sun, 13 Aug 2017 17:29:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=D3d6+pDFAFAnWN8goCtLMSDjI6HdiHpHBMjDsggfUoU=; b=IHpRBv1cpwcW7zVrBxDrj/WKH1x27IU6/mEQCNBBChP2xT0Kanw1odYdbzG0G3WwT+ +5N/5/2ovhORR0e0obOgJ4w9DtwLo196TTiHrSTSYqn5KmzDlrR2pGIbLLJO25NlDHy8 0ew7oUNDOrvrgZmFHd/cHW1BM9PfYkFPGyX6qnS+AYJ7JK8RhKt+lW8e4sEPJrxXoNaH d97T3eg9koeXquiJmhSX1ZR+r+wYQQF/h+FqpyNk3IgAjwmiD0LHQpONLSLqRALgc0sk Jtky+ivbHDXiB+VbyNJHM4Uc/EZV1jUt7dTl2fDaf6f04JEsECJgpp+woV5v+zDnIdNe xkYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=D3d6+pDFAFAnWN8goCtLMSDjI6HdiHpHBMjDsggfUoU=; b=kClP3VTbbIcFrA0MyRovAF7tXv/brmC5QBa/Y0AnFW7XYQu/+iArjnuZnARG/W1wR2 krVI06NMNUlUqRl65agq9DS8xmKwcjjWvbyatQpc6HroDqXRr2RI1hYwJBEQP6uWnqE1 kTpH3HgFHdY0lUoAwyKnFdHX/TWSaw4+YZCAhA31Sl+RJPi4p2BIeZnKNkCeoJ95OkCd SOmr47RrnsNNVaVZmjxvwZRfCUQJqfVAcPPlKYXu3o3WvPj6J+RHiDvHqWvNh0/tklfk PvqNt3N1vae4ra0eqSFxsF6vw1NQ4tV5KXrVRowzqzUUBMI3F29mBNNWviary8bRnlvz wNgw== X-Gm-Message-State: AHYfb5gXf9tABxvzowtaHC4Q1MHbqSmevDKskYg9JICTnGuCYCFA87nI +elZBvlAP7s4Z4zUErmMuwkUVKR6Ww== X-Received: by 10.46.2.212 with SMTP id y81mr1485057lje.161.1502670595679; Sun, 13 Aug 2017 17:29:55 -0700 (PDT) Original-Received: by 10.46.64.86 with HTTP; Sun, 13 Aug 2017 17:29:15 -0700 (PDT) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4010:c07::22a X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:217528 Archived-At: > My view is that -pkg.el should be built from metadata stored > elsewhere (that's what we do in elpa.git). That contradicts common usage, as reflected in the following Emacs packages which ship a -pkg.el file containing their metadata: $ find . -name '*-pkg.el' ./cucumber.el/feature-mode-pkg.el ./ecukes/ecukes-pkg.el ./elisp-finalize/finalize-pkg.el ./emacs-async/async-pkg.el ./emacs-eclim/eclim-pkg.el ./ESS/lisp/ess-pkg.el ./evil/evil-pkg.el ./haskell-mode/haskell-mode-pkg.el ./markdown-toc/markdown-toc-pkg.el ./multiple-cursors.el/multiple-cursors-pkg.el ./skewer-mode/skewer-mode-pkg.el ./smartparens/smartparens-pkg.el ./zone-matrix/zone-matrix-pkg.el The current manual [1] implies that it is the responsibility of the upstream package maintainer to create a -pkg.el file when packaging their package. If that wasn't the intention, the documentation should make this clear. But at this point, I would advise against trying to enforce a new convention, given how useful it is to have a consistent one established. > I was thinking about it in the context of packages which can come > from the user's config as well as from the system (i.e. installed by > the sysadmin). In this case, wouldn't the multiple versions be installed in entirely different directories anyway? > There are other cases where it can make sense. E.g. have both > foo-1.0 and foo-2.0 installed at the same time, because foo-2.0 > requires Emacs-25 and you use both Emacs-25 and Emacs-24. Again, in package.el, what you say makes perfect sense. But in a source-based package manager, it seems better to handle this by having the package manager check out the correct revision. > I agree that these needs aren't the most common ones, but you don't > gain anything by disallowing them, really. I think you gain something by enforcing a simpler directory naming convention. But of course this is still only relevant to source-based package managers, and not to package.el. > These look like info about the internals, and indeed we don't > document them very much. Some of them are purposefully not > documented ("use the source, 'cause it might change"). But there's a > a lot of room for improvement anyway. I'll see if I can improve on > that. Sounds good. I agree that these things are sort-of internal, but the problem is that users frequently encounter problems with package.el ("try deleting your ~/.emacs.d/elpa directory"; "try running package-refresh-contents again"; "you put package-initialize in the wrong place"), so it's important to have documentation that allows debugging such problems. IMO, the package installation and loading process doesn't need to be complicated. Everyone should be able to understand it. (In other words, "it should only be a black box if it always works".) > AFAIK this is documented somewhere. The "somewhere" part is a big part of the issue. If I want to know something about package.el, maybe I should go to the Packages section of the manual, or maybe the Packaging section, or maybe elpa.gnu.org, or maybe ELPA's README, or maybe the Commentary in the source code of package.el, or the docstrings, or the comments ??? This is very confusing for a new user who only wants to know why their packages aren't being loaded. Regarding the format of packages on the server, I think this documentation is supposed to be at [2], but it doesn't seem to actually be there. > Actually "adding a directory to the load path" is only there by > accident: it should be done by the autoloads file. Does that mean M-x update-directory-autoloads will start inserting load-path manipulation calls into the generated autoload files? Or should package.el roll its own autoload generation routine? > > * Why is package-initialize so slow? > > I don't know that it is, so I can't answer the question. > > > What are the bottlenecks when you > > activate lots of packages? > > I haven't looked into. If you find performance problems, > M-x report-emacs-bug is probably the best option, so we can try to find > how to improve the situation. Sorry, I didn't mean to imply there was a bug. I just meant that package-initialize is slow when you have lots and lots of packages, simply because a lot of work is being done. The problem is that package-initialize is a black box, so the user doesn't know where to start if they want to optimize their init. It'd be helpful to know if the bottleneck was: * evaluating autoloads * processing dependencies * loading caches * activating packages that are no longer referenced in the init-file ?? * etc I know at least one person who was frustrated that a quarter of their init-file was a black-box package-initialize call (at least before they switched to straight.el). > The source code of package.el is not supposed to be opaque. I wasn't > the original author and I didn't find it opaque when I looked at it. > So if it's opaque, maybe it's my fault, but I'm probably not the > right guy to fix it, then ;-) When I look at package.el, I see things like: * not all the functions and variables have docstrings * quite a few FIXME notices * a method called "package--make-autoloads-and-stuff" (???) * a lack of high-level explanation as to how things are organized and what the various formats are Maybe if I worked with package.el more, I would not be as confused. And maybe I have unrealistic expectations. But I think it would really help if there were a comprehensive explanation of exactly how package.el thinks about package installation and loading. For reference, [3] is the corresponding explanation for straight.el. Things like the meaning of "package", "installation", "deletion", "version", "activate", etc. most likely seem obvious to everyone working on package.el, but in fact there are many different ways to handle these things (they are done very differently in straight.el than in package.el, for example), so documentation is really necessary. [1]: https://www.gnu.org/software/emacs/manual/html_node/elisp/Multi_002dfile-Packages.html [2]: https://www.gnu.org/software/emacs/manual/html_node/elisp/Package-Archives.html#Package-Archives [3]: https://github.com/raxod502/straight.el#conceptual-overview