From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Radon Rosborough Newsgroups: gmane.emacs.devel Subject: Re: Interoperation between package managers Date: Sun, 13 Aug 2017 15:42:48 -0700 Message-ID: References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Trace: blaine.gmane.org 1502664222 3906 195.159.176.226 (13 Aug 2017 22:43:42 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 13 Aug 2017 22:43:42 +0000 (UTC) Cc: emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Aug 14 00:43:36 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dh1bb-0000ba-Ou for ged-emacs-devel@m.gmane.org; Mon, 14 Aug 2017 00:43:35 +0200 Original-Received: from localhost ([::1]:43216 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dh1bi-0001do-AM for ged-emacs-devel@m.gmane.org; Sun, 13 Aug 2017 18:43:42 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:46917) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dh1ba-0001cW-8H for emacs-devel@gnu.org; Sun, 13 Aug 2017 18:43:35 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dh1bY-0006LK-KT for emacs-devel@gnu.org; Sun, 13 Aug 2017 18:43:34 -0400 Original-Received: from mail-lf0-x233.google.com ([2a00:1450:4010:c07::233]:36338) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dh1bY-0006Jr-9G for emacs-devel@gnu.org; Sun, 13 Aug 2017 18:43:32 -0400 Original-Received: by mail-lf0-x233.google.com with SMTP id o85so33263328lff.3 for ; Sun, 13 Aug 2017 15:43:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=FdG8S/qn5stwhNuXt0Hfci7KJnuXdbRNC7w/qGKVDpE=; b=uKPlN7Dfz0/TSlFq0GAXma4/5DAwBSH8P+hwHZFQUFdO8F3bbWZS08jIdcSLNyiM93 U9DxJMkms3QJyIWL/B5FuZuqathovXXjKGINAR29hqDdBRl9gKH65lSeYG/6Vbmc12P1 6eubYpuYrRtOWBRrr4QknlcGAsCG9oI5cEsDi5bB3KbaePb4WRvTE2H16b75JUR8gnul pA1JrY0Z4NDJaaNy7ALrriPFblmtOPuK8FzdfPKg+ZKkdxa7Zx+jx9T5oFsrpN0YGxY1 w1uc1YLhtPsRuxJhAZwWrvp6bjEKWCzKQZ5WyndFO2KxmvmFifZLOx3ei7yIhIpZACHx GNrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=FdG8S/qn5stwhNuXt0Hfci7KJnuXdbRNC7w/qGKVDpE=; b=OITCvSBkXIuSjp1/j3Zk3Rn55NV9hynfQrJed4PnlvmmXykYQbp4vrSazDcDRRKgnE Yxf67cpQWmRPl5xnhHgXkSMQiSZJ4BybyZODJL14vqzqeHWerBmluQQCvGQ7yil+GjLr gVGcfNXDqG5E7aGdfeKUCxp9EHlcBCNWrXohJ4xa4pq5EezxqGOtWr8wiZ1QHJd/6TE7 VVK/BD1JhXO6kg5YhY5ITAAWxu07dzJ3wfiZjDCE7XFPFp+zDk2s9A7Apyo9YvBeSHa1 sqnRynG+zSw/vbbkC5Js7tNiw42/V5m3CcQ5VT/3tzq+Pm1rN/sOv/gqnTTTCwavL/iO wpBw== X-Gm-Message-State: AHYfb5hyOEvLWHXUXMavJDpSsts6oAlEc/B3uDPr+GzNfoAQwOQNPRtG 69KgPNrLRNdRXdZZsH5iBqfGQvCKaQtWdJw= X-Received: by 10.25.27.20 with SMTP id b20mr7399169lfb.131.1502664208767; Sun, 13 Aug 2017 15:43:28 -0700 (PDT) Original-Received: by 10.46.64.86 with HTTP; Sun, 13 Aug 2017 15:42:48 -0700 (PDT) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4010:c07::233 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:217524 Archived-At: > I'd probably want to also unify the two files into one (which would > likely hold the concatenation of -pkg.el and > -autoloads.el). I'm a little wary of this idea. It seems to me like -autoloads.el is purely an implementation detail (or "build artifact") of the package manager, whereas -pkg.el contains author-maintained metadata that should be checked into version control in the upstream repository. To give some perspective on why this might be important, consider how source-based package managers work: * clone upstream repository * inspect -pkg.el (or in-file headers) to determine dependencies, etc. * generate autoloads and byte-compile on the client-side In this workflow it makes no sense to combine the package metadata with the autoloads. Now, straight.el doesn't make any demands on how the upstream formats things, which I consider one of its strengths. So as long as the metadata is available in some kind of standard format, I don't think there will be a problem. Most likely the situation will be similar for other source-based package managers. > I think it's very important to be able to have several versions of a > given package installed at the same time. This is another concept which makes a lot of sense in the tarball-from-ELPA workflow, but not much sense in the source-based workflow (where you don't need to save old versions because you can revert to any version at any time). I don't know what the best solution is for package.el to accommodate both of these workflows (presuming that this is desired). > > I think the biggest problem is the documentation. > > To me, all of that is pretty "obvious" because I've spent enough > time both in the design and in the code, which makes it difficult to > figure out what people might need to know. Yes, that's the problem. I wrote a package manager too, and I have exactly the same experience of everything seeming obvious to me. I think the best solution is to just sit down and explain everything. Have you seen the README for straight.el [1]? That is the kind of documentation that I would like to see from every library I use. (I know it's long; splitting into multiple documents is on my to-do list.) > Specific requests (especially patches) are very welcome here. Unfortunately, I can't contribute patches since I am already too busy maintaining my own package manager. However, I can give some suggestions for things that need to be explained. * What is the format of a package on the server? What is the format on the client? Be specific. * What does package.el do to a package? I know that it generates autoloads and byte-compiles, but most people don't know that. * What does activating a package mean? Again, I know that it entails evaluating the autoloads file and adding a directory to the load path, but most people don't. * Why is package-initialize so slow? What are the bottlenecks when you activate lots of packages? * What exactly is package-refresh-contents doing? Where is it saving things? What are the error conditions if you forget to run it? What commands is it automatically run by? * Why does package-initialize keep getting inserted into my init-file? What's up with package-selected-packages? * How am I supposed to load a single package and its dependencies for a bug report, without loading everything else? * What happens when I need to run a package directly from its Git repository? How do I make local changes to a package installed from package.el? How do I contribute changes upstream? * How do I sync my configuration between machines? How can I make sure that package.el creates a reproducible configuration that won't break if I move my configuration to a new machine? * What's this business with package-install-file? How does it differ from other ways of loading a file into my Emacs configuration? * How would I host my own ELPA server? Would I want to do such a thing? How do I deal with needing to install packages that aren't available in the default archives? * How do I install old versions of packages? Is this even possible? * What's the deal with built-in packages? What does that mean? Are all the lisp/ files built-in packages? None of them? An arbitrary subset? * How do I install a package if there's also a built-in version of it? * What are the known limitations of package.el? Clearly there are a lot of them, judging by the number of FIXME notices in the code. * What are the interactions between installing different versions of packages with different dependencies from different archives? What rules are used to resolve all of this? * What are the various states the package management system can be in? How does calling package-activate and/or package-initialize in various places during init or later behave, especially if the backing data on disk has changed in the interim? * How strict are the version requirement rules? Are they always guaranteed to find a working set of versions if such a set exists? (No, as I found out while looking at FIXME notices in the source code.) This is just a few simple things I thought of. I'm sure if I had a better understanding of package.el, I would have more questions, and more specific ones. Especially given all the features you told me about in the last few emails, which I had no idea even existed, and therefore didn't even consider asking questions about. Maybe package.el is simple, but unless there's a clear statement that "this documentation covers exactly what package.el does", there's always the concern that more magic is going on, given how opaque the source code is. > What (w|c)ould be such a "more comprehensible source of > information"? The problem is that no such source exists at the moment. We need to create one. What I mean by "comprehensible" is that any random Emacs user can visit a nicely formatted modern web page and understand in just a few minutes: * what package.el does * where packages live (on the server, in ~/.emacs.d, etc.) * what package-initialize does * where they should put code to configure packages * commonly-asked questions and troubleshooting information Then after the short summary, there should be comprehensive documentation on everything. Again, see my documentation [1] which I think is a pretty comprehensive description of both the concepts and usage of straight.el. I want my source code to be as readable as prose, but nevertheless try to write my documentation to be such that my users never *have* to look at the source code. I don't think it would be a bad thing if package.el did the same. Best, Radon [1]: https://github.com/raxod502/straight.el