From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Jonas Bernoulli Newsgroups: gmane.emacs.devel Subject: Re: Interoperation between package managers Date: Sat, 12 Aug 2017 22:53:01 +0200 Message-ID: <877ey8cxua.fsf@bernoul.li> References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1502571236 8803 195.159.176.226 (12 Aug 2017 20:53:56 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 12 Aug 2017 20:53:56 +0000 (UTC) User-Agent: mu4e 0.9.19; emacs 25.2.1 Cc: Stefan Monnier , emacs-devel@gnu.org To: Radon Rosborough Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Aug 12 22:53:50 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dgdPm-0001Ux-HN for ged-emacs-devel@m.gmane.org; Sat, 12 Aug 2017 22:53:46 +0200 Original-Received: from localhost ([::1]:49321 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dgdPp-00007W-IX for ged-emacs-devel@m.gmane.org; Sat, 12 Aug 2017 16:53:49 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:46598) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dgdPB-00007F-T2 for emacs-devel@gnu.org; Sat, 12 Aug 2017 16:53:11 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dgdP7-0007Wf-TI for emacs-devel@gnu.org; Sat, 12 Aug 2017 16:53:09 -0400 Original-Received: from mail.hostpark.net ([212.243.197.30]:48348) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dgdP7-0007Uv-JW for emacs-devel@gnu.org; Sat, 12 Aug 2017 16:53:05 -0400 Original-Received: from localhost (localhost [127.0.0.1]) by mail.hostpark.net (Postfix) with ESMTP id F1DEC16544; Sat, 12 Aug 2017 22:53:02 +0200 (CEST) X-Virus-Scanned: by Hostpark/NetZone Mailprotection at hostpark.net Original-Received: from mail.hostpark.net ([127.0.0.1]) by localhost (mail0.hostpark.net [127.0.0.1]) (amavisd-new, port 10124) with ESMTP id 8SXNXjMPSyH7; Sat, 12 Aug 2017 22:53:02 +0200 (CEST) Original-Received: from desktop (77-58-214-193.dclient.hispeed.ch [77.58.214.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.hostpark.net (Postfix) with ESMTPSA id 4A474164B4; Sat, 12 Aug 2017 22:53:01 +0200 (CEST) In-reply-to: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 212.243.197.30 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:217500 Archived-At: >> As mentioned above, I haven't thought about what package.el could do >> to make it easier to automate what I do manually, so if you have >> suggestions for things that we could change in (or add to) >> package.el to make it easier for Borg to interoperate with it, >> please send them along, > > You should talk to Jonas (cc'd) about this as he's the author of Borg. > I can't really provide much useful help beyond this point since I have > no desire to use package.el for anything. (This has gotten a bit long, you may skip to "What Borg needs from package.el".) I would like to eventually allow users to use Borg alongside package.el. The idea behind Borg is that for some people using a package is only the first step, the second is to contribute to it. Borg installs packages by cloning the respective git repository and then byte-compiling the libraries, generating the autoload file, generating the info files, and adding the appropriate directories to `load-path' and `Info-directory-list'. That works surprisingly well, because most packages follow some informal common sense conventions. But Borg also has features to deal with packages that stray for the common path. The main difference from other package managers is that in the case of Borg, those packages are not brought into line at the time a package is created (because there is neither a "package" not a "package maintainer" just the upstream git repository). But like Melpa or El-Get, Bort does support "recipes"; the only difference is when and where the recipe is "applied". In the case of Borg it is applied on the users machine at the time of installation (and update). So as I said, Borg is primarily intended for people who contribute to many of the package that they use. In that sense it is similar to adding the local clone of elpa.git to `package-directory-list. One big difference though is that in the case of Borg each package continues to live in its own repository. It is already possible to use Borg and package at the same time, but I intend to make it a fully supported use-case. All you have to (and can) do currently to use both package managers at the same time is to preserve this order in the init file: (package-initialize) (borg-initialize) This guarantees that Borg wins in case a package is installed using both package.el and Borg. This allows users to install most packages using package.el and only install the packages that they contribute to using Borg. The main benefit of using Borg to *install* some packages instead of just *using* the package.el-installed version and *working* in a manually cloned git repository, is that after restarting Emacs, the locally modified version is loaded without having to manually adjust the `load-path'. Of course there are other benefits, for example `borg-install' only needs the name of a package because it knows the location of thousands of upstream repositories. `magit-clone' on the other hand expects an url. In addition to cloning package repositories, it adds the clones as git submodules of the ~/.emacs.d repository. This makes sense if one only uses Borg, but when also using package.el then probably not so much. This is the main change required on Borg's side to make it more useful for people who also use package.el. (It is already possible to clone a package repository without adding it as a submodule, but currently this also means that Borg forgoes actually installing the package by compiling it and adding it to the `load-path'. --- What Borg needs from package.el? If you initialize package.el before Borg, then the Borg-installed version of a package shadows the package.el-installed version. It would be nicer if package.el could be told "this is a list of packages that are somehow installed, do not install those packages, just assume they are available". I don't think that adding the directory that contains the Borg-installed packages to `package-directory-list' would do because: * That would cause package.el to load those packages, resulting in them being loaded twice. (At least they would be added to the `load-path' twice.) * While some of these directories follow the format expected by package.el, others do not. For example in many repositories libraries live in ./lisp/ instead of ./. So really, all that Borg needs is option `package-assume-installed'. Maybe that already exists, I haven't checked, because I haven't actually gotten around to making Borg package.el compatible yet. --- And since that is what started this whole conversation, I should note that I don't really care that users who use Borg exclusively have to put (setq package-enable-at-startup nil) in their init file. While I haven't followed this conversation, I have thought a bit about how package.el is initialized, and can appreciate how the current solution has grown out of necessity. It tries to satisfy users with very different needs and especially varying levels of understanding. The only thing that annoys me a bit is that when I use `emacs -Q' to debug some issue without my configuration, then I end up with ~/.emacs.d/elpa being created. It would nice if `package-initialize' did not do that and would only *update* the local metadata if it is already there. Later when the user calls `package-list-package' or `package-install', then the initial local copy of the metadata could be downloaded. But package.el is not unique in this regard. When I use `emacs -Q', then some other build in packages also create files in ~/.emacs.d that would be different places if my personal init.el had been loaded. I can live with that and just cleanup manually after an `emacs -Q'.