From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync. Date: Thu, 25 Feb 2021 10:12:37 -0500 Message-ID: References: <20210225102521.11653.64611@vcs0.savannah.gnu.org> <20210225102523.7CEF420B28@vcs0.savannah.gnu.org> <87h7m0z07r.fsf@tcd.ie> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18813"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: emacs-devel@gnu.org, Oleh Krehel To: "Basil L. Contovounesios" Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Feb 25 16:13:37 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lFIKf-0004mp-Ca for ged-emacs-devel@m.gmane-mx.org; Thu, 25 Feb 2021 16:13:37 +0100 Original-Received: from localhost ([::1]:47284 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lFIKe-0000Nc-F3 for ged-emacs-devel@m.gmane-mx.org; Thu, 25 Feb 2021 10:13:36 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:43820) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lFIJt-0008NV-Cp for emacs-devel@gnu.org; Thu, 25 Feb 2021 10:12:49 -0500 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:24480) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lFIJq-0006ZC-JN for emacs-devel@gnu.org; Thu, 25 Feb 2021 10:12:48 -0500 Original-Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id D97BC10022F; Thu, 25 Feb 2021 10:12:44 -0500 (EST) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id E50771000F4; Thu, 25 Feb 2021 10:12:38 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1614265958; bh=JS6cXle+0AxceLt7tP4lojlIuo5IgAgRQKL4F1YFrFw=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=L/Axi29TkrrbzZYk794tJWr3WYAl4mn72q9KAMi+9tjxGsh4ua07V6H6LmuSY8rRN NhpNBuHXlKpzcQUFhQkFed+H1c+hWuGFIW6GftMFLLJPkWfhVl0B87N7wrc7vx5wWk GexfQX6wleYu8R43WNDzVDiHeU3mjv9gMxdeJIeHCaTbP9U66Oi/dWCaZI1eSKE+9V CTVVGUyDyoHmYB+U1uGWUhqOtav+VzX2u3+H6m+CmtXbGNknidkCr++8o2o6k1iq1O bv5jD6EIsm0cxWFl4XdDHAfk/MP/xRPGjIQ1KjVvIItgU+Ct0dYiuOSZToN6gKYZUW iC+psB65Ni3KA== Original-Received: from alfajor (unknown [216.154.41.47]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 869E612023F; Thu, 25 Feb 2021 10:12:38 -0500 (EST) In-Reply-To: <87h7m0z07r.fsf@tcd.ie> (Basil L. Contovounesios's message of "Thu, 25 Feb 2021 14:33:12 +0000") Received-SPF: pass client-ip=132.204.25.50; envelope-from=monnier@iro.umontreal.ca; helo=mailscanner.iro.umontreal.ca X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:265622 Archived-At: >> * elpa-packages (counsel, ivy, swiper): Auto-sync. > Sorry, I realised only after the fact that the corresponding externals > in elpa.git had significant local changes in the form of deleted files, > diverging Version headers, etc., so I'm guessing :auto-sync won't work > for these packages. Indeed. When I saw that commit this morning I thought (naively?) that maybe you had sorted it out :-( > Should I revert this commit in the meantime? I think so, yes (it's not terribly harmful because the sync will just fail with "can't fast forward", but it's not a very valuable use of resources ;-) > It gets even more complicated: the existing ivy-hydra.el and the new > ivy-avy.el in swiper.git were apparently designed as optional separate > packages, so I've now created the corresponding external branches in > elpa.git (but not yet added them to elpa-packages). > > What is the best way to handle this complexity (that of having multiple > packages provided by a single upstream repository) in elpa.git? > > For example, should swiper.git have separate branches for each package's > subtree, so that elpa.git can (hopefully) just fetch and fast-forward to > them automatically? > > Should things continue as they are now, with elpa.git deleting files > locally and swiper.git blissfully unaware of the fact? Good question. The scripts used for GNU ELPA were designed with a dual-use in mind, where OT1H they can take the elpa.git and generate the ELPA tarballs and OTOH they can take an elpa.git clone and build the packages for use in-place. The "use in place" feature is not used very commonly, but it is the most natural use case for the developer (which reduces the difference between what the developer uses and what the users use, thus reducing the risk for errors introduced by the packaging) so I find it important. To me the two best options are: - Split the upstream repository so we get one repository per ELPA package. - Get rid of the split between the ELPA packages (so you again get one ELPA package per repository). Another option is to have separate manually-synced branches in the upstream, one per ELPA package. This is basically the same as what we have currently, except that the manual syncing is done between the "main" branch" and the "for-elpa branches" (all within the upstream repository) rather than between the main branch in the upstream and the elpa branches in `elpa.git`. To me, it seems to bring no benefit, but I guess depending on your workflow (and access rights) it could make a difference. There's another option, which I dislike and introduces inefficiencies, which is to make all those packages have the exact same content (so they all have the same upstream branch and we can fast-forward it) and then rely on the `:ignored-files` parameter in `elpa-packages` to filter out the files we don't want in the tarballs. Downsides: - A full elpa.git checkout (like the one elpa.gnu.org keeps to build the tarballs, or the one I keep on my machines to "install in place" all the GNU ELPA packages) would contain duplicate copies. - The `:ignored-files` doesn't have any notion of "negation" so you can't say "ignore all but counsel.el", making those lists of ignored files annoying to maintain. - For the "install in place" case, those duplicate copies get redundantly byte-compiled as well. - For the "install in place" case, the "ivy.el" file loaded by Emacs may not be the one I think: instead of `packages/ivy/ivy.el` the `load-path` may direct Emacs to choose `packages/counsel/ivy.el` instead (so I'd have to be careful to ask Emacs which file it's using before I start hacking on it, lest I'd start modifying one file which turns out not be used, which could lead me to tearing out my hair for a while). > Or maybe adding something like MELPA's ability to specify which exact > files to consider from the upstream repository? That could be added, and could reduce the burden of managing the `:ignored-filed`, yes. It'd likely have to be done in this part of the elpa-admin.el code: (apply #'elpaa--call nil "tar" `("--exclude-vcs" ,@(cond (ignores (mapcar (lambda (i) (format "--exclude=packages/%s/%s" pkgname i)) ignores)) ((file-readable-p elpaignore) `("-X" ,elpaignore))) ,@(mapcar (lambda (r) (elpaa--make-tar-transform pkgname r)) renames) "--transform" ,(format "s|^packages/%s|%s-%s|" pkgname pkgname vers) "-chf" ,tarball ,(concat "packages/" pkgname))) right at the end here, you'd probably want to list each part individually instead of telling tar to include the whole subdirectory. Stefan