Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Basil L. Contovounesios]

blc@savannah.gnu.org (Basil L. Contovounesios) writes:

> branch: main
> commit 8f4cb595fb1f34059b22a3f3e0ef71cb310e08d5
> Author: Basil L. Contovounesios <contovob@tcd.ie>
> Commit: Basil L. Contovounesios <contovob@tcd.ie>
>
>     * elpa-packages (counsel, ivy, swiper): Auto-sync.

Sorry, I realised only after the fact that the corresponding externals
in elpa.git had significant local changes in the form of deleted files,
diverging Version headers, etc., so I'm guessing :auto-sync won't work
for these packages.  Should I revert this commit in the meantime?

It gets even more complicated: the existing ivy-hydra.el and the new
ivy-avy.el in swiper.git were apparently designed as optional separate
packages, so I've now created the corresponding external branches in
elpa.git (but not yet added them to elpa-packages).

What is the best way to handle this complexity (that of having multiple
packages provided by a single upstream repository) in elpa.git?

For example, should swiper.git have separate branches for each package's
subtree, so that elpa.git can (hopefully) just fetch and fast-forward to
them automatically?

Should things continue as they are now, with elpa.git deleting files
locally and swiper.git blissfully unaware of the fact?

Either way, it currently looks to me like there will always be manual
merging required in either swiper.git or elpa.git, with no auto-sync,
fast-foward, DTRT option.

Is there a better way, e.g. reverting the local deletions in elpa.git,
keeping 5 full copies of swiper.git as externals (one for each package),
and tightening each package's :ignored-files entry?

Or maybe adding something like MELPA's ability to specify which exact
files to consider from the upstream repository?

I'd be grateful for any guidance in how to proceed with the update of
ivy, ivy-avy, ivy-hydra, swiper, and counsel in elpa.git.

Thanks,

-- 
Basil

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Stefan Monnier]

>>     * elpa-packages (counsel, ivy, swiper): Auto-sync.
> Sorry, I realised only after the fact that the corresponding externals
> in elpa.git had significant local changes in the form of deleted files,
> diverging Version headers, etc., so I'm guessing :auto-sync won't work
> for these packages.

Indeed.  When I saw that commit this morning I thought (naively?)
that maybe you had sorted it out :-(

> Should I revert this commit in the meantime?

I think so, yes (it's not terribly harmful because the sync will just
fail with "can't fast forward", but it's not a very valuable use of
resources ;-)

> It gets even more complicated: the existing ivy-hydra.el and the new
> ivy-avy.el in swiper.git were apparently designed as optional separate
> packages, so I've now created the corresponding external branches in
> elpa.git (but not yet added them to elpa-packages).
>
> What is the best way to handle this complexity (that of having multiple
> packages provided by a single upstream repository) in elpa.git?
>
> For example, should swiper.git have separate branches for each package's
> subtree, so that elpa.git can (hopefully) just fetch and fast-forward to
> them automatically?
>
> Should things continue as they are now, with elpa.git deleting files
> locally and swiper.git blissfully unaware of the fact?

Good question.  The scripts used for GNU ELPA were designed with
a dual-use in mind, where OT1H they can take the elpa.git and generate
the ELPA tarballs and OTOH they can take an elpa.git clone and build the
packages for use in-place.

The "use in place" feature is not used very commonly, but it is the most
natural use case for the developer (which reduces the difference between
what the developer uses and what the users use, thus reducing the risk
for errors introduced by the packaging) so I find it important.

To me the two best options are:

- Split the upstream repository so we get one repository per ELPA package.

- Get rid of the split between the ELPA packages (so you again get one
  ELPA package per repository).

Another option is to have separate manually-synced branches in the
upstream, one per ELPA package.  This is basically the same as what we
have currently, except that the manual syncing is done between the "main"
branch" and the "for-elpa branches" (all within the upstream repository)
rather than between the main branch in the upstream and the elpa
branches in `elpa.git`.  To me, it seems to bring no benefit, but
I guess depending on your workflow (and access rights) it could make
a difference.

There's another option, which I dislike and introduces inefficiencies,
which is to make all those packages have the exact same content (so they
all have the same upstream branch and we can fast-forward it) and then
rely on the `:ignored-files` parameter in `elpa-packages` to filter out the
files we don't want in the tarballs.
Downsides:
- A full elpa.git checkout (like the one elpa.gnu.org keeps to build the tarballs, or the
  one I keep on my machines to "install in place" all the GNU ELPA packages)
  would contain duplicate copies.
- The `:ignored-files` doesn't have any notion of "negation" so you
  can't say "ignore all but counsel.el", making those lists of
  ignored files annoying to maintain.
- For the "install in place" case, those duplicate copies get
  redundantly byte-compiled as well.
- For the "install in place" case, the "ivy.el" file loaded by Emacs
  may not be the one I think: instead of `packages/ivy/ivy.el` the
  `load-path` may direct Emacs to choose `packages/counsel/ivy.el`
  instead (so I'd have to be careful to ask Emacs which file it's using
  before I start hacking on it, lest I'd start modifying one file which
  turns out not be used, which could lead me to tearing out my hair for
  a while).

> Or maybe adding something like MELPA's ability to specify which exact
> files to consider from the upstream repository?

That could be added, and could reduce the burden of managing the
`:ignored-filed`, yes.

It'd likely have to be done in this part of the elpa-admin.el code:

        (apply #'elpaa--call
               nil "tar"
               `("--exclude-vcs"
                 ,@(cond
                    (ignores
                     (mapcar (lambda (i) (format "--exclude=packages/%s/%s" pkgname i))
                             ignores))
                    ((file-readable-p elpaignore) `("-X" ,elpaignore)))
                 ,@(mapcar (lambda (r) (elpaa--make-tar-transform pkgname r))
                           renames)
                 "--transform"
                 ,(format "s|^packages/%s|%s-%s|" pkgname pkgname vers)
                 "-chf" ,tarball
                 ,(concat "packages/" pkgname)))

right at the end here, you'd probably want to list each part
individually instead of telling tar to include the whole subdirectory.


        Stefan

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Basil L. Contovounesios]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> To me the two best options are:
>
> - Split the upstream repository so we get one repository per ELPA package.

No objections from me, but that's Oleh's call and I think
ivy/swiper/counsel are too tightly coupled anyway.

> - Get rid of the split between the ELPA packages (so you again get one
>   ELPA package per repository).

This sounds like the path of least resistance for everyone, and I'm in
favour of it, but let's see what Oleh thinks.

This would involve deleting the other packages, though, right?  (Since
there would otherwise be multiple different copies of the same files in
elpa.git.)  Wouldn't users complain?

> Another option is to have separate manually-synced branches in the
> upstream, one per ELPA package.  This is basically the same as what we
> have currently, except that the manual syncing is done between the "main"
> branch" and the "for-elpa branches" (all within the upstream repository)
> rather than between the main branch in the upstream and the elpa
> branches in `elpa.git`.  To me, it seems to bring no benefit, but
> I guess depending on your workflow (and access rights) it could make
> a difference.

For better or worse I have access rights in both repositories, and I
don't mind maintaining these branches in swiper.git.  To me, the main
benefit of this would be not having to switch between different
repositories for development or Git trickery, and then elpa.git would
remain more declarative.  But ultimately I'm happy to do either.

BTW, is it possible/kosher to force-push to external branches in
elpa.git?

> There's another option, which I dislike and introduces inefficiencies,
> which is to make all those packages have the exact same content (so they
> all have the same upstream branch and we can fast-forward it) and then
> rely on the `:ignored-files` parameter in `elpa-packages` to filter out the
> files we don't want in the tarballs.
> Downsides:
> - A full elpa.git checkout (like the one elpa.gnu.org keeps to build the tarballs, or the
>   one I keep on my machines to "install in place" all the GNU ELPA packages)
>   would contain duplicate copies.
> - The `:ignored-files` doesn't have any notion of "negation" so you
>   can't say "ignore all but counsel.el", making those lists of
>   ignored files annoying to maintain.
> - For the "install in place" case, those duplicate copies get
>   redundantly byte-compiled as well.
> - For the "install in place" case, the "ivy.el" file loaded by Emacs
>   may not be the one I think: instead of `packages/ivy/ivy.el` the
>   `load-path` may direct Emacs to choose `packages/counsel/ivy.el`
>   instead (so I'd have to be careful to ask Emacs which file it's using
>   before I start hacking on it, lest I'd start modifying one file which
>   turns out not be used, which could lead me to tearing out my hair for
>   a while).

Anything to protect your hair!

>> Or maybe adding something like MELPA's ability to specify which exact
>> files to consider from the upstream repository?
>
> That could be added, and could reduce the burden of managing the
> `:ignored-filed`, yes.

It would be nice to have, but it doesn't address the hair tearing issue,
so I won't be bothering with it just yet.

Thanks,

-- 
Basil

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Stefan Monnier]

> BTW, is it possible/kosher to force-push to external branches in elpa.git?

It's strongly discouraged (since there are users and a few automated
systems which track those branches) but possible.  So if necessary it
can be considered for a "one time thing" if it comes with significant
long term benefits.

> Anything to protect your hair!

OTOH I think it's healthy enough to withstand such attacks anyway.
[ Beside, Covid made me miss my yearly appointment with my hair dresser
  (who see me rarely enough that they doesn't know they're my hair
  dresser) before the summer, so I have a fair bit of "excess hair".  ]


        Stefan

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Basil L. Contovounesios]

[-- Attachment #1: Type: text/plain, Size: 1574 bytes --]

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> Another option is to have separate manually-synced branches in the
>> upstream, one per ELPA package.  This is basically the same as what we
>> have currently, except that the manual syncing is done between the "main"
>> branch" and the "for-elpa branches" (all within the upstream repository)
>> rather than between the main branch in the upstream and the elpa
>> branches in `elpa.git`.  To me, it seems to bring no benefit, but
>> I guess depending on your workflow (and access rights) it could make
>> a difference.
>
> For better or worse I have access rights in both repositories, and I
> don't mind maintaining these branches in swiper.git.  To me, the main
> benefit of this would be not having to switch between different
> repositories for development or Git trickery, and then elpa.git would
> remain more declarative.  But ultimately I'm happy to do either.

I've now bumped all the swiper.git versions to 0.13.3, and merged the
upstream changes into the five corresponding branches in elpa.git.

The elpa-admin release detection doesn't seem happy with it though:
'make build/ivy' git-resets to the literal commit in swiper.git that
changed the Version header, which means it discards all the local
deletions etc. in elpa.git.  I attach a debug log of this in action,
where the presence of files from swiper.git that were meant to be
deleted in elpa.git causes the copyright check to barf.

The following seems to fix the revision detection locally:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Handle-merges-when-looking-for-release-revision.patch --]
[-- Type: text/x-diff, Size: 1334 bytes --]

From 5d2120a287ae781e778e05383d8e8d223c49999d Mon Sep 17 00:00:00 2001
From: "Basil L. Contovounesios" <contovob@tcd.ie>
Date: Tue, 9 Mar 2021 21:53:34 +0000
Subject: [PATCH] Handle merges when looking for release revision

For some discussion, see the following thread:
https://lists.gnu.org/r/emacs-devel/2021-02/msg01935.html

* elpa-admin.el (elpaa--get-release-revision): Follow only the first
parent of merge commits.  This handles cases where the Version
header is updated in a different branch to the current one, as is
the case with, e.g., ivy/swiper/counsel, whose external branches in
elpa.git have local deletions not present in the upstream.
---
 elpa-admin.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/elpa-admin.el b/elpa-admin.el
index 5cb3d261c5..17b3bb1bd4 100644
--- a/elpa-admin.el
+++ b/elpa-admin.el
@@ -170,7 +170,7 @@ elpaa--get-release-revision
                      (elpaa--call
                       (current-buffer)
                       "git" "log" "-n1" "--oneline" "--no-patch"
-                      "--pretty=format:%H"
+                      "--pretty=format:%H" "--first-parent"
                       "-L" (concat "/^;;* *\\(Package-\\)\\?Version:/,+1:"
                                    (elpaa--main-file pkg-spec))))
                     (buffer-string)
-- 
2.30.1


[-- Attachment #3: Type: text/plain, Size: 372 bytes --]


It seems to work with ivy and ivy-avy, which introduce releases via
merge commits, as well as dash, which doesn't.  WDYT?

Is there a better/different way?  The only alternative I can think of is
to enter revisions manually in the :version-map, but that doesn't seem
desirable in the long run.

P.S. Sorry if this breaks anything on elpa.gnu.org :(.

Thanks,

-- 
Basil


[-- Attachment #4: Ivy build log --]
[-- Type: text/plain, Size: 26484 bytes --]

-*- mode: compilation; default-directory: "~/.local/src/elpa/" -*-
Compilation started at Tue Mar  9 21:51:00

make build/ivy
emacs --batch -l /home/blc/.local/src/elpa/admin/elpa-admin.el	\
         -f elpaa-batch-make-one-package ivy
Checking package ivy for updates...
ivy up-to-date
pkg-spec for ivy: ("ivy" :url "https://github.com/abo-abo/swiper" :renames (("doc/" "")) :doc "doc/ivy.texi")
metadata = (nil "0.13.3" "Incremental Vertical completYon" ((emacs (24 5))) ((:authors ("Oleh Krehel" . "ohwoeowho@gmail.com")) (:maintainer "Oleh Krehel" . "ohwoeowho@gmail.com") (:keywords "matching") (:url . "https://github.com/abo-abo/swiper")))
call-process git ("show" "--pretty=format:%cI" "--no-patch")
call-process date ("-d" "2021-03-09T19:30:39+00:00" "+%Y%m%d.%H%M%S")
Building tarball archive-devel/ivy-0.13.3.0.20210309.193039.tar...
call-process git ("status" "--porcelain")
call-process git ("clean" "-x" "-d" "-f")
call-sandboxed ("makeinfo" "--no-split" "doc/ivy.texi" "-o" "ivy.info")
call-process bwrap ("--unshare-all" "--dev" "/dev" "--proc" "/proc" "--tmpfs" "/tmp" "--ro-bind" "/etc/emacs" "/etc/emacs" "--ro-bind" "/etc/alternatives" "/etc/alternatives" "--ro-bind" "/usr" "/usr" "--ro-bind" "/bin" "/bin" "--ro-bind" "/lib64" "/lib64" "--ro-bind" "/lib" "/lib" "--bind" "/home/blc/.local/src/elpa/packages/ivy/" "/home/blc/.local/src/elpa/packages/ivy/" "makeinfo" "--no-split" "doc/ivy.texi" "-o" "ivy.info")

call-sandboxed ("install-info" "--dir=/home/blc/.local/src/elpa/packages/ivy/dir" "ivy.info")
call-process bwrap ("--unshare-all" "--dev" "/dev" "--proc" "/proc" "--tmpfs" "/tmp" "--ro-bind" "/etc/emacs" "/etc/emacs" "--ro-bind" "/etc/alternatives" "/etc/alternatives" "--ro-bind" "/usr" "/usr" "--ro-bind" "/bin" "/bin" "--ro-bind" "/lib64" "/lib64" "--ro-bind" "/lib" "/lib" "--bind" "/home/blc/.local/src/elpa/packages/ivy/" "/home/blc/.local/src/elpa/packages/ivy/" "install-info" "--dir=/home/blc/.local/src/elpa/packages/ivy/dir" "ivy.info")

call-process tar ("--exclude-vcs" "-X" "/home/blc/.local/src/elpa/packages/ivy/.elpaignore" "--transform=s|^packages/ivy/doc\\($\\|/\\)|packages/ivy/|" "--transform" "s|^packages/ivy|ivy-0.13.3.0.20210309.193039|" "-chf" "archive-devel/ivy-0.13.3.0.20210309.193039.tar" "packages/ivy")
ivy: (ivy . [(0 13 3 0 20210309 193039) ((emacs (24 5))) "Incremental Vertical completYon" tar ((:url . "https://github.com/abo-abo/swiper") (:keywords "matching") (:maintainer "Oleh Krehel" . "ohwoeowho@gmail.com") (:authors ("Oleh Krehel" . "ohwoeowho@gmail.com")))])
current AC: (1)
new AC: (1 (ivy . [(0 13 3 0 20210309 193039) ((emacs (24 5))) "Incremental Vertical completYon" tar ((:url . "https://github.com/abo-abo/swiper") (:keywords "matching") (:maintainer "Oleh Krehel" . "ohwoeowho@gmail.com") (:authors ("Oleh Krehel" . "ohwoeowho@gmail.com")))]))
maint=("Oleh Krehel" . "ohwoeowho@gmail.com")
Built new package archive-devel/ivy-0.13.3.0.20210309.193039.tar!
Deleting temp files: ("/home/blc/.local/src/elpa/packages/ivy/ivy-pkg.el" "/home/blc/.local/src/elpa/packages/ivy/dir" "/home/blc/.local/src/elpa/packages/ivy/ivy.info" (closure ((temp-buffer . #<killed buffer>) (generated-files) (dir . "/home/blc/.local/src/elpa/packages/ivy") elpaa--temp-files t) nil (let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (let* ((default-directory (elpaa--dirname dir))) (elpaa--call t "git" "clean" "-x" "-d" "-f") (elpaa--call t "git" "checkout" "--" ".")) (elpaa--message "%s" (buffer-string))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))))
call-process git ("clean" "-x" "-d" "-f")
call-process git ("checkout" "--" ".")

Building tarball archive/ivy-0.13.3.tar...
call-process git ("status" "--porcelain")
call-process git ("clean" "-x" "-d" "-f")
call-process git ("log" "-n1" "--oneline" "--no-patch" "--pretty=format:%H" "-L" "/^;;* *\\(Package-\\)\\?Version:/,+1:ivy.el")
Found release rev: "9eb63d5b48f1958f814a4d684b8498c12a7abe6b"
call-process git ("reset" "--merge" "9eb63d5b48f1958f814a4d684b8498c12a7abe6b")
Reverted to release revision 9eb63d5b48f1958f814a4d684b8498c12a7abe6b

Problem with copyright notices:
Missing copyright notice in ivy/targets/plain.el
Missing copyright notice in ivy/targets/obsolete-config.el
Missing copyright notice in ivy/targets/install-deps.el
Missing copyright notice in ivy/targets/checkdoc.el
Missing copyright notice in ivy/elpa.el
Debugger entered--Lisp error: (error "Abort")
  signal(error ("Abort"))
  error("Abort")
  (progn (message "Problem with copyright notices:\n%s" (mapconcat #'(lambda (line) (if (string-match ":" line) line (concat "Missing copyright notice in " line))) filtered "\n")) (error "Abort"))
  (if filtered (progn (message "Problem with copyright notices:\n%s" (mapconcat #'(lambda (line) (if (string-match ":" line) line (concat "Missing copyright notice in " line))) filtered "\n")) (error "Abort")))
  (let* ((collected (elpaa--copyright-collect pkg-spec)) (filtered (elpaa--copyright-filter collected))) (if filtered (progn (message "Problem with copyright notices:\n%s" (mapconcat #'(lambda (line) (if ... line ...)) filtered "\n")) (error "Abort"))))
  (progn (let* ((collected (elpaa--copyright-collect pkg-spec)) (filtered (elpaa--copyright-filter collected))) (if filtered (progn (message "Problem with copyright notices:\n%s" (mapconcat #'(lambda ... ...) filtered "\n")) (error "Abort")))))
  (if (file-readable-p elpaa--copyright-file) (progn (let* ((collected (elpaa--copyright-collect pkg-spec)) (filtered (elpaa--copyright-filter collected))) (if filtered (progn (message "Problem with copyright notices:\n%s" (mapconcat #'... filtered "\n")) (error "Abort"))))))
  elpaa--copyright-check(("ivy" :url "https://github.com/abo-abo/swiper" :renames (("doc/" "")) :doc "doc/ivy.texi"))
  (let* ((destdir (file-name-directory tarball)) (pkgname (car pkg-spec)) (_ (if (and destdir (not (file-directory-p destdir))) (progn (make-directory destdir)))) (vers (nth 1 metadata)) (elpaignore (expand-file-name ".elpaignore" dir)) (ignores (elpaa--spec-get pkg-spec :ignored-files)) (renames (elpaa--spec-get pkg-spec :renames)) (ldir (elpaa--spec-get pkg-spec :lisp-dir)) (re (concat "\\`" (regexp-quote pkgname) "-\\([0-9].*\\)\\.\\(tar\\|el\\)\\(\\.[a-z]*z\\)?\\'")) (oldtarballs (if one-tarball nil (mapcar #'(lambda (file) (string-match re file) (cons ... file)) (directory-files destdir nil re))))) (if ldir (progn (setq renames (let* ((va (list ... ""))) (if (member va renames) renames (cons va renames)))))) (if revision-function (progn (elpaa--select-revision dir pkg-spec (funcall revision-function)))) (elpaa--copyright-check pkg-spec) (elpaa--make pkg-spec dir) (elpaa--build-Info pkg-spec dir) (elpaa--write-pkg-file dir pkgname metadata) (progn (or (not (string-match "[][*\\|?]" pkgname)) (cl--assertion-failed '(not (string-match "[][*\\|?]" pkgname)))) nil) (progn (or (not (string-match "[][*\\|?]" vers)) (cl--assertion-failed '(not (string-match "[][*\\|?]" vers)))) nil) (apply #'elpaa--call nil "tar" (cons "--exclude-vcs" (append (cond (ignores (mapcar #'... ignores)) ((file-readable-p elpaignore) (list "-X" elpaignore))) (mapcar #'(lambda (r) (elpaa--make-tar-transform pkgname r)) renames) (list "--transform" (format "s|^packages/%s|%s-%s|" pkgname pkgname vers) "-chf" tarball (concat "packages/" pkgname))))) (if one-tarball nil (let* ((pkgdesc (elpaa--process-multi-file-package dir pkgname 'dont-rename))) (elpaa--message "%s: %S" pkgname pkgdesc) (elpaa--update-archive-contents pkgdesc destdir) (if (and nil revision-function) (progn (let ((default-directory ...)) (elpaa--call nil "git" "tag" "-f" (format "%s-release/%s-%s" elpaa--name pkgname vers))))) (let ((link (expand-file-name (format "%s.tar" pkgname) destdir))) (if (file-symlink-p link) (progn (delete-file link))) (make-symbolic-link (file-name-nondirectory tarball) link)) (setq oldtarballs (elpaa--prune-old-tarballs tarball oldtarballs destdir)) (let* ((default-directory (expand-file-name destdir))) (elpaa--html-make-pkg pkgdesc pkg-spec (cons (cons vers (file-name-nondirectory tarball)) oldtarballs) dir)) (message "Built new package %s!" tarball) 'new)))
  (closure ((one-tarball) (revision-function closure ((tarball . "archive/ivy-0.13.3.tar") (new . new) (tarball . "archive-devel/ivy-0.13.3.0.20210309.193039.tar") (devel-vers . "0.13.3.0.20210309.193039") (date-version . "20210309.193039") (vers . "0.13.3") (metadata nil "0.13.3" "Incremental Vertical completYon" ((emacs (24 5))) ((:authors ("Oleh Krehel" . "ohwoeowho@gmail.com")) (:maintainer "Oleh Krehel" . "ohwoeowho@gmail.com") (:keywords "matching") (:url . "https://github.com/abo-abo/swiper"))) (_ . "pkg-spec for ivy: (\"ivy\" :url \"https://github.com/...") (_) (dir . "/home/blc/.local/src/elpa/packages/ivy") (pkgname . "ivy") (one-tarball) (pkg-spec "ivy" :url "https://github.com/abo-abo/swiper" :renames (("doc/" "")) :doc "doc/ivy.texi") elpaa--temp-files t) nil (elpaa--get-release-revision dir pkg-spec vers (plist-get (cdr pkg-spec) :version-map))) (metadata nil "0.13.3" "Incremental Vertical completYon" ((emacs (24 5))) ((:authors ("Oleh Krehel" . "ohwoeowho@gmail.com")) (:maintainer "Oleh Krehel" . "ohwoeowho@gmail.com") (:keywords "matching") (:url . "https://github.com/abo-abo/swiper"))) (pkg-spec "ivy" :url "https://github.com/abo-abo/swiper" :renames (("doc/" "")) :doc "doc/ivy.texi") (dir . "/home/blc/.local/src/elpa/packages/ivy") (tarball . "archive/ivy-0.13.3.tar") elpaa--temp-files t) nil (let* ((destdir (file-name-directory tarball)) (pkgname (car pkg-spec)) (_ (if (and destdir (not (file-directory-p destdir))) (progn (make-directory destdir)))) (vers (nth 1 metadata)) (elpaignore (expand-file-name ".elpaignore" dir)) (ignores (elpaa--spec-get pkg-spec :ignored-files)) (renames (elpaa--spec-get pkg-spec :renames)) (ldir (elpaa--spec-get pkg-spec :lisp-dir)) (re (concat "\\`" (regexp-quote pkgname) "-\\([0-9].*\\)\\.\\(tar\\|el\\)\\(\\.[a-z]*z\\)?\\'")) (oldtarballs (if one-tarball nil (mapcar #'(lambda ... ... ...) (directory-files destdir nil re))))) (if ldir (progn (setq renames (let* ((va ...)) (if (member va renames) renames (cons va renames)))))) (if revision-function (progn (elpaa--select-revision dir pkg-spec (funcall revision-function)))) (elpaa--copyright-check pkg-spec) (elpaa--make pkg-spec dir) (elpaa--build-Info pkg-spec dir) (elpaa--write-pkg-file dir pkgname metadata) (progn (or (not (string-match "[][*\\|?]" pkgname)) (cl--assertion-failed '(not (string-match "[][*\\|?]" pkgname)))) nil) (progn (or (not (string-match "[][*\\|?]" vers)) (cl--assertion-failed '(not (string-match "[][*\\|?]" vers)))) nil) (apply #'elpaa--call nil "tar" (cons "--exclude-vcs" (append (cond (ignores (mapcar ... ignores)) ((file-readable-p elpaignore) (list "-X" elpaignore))) (mapcar #'(lambda ... ...) renames) (list "--transform" (format "s|^packages/%s|%s-%s|" pkgname pkgname vers) "-chf" tarball (concat "packages/" pkgname))))) (if one-tarball nil (let* ((pkgdesc (elpaa--process-multi-file-package dir pkgname 'dont-rename))) (elpaa--message "%s: %S" pkgname pkgdesc) (elpaa--update-archive-contents pkgdesc destdir) (if (and nil revision-function) (progn (let (...) (elpaa--call nil "git" "tag" "-f" ...)))) (let ((link (expand-file-name ... destdir))) (if (file-symlink-p link) (progn (delete-file link))) (make-symbolic-link (file-name-nondirectory tarball) link)) (setq oldtarballs (elpaa--prune-old-tarballs tarball oldtarballs destdir)) (let* ((default-directory (expand-file-name destdir))) (elpaa--html-make-pkg pkgdesc pkg-spec (cons (cons vers ...) oldtarballs) dir)) (message "Built new package %s!" tarball) 'new))))()
  funcall((closure ((one-tarball) (revision-function closure ((tarball . "archive/ivy-0.13.3.tar") (new . new) (tarball . "archive-devel/ivy-0.13.3.0.20210309.193039.tar") (devel-vers . "0.13.3.0.20210309.193039") (date-version . "20210309.193039") (vers . "0.13.3") (metadata nil "0.13.3" "Incremental Vertical completYon" ((emacs ...)) ((:authors ...) (:maintainer "Oleh Krehel" . "ohwoeowho@gmail.com") (:keywords "matching") (:url . "https://github.com/abo-abo/swiper"))) (_ . "pkg-spec for ivy: (\"ivy\" :url \"https://github.com/...") (_) (dir . "/home/blc/.local/src/elpa/packages/ivy") (pkgname . "ivy") (one-tarball) (pkg-spec "ivy" :url "https://github.com/abo-abo/swiper" :renames (("doc/" "")) :doc "doc/ivy.texi") elpaa--temp-files t) nil (elpaa--get-release-revision dir pkg-spec vers (plist-get (cdr pkg-spec) :version-map))) (metadata nil "0.13.3" "Incremental Vertical completYon" ((emacs (24 5))) ((:authors ("Oleh Krehel" . "ohwoeowho@gmail.com")) (:maintainer "Oleh Krehel" . "ohwoeowho@gmail.com") (:keywords "matching") (:url . "https://github.com/abo-abo/swiper"))) (pkg-spec "ivy" :url "https://github.com/abo-abo/swiper" :renames (("doc/" "")) :doc "doc/ivy.texi") (dir . "/home/blc/.local/src/elpa/packages/ivy") (tarball . "archive/ivy-0.13.3.tar") elpaa--temp-files t) nil (let* ((destdir (file-name-directory tarball)) (pkgname (car pkg-spec)) (_ (if (and destdir (not ...)) (progn (make-directory destdir)))) (vers (nth 1 metadata)) (elpaignore (expand-file-name ".elpaignore" dir)) (ignores (elpaa--spec-get pkg-spec :ignored-files)) (renames (elpaa--spec-get pkg-spec :renames)) (ldir (elpaa--spec-get pkg-spec :lisp-dir)) (re (concat "\\`" (regexp-quote pkgname) "-\\([0-9].*\\)\\.\\(tar\\|el\\)\\(\\.[a-z]*z\\)?\\'")) (oldtarballs (if one-tarball nil (mapcar #'... (directory-files destdir nil re))))) (if ldir (progn (setq renames (let* (...) (if ... renames ...))))) (if revision-function (progn (elpaa--select-revision dir pkg-spec (funcall revision-function)))) (elpaa--copyright-check pkg-spec) (elpaa--make pkg-spec dir) (elpaa--build-Info pkg-spec dir) (elpaa--write-pkg-file dir pkgname metadata) (progn (or (not (string-match "[][*\\|?]" pkgname)) (cl--assertion-failed '(not ...))) nil) (progn (or (not (string-match "[][*\\|?]" vers)) (cl--assertion-failed '(not ...))) nil) (apply #'elpaa--call nil "tar" (cons "--exclude-vcs" (append (cond (ignores ...) (... ...)) (mapcar #'... renames) (list "--transform" (format "s|^packages/%s|%s-%s|" pkgname pkgname vers) "-chf" tarball (concat "packages/" pkgname))))) (if one-tarball nil (let* ((pkgdesc (elpaa--process-multi-file-package dir pkgname ...))) (elpaa--message "%s: %S" pkgname pkgdesc) (elpaa--update-archive-contents pkgdesc destdir) (if (and nil revision-function) (progn (let ... ...))) (let ((link ...)) (if (file-symlink-p link) (progn ...)) (make-symbolic-link (file-name-nondirectory tarball) link)) (setq oldtarballs (elpaa--prune-old-tarballs tarball oldtarballs destdir)) (let* ((default-directory ...)) (elpaa--html-make-pkg pkgdesc pkg-spec (cons ... oldtarballs) dir)) (message "Built new package %s!" tarball) 'new)))))
  (progn (elpaa--clean dir) (funcall f))
  (unwind-protect (progn (elpaa--clean dir) (funcall f)) (elpaa--message "Deleting temp files: %S" elpaa--temp-files) (let ((--dolist-tail-- elpaa--temp-files)) (while --dolist-tail-- (let ((f (car --dolist-tail--))) (if (stringp f) (delete-file f) (funcall f)) (setq --dolist-tail-- (cdr --dolist-tail--))))))
  (let ((elpaa--temp-files nil)) (unwind-protect (progn (elpaa--clean dir) (funcall f)) (elpaa--message "Deleting temp files: %S" elpaa--temp-files) (let ((--dolist-tail-- elpaa--temp-files)) (while --dolist-tail-- (let ((f (car --dolist-tail--))) (if (stringp f) (delete-file f) (funcall f)) (setq --dolist-tail-- (cdr --dolist-tail--)))))))
  elpaa--call-with-temp-files("/home/blc/.local/src/elpa/packages/ivy" (closure ((one-tarball) (revision-function closure ((tarball . "archive/ivy-0.13.3.tar") (new . new) (tarball . "archive-devel/ivy-0.13.3.0.20210309.193039.tar") (devel-vers . "0.13.3.0.20210309.193039") (date-version . "20210309.193039") (vers . "0.13.3") (metadata nil "0.13.3" "Incremental Vertical completYon" ((emacs ...)) ((:authors ...) (:maintainer "Oleh Krehel" . "ohwoeowho@gmail.com") (:keywords "matching") (:url . "https://github.com/abo-abo/swiper"))) (_ . "pkg-spec for ivy: (\"ivy\" :url \"https://github.com/...") (_) (dir . "/home/blc/.local/src/elpa/packages/ivy") (pkgname . "ivy") (one-tarball) (pkg-spec "ivy" :url "https://github.com/abo-abo/swiper" :renames (("doc/" "")) :doc "doc/ivy.texi") elpaa--temp-files t) nil (elpaa--get-release-revision dir pkg-spec vers (plist-get (cdr pkg-spec) :version-map))) (metadata nil "0.13.3" "Incremental Vertical completYon" ((emacs (24 5))) ((:authors ("Oleh Krehel" . "ohwoeowho@gmail.com")) (:maintainer "Oleh Krehel" . "ohwoeowho@gmail.com") (:keywords "matching") (:url . "https://github.com/abo-abo/swiper"))) (pkg-spec "ivy" :url "https://github.com/abo-abo/swiper" :renames (("doc/" "")) :doc "doc/ivy.texi") (dir . "/home/blc/.local/src/elpa/packages/ivy") (tarball . "archive/ivy-0.13.3.tar") elpaa--temp-files t) nil (let* ((destdir (file-name-directory tarball)) (pkgname (car pkg-spec)) (_ (if (and destdir (not ...)) (progn (make-directory destdir)))) (vers (nth 1 metadata)) (elpaignore (expand-file-name ".elpaignore" dir)) (ignores (elpaa--spec-get pkg-spec :ignored-files)) (renames (elpaa--spec-get pkg-spec :renames)) (ldir (elpaa--spec-get pkg-spec :lisp-dir)) (re (concat "\\`" (regexp-quote pkgname) "-\\([0-9].*\\)\\.\\(tar\\|el\\)\\(\\.[a-z]*z\\)?\\'")) (oldtarballs (if one-tarball nil (mapcar #'... (directory-files destdir nil re))))) (if ldir (progn (setq renames (let* (...) (if ... renames ...))))) (if revision-function (progn (elpaa--select-revision dir pkg-spec (funcall revision-function)))) (elpaa--copyright-check pkg-spec) (elpaa--make pkg-spec dir) (elpaa--build-Info pkg-spec dir) (elpaa--write-pkg-file dir pkgname metadata) (progn (or (not (string-match "[][*\\|?]" pkgname)) (cl--assertion-failed '(not ...))) nil) (progn (or (not (string-match "[][*\\|?]" vers)) (cl--assertion-failed '(not ...))) nil) (apply #'elpaa--call nil "tar" (cons "--exclude-vcs" (append (cond (ignores ...) (... ...)) (mapcar #'... renames) (list "--transform" (format "s|^packages/%s|%s-%s|" pkgname pkgname vers) "-chf" tarball (concat "packages/" pkgname))))) (if one-tarball nil (let* ((pkgdesc (elpaa--process-multi-file-package dir pkgname ...))) (elpaa--message "%s: %S" pkgname pkgdesc) (elpaa--update-archive-contents pkgdesc destdir) (if (and nil revision-function) (progn (let ... ...))) (let ((link ...)) (if (file-symlink-p link) (progn ...)) (make-symbolic-link (file-name-nondirectory tarball) link)) (setq oldtarballs (elpaa--prune-old-tarballs tarball oldtarballs destdir)) (let* ((default-directory ...)) (elpaa--html-make-pkg pkgdesc pkg-spec (cons ... oldtarballs) dir)) (message "Built new package %s!" tarball) 'new)))))
  (if (or (file-readable-p tarball) (file-readable-p (replace-regexp-in-string "\\.tar\\'" ".el" tarball))) (progn (elpaa--message "Tarball %s already built!" tarball) nil) (elpaa--call-with-temp-files dir #'(lambda nil (let* ((destdir (file-name-directory tarball)) (pkgname (car pkg-spec)) (_ (if ... ...)) (vers (nth 1 metadata)) (elpaignore (expand-file-name ".elpaignore" dir)) (ignores (elpaa--spec-get pkg-spec :ignored-files)) (renames (elpaa--spec-get pkg-spec :renames)) (ldir (elpaa--spec-get pkg-spec :lisp-dir)) (re (concat "\\`" ... "-\\([0-9].*\\)\\.\\(tar\\|el\\)\\(\\.[a-z]*z\\)?\\'")) (oldtarballs (if one-tarball nil ...))) (if ldir (progn (setq renames ...))) (if revision-function (progn (elpaa--select-revision dir pkg-spec ...))) (elpaa--copyright-check pkg-spec) (elpaa--make pkg-spec dir) (elpaa--build-Info pkg-spec dir) (elpaa--write-pkg-file dir pkgname metadata) (progn (or (not ...) (cl--assertion-failed ...)) nil) (progn (or (not ...) (cl--assertion-failed ...)) nil) (apply #'elpaa--call nil "tar" (cons "--exclude-vcs" (append ... ... ...))) (if one-tarball nil (let* (...) (elpaa--message "%s: %S" pkgname pkgdesc) (elpaa--update-archive-contents pkgdesc destdir) (if ... ...) (let ... ... ...) (setq oldtarballs ...) (let* ... ...) (message "Built new package %s!" tarball) 'new))))))
  elpaa--make-one-tarball("archive/ivy-0.13.3.tar" "/home/blc/.local/src/elpa/packages/ivy" ("ivy" :url "https://github.com/abo-abo/swiper" :renames (("doc/" "")) :doc "doc/ivy.texi") (nil "0.13.3" "Incremental Vertical completYon" ((emacs (24 5))) ((:authors ("Oleh Krehel" . "ohwoeowho@gmail.com")) (:maintainer "Oleh Krehel" . "ohwoeowho@gmail.com") (:keywords "matching") (:url . "https://github.com/abo-abo/swiper"))) (closure ((tarball . "archive/ivy-0.13.3.tar") (new . new) (tarball . "archive-devel/ivy-0.13.3.0.20210309.193039.tar") (devel-vers . "0.13.3.0.20210309.193039") (date-version . "20210309.193039") (vers . "0.13.3") (metadata nil "0.13.3" "Incremental Vertical completYon" ((emacs (24 5))) ((:authors ("Oleh Krehel" . "ohwoeowho@gmail.com")) (:maintainer "Oleh Krehel" . "ohwoeowho@gmail.com") (:keywords "matching") (:url . "https://github.com/abo-abo/swiper"))) (_ . "pkg-spec for ivy: (\"ivy\" :url \"https://github.com/...") (_) (dir . "/home/blc/.local/src/elpa/packages/ivy") (pkgname . "ivy") (one-tarball) (pkg-spec "ivy" :url "https://github.com/abo-abo/swiper" :renames (("doc/" "")) :doc "doc/ivy.texi") elpaa--temp-files t) nil (elpaa--get-release-revision dir pkg-spec vers (plist-get (cdr pkg-spec) :version-map))))
  (if (elpaa--make-one-tarball tarball dir pkg-spec metadata #'(lambda nil (elpaa--get-release-revision dir pkg-spec vers (plist-get (cdr pkg-spec) :version-map)))) (progn (elpaa--release-email pkg-spec metadata dir)))
  (let ((tarball (concat elpaa--release-subdir (format "%s-%s.tar" pkgname vers)))) (if (elpaa--make-one-tarball tarball dir pkg-spec metadata #'(lambda nil (elpaa--get-release-revision dir pkg-spec vers (plist-get (cdr pkg-spec) :version-map)))) (progn (elpaa--release-email pkg-spec metadata dir))))
  (cond (one-tarball nil) ((or (equal vers "0") (member '-4 (version-to-list vers))) (cond ((equal vers "0") (elpaa--message "Package %s not released yet!" pkgname)) ((not new) (elpaa--message "Nothing new for package %s!" pkgname)) (t (let* ((last-rel (elpaa--get-last-release pkg-spec)) (tarball (concat elpaa--release-subdir ...)) (metadata (cons nil ...))) (if (not last-rel) (elpaa--message "Package %s not released yet!" pkgname) (if (elpaa--make-one-tarball tarball dir pkg-spec metadata ...) (progn ...))))))) (t (let ((tarball (concat elpaa--release-subdir (format "%s-%s.tar" pkgname vers)))) (if (elpaa--make-one-tarball tarball dir pkg-spec metadata #'(lambda nil (elpaa--get-release-revision dir pkg-spec vers ...))) (progn (elpaa--release-email pkg-spec metadata dir))))))
  (let* ((date-version (elpaa--get-devel-version dir pkg-spec)) (devel-vers (concat vers (if (string-match "[0-9]\\'" vers) ".") "0." date-version)) (tarball (or one-tarball (concat elpaa--devel-subdir (format "%s-%s.tar" pkgname devel-vers)))) (new (let ((elpaa--name (concat elpaa--name "-devel"))) (elpaa--make-one-tarball tarball dir pkg-spec (cons nil (cons devel-vers (nthcdr 2 metadata))) nil one-tarball)))) (cond (one-tarball nil) ((or (equal vers "0") (member '-4 (version-to-list vers))) (cond ((equal vers "0") (elpaa--message "Package %s not released yet!" pkgname)) ((not new) (elpaa--message "Nothing new for package %s!" pkgname)) (t (let* ((last-rel ...) (tarball ...) (metadata ...)) (if (not last-rel) (elpaa--message "Package %s not released yet!" pkgname) (if ... ...)))))) (t (let ((tarball (concat elpaa--release-subdir (format "%s-%s.tar" pkgname vers)))) (if (elpaa--make-one-tarball tarball dir pkg-spec metadata #'(lambda nil ...)) (progn (elpaa--release-email pkg-spec metadata dir)))))))
  (if (null metadata) (error "No metadata found for package: %s" pkgname) (let* ((v metadata)) (setcar v nil)) (let* ((date-version (elpaa--get-devel-version dir pkg-spec)) (devel-vers (concat vers (if (string-match "[0-9]\\'" vers) ".") "0." date-version)) (tarball (or one-tarball (concat elpaa--devel-subdir (format "%s-%s.tar" pkgname devel-vers)))) (new (let ((elpaa--name (concat elpaa--name "-devel"))) (elpaa--make-one-tarball tarball dir pkg-spec (cons nil (cons devel-vers ...)) nil one-tarball)))) (cond (one-tarball nil) ((or (equal vers "0") (member '-4 (version-to-list vers))) (cond ((equal vers "0") (elpaa--message "Package %s not released yet!" pkgname)) ((not new) (elpaa--message "Nothing new for package %s!" pkgname)) (t (let* (... ... ...) (if ... ... ...))))) (t (let ((tarball (concat elpaa--release-subdir ...))) (if (elpaa--make-one-tarball tarball dir pkg-spec metadata #'...) (progn (elpaa--release-email pkg-spec metadata dir))))))))
  (let* ((pkgname (car pkg-spec)) (dir (expand-file-name pkgname "packages")) (_ (cond (one-tarball nil) ((eq (nth 1 pkg-spec) :core) (elpaa--core-package-sync pkg-spec)) (t (elpaa--worktree-sync pkg-spec)))) (_ (elpaa--message "pkg-spec for %s: %S" pkgname pkg-spec)) (metadata (elpaa--metadata dir pkg-spec)) (vers (nth 1 metadata))) (elpaa--message "metadata = %S" metadata) (if (null metadata) (error "No metadata found for package: %s" pkgname) (let* ((v metadata)) (setcar v nil)) (let* ((date-version (elpaa--get-devel-version dir pkg-spec)) (devel-vers (concat vers (if (string-match "[0-9]\\'" vers) ".") "0." date-version)) (tarball (or one-tarball (concat elpaa--devel-subdir (format "%s-%s.tar" pkgname devel-vers)))) (new (let ((elpaa--name ...)) (elpaa--make-one-tarball tarball dir pkg-spec (cons nil ...) nil one-tarball)))) (cond (one-tarball nil) ((or (equal vers "0") (member '-4 (version-to-list vers))) (cond ((equal vers "0") (elpaa--message "Package %s not released yet!" pkgname)) ((not new) (elpaa--message "Nothing new for package %s!" pkgname)) (t (let* ... ...)))) (t (let ((tarball ...)) (if (elpaa--make-one-tarball tarball dir pkg-spec metadata ...) (progn ...))))))))
  elpaa--make-one-package(("ivy" :url "https://github.com/abo-abo/swiper" :renames (("doc/" "")) :doc "doc/ivy.texi"))
  (while command-line-args-left (elpaa--make-one-package (elpaa--get-package-spec (car-safe (prog1 command-line-args-left (setq command-line-args-left (cdr command-line-args-left)))))))
  elpaa-batch-make-one-package()
  command-line-1(("-l" "/home/blc/.local/src/elpa/admin/elpa-admin.el" "-f" "elpaa-batch-make-one-package" "ivy"))
  command-line()
  normal-top-level()

make: *** [GNUmakefile:21: build/ivy] Error 255

Compilation exited abnormally with code 2 at Tue Mar  9 21:51:01

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Stefan Monnier]

> I've now bumped all the swiper.git versions to 0.13.3, and merged the
> upstream changes into the five corresponding branches in elpa.git.
>
> The elpa-admin release detection doesn't seem happy with it though:

Indeed.  It would want first merging and then updating the `Version:` line.

> The following seems to fix the revision detection locally:

[...]

> @@ -170,7 +170,7 @@ elpaa--get-release-revision
>                       (elpaa--call
>                        (current-buffer)
>                        "git" "log" "-n1" "--oneline" "--no-patch"
> -                      "--pretty=format:%H"
> +                      "--pretty=format:%H" "--first-parent"
>                        "-L" (concat "/^;;* *\\(Package-\\)\\?Version:/,+1:"
>                                     (elpaa--main-file pkg-spec))))
>                      (buffer-string)

[...]

> It seems to work with ivy and ivy-avy, which introduce releases via
> merge commits, as well as dash, which doesn't.  WDYT?

The problem is that which parent is "first" is very much arbitrary in
Git (contrary to Bzr where this was made much more visible), so this is
just a heuristic that will fail in other cases.

> Is there a better/different way?  The only alternative I can think of is
> to enter revisions manually in the :version-map, but that doesn't seem
> desirable in the long run.

There are many different ways, some of which are likely better in some
respects, but I'm not sure which is the better one overall.

Suggestions:

- change the versions after merging rather than before (as mentioned
  above).  It's an easy low-tech solution, but it involves N times more
  work if a single .git upstream version leads to N downstream releases.

- if there's really only one version number shared by all the
  sub-packages, I'd tend to argue that maybe there should only be one
  package ;-)

- as discussed earlier, we could have each subpackage have the complete
  upstream Git and only create the subpackages via `elpa-package`
  filtering out the other files.

- split the upstream repository.

- add some other way than :version-map to specify which commit to use.
  This could be a new :version-first-parent boolean option, or maybe
  a "don't look for commit in history" (after all, that's what the old
  GNU ELPA script used to do: it used whichever commit was
  HEAD when we discovered that the `Version:` "had changed" (meaning
  the version in HEAD corresponds to not-yet-built file)).

> P.S. Sorry if this breaks anything on elpa.gnu.org :(.

I haven't noticed any problem so far.


        Stefan

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Basil L. Contovounesios]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Suggestions:
>
> - change the versions after merging rather than before (as mentioned
>   above).  It's an easy low-tech solution, but it involves N times more
>   work if a single .git upstream version leads to N downstream releases.

O(2N) is still O(N) though, so not much worse than the status quo.  I've
gone with this for now so that the packages at least function properly.

[Hopefully this is the last version bump this month...]

> - if there's really only one version number shared by all the
>   sub-packages, I'd tend to argue that maybe there should only be one
>   package ;-)

I tend to agree ;).  Not my call to make, though, and that ship is
either getting smaller or sailing away.

> - as discussed earlier, we could have each subpackage have the complete
>   upstream Git and only create the subpackages via `elpa-package`
>   filtering out the other files.

That would be nice to have in general, but it doesn't solve the
inefficiency when multiple packages are bundled from the same repo.

Unless, of course, we either stop caring about the inefficiency (run out
of hair to pull?) or significantly complicate the Git dance, e.g.:
https://stackoverflow.com/q/600079/3084001

> - split the upstream repository.

Not my call to make.

> - add some other way than :version-map to specify which commit to use.
>   This could be a new :version-first-parent boolean option, or maybe

This would obviously "fix" part of my use-case, but I don't know how
good a solution it is.

>   a "don't look for commit in history" (after all, that's what the old
>   GNU ELPA script used to do: it used whichever commit was
>   HEAD when we discovered that the `Version:` "had changed" (meaning
>   the version in HEAD corresponds to not-yet-built file)).

The bump-after-merge works better for me personally, as I don't have to
worry about when to push which commits (I just have to repeat the same
dance 5 times, which at least is somewhat streamlined thanks to the
Project and Magit packages).

I guess another alternative to :version-map would be Git tags?
E.g. using an N.N.N-elpa scheme or something like that.

Thanks,

-- 
Basil

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Stefan Monnier]

>> - change the versions after merging rather than before (as mentioned
>>   above).  It's an easy low-tech solution, but it involves N times more
>>   work if a single .git upstream version leads to N downstream releases.
> O(2N) is still O(N) though, so not much worse than the status quo.

;-)

>> - as discussed earlier, we could have each subpackage have the complete
>>   upstream Git and only create the subpackages via `elpa-package`
>>   filtering out the other files.
> That would be nice to have in general, but it doesn't solve the
> inefficiency when multiple packages are bundled from the same repo.

I don't understand the "when ..." qualification, since my suggestion is
only meaningful in that precise case.  As for the inefficiency: as long
as the upstream is not enormous and that we don't have too many packages
following that model, it's bearable.

> Unless, of course, we either stop caring about the inefficiency (run out
> of hair to pull?) or significantly complicate the Git dance, e.g.:
> https://stackoverflow.com/q/600079/3084001

No subtree: I really mean the whole Git branch being duplicated
will-nilly for each subpackage.

>> - if there's really only one version number shared by all the
>>   sub-packages, I'd tend to argue that maybe there should only be one
>>   package ;-)
> I tend to agree ;).  Not my call to make, though, and that ship is
> either getting smaller or sailing away.
>> - split the upstream repository.
> Not my call to make.

Of course it can also be a mix of the two and it can be done in bits and
pieces: every bit helps.  So maybe we can start by lobbying the upstream
for one specific subpackage we identified as being a good candidate for
either a separate upstream repository or for having the subpackage be
merged with another.

>>   a "don't look for commit in history" (after all, that's what the old
>>   GNU ELPA script used to do: it used whichever commit was
>>   HEAD when we discovered that the `Version:` "had changed" (meaning
>>   the version in HEAD corresponds to not-yet-built file)).
>
> The bump-after-merge works better for me personally, as I don't have to
> worry about when to push which commits.

I know what you mean: that's exactly why the new code goes to the
trouble of checking the commit that modified the `Version:` line.
It's all too common for me to want to bump the version right before
installing experimental new code.

> I guess another alternative to :version-map would be Git tags?
> E.g. using an N.N.N-elpa scheme or something like that.

Fetching tags from upstream is problematic (because we have a single
elpa.git repository for all packages), but we could use tags manually
pushed to elpa.git, yes.


        Stefan

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Basil L. Contovounesios]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> - as discussed earlier, we could have each subpackage have the complete
>>>   upstream Git and only create the subpackages via `elpa-package`
>>>   filtering out the other files.
>> That would be nice to have in general, but it doesn't solve the
>> inefficiency when multiple packages are bundled from the same repo.
> I don't understand the "when ..." qualification, since my suggestion is
> only meaningful in that precise case.  As for the inefficiency: as long
> as the upstream is not enormous and that we don't have too many packages
> following that model, it's bearable.

If that's acceptable (albeit discouraged), then that could work, yes.

>>> - if there's really only one version number shared by all the
>>>   sub-packages, I'd tend to argue that maybe there should only be one
>>>   package ;-)
>> I tend to agree ;).  Not my call to make, though, and that ship is
>> either getting smaller or sailing away.
>>> - split the upstream repository.
>> Not my call to make.
>
> Of course it can also be a mix of the two and it can be done in bits and
> pieces: every bit helps.  So maybe we can start by lobbying the upstream
> for one specific subpackage we identified as being a good candidate for
> either a separate upstream repository or for having the subpackage be
> merged with another.

Consider this the lobby, then :).  (Oleh is already CCed.)

Just today I had to bump only ivy-hydra's version, because it was
unnecessarily requiring a newer version of hydra that hasn't yet been
pushed to elpa.git.

The packages ivy-avy and ivy-hydra are considered optional integrations
with the separate packages avy and hydra, respectively, so they seem
like the best candidates for splitting into separate packages (or
merging with the avy and hydra packages, I think).

ivy, swiper, and counsel are too coupled to split, I think.  So my vote
is for either distributing ivy, swiper, counsel, ivy-avy, and ivy-hydra
as a single superpackage that mirrors swiper.git development, or
splitting ivy-avy and ivy-hydra out of swiper.git.

>> I guess another alternative to :version-map would be Git tags?
>> E.g. using an N.N.N-elpa scheme or something like that.
>
> Fetching tags from upstream is problematic (because we have a single
> elpa.git repository for all packages), but we could use tags manually
> pushed to elpa.git, yes.

I was thinking of the latter indeed, as a more "Git-native" way of
pointing to a release.

I can't promise to implement any of these features for another few
months though (at least not without a guilty conscience ;).

Thanks,

-- 
Basil

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Stefan Monnier]

> The packages ivy-avy and ivy-hydra are considered optional integrations
> with the separate packages avy and hydra, respectively, so they seem
> like the best candidates for splitting into separate packages (or
> merging with the avy and hydra packages, I think).

`ivy-hydra` seems to be "extra `ivy` functionality, using a hydra",
rather than "`ivy` support for hydras", so I don't think it belongs with
`hydra.el`.

OTOH, I think it should be included into `ivy`.  IIUC this is currently
not done because it would require adding `hydra` as a dependency to
`ivy`, but my counter argument would be that the better option is to add
to `hydra` a new *function* to replace/complement the `defhydra`
*macro*, so that `ivy-hydra.el` can be included into `ivy` (and compiled
correctly) without needing to add a dependency on `hydra`.

For `ivy-avy` I think the argument is almost the same, except I don't
see any reason why it isn't done yet (there doesn't seem to be any need
for `avy` to be around to compile correctly `ivy-avy`).

So, let's start with this: Could we merge `ivy-avy` into `ivy`?

>> Fetching tags from upstream is problematic (because we have a single
>> elpa.git repository for all packages), but we could use tags manually
>> pushed to elpa.git, yes.
> I was thinking of the latter indeed, as a more "Git-native" way of
> pointing to a release.

That'd be fine by me.

> I can't promise to implement any of these features for another few
> months though (at least not without a guilty conscience ;).

Oh, if you need an extra guilty conscience in order to get to work,
I can help, I have plenty of those here.


        Stefan

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Basil L. Contovounesios]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> The packages ivy-avy and ivy-hydra are considered optional integrations
>> with the separate packages avy and hydra, respectively, so they seem
>> like the best candidates for splitting into separate packages (or
>> merging with the avy and hydra packages, I think).
>
> `ivy-hydra` seems to be "extra `ivy` functionality, using a hydra",
> rather than "`ivy` support for hydras", so I don't think it belongs with
> `hydra.el`.
>
> OTOH, I think it should be included into `ivy`.  IIUC this is currently
> not done because it would require adding `hydra` as a dependency to
> `ivy`, but my counter argument would be that the better option is to add
> to `hydra` a new *function* to replace/complement the `defhydra`
> *macro*, so that `ivy-hydra.el` can be included into `ivy` (and compiled
> correctly) without needing to add a dependency on `hydra`.

Right.

> For `ivy-avy` I think the argument is almost the same, except I don't
> see any reason why it isn't done yet (there doesn't seem to be any need
> for `avy` to be around to compile correctly `ivy-avy`).
>
> So, let's start with this: Could we merge `ivy-avy` into `ivy`?

That's where it was until last year:

https://git.savannah.gnu.org/cgit/emacs/elpa.git/commit/?id=6a576cba564ab8860142f9ec343b13ec89cc2440

Thanks,

-- 
Basil

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Stefan Monnier]

Basil L. Contovounesios [2021-03-11 19:25:57] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> For `ivy-avy` I think the argument is almost the same, except I don't
>> see any reason why it isn't done yet (there doesn't seem to be any need
>> for `avy` to be around to compile correctly `ivy-avy`).
>>
>> So, let's start with this: Could we merge `ivy-avy` into `ivy`?
> That's where it was until last year:
>
> https://git.savannah.gnu.org/cgit/emacs/elpa.git/commit/?id=6a576cba564ab8860142f9ec343b13ec89cc2440

Hmm... that commit's just moves the code to a separate file, but that's
not the same as separating it into its own subpackage.

Oleh, what was the motivation for splitting ivy-avy.el into its own
(sub)package instead of bundling it with the `ivy` package?


        Stefan

Re: [elpa] main 8f4cb59: * elpa-packages (counsel, ivy, swiper): Auto-sync.

[Basil L. Contovounesios]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Basil L. Contovounesios [2021-03-11 19:25:57] wrote:
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>> So, let's start with this: Could we merge `ivy-avy` into `ivy`?
>> That's where it was until last year:
>> https://git.sv.gnu.org/cgit/emacs/elpa.git/commit/?id=6a576cba564ab8860142f9ec343b13ec89cc2440
> Hmm... that commit's just moves the code to a separate file, but that's
> not the same as separating it into its own subpackage.

Well, it was packaged separately on MELPA a month later:
https://github.com/melpa/melpa/pull/6951

-- 
Basil