[ELPA] Wrong dependency headers for Ement v0.4

[ELPA] Wrong dependency headers for Ement v0.4

[Adam Porter]

Hi Stefan, et al,

I've encountered a strange situation: I recently published v0.4 of 
Ement.  The describe-package command now shows this:

   Package ement is incompatible.

      Status: Incompatible because it depends on uninstallable packages.
     Archive: gnu
     Version: 0.4
      Commit: b06c78d1ba700857330520bc796c5a29018b7ec5
     Summary: Matrix client
    Requires: emacs-27.1, map-2.1, plz-0.2, taxy-0.12.1 (not available),
                taxy-magit-section-0.9, svg-lib-0.2.5, transient-0.3.7
     Website: https://github.com/alphapapa/ement.el

However, these package headers do not match those in the listed commit, 
i.e. 
https://github.com/alphapapa/ement.el/blob/b06c78d1ba700857330520bc796c5a29018b7ec5/ement.el 
shows:

   ;; Package-Requires: ((emacs "27.1") (map "2.1") (plz "0.2") (taxy 
"0.9") (taxy-magit-section "0.9") (svg-lib "0.2.5") (transient "0.3.7"))

The package headers listed on ELPA correspond to a different commit I 
added later, which had that mistake in it 
(https://github.com/alphapapa/ement.el/commit/529c6b21e7e2c18d2dbcf35359829c3a907a41bc), 
which I've since corrected on git master 
(https://github.com/alphapapa/ement.el/commit/81caaae8fd33b67759bbd7403e3e98143acb2915).

I'd like to push a v0.4.1, just hoping that it would trigger ELPA to use 
the correct dependency headers, but I've since pushed a v0.5-pre version 
to git master, in which I'm preparing the next stable release, so I 
can't make a v0.4.1 release for ELPA, since it reads from the same branch.

I could go ahead and release v0.5, but I was planning to work on it 
more, to fix some bugs that I didn't want to release yet.  So I'm not 
sure what to do now.

Aside: This is one way in which MELPA is a bit easier to work with: 
although MELPA Stable is generally not recommended for use (due to 
various issues), it builds packages from version tags rather than git 
master (regular MELPA builds from git master); if ELPA worked the same 
way, I could easily tag a v0.4.1 independently of what's on the master 
branch.

Given that, would it be possible to have ELPA build from a separate 
branch?  That's how I organize some of my other packages, with a 
"stable" branch and a master branch.  If ELPA pulled from a "stable" 
branch, I could push a fix to that while the next major version is still 
in development.

Thanks for your help.

Adam

Re: [ELPA] Wrong dependency headers for Ement v0.4

[Stefan Monnier]

> I've encountered a strange situation: I recently published v0.4 of Ement.
> The describe-package command now shows this:
>
>   Package ement is incompatible.
>
>      Status: Incompatible because it depends on uninstallable packages.
>     Archive: gnu
>     Version: 0.4
>      Commit: b06c78d1ba700857330520bc796c5a29018b7ec5
>     Summary: Matrix client
>    Requires: emacs-27.1, map-2.1, plz-0.2, taxy-0.12.1 (not available),
>                taxy-magit-section-0.9, svg-lib-0.2.5, transient-0.3.7
>     Website: https://github.com/alphapapa/ement.el
>
> However, these package headers do not match those in the listed commit,
> i.e. https://github.com/alphapapa/ement.el/blob/b06c78d1ba700857330520bc796c5a29018b7ec5/ement.el
> shows:

Hmm... this seems to clearly point to a bug in `elpa-admin.el`.

I wonder how we end up generating a `ement-pkg.el` that contains this
Git revision id, with a `0.4` version, the correct files, but the wrong
set of requires.

[... reading the code...]

Hmm... it looks like we get most of the metadata for `<PKG>-pkg.el` from
the HEAD before we select the actual revision from which we'll build the
tarball :-(

This is ugly.  It probably messed up several existing tarballs, actually
(just in a way sufficiently "harmless" that noone complained loudly
enough yet).

> Aside: This is one way in which MELPA is a bit easier to work with: although
> MELPA Stable is generally not recommended for use (due to various issues),
> it builds packages from version tags rather than git master (regular MELPA
> builds from git master); if ELPA worked the same way, I could easily tag
> a v0.4.1 independently of what's on the master branch.

We can actually do something similar currently by adding
a `:version-map` in the `elpa-packages` spec to make a release
from an arbitrary Git commit (it's better avoided but was introduced to
handle cases where the upstream doesn't have any `Version:` header, as
well as to allow building a new tarball with a different version number
in case the original build was broken for some reason, as in our case).

> Given that, would it be possible to have ELPA build from a separate branch?
> That's how I organize some of my other packages, with a "stable" branch and
> a master branch.  If ELPA pulled from a "stable" branch, I could push a fix
> to that while the next major version is still in development.

We can also do that, yes, using the `:release-branch`.


        Stefan

Re: [ELPA] Wrong dependency headers for Ement v0.4

[Adam Porter]

[-- Attachment #1: Type: text/plain, Size: 2181 bytes --]

Hi Stefan,

On 10/24/22 15:51, Stefan Monnier wrote:

> Hmm... this seems to clearly point to a bug in `elpa-admin.el`.
> 
> I wonder how we end up generating a `ement-pkg.el` that contains this
> Git revision id, with a `0.4` version, the correct files, but the wrong
> set of requires.
> 
> [... reading the code...]
> 
> Hmm... it looks like we get most of the metadata for `<PKG>-pkg.el` from
> the HEAD before we select the actual revision from which we'll build the
> tarball :-(
> 
> This is ugly.  It probably messed up several existing tarballs, actually
> (just in a way sufficiently "harmless" that noone complained loudly
> enough yet).

Well, it's good that it was finally discovered.  :)

>> Aside: This is one way in which MELPA is a bit easier to work with: although
>> MELPA Stable is generally not recommended for use (due to various issues),
>> it builds packages from version tags rather than git master (regular MELPA
>> builds from git master); if ELPA worked the same way, I could easily tag
>> a v0.4.1 independently of what's on the master branch.
> 
> We can actually do something similar currently by adding
> a `:version-map` in the `elpa-packages` spec to make a release
> from an arbitrary Git commit (it's better avoided but was introduced to
> handle cases where the upstream doesn't have any `Version:` header, as
> well as to allow building a new tarball with a different version number
> in case the original build was broken for some reason, as in our case).
> 
>> Given that, would it be possible to have ELPA build from a separate branch?
>> That's how I organize some of my other packages, with a "stable" branch and
>> a master branch.  If ELPA pulled from a "stable" branch, I could push a fix
>> to that while the next major version is still in development.
> 
> We can also do that, yes, using the `:release-branch`.

Okay, that sounds good to me.  Please see the attached patch to 
elpa-packages, which sets the release-branch to "stable".  I've already 
pushed such a branch to the repo, at the v0.4 commit.

Will this patch cause the package to be rebuilt for v0.4, or do I need 
to make a v0.4.1 release to trigger that?

Thanks,
Adam

[-- Attachment #2: 0001-elpa-packages-ement-Set-release-branch.patch --]
[-- Type: text/x-patch, Size: 694 bytes --]

From b232378c68c36634a9c25e11ec035b0d8820cbfd Mon Sep 17 00:00:00 2001
From: Adam Porter <adam@alphapapa.net>
Date: Thu, 27 Oct 2022 12:43:09 -0500
Subject: [PATCH] * elpa-packages (ement): Set :release-branch

---
 elpa-packages | 1 +
 1 file changed, 1 insertion(+)

diff --git a/elpa-packages b/elpa-packages
index 024253e5c0..4d063d1790 100644
--- a/elpa-packages
+++ b/elpa-packages
@@ -286,6 +286,7 @@
   :ignored-files ("embark.el" "embark.texi" "avy-embark-collect.el")
   :auto-sync t)
  ("ement"		:url "https://github.com/alphapapa/ement.el.git"
+  :release-branch "stable"
   :doc "README.org"
   :auto-sync t)
  ("emms"		:url "https://git.savannah.gnu.org/git/emms.git"
-- 
2.34.0

Re: [ELPA] Wrong dependency headers for Ement v0.4

[Stefan Monnier]

> Well, it's good that it was finally discovered.  :)

And fixed, even!

>>> Given that, would it be possible to have ELPA build from a separate branch?
>>> That's how I organize some of my other packages, with a "stable" branch and
>>> a master branch.  If ELPA pulled from a "stable" branch, I could push a fix
>>> to that while the next major version is still in development.
>> We can also do that, yes, using the `:release-branch`.
>
> Okay, that sounds good to me.  Please see the attached patch to
> elpa-packages, which sets the release-branch to "stable".

Done, thanks.

> Will this patch cause the package to be rebuilt for v0.4, or do I need to
> make a v0.4.1 release to trigger that?

There's already a 0.4 package, and by design, packages never(!) get
rebuilt, so you'll need to change `Version:`, yes.
[ And currently, changes on the release branch are only pulled as
  a side-effect of pulling changes from the devel branch, so if you only
  commit to the release branch, the package may wait for a while until
  it's actually released.  :-(  ]


        Stefan

Re: [ELPA] Wrong dependency headers for Ement v0.4

[Adam Porter]

On 10/27/22 15:18, Stefan Monnier wrote:
>> Well, it's good that it was finally discovered.  :)
> 
> And fixed, even!

:D

>>>> Given that, would it be possible to have ELPA build from a separate branch?
>>>> That's how I organize some of my other packages, with a "stable" branch and
>>>> a master branch.  If ELPA pulled from a "stable" branch, I could push a fix
>>>> to that while the next major version is still in development.
>>> We can also do that, yes, using the `:release-branch`.
>>
>> Okay, that sounds good to me.  Please see the attached patch to
>> elpa-packages, which sets the release-branch to "stable".
> 
> Done, thanks.
> 
>> Will this patch cause the package to be rebuilt for v0.4, or do I need to
>> make a v0.4.1 release to trigger that?
> 
> There's already a 0.4 package, and by design, packages never(!) get
> rebuilt, so you'll need to change `Version:`, yes.
> [ And currently, changes on the release branch are only pulled as
>    a side-effect of pulling changes from the devel branch, so if you only
>    commit to the release branch, the package may wait for a while until
>    it's actually released.  :-(  ]

Understood, so I just pushed v0.4.1 to the stable branch and then a new 
commit to the master branch, so IIUC that will guarantee a rebuild with 
v0.4.1 being released.

Thanks for all your work!