Re: Poetry upgrade and related packages

[Marius Bakke <marius@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 4938 bytes --]

Tanguy LE CARROUR <tanguy@bioneland.org> skriver:

> Hi Marius,
>
>
> Excerpts from Marius Bakke's message of December 7, 2020 11:59 pm:
>> Tanguy LE CARROUR <tanguy@bioneland.org> skriver:
>>> Excerpts from Ludovic Courtès's message of December 5, 2020 4:44 pm:
>>>> Tanguy LE CARROUR <tanguy@bioneland.org> skribis:
>>>> 
>>>>> It's not yet clear to me how to handle (python) package updates:
>>>>> - when to update;
>>>>> - when not to update;
>>>>> - when to introduce "versionned" (`-x.y` suffix) package definitions;
>>>>> - when to introduce "next" (`/next` suffix) package definitions;
>>>>> - when to remove the two above suffixes;
>>>>> - …
>>>>>
>>>>> So I'm looking forward to reading the answers to this thread! :-)
>>>> 
>>>> When a change introduces too many rebuilds, the convention is to make
>>>> that change on a branch that will be merged “later” rather than on
>>>> ‘master’; this is bullet 8 here:
>>>> 
>>>>   https://guix.gnu.org/manual/devel/en/html_node/Submitting-Patches.html
>>>
>>> Thanks for pointing at, but this "just" tells me on which branch to put
>>> the changeset, not which of the above options should be used when a
>>> package needs to be updated.
>> 
>> There is no "one size fits all" rule.  With rare exceptions, Guix
>> "wants" to have only have a single version of each package (mainly to
>> ease maintenance).  As you found, that's not always feasible.
>> 
>> If a package depends on a newer version of something "deep in the graph"
>> such as Pytest, it's always OK to add a "/next" or "-x.y" variant
>> (though a convention about which to use would probably be a good idea).
>>
>> If something depends on a *specific* (older) version of Pytest, it's
>> better to try and make it work with the newer version; but failing that,
>> adding a "-x.y" is fine too.
>
> `-x.y` packages are here to stay.
> `/next` are temporary packages.

In most cases, yes.  :-)

> Is it safe to remove all `-x.y` packages that are not used as inputs?
> Is there a (programmatic) way to find those packages?

Good questions.  It should be safe to remove unused and versioned "leaf"
packages.  I don't have good answers on how to discover them though.

> When `/next` become "current|default|latest", who is in charge of patching
> all the package definitions that were using it?!

Typically those occur in the same or (closely related) commits.  But it
happens that a "/latest" has been removed on 'core-updates', yet keeps
gaining uses on 'master'.  In those cases deprecating (instead of
removing) the variable on the 'core-updates' branch is the best option,
so that merging master->core-updates won't break the branch.

There is no one "in charge" though, just "someone" has to do it.  :-)

>>>> Yet, sometimes we want to introduce new versions that people can get in
>>>> their profile, even if the “default” one remains the older version to
>>>> avoid world rebuilds.
>>>
>>> That's exactly my point! If the default one lags behind, then after some
>>> time, nobody will use it any more and we will have introduced one or more
>>> `-x.y` package definitions!
>>> I would consider it to be a "saner" approach to have the default always
>>> "point" to the latest version, but then we would have to "fix" package
>>> depending on older versions by introducing `-x.y` package definitions
>>> for them.
>>>
>>> Or am I missing something?!
>> 
>> You got it right.  It might be saner to make the unversioned variable
>> refer to the newest version, but it would often require "pinning"
>> hundreds of packages to the old version to avoid rebuilds.  Thus, it's
>> typically more practical to use the "/next" variant until the next
>> rebuild cycle.
>
> Then, during the rebuild cycle those hundreds of packages get rebuilt
> and… some of them fail because they depend on the older version, right?!
> But I'm pretty sure it's a problem all distributions have to face.

Yes, often you'll get an "upgrading cascade", which may bring new
problems such as circular dependencies, etc (looking at you, Python
ecosystem).

The only remedy I know for that is copious amounts of coffee.

> Wasn't it discussed somewhere else that one should get notified (by email)
> when their favourite packages were broken?

I think cbaines is working on something along those lines.

>> Again there is no hard rule here, I did such a change for 'libcap' in
>> 9e1f5a263e4f6df4d075901c9b58a56f80c8b452 because only two packages
>> needed to pin the old version.
>
>> Hope this clears things up a little more.  :-)
>
> Yes, thanks! But I guess I need to go through more of those situations
> to make sure I understand everything.

For some practical experience, you can try updating to Python 3.9.1 and
the latest Pytest (+ dependencies).  This needs to be done soon for the
'core-updates' branch anyway.  ;-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]