Python 2 EOL starts to break packages

Python 2 EOL starts to break packages

[zimoun]

Hi,

The bug#46132 [1] reports the failure of the package
’python2-setuptools’.  It is expected because this Python 2 is inherited
via ’package-with-python2’ from Python 3 and the current
’python-setuptools’ is at v52.  And, Setuptools dropped the support of
Python 2 at v47.  See their changelog [2,3]:

--8<---------------cut here---------------start------------->8---
 v47.0.0

 28 May 2020
 Breaking Changes

     #2094: Setuptools now actively crashes under Python 2. Python 3.5 or later is required. Users of Python 2 should use setuptools<45.
--8<---------------cut here---------------end--------------->8---


Because Python 2 is End-Of-Life, these cases will happen more and more.
It is not new and “we” (at least me) are aware this would happen–see
bug#38420 [4] from Nov 2019; but “we“ have not really work on it.
Anyway. :-)

Removing ’python2-setuptools’ from Guix could break some other packages.
Maybe?  Who knows? ;-)

Instead, a plan could be:

 a) move all the ’python2-<package>’ to their own module
    say (gnu packages python2-xyz)
 b) pin the ones which breaks; pin meaning «not use package-with-python2» 
 c) fix a deadline announcing the remove of availability at user level
 d) after this date, hide all the python2 packages
 e) remove case per case these hidden python2 packages

Today, it does not make sense to have ’python2-setuptools’ in a manifest
file or type ‘guix install python2-setuptools’.  So it should not be
possible and raise “package does not exist”.  However, I could have a
custom package that depends on ’python2-setuptools’ so it should be
accessible via “#:use-modules (gnu packages python2-xyz)”.


WDYT?

1: <http://issues.guix.gnu.org/issue/46132>
2: <https://setuptools.readthedocs.io/en/latest/history.html>
3: <https://github.com/pypa/setuptools/issues/2094>
4: <http://issues.guix.gnu.org/issue/38420>


All the best,
simon

Re: Python 2 EOL starts to break packages

[Maxim Cournoyer]

Hi Simon,

zimoun <zimon.toutoune@gmail.com> writes:

[...]

> Removing ’python2-setuptools’ from Guix could break some other packages.
> Maybe?  Who knows? ;-)
>
> Instead, a plan could be:
>
>  a) move all the ’python2-<package>’ to their own module
>     say (gnu packages python2-xyz)
>  b) pin the ones which breaks; pin meaning «not use package-with-python2» 
>  c) fix a deadline announcing the remove of availability at user level
>  d) after this date, hide all the python2 packages
>  e) remove case per case these hidden python2 packages

I think that's more involved than it needs to be.  Here's how I've been
slowly getting at the Python 2 issues:

Whenever I update a Python package that breaks its Python 2 variant, I
look at the dependencies of those variants with 'guix refresh -l
python2-something'.  If all the dependents are purely python2 variants
themselves, then I proceed to remove them all from the package
collection.  If instead some non-python2 variant package
depends on it, I research if that package could be updated to make it
Python 3 compatible.  If yes, then problem solved.  Otherwise we need to
ping upstream and keep it for a little longer before it gets removed.

> Today, it does not make sense to have ’python2-setuptools’ in a manifest
> file or type ‘guix install python2-setuptools’.  So it should not be
> possible and raise “package does not exist”.  However, I could have a
> custom package that depends on ’python2-setuptools’ so it should be
> accessible via “#:use-modules (gnu packages python2-xyz)”.

Let's apply this to the current case:

$ guix refresh -l python2-setuptools

Building the following 3 packages would ensure 7 dependent packages are
rebuilt: python2-behave-web-api@1.0.6 grocsvs@0.2.6.1-1.ecd956a
syncthing-gtk@0.9.4.4

The python2-behave-web-api is not a concern (there's an existing
python-behave-web-api package), but the others are.  There's an issue
about syncthing-gtk reliance on Python 2 at
https://github.com/kozec/syncthing-gtk/issues/568.  For grocsvs I've
just commented at https://github.com/grocsvs/grocsvs/issues/6.  If the
upstream are not willing to budge, the packages can be removed after a
grace period (6 months, I'd say).

Does that help?

Maxim

Re: Python 2 EOL starts to break packages

[zimoun]

HI Maxim,

Thanks for your answer.

On Fri, 29 Jan 2021 at 04:49, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
> zimoun <zimon.toutoune@gmail.com> writes:

> Whenever I update a Python package that breaks its Python 2 variant, I
> look at the dependencies of those variants with 'guix refresh -l
> python2-something'.  If all the dependents are purely python2 variants
> themselves, then I proceed to remove them all from the package
> collection.  If instead some non-python2 variant package
> depends on it, I research if that package could be updated to make it
> Python 3 compatible.  If yes, then problem solved.  Otherwise we need to
> ping upstream and keep it for a little longer before it gets removed.

By "keep it for a little longer", do you mean update the Python 3
version and explicitly define the Python 2 variant ? Or do you mean do
not update at all?

In other words, should 3142daccbe be reverted?  Or should 3142daccbe
keep as it is and the variant 'python2-setuptools' defined explicitly
with the previous version?

Noting that people submitting and reviewing patches should both apply
your methodology.  Otherwise, the janitor work is more burden.
Contrary to what I am proposing, i.e., move all the python2-* variants
inside a specific module, where the janitor work can be done
asynchronously: open this module and address package one after the
other, mainly once a while by batch.  Instead of greping, etc.  I can
tell you by janitoring Bioconductor packages that grepping becomes
really annoying when you address more packages than a child is able to
count. ;-)

In other words, I am proposing to separate update and janitor with an
easier janitor task.

Well, all are fine with me.  The aim of my email is simply to roughly
agree on a "way" to collectively ease this boring task.

> The python2-behave-web-api is not a concern (there's an existing
> python-behave-web-api package), but the others are.  There's an issue
> about syncthing-gtk reliance on Python 2 at
> https://github.com/kozec/syncthing-gtk/issues/568.  For grocsvs I've
> just commented at https://github.com/grocsvs/grocsvs/issues/6.  If the
> upstream are not willing to budge, the packages can be removed after a
> grace period (6 months, I'd say).

Just to fix the idea, the last release of syncthing-gtk is from 4 Aug
2019 and the last commit in master from 30 Oct 2019.  The issue you
mention is from 30 Nov 2020 but the real first one is #487 [1] from 22
Oct 2018 and mentioning a port to Python 3 by Debian folks [2] (not
tried the status).  Hum, the 6 months grace period is far beyond. ;-)
(I am not asking any remove.)

And I guess your are Apteryks that asked to reopen the issue of grocsvs. :-)
Well, since this package is Bioinformatics, I can tell you that it
will not be ported, another story.
However, this Python 2 package had been added in Guix on 2020-05-05,
i.e., post Python 2 EOL.  I think it is a "mistake" and instead should
live in the channel "guix-past", but at the time, this channel did not
exist, if I remember correctly.

Anyway, that's a discussion for bug#46132.

Just to understand, does your suggestion is: should the author of
3142daccbe updating python-setuptools have done these 2 checks?
(Maybe more for some cases.)

In my mind, no.  And having a separate file containing all the python2
variants ease the review of each and then janitor.

1: https://github.com/kozec/syncthing-gtk/issues/487
2: https://salsa.debian.org/debian/syncthing-gtk/-/tree/python3-port


Cheers,
simon

Re: Python 2 EOL starts to break packages

[Maxim Cournoyer]

Hi,

zimoun <zimon.toutoune@gmail.com> writes:

> HI Maxim,
>
> Thanks for your answer.
>

[...]

> In other words, should 3142daccbe be reverted?  Or should 3142daccbe
> keep as it is and the variant 'python2-setuptools' defined explicitly
> with the previous version?

I wouldn't revert; Python 2 should not hold anyone from keeping packages
up to date.

> Noting that people submitting and reviewing patches should both apply
> your methodology.  Otherwise, the janitor work is more burden.
> Contrary to what I am proposing, i.e., move all the python2-* variants
> inside a specific module, where the janitor work can be done
> asynchronously: open this module and address package one after the
> other, mainly once a while by batch.  Instead of greping, etc.  I can
> tell you by janitoring Bioconductor packages that grepping becomes
> really annoying when you address more packages than a child is able to
> count. ;-)
>
> In other words, I am proposing to separate update and janitor with an
> easier janitor task.
>
> Well, all are fine with me.  The aim of my email is simply to roughly
> agree on a "way" to collectively ease this boring task.

My opinion is that moving packages about to be removed is not worth the
hassle.  Most Python 2 packages are already in a single module: the
gigantic python-xyz.scm ;-)

>> The python2-behave-web-api is not a concern (there's an existing
>> python-behave-web-api package), but the others are.  There's an issue
>> about syncthing-gtk reliance on Python 2 at
>> https://github.com/kozec/syncthing-gtk/issues/568.  For grocsvs I've
>> just commented at https://github.com/grocsvs/grocsvs/issues/6.  If the
>> upstream are not willing to budge, the packages can be removed after a
>> grace period (6 months, I'd say).
>
> Just to fix the idea, the last release of syncthing-gtk is from 4 Aug
> 2019 and the last commit in master from 30 Oct 2019.  The issue you
> mention is from 30 Nov 2020 but the real first one is #487 [1] from 22
> Oct 2018 and mentioning a port to Python 3 by Debian folks [2] (not
> tried the status).  Hum, the 6 months grace period is far beyond. ;-)
> (I am not asking any remove.)

I just committed a series of python2-* package removal, along with using
this port for our syncthing-gtk package.  It now builds, at least!  See
commit de8dc9039b.

> And I guess your are Apteryks that asked to reopen the issue of grocsvs. :-)
> Well, since this package is Bioinformatics, I can tell you that it
> will not be ported, another story.
> However, this Python 2 package had been added in Guix on 2020-05-05,
> i.e., post Python 2 EOL.  I think it is a "mistake" and instead should
> live in the channel "guix-past", but at the time, this channel did not
> exist, if I remember correctly.

Fair enough.  Removed in 23a5dcce1d.

Thanks!

Maxim