Restarting 'python-updates'

Restarting 'python-updates'

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 992 bytes --]

Greetings Guix!

Python 3.6.2 is scheduled to be released in about a week[0].

Our current Python 3.5 is EOL[1], so we should upgrade ASAP. 3.6.2 seems
like a good excuse. It would be good to start building a branch based on
it once it is released (or core-updates is merged, whichever is last).

Are there any objections to deleting the current 'python-updates' branch
and starting a fresh one? Looking at `git log master..python-updates`,
most of the commits seem to have made their way to 'master' in some form.

I would also like to try making the newer versions of 'pytest' and
'mock' the default and instead add lower versions as necessary. Would
that be too much for one cycle?

Updating the heavy-impact python libraries will require upgrading many
collateral packages and solving new dependency cycles, so I'm hoping we
can start this effort earlier to make the process smoother. Thoughts?

[0] https://www.python.org/dev/peps/pep-0494/
[1] https://www.python.org/dev/peps/pep-0478/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Restarting 'python-updates'

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1703 bytes --]

On Fri, Jun 23, 2017 at 06:36:09PM +0200, Marius Bakke wrote:
> Python 3.6.2 is scheduled to be released in about a week[0].
> 
> Our current Python 3.5 is EOL[1], so we should upgrade ASAP. 3.6.2 seems
> like a good excuse. It would be good to start building a branch based on
> it once it is released (or core-updates is merged, whichever is last).

We could also start sooner with Python 3.6.1, if people are eager to
begin working :)

> Are there any objections to deleting the current 'python-updates' branch
> and starting a fresh one? Looking at `git log master..python-updates`,
> most of the commits seem to have made their way to 'master' in some form.

No objections!

> I would also like to try making the newer versions of 'pytest' and
> 'mock' the default and instead add lower versions as necessary. Would
> that be too much for one cycle?
> 
> Updating the heavy-impact python libraries will require upgrading many
> collateral packages and solving new dependency cycles, so I'm hoping we
> can start this effort earlier to make the process smoother. Thoughts?

It will be quite a bit of work to upgrade the "core" Python libraries,
but we have to do it. Your suggestion to reverse the "default" versions
is a good one.

I thought this Nix proposal to treat Python tests as their own
derivations is an interesting idea for cutting the Gordian knot of
Python dependency cycles:

https://github.com/NixOS/nixpkgs/issues/26400

But, I don't mind if we skip this for now. It depends on peoples'
motivation.

However, I want to fix the bytecode caching issue in this cycle. It's a
bad performance bug :(

https://github.com/NixOS/nixpkgs/issues/22569

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Restarting 'python-updates'

[Mark H Weaver]

Marius Bakke <mbakke@fastmail.com> writes:

> Greetings Guix!
>
> Python 3.6.2 is scheduled to be released in about a week[0].
>
> Our current Python 3.5 is EOL[1], so we should upgrade ASAP. 3.6.2 seems
> like a good excuse. It would be good to start building a branch based on
> it once it is released (or core-updates is merged, whichever is last).
>
> Are there any objections to deleting the current 'python-updates' branch
> and starting a fresh one? Looking at `git log master..python-updates`,
> most of the commits seem to have made their way to 'master' in some form.

I think this should wait until after core-updates is merged.  Our build
farm capacity is already stretched quite thin, and "guix refresh -l
python@3" estimates that 2886 rebuilds would be required per
architecture.  Given the limitations of "guix refresh -l", that might
well be an underestimate.  So that's at least 8600 builds total, or
about half of the entire jobset on Hydra.

I don't think we can afford to build this out at the same time as
'core-updates' is being built, and 'master' is being kept up-to-date.
Even keeping 'master' up-to-date requires significant resources, given
the frequency of updates that require rebuilding most of GNOME, Qt,
IceCat, Libreoffice, Webkitgtk, etc.

       Mark