[PATCH 0/2] Update Python to current versions

[PATCH 0/2] Update Python to current versions

[Hartmut Goebel]

Update Python to current minor version

Hartmut Goebel (2):
  gnu: python-2: Update to 2.7.11
  gnu: python: Update to 3.4.4

 gnu/packages/python.scm | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

-- 
2.7.4

[PATCH 1/2] gnu: python-2: Update to 2.7.11

[Hartmut Goebel]

* gnu/packages/python.scm (python-2): Update to 2.7.11
---
 gnu/packages/python.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 0379352..ba9ac94 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -91,7 +91,7 @@
 (define-public python-2
   (package
     (name "python")
-    (version "2.7.10")
+    (version "2.7.11")
     (source
      (origin
       (method url-fetch)
@@ -99,7 +99,7 @@
                           version "/Python-" version ".tar.xz"))
       (sha256
        (base32
-        "1h7zbrf9pkj29hlm18b10548ch9757f75m64l47sy75rh43p7lqw"))
+        "0iiz844riiznsyhhyy962710pz228gmhv8qi3yk4w4jhmx2lqawn"))
       (patches (search-patches
                 "python-2.7-search-paths.patch"
                 "python-2-deterministic-build-info.patch"
-- 
2.7.4

Re: [PATCH 1/2] gnu: python-2: Update to 2.7.11

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1286 bytes --]

On Thu, Apr 21, 2016 at 10:18:45PM +0200, Hartmut Goebel wrote:
> * gnu/packages/python.scm (python-2): Update to 2.7.11
> ---
>  gnu/packages/python.scm | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
> index 0379352..ba9ac94 100644
> --- a/gnu/packages/python.scm
> +++ b/gnu/packages/python.scm
> @@ -91,7 +91,7 @@
>  (define-public python-2
>    (package
>      (name "python")
> -    (version "2.7.10")
> +    (version "2.7.11")
>      (source
>       (origin
>        (method url-fetch)
> @@ -99,7 +99,7 @@
>                            version "/Python-" version ".tar.xz"))
>        (sha256
>         (base32
> -        "1h7zbrf9pkj29hlm18b10548ch9757f75m64l47sy75rh43p7lqw"))
> +        "0iiz844riiznsyhhyy962710pz228gmhv8qi3yk4w4jhmx2lqawn"))
>        (patches (search-patches
>                  "python-2.7-search-paths.patch"
>                  "python-2-deterministic-build-info.patch"
> -- 
> 2.7.4
> 

This one is already updated in core-updates

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 1/2] gnu: python-2: Update to 2.7.11

[Hartmut Goebel]

Am 21.04.2016 um 23:20 schrieb Efraim Flashner:
> This one is already updated in core-updates

What is this "core-updates" thing?

SCNR I assume this is a branch. But the manual is not wasting a single
word about branches in the repo. So how I am expected to know this.


And why is a minor update in "core-updates" and not on the main-line?

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: [PATCH 1/2] gnu: python-2: Update to 2.7.11

[Ricardo Wurmus]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 21.04.2016 um 23:20 schrieb Efraim Flashner:
>> This one is already updated in core-updates
>
> What is this "core-updates" thing?
>
> SCNR I assume this is a branch. But the manual is not wasting a single
> word about branches in the repo. So how I am expected to know this.

It’s a branch.  It’s not in the manual because it’s just an operational
detail of how we use the build farm at hydra.gnu.org.

> And why is a minor update in "core-updates" and not on the main-line?

Rebuilding the Python package would result in a forced rebuild of all
dependent packages.  Our build farm isn’t fast enough to rebuild that
much in time, so we would have an uncomfortable window during which a
lot of packages would have to be built from source.  By having a
separate branch for updates that cause a lot of rebuilds we can build
substitutes in the background.  Once most of the substitutes have been
built we merge “core-updates” into “master”, so that people upgrading to
latest master will be able to download substitutes.

~~ Ricardo

Re: [PATCH 1/2] gnu: python-2: Update to 2.7.11

[Hartmut Goebel]

Am 22.04.2016 um 08:52 schrieb Ricardo Wurmus:
> Rebuilding the Python package would result in a forced rebuild of all
> dependent packages.  Our build farm isn’t fast enough to rebuild that
> much in time, so we would have an uncomfortable window during which a
> lot of packages would have to be built from source.  By having a

Thanks for the explanation, this is quite reasonable. I'll check core
updates the next time.

Regarding the bug-fix updates for Python: I'd not expect any problems,
since these updates are bug-fixes only and the byte-code does not change
within a series (2.7.x, 3.4.x). So grafting could be an option here, too.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: [PATCH 1/2] gnu: python-2: Update to 2.7.11

[Ludovic Courtès]

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

> Am 22.04.2016 um 08:52 schrieb Ricardo Wurmus:
>> Rebuilding the Python package would result in a forced rebuild of all
>> dependent packages.  Our build farm isn’t fast enough to rebuild that
>> much in time, so we would have an uncomfortable window during which a
>> lot of packages would have to be built from source.  By having a
>
> Thanks for the explanation, this is quite reasonable. I'll check core
> updates the next time.

“python updates” even.  :-)

> Regarding the bug-fix updates for Python: I'd not expect any problems,
> since these updates are bug-fixes only and the byte-code does not change
> within a series (2.7.x, 3.4.x). So grafting could be an option here, too.

I would prefer to restrict grafts to security fixes, primarily because
they have possibly surprising side effects on the UI.

Ludo’.

[PATCH 2/2] gnu: python: Update to 3.4.4

[Hartmut Goebel]

* gnu/packages/python.scm (python-2): Update to 3.4.4
---
 gnu/packages/python.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index ba9ac94..65210e0 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -262,7 +262,7 @@ data types.")
 
 (define-public python
   (package (inherit python-2)
-    (version "3.4.3")
+    (version "3.4.4")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://www.python.org/ftp/python/"
@@ -276,7 +276,7 @@ data types.")
               (patch-flags '("-p0"))
               (sha256
                (base32
-                "1f4nm4z08sy0kqwisvv95l02crv6dyysdmx44p1mz3bn6csrdcxm"))))
+                "18kb5c29w04rj4gyz3jngm72sy8izfnbjlm6ajv6rv2m061d75x7"))))
     (arguments (substitute-keyword-arguments (package-arguments python-2)
                  ((#:tests? _) #t)))
     (native-search-paths
-- 
2.7.4

Re: [PATCH 0/2] Update Python to current versions

[Cyril Roelandt]

On 04/21/2016 10:18 PM, Hartmut Goebel wrote:
>   gnu: python: Update to 3.4.4
I think the latest Python 3.x is Python 3.5.1 (see
https://www.python.org/). Maybe we should forget about 3.4.x, WDYT?

Cyril.

Re: [PATCH 0/2] Update Python to current versions

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 588 bytes --]

On Fri, Apr 22, 2016 at 01:12:38AM +0200, Cyril Roelandt wrote:
> On 04/21/2016 10:18 PM, Hartmut Goebel wrote:
> >   gnu: python: Update to 3.4.4
> I think the latest Python 3.x is Python 3.5.1 (see
> https://www.python.org/). Maybe we should forget about 3.4.x, WDYT?
> 
> Cyril.
> 

I know there were a couple of people working on getting 3.5.1 to work
nicely.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 0/2] Update Python to current versions

[Hartmut Goebel]

[-- Attachment #1: Type: text/plain, Size: 720 bytes --]

Am 22.04.2016 um 01:12 schrieb Cyril Roelandt:
> I think the latest Python 3.x is Python 3.5.1 (see
> https://www.python.org/). Maybe we should forget about 3.4.x, WDYT?

Python 3.4.4 is the latest version fon the 3.4 series. So we should
provide this too. (Esp. as long as 3.5 ist not packaged yet).


-- 
Schönen Gruß
Hartmut Goebel
Dipl.-Informatiker (univ), CISSP, CSSLP, ISO 27001 Lead Implementer
Information Security Management, Security Governance, Secure Software
Development

Goebel Consult, Landshut
http://www.goebel-consult.de

Blog:
http://www.goebel-consult.de/blog/vorratsdatenspeicherung-jetzt-verfassungsbeschwerde-unterschreiben

Kolumne: http://www.cissp-gefluester.de/2011-11-in-troja-nichts-neues


[-- Attachment #2: Type: text/html, Size: 1955 bytes --]

Re: [PATCH 0/2] Update Python to current versions

[Leo Famulari]

On Fri, Apr 22, 2016 at 08:41:04AM +0200, Hartmut Goebel wrote:
> Am 22.04.2016 um 01:12 schrieb Cyril Roelandt:
> > I think the latest Python 3.x is Python 3.5.1 (see
> > https://www.python.org/). Maybe we should forget about 3.4.x, WDYT?
> 
> Python 3.4.4 is the latest version fon the 3.4 series. So we should
> provide this too. (Esp. as long as 3.5 ist not packaged yet).

As mentioned by others, updates to packages with many dependents are
very computationally expensive, and so we try to do many of them in
batches using the 'core-updates' branch of our Git repository.

I agree that we should try to provide the latest version of whatever
series we have packaged. The changelog mentions many bug fixes [0].

Can you take a look at the work that's already been done towards 3.5?
It starts here, and the discussion continues in March:
http://lists.gnu.org/archive/html/guix-devel/2016-02/msg01265.html

If we can successfully update to 3.5, that would be best. Otherwise, we
should consider using 3.4.4.

[0]
https://docs.python.org/3.4/whatsnew/changelog.html

Re: [PATCH 0/2] Update Python to current versions

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> On Fri, Apr 22, 2016 at 08:41:04AM +0200, Hartmut Goebel wrote:
>> Am 22.04.2016 um 01:12 schrieb Cyril Roelandt:
>> > I think the latest Python 3.x is Python 3.5.1 (see
>> > https://www.python.org/). Maybe we should forget about 3.4.x, WDYT?
>> 
>> Python 3.4.4 is the latest version fon the 3.4 series. So we should
>> provide this too. (Esp. as long as 3.5 ist not packaged yet).
>
> As mentioned by others, updates to packages with many dependents are
> very computationally expensive, and so we try to do many of them in
> batches using the 'core-updates' branch of our Git repository.

I’d suggest a ‘python-updates’ branch where we’d be able to pinpoint
breakage due specifically to these updates.

Unfortunately, this will have to be scheduled after we’re done with
‘gnome-updates’, and possibly after ‘core-updates’.

Thanks,
Ludo’.

Re: [PATCH 0/2] Update Python to current versions

[Leo Famulari]

On Sun, Apr 24, 2016 at 11:43:09PM +0200, Ludovic Courtès wrote:
> Leo Famulari <leo@famulari.name> skribis:
> 
> > On Fri, Apr 22, 2016 at 08:41:04AM +0200, Hartmut Goebel wrote:
> >> Am 22.04.2016 um 01:12 schrieb Cyril Roelandt:
> >> > I think the latest Python 3.x is Python 3.5.1 (see
> >> > https://www.python.org/). Maybe we should forget about 3.4.x, WDYT?
> >> 
> >> Python 3.4.4 is the latest version fon the 3.4 series. So we should
> >> provide this too. (Esp. as long as 3.5 ist not packaged yet).
> >
> > As mentioned by others, updates to packages with many dependents are
> > very computationally expensive, and so we try to do many of them in
> > batches using the 'core-updates' branch of our Git repository.
> 
> I’d suggest a ‘python-updates’ branch where we’d be able to pinpoint
> breakage due specifically to these updates.

In that case, we should also try to update python-setuptools, and
possibly some other "core" Python packages.

I've noticed some upstream "minimum version of setuptools" requirements
creepings towards our version.

Re: [PATCH 0/2] Update Python to current versions

[Hartmut Goebel]

Am 24.04.2016 um 02:18 schrieb Leo Famulari:
> If we can successfully update to 3.5, that would be best. Otherwise, we
> should consider using 3.4.4.

I disagree.

While Python typically is backward-compatible (except for the big change
between Python 2 and Python 3), some minor incompatibilities are common
when upgrading from e.g. 3.4 to 3.5. E.g. in 3.5 some deprecated things
have been removed [1].

I suggest:

- Rename "python" (python@3.4.x) to "python-3.4"
- Add a new variable "python" as an alias for "python-3.4"
- Add "python-3.5" (@3.5.1) (for now independent from "python"
- Update python-3.4 to 3.4.5

So in feature we can easily provide two series (3.4, 3.5), which are
still available for those who need it. At the same time we can easily
switch "python" to the series we decide to be the "main" python for this
release. This is much like we already do for icedtea.

[1] https://docs.python.org/3.5/whatsnew/3.5.html#removed

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: [PATCH 0/2] Update Python to current versions

[Leo Famulari]

On Tue, Apr 26, 2016 at 11:28:53AM +0200, Hartmut Goebel wrote:
> Am 24.04.2016 um 02:18 schrieb Leo Famulari:
> > If we can successfully update to 3.5, that would be best. Otherwise, we
> > should consider using 3.4.4.
> 
> I disagree.
> 
> While Python typically is backward-compatible (except for the big change
> between Python 2 and Python 3), some minor incompatibilities are common
> when upgrading from e.g. 3.4 to 3.5. E.g. in 3.5 some deprecated things
> have been removed [1].

Okay.

> I suggest:
> 
> - Rename "python" (python@3.4.x) to "python-3.4"
> - Add a new variable "python" as an alias for "python-3.4"
> - Add "python-3.5" (@3.5.1) (for now independent from "python"

It should be possible to do these 3 actions without causing any packages
to be rebuilt. I think the first 2 steps should be in one commit, and
the second step in a second commit.

> - Update python-3.4 to 3.4.5

This will require every python-3 package to be rebuilt, so it should be
done on a python-updates branch.

That branch could also include python-3.5, updates to setuptools, sphinx
[0], and other core Python packages.

It would also be good to address these reproducibility bugs in Python's
bytecode compiler:
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=22010
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=22533

What does everyone think of this plan?

[0] We need to make Sphinx stop embedding build timestamps:
https://wiki.debian.org/ReproducibleBuilds/TimestampsInManpagesGeneratedBySphinx

Re: [PATCH 0/2] Update Python to current versions

[Hartmut Goebel]

Am 27.04.2016 um 18:55 schrieb Leo Famulari:
> It should be possible to do these 3 actions without causing any packages
> to be rebuilt. I think the first 2 steps should be in one commit, and
> the second step in a second commit.
>
> [...]
> This will require every python-3 package to be rebuilt, so it should be
> done on a python-updates branch.
>
> That branch could also include python-3.5, updates to setuptools, sphinx
> [0], and other core Python packages.
I can take care of this, but it will be a few days. (and I ought to
close my other python patches first, I think.)

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |