[bug#54866] [PATCH] docker-compose, python-pyyaml

[bug#54866] [PATCH] docker-compose, python-pyyaml

[daniel.herzig]

[-- Attachment #1.1: Type: text/plain, Size: 608 bytes --]

Hi Guix,

I noticed that after my last pull to 6413d0898b92, docker-compose did
not build anymore. The backtrace showed that it fails in the sanity-
check-phase due to the requirement python-pyyaml <6.

As it builds nicely on guix 9bd4ed3, which still features
python-pyyaml@5.4.1 I took the definition from there, re-added it to
python-xyz.scm and adjusted docker-compose in docker.scm.  You find the
patchfile attached, as evolution seems to mess with line-breaks
(docker-compose build tested on x86_64 only, as I don't have anything
else around).

Best & thanks for your great work,
Daniel

[-- Attachment #1.2: 0001-re-adding-python-pyyaml-5.4.1-for-compatibility-with.patch --]
[-- Type: text/x-patch, Size: 2191 bytes --]

From eab4590fe2c9726920dbe07202616908065dad62 Mon Sep 17 00:00:00 2001
From: Daniel <daniel.herzig@outlook.at>
Date: Tue, 12 Apr 2022 00:04:27 +0200
Subject: [PATCH] re-adding python-pyyaml@5.4.1 for compatibility with
 docker-compose@1.29.2

---
 gnu/packages/docker.scm     |  2 +-
 gnu/packages/python-xyz.scm | 24 ++++++++++++++++++++++++
 2 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index 3f52f4f8db..07731886ae 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -122,7 +122,7 @@ (define-public docker-compose
            python-docopt
            python-dotenv
            python-jsonschema
-           python-pyyaml
+           python-pyyaml-5
            python-requests
            python-six
            python-texttable
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index e83eb017f5..12b538594e 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -3928,6 +3928,30 @@ (define-public python-pyyaml
     (license license:expat)
     (properties `((python2-variant . ,(delay python2-pyyaml))))))
 
+;; for docker-compose 1.29.2 compatibility
+(define-public python-pyyaml-5
+  (package
+    (name "python-pyyaml")
+    (version "5.4.1")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "PyYAML" version))
+       (sha256
+	(base32
+	 "0pm440pmpvgv5rbbnm8hk4qga5a292kvlm1bh3x2nwr8pb5p8xv0"))))
+    (build-system python-build-system)
+    (inputs
+     (list libyaml python-cython))
+    (home-page "https://pyyaml.org")
+    (synopsis "YAML parser and emitter for Python")
+    (description
+     "PyYAML is a YAML parser and emitter for Python.  PyYAML features a
+complete YAML 1.1 parser, Unicode support, pickle support, capable extension
+API, and sensible error messages.  PyYAML supports standard YAML tags and
+provides Python-specific tags that represent an arbitrary Python object.")
+    (license license:expat)))
+
 (define-public python2-pyyaml
   (let ((base (package-with-python2 (strip-python2-variant python-pyyaml))))
     (package

base-commit: 6413d0898b92efda8213b3ced0b6d1c736726b89
-- 
2.35.1


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 695 bytes --]

[bug#54866] [PATCH] docker-compose, python-pyyaml

[zimoun]

Hi,

On mar., 12 avril 2022 at 00:28, daniel.herzig@outlook.at wrote:

> I noticed that after my last pull to 6413d0898b92, docker-compose did
> not build anymore. The backtrace showed that it fails in the sanity-
> check-phase due to the requirement python-pyyaml <6.

Thanks for the report.  Indeed, commit
27720d0fe14ee4cd413a6745e0782a698a011924 updates python-pyyaml from 5 to
6 and thus breaks docker-compose.

Well, this update is for core-updates (or maybe staging) and not for
master directly.

--8<---------------cut here---------------start------------->8---
$ guix refresh -l python-pyyaml | cut -f1 -d':'
Building the following 1453 packages would ensure 2671 dependent packages are rebuilt
--8<---------------cut here---------------end--------------->8---

Examining the failures, it looks like more than 66 packages are broken;
some below.

Ricardo, I suggest to keep the symbol 'python-pyyaml' pointing to
version 5 and use 'python-pyyaml-next' for pointing to version 6.

WDYT?


Cheers,
simon


guix build: error: build of `aria-maestosa-1.4.13.drv' failed
guix build: error: build of `awscli-1.21.11.drv' failed
guix build: error: build of `balsa-2.6.3.drv' failed
guix build: error: build of `bap-2.0.0.drv' failed
guix build: error: build of `barrier-2.4.0.drv' failed
guix build: error: build of `bitcoin-abc-0.21.12.drv' failed
guix build: error: build of `bitcoin-core-0.20.2.drv' failed
guix build: error: build of `cava-0.7.4.drv' failed
guix build: error: build of `conan-1.42.0.drv' failed
guix build: error: build of `csound-6.14.0.drv' failed
guix build: error: build of `docker-compose-1.29.2.drv' failed
guix build: error: build of `ecl-dufy-0.4.1.drv' failed
guix build: error: build of `emacsy-0.4.1.drv' failed
guix build: error: build of `flowee-2020.04.1.drv' failed
guix build: error: build of `fprintd-1.94.1.drv' failed
guix build: error: build of `fulcrum-1.1.1.drv' failed
guix build: error: build of `gens-gs-7.drv' failed
guix build: error: build of `gplates-2.3.0.drv' failed
guix build: error: build of `gzochi-0.13.drv' failed
guix build: error: build of `hugin-2020.0.0.drv' failed
guix build: error: build of `inkbox-1.7.drv' failed
guix build: error: build of `itk-snap-3.8.0.drv' failed
guix build: error: build of `js-of-ocaml-4.0.0.drv' failed
guix build: error: build of `knights-025.drv' failed
guix build: error: build of `libcamera-0.0.0.drv' failed
guix build: error: build of `libgrss-0.7.0.drv' failed
guix build: error: build of `libvisual-plugins-0.4.0.drv' failed
guix build: error: build of `libxmlb-0.1.15.drv' failed
guix build: error: build of `lightdm-gtk-greeter-2.0.8.drv' failed
guix build: error: build of `lush2-2.0.1.drv' failed
guix build: error: build of `markets-0.5.3.drv' failed
guix build: error: build of `memphis-0.2.3.drv' failed
guix build: error: build of `meshlab-2020.06.drv' failed
guix build: error: build of `muse-sequencer-4.0.0.drv' failed
guix build: error: build of `nextpnr-ice40-0.0.0-1.fbe486d.drv' failed
guix build: error: build of `nip2-8.7.1.drv' failed
guix build: error: build of `ocaml-craml-1.0.0.drv' failed
guix build: error: build of `ocaml-frontc-4.1.0.drv' failed
guix build: error: build of `ocaml-merlin-4.4-413.drv' failed
guix build: error: build of `ocaml-sedlex-2.5.drv' failed
guix build: error: build of `ocaml-sqlite3-5.1.0.drv' failed
guix build: error: build of `ocamlformat-0.20.1.drv' failed
guix build: error: build of `offlate-0.5.drv' failed
guix build: error: build of `phodav-2.5.drv' failed
guix build: error: build of `pokerth-1.1.2.drv' failed
guix build: error: build of `povray-3.7.0.8.drv' failed
guix build: error: build of `prboom-plus-2.5.1.4.drv' failed
guix build: error: build of `python-capablerobot-usbhub-0.2.7.drv' failed
guix build: error: build of `python-cgatcore-0.6.7.drv' failed
guix build: error: build of `python-faiss-1.5.0.drv' failed
guix build: error: build of `python-falcon-cors-1.1.7.drv' failed
guix build: error: build of `python-mailman-hyperkitty-1.2.0.drv' failed
guix build: error: build of `python-multipart-0.0.5.drv' failed
guix build: error: build of `python-numpy-documentation-1.20.3.drv' failed
guix build: error: build of `python-woob-3.0.drv' failed
guix build: error: build of `qtcolorwidgets-0-1.a95f72e.drv' failed
guix build: error: build of `scorep-openmpi-3.1.drv' failed
guix build: error: build of `setzer-0.4.1.drv' failed
guix build: error: build of `slade-3.1.13.drv' failed
guix build: error: build of `slimevolley-2.4.2.drv' failed
guix build: error: build of `smb4k-3.0.6.drv' failed
guix build: error: build of `snapcast-0.24.0.drv' failed
guix build: error: build of `stcgal-1.6.drv' failed
guix build: error: build of `uim-qt-1.8.8.drv' failed
guix build: error: build of `xfce4-embed-plugin-1.6.0.drv' failed
guix build: error: build of `xfce4-equake-plugin-1.3.8.drv' failed

[bug#54866] [PATCH] docker-compose, python-pyyaml

[Ricardo Wurmus]

Hi,

zimoun <zimon.toutoune@gmail.com> writes:
>
> On mar., 12 avril 2022 at 00:28, daniel.herzig@outlook.at wrote:
>
>> I noticed that after my last pull to 6413d0898b92, docker-compose did
>> not build anymore. The backtrace showed that it fails in the sanity-
>> check-phase due to the requirement python-pyyaml <6.
>
> Thanks for the report.  Indeed, commit
> 27720d0fe14ee4cd413a6745e0782a698a011924 updates python-pyyaml from 5 to
> 6 and thus breaks docker-compose.
>
> Well, this update is for core-updates (or maybe staging) and not for
> master directly.
>
> $ guix refresh -l python-pyyaml | cut -f1 -d':'
> Building the following 1453 packages would ensure 2671 dependent packages are rebuilt

Yes, that’s why I pushed wip-pyyaml and had ci.guix.gnu.org build it.
On IRC I asked whether it could be merged after it had finished and I
had fixed obvious problems.  Then I waited a few days, asked again, and
eventually rebased, built it out again, and merged.

It’s hard to compare failures across branches with cuirass, and it is
now obvious that I missed a few new failures.

I really don’t think that upgrading something as lowly as pyyaml
deserves to be put on core-updates along with all the world-breaking
upgrades.  pyyaml 6 came out in Oct 2021, and the changelog compared to
the last 5.x release is *very* short:

  https://github.com/yaml/pyyaml/blob/master/CHANGES#L7

Perhaps we can just change packages that want version 5 and relax their
requirements.

> Examining the failures, it looks like more than 66 packages are broken;
> some below.
>
> Ricardo, I suggest to keep the symbol 'python-pyyaml' pointing to
> version 5 and use 'python-pyyaml-next' for pointing to version 6.
>
> WDYT?

I don’t think that’s a better idea than fixing what’s broken.  

> guix build: error: build of `aria-maestosa-1.4.13.drv' failed

Aria Maestosa seems to have been abandoned.

> guix build: error: build of `awscli-1.21.11.drv' failed

Fixed.

> guix build: error: build of `docker-compose-1.29.2.drv' failed

The current version is 2.4.1.  1.29.2 is from May 2021.

> guix build: error: build of `itk-snap-3.8.0.drv' failed

There have been no successful builds since at least January 2022
according to https://ci.guix.gnu.org/build/617429/details

> guix build: error: build of `knights-025.drv' failed

There have been no successful builds sinc at least Dec 2021 according to
https://ci.guix.gnu.org/build/618536/details

-- 
Ricardo

[bug#54866] [PATCH] docker-compose, python-pyyaml

[zimoun]

Hi Ricardo,

On Tue, 12 Apr 2022 at 23:42, Ricardo Wurmus <rekado@elephly.net> wrote:

>> Well, this update is for core-updates (or maybe staging) and not for
>> master directly.

[...]

> Yes, that’s why I pushed wip-pyyaml and had ci.guix.gnu.org build it.
> On IRC I asked whether it could be merged after it had finished and I
> had fixed obvious problems.  Then I waited a few days, asked again, and
> eventually rebased, built it out again, and merged.

Cool for proposing such workflow.  I think it is better to update that
way instead of going via (always too) long core-updates cycles. :-)


> It’s hard to compare failures across branches with cuirass, and it is
> now obvious that I missed a few new failures.

It is what I manually did, well partially though – that’s why some
packages listed below were inaccurate.

Therefore, I am building all the packages returned by “guix refresh -l
python-pyyaml” for 2 Guix revisions.  And then we will have the complete
list of packages broken by the upgrade.


> Perhaps we can just change packages that want version 5 and relax their
> requirements.
>
>> Examining the failures, it looks like more than 66 packages are broken;
>> some below.
>>
>> Ricardo, I suggest to keep the symbol 'python-pyyaml' pointing to
>> version 5 and use 'python-pyyaml-next' for pointing to version 6.
>>
>> WDYT?
>
> I don’t think that’s a better idea than fixing what’s broken.

The list of broken packages is large, from my partial observation.  And
thus it will take some time to fix all – from one day to weeks.

That’s why I propose to revert and introduce python-pyyaml-next for the
ones requiring it because it maximizes the number of non-broken
packages.


Here you examined for 5 packages, the partial list was of 66!  and I
hope to get the complete list today.

Well, it takes time to fix package by package and let the current
situation is unsatisfactory, IMHO.


Deal with all that will take some time…

>> guix build: error: build of `aria-maestosa-1.4.13.drv' failed
>
> Aria Maestosa seems to have been abandoned.

…for removing the obsolete packages.


>> guix build: error: build of `awscli-1.21.11.drv' failed
>
> Fixed.

…for fixing.


>> guix build: error: build of `docker-compose-1.29.2.drv' failed
>
> The current version is 2.4.1.  1.29.2 is from May 2021.

…for updating.


>> guix build: error: build of `itk-snap-3.8.0.drv' failed
>
> There have been no successful builds since at least January 2022
> according to https://ci.guix.gnu.org/build/617429/details

…for filtering some false-positive.


etc.


Cheers,
simon

[bug#54866] [PATCH] docker-compose, python-pyyaml

[Michael Rohleder]

[-- Attachment #1: Type: text/plain, Size: 197 bytes --]

Ricardo has added a 5.4.1 python-pyyaml with python-pyyaml-for-awscli in
7fcb16e9d4.
Maybe docker-compose can also use this version?

-- 
Listen and silent are spelled with the same letters.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

[bug#54866] [PATCH] docker-compose, python-pyyaml

[zimoun]

Hi Ricardo,

Please see #54934.  Sorry, I forgot to mention Daniel as bug reporter.
Please, could you add the mention in the commit message?

On Tue, 12 Apr 2022 at 23:42, Ricardo Wurmus <rekado@elephly.net> wrote:

> Perhaps we can just change packages that want version 5 and relax their
> requirements.

I did for 3 packages.  For some, it is not straightforward.  See the
package orange for instance.


Once #54934 merged, feel free to close this one.

Cheers,
simon