[bug#74345] Update awscli

[bug#74345] Update awscli

[Ricardo Wurmus]

This series of patches updates awscli and associated Python libraries.

-- 
Ricardo

[bug#74345] [PATCH 1/9] gnu: python-s3transfer: Update to 0.10.3.

[Ricardo Wurmus]

* gnu/packages/python-web.scm (python-s3transfer): Update to 0.10.3.
[build-system]: Use pyproject-build-system.
[arguments]: Specify test-flags instead of replacing 'check phase.
[native-inputs]: Remove python-docutils, python-mock, and python-nose; add
python-pytest.
[propagated-inputs]: Remove python-urllib3.

Change-Id: Idd705b6968577c49f6b99adae40008f67bc3db23
---
 gnu/packages/python-web.scm | 23 ++++++++++-------------
 1 file changed, 10 insertions(+), 13 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 7bbf05f3681..e93b32e3b1d 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4718,27 +4718,24 @@ (define-public python-betamax-serializers
 (define-public python-s3transfer
   (package
     (name "python-s3transfer")
-    (version "0.5.0")
+    (version "0.10.3")
     (source (origin
               (method url-fetch)
               (uri (pypi-uri "s3transfer" version))
               (sha256
                (base32
-                "0k6sc956yrrv9b4laa0r79jhxajpyxr21jcd1ka8m1n53lz85vah"))))
-    (build-system python-build-system)
+                "032bjky1q8r5x80mvb0ah60g0zq4snwf0xa4c7779m44mdsfsl2g"))))
+    (build-system pyproject-build-system)
     (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (replace 'check
-           (lambda* (#:key tests? #:allow-other-keys)
-             (when tests?
-               ;; Some of the 'integration' tests require network access or
-               ;; login credentials.
-               (invoke "nosetests" "--exclude=integration")))))))
+     (list
+      #:test-flags
+      ;; Some of the 'integration' tests require network access or
+      ;; login credentials.
+      '(list "--ignore-glob=tests/integration/*")))
     (native-inputs
-     (list python-docutils python-mock python-nose))
+     (list python-pytest))
     (propagated-inputs
-     (list python-botocore python-urllib3))
+     (list python-botocore))
     (synopsis "Amazon S3 Transfer Manager")
     (description "S3transfer is a Python library for managing Amazon S3
 transfers.")

base-commit: 07cd046c209f192032b425d2ee44f6025bff63a1
-- 
2.46.0

[bug#74345] [PATCH 2/9] gnu: python-botocore: Update to 1.35.59.

[Ricardo Wurmus]

* gnu/packages/python-xyz.scm (python-botocore): Update to 1.35.59.
[build-system]: Use pyproject-build-system.
[arguments]: Enable most tests.
[native-inputs]: Add python-jsonschema and python-pytest.

Change-Id: Iee873e6a84d582271fcbcd39e78521583574fb40
---
 gnu/packages/python-xyz.scm | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 5083473c080..3be34786be5 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -19364,20 +19364,24 @@ (define-public python-botocore
   ;; are compatible.
   (package
     (name "python-botocore")
-    (version "1.24.35")
+    (version "1.35.59")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "botocore" version))
        (sha256
         (base32
-         "0rv8mvhq5s373zdjs2yb45hzvqcqdh2lp2rbb21jjc8ciwnl5d9n"))))
-    (build-system python-build-system)
+         "161wp1ribgkc23w6wcfs6zzig2j84ava7ylxhs3jrh6zzrayc36y"))))
+    (build-system pyproject-build-system)
     (arguments
-     ;; FIXME: Many tests are failing.
-     '(#:tests? #f))
+     (list
+      #:test-flags
+      ;; FIXME: Many tests are failing.
+      '(list "--ignore-glob=tests/functional/leak/*"
+             "--ignore-glob=tests/integration/*")))
     (propagated-inputs
      (list python-dateutil python-jmespath python-urllib3))
+    (native-inputs (list python-jsonschema python-pytest))
     (home-page "https://github.com/boto/botocore")
     (synopsis "Low-level interface to AWS")
     (description "Botocore is a Python library that provides a low-level
-- 
2.46.0

[bug#74345] [PATCH 3/9] gnu: python-boto3: Update to 1.35.59.

[Ricardo Wurmus]

* gnu/packages/python-xyz.scm (python-boto3): Update to 1.35.59.
[build-system]: Use pyproject-build-system.
[arguments]: Use #:test-flags instead of custom build phases.

Change-Id: Iafc22bafbf6ef185c2d9f4907386bd904ebaecbd
---
 gnu/packages/python-xyz.scm | 20 +++++++-------------
 1 file changed, 7 insertions(+), 13 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 3be34786be5..67219049ff2 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -19391,7 +19391,7 @@ (define-public python-botocore
 (define-public python-boto3
   (package
     (name "python-boto3")
-    (version "1.21.35")
+    (version "1.35.59")
     (home-page "https://github.com/boto/boto3")
     (source (origin
               (method git-fetch)
@@ -19399,19 +19399,13 @@ (define-public python-boto3
               (file-name (git-file-name name version))
               (sha256
                (base32
-                "1kdyf238rpfldnpzs0rdh3nhjn6hwfym4faskyhzlgzkf1smmbg1"))))
+                "10bdzdaw7qg2m5n5ivb2zzsdl7wgjmz05xyxajd4cmk629ick95m"))))
+    (build-system pyproject-build-system)
     (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'delete-network-tests
-           ;; Deleting integration tests because they are trying to connect to AWS.
-           (lambda _
-             (delete-file-recursively "tests/integration")))
-         (replace 'check
-           (lambda* (#:key tests? #:allow-other-keys)
-             (when tests?
-               (invoke "pytest" "-v")))))))
-    (build-system python-build-system)
+     (list
+      #:test-flags
+      ;; Integration tests are trying to connect to AWS.
+      '(list "--ignore-glob=tests/integration/*")))
     (native-inputs
      (list python-nose python-mock python-pytest))
     (propagated-inputs
-- 
2.46.0

[bug#74345] [PATCH 4/9] gnu: awscli: Update to 1.36.0.

[Ricardo Wurmus]

* gnu/packages/python-web.scm (awscli): Update to 1.36.0.
[build-system]: Use pyproject-build-system.
[arguments]: Remove build phase 'use-recent-pyyaml; adjust phase
'fix-reference-to-groff.

Change-Id: Ife0dd9d4439e4421275ce658e1073e487cec6e80
---
 gnu/packages/python-web.scm | 15 ++++-----------
 1 file changed, 4 insertions(+), 11 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index e93b32e3b1d..c4c7586e783 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4136,31 +4136,24 @@ (define-public awscli
   (package
     ;; Note: updating awscli typically requires updating botocore as well.
     (name "awscli")
-    (version "1.22.90")
+    (version "1.36.0")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri name version))
        (sha256
         (base32
-         "0ky4ax4xh7s8w1l0hwc7w9ii8afvh9nib3kz09qhiqdinxzrlv54"))))
-    (build-system python-build-system)
+         "0jhak5bfw2mll333xgnrvaak4wh6ll4alla0mqd073nlkl0q7jcd"))))
+    (build-system pyproject-build-system)
     (arguments
      ;; FIXME: The 'pypi' release does not contain tests.
      '(#:tests? #f
        #:phases
        (modify-phases %standard-phases
-         (add-after 'unpack 'use-recent-pyyaml
-           (lambda _
-             (substitute* '("awscli.egg-info/requires.txt"
-                            "setup.cfg"
-                            "setup.py")
-               (("<5.5") "<=6"))))
          (add-after 'unpack 'fix-reference-to-groff
            (lambda* (#:key inputs #:allow-other-keys)
              (substitute* "awscli/help.py"
-               (("if not self._exists_on_path\\('groff'\\):") "")
-               (("raise ExecutableNotFoundError\\('groff'\\)") "")
+               (("if self._exists_on_path\\('groff'\\):") "if True:")
                (("cmdline = \\['groff'")
                 (string-append "cmdline = ['"
                                (search-input-file inputs "bin/groff")
-- 
2.46.0

[bug#74345] [PATCH 5/9] gnu: awscli-2: Add missing inputs.

[Ricardo Wurmus]

* gnu/packages/python-web.scm (awscli-2)[native-inputs]: Add
python-poetry-core and python-pytest.

Change-Id: Ia0d160eca47f9e39ba902481c7146ad85b848c2a
---
 gnu/packages/python-web.scm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index c4c7586e783..238c0c77efd 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4189,7 +4189,10 @@ (define-public awscli-2
      '(#:tests? #f))
     (inputs
      (list python-importlib-resources
-           python-executor))))
+           python-executor))
+    (native-inputs
+     (list python-poetry-core
+           python-pytest))))
 
 
 (define-public python-wsgiproxy2
-- 
2.46.0

[bug#74345] [PATCH 6/9] gnu: awscli-2: Update to 2.3.1.

[Ricardo Wurmus]

* gnu/packages/python-web.scm (awscli-2): Update to 2.3.1.
[name]: Rename to awscliv2.

Change-Id: Ifd9456300a7f9008e326a156a58dd5b7094639cd
---
 gnu/packages/python-web.scm | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 238c0c77efd..cbec6bf900d 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4172,18 +4172,20 @@ (define-public awscli
 Amazon Web Services (AWS) API.")
     (license license:asl2.0)))
 
+;; This is not an official release of awscli version 2, so it should not be
+;; named awscli.
 (define-public awscli-2
   (package
     (inherit awscli)
-    (name "awscli")
-    (version "2.2.0")
+    (name "awscliv2")
+    (version "2.3.1")
     (source
      (origin
        (method url-fetch)
-       (uri (pypi-uri (string-append name "v2") version))
+       (uri (pypi-uri name version))
        (sha256
         (base32
-         "0g1icsy2l4n540gnhliypy830dfp08hpfc3rk12dlxgc9v3ra4wl"))))
+         "1bpp6kmb75qdhgzsx69ki04345bfkzwnmg84y5x6nyfpph2g3fsz"))))
     (arguments
      ;; FIXME: The 'pypi' release does not contain tests.
      '(#:tests? #f))
-- 
2.46.0

[bug#74345] [PATCH 7/9] gnu: Add python-awscrt.

[Ricardo Wurmus]

* gnu/packages/python-web.scm (python-awscrt): New variable.

Change-Id: I1c08a06fdb1ea9811c183d3711f092edd12cdca7
---
 gnu/packages/python-web.scm | 74 +++++++++++++++++++++++++++++++++++++
 1 file changed, 74 insertions(+)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index cbec6bf900d..3160c807a1d 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -100,6 +100,7 @@ (define-module (gnu packages python-web)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages certs)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages cmake)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages crates-io)
   #:use-module (gnu packages crates-web)
@@ -4132,6 +4133,79 @@ (define-public python-urllib3-next
 supports url redirection and retries, and also gzip and deflate decoding.")
     (license license:expat)))
 
+(define-public python-awscrt
+  (package
+    (name "python-awscrt")
+    (version "0.23.0")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "awscrt" version))
+       (sha256
+        (base32 "0a669xxfmgw3g6xpcnm64pbmlrbxw5wf3jcrivixscl2glapdxgx"))))
+    (build-system pyproject-build-system)
+    (arguments
+     (list
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'disable-broken-tests
+            (lambda _
+              ;; Disable broken tests.  These tests fail because the
+              ;; certificate bundle at the default location does not exist.
+              (substitute* "test/test_auth.py"
+                (("def test_default_provider")
+                 "def _test_default_provider"))
+              (substitute* "test/test_http_client.py"
+                (("def test_h2_client")
+                 "def _test_h2_client"))
+              (substitute* "test/test_s3.py"
+                (("def test_sanity")
+                 "def _test_sanity")
+                (("def test_sanity_secure")
+                 "def _test_sanity_secure")
+                (("def test_wait_shutdown")
+                 "def _test_wait_shutdown"))))
+          (add-after 'unpack 'override-cert-bundle-location
+            (lambda* (#:key inputs #:allow-other-keys)
+              (let ((bundle (search-input-file inputs
+                                               "/etc/ssl/certs/ca-certificates.crt")))
+                (setenv "SSL_CERT_FILE" bundle)
+                (substitute* "awscrt/io.py"
+                  (("( +)opt = TlsContextOptions\\(\\)" m indent)
+                   (string-append m "\n"
+                                  indent "import os\n"
+                                  indent "\
+opt.override_default_trust_store_from_path(None, os.getenv('SSL_CERT_FILE')) if os.getenv('SSL_CERT_FILE') else None\n")))
+                (substitute* "test/appexit_http.py"
+                  (("( +)tls_ctx_opt = awscrt.io.TlsContextOptions.*" m indent)
+                   (string-append m indent
+                                  "tls_ctx_opt.override_default_trust_store_from_path(None, '"
+                                  bundle "')\n")))
+                (substitute* "test/test_io.py"
+                  (("( +)opt = TlsContextOptions\\(\\).*" m indent)
+                   (string-append m indent
+                                  "opt.override_default_trust_store_from_path(None, '"
+                                  bundle "')\n"))))))
+          (add-after 'unpack 'use-system-libraries
+            (lambda _
+              (setenv "AWS_CRT_BUILD_USE_SYSTEM_LIBCRYPTO" "1")))
+          (replace 'check
+            (lambda* (#:key tests? #:allow-other-keys)
+              (when tests?
+                (invoke "python3" "-m" "unittest"
+                        "discover" "--verbose")))))))
+    (inputs (list openssl))
+    (native-inputs (list cmake-minimal
+                         ;; For tests only
+                         nss-certs-for-test
+                         python-boto3
+                         python-websockets))
+    (home-page "https://github.com/awslabs/aws-crt-python")
+    (synopsis "Common runtime for AWS Python projects")
+    (description
+     "This package provides a common runtime for AWS Python projects.")
+    (license license:asl2.0)))
+
 (define-public awscli
   (package
     ;; Note: updating awscli typically requires updating botocore as well.
-- 
2.46.0

[bug#74345] [PATCH 8/9] gnu: Add python-awscrt-for-awscli.

[Ricardo Wurmus]

* gnu/packages/python-web.scm (python-awscrt-for-awscli): New variable.

Change-Id: Iea7f6467d01eee447d6605390be1066589537aae
---
 gnu/packages/python-web.scm | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 3160c807a1d..4000fdfcdd7 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4206,6 +4206,17 @@ (define-public python-awscrt
      "This package provides a common runtime for AWS Python projects.")
     (license license:asl2.0)))
 
+(define-public python-awscrt-for-awscli
+  (package
+    (inherit python-awscrt)
+    (version "0.22.0")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "awscrt" version))
+       (sha256
+        (base32 "0w6pw42jbznrxh92cd97p96dg2nz698mcbfy7md3zw18jfsb18jc"))))))
+
 (define-public awscli
   (package
     ;; Note: updating awscli typically requires updating botocore as well.
-- 
2.46.0

[bug#74345] [PATCH 9/9] gnu: Add awscli-2.

[Ricardo Wurmus]

* gnu/packages/python-web.scm (awscli-2): Rename to...
(awscliv2): ...this new variable.
(awscli-2): New variable.

Change-Id: I24158ab8125d1fec15370841334bf089f0bf7978
---
 gnu/packages/python-web.scm | 71 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 70 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 4000fdfcdd7..9127444d62f 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4257,9 +4257,78 @@ (define-public awscli
 Amazon Web Services (AWS) API.")
     (license license:asl2.0)))
 
+(define-public awscli-2
+  (package
+    (inherit awscli)
+    (name "awscli")
+    (version "2.20.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/aws/aws-cli")
+             (commit version)))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0hyr9gmcfk7nzkgs0v6wgkh8k15dyhknqzfymbc9a9sa2dblc40q"))))
+    (build-system pyproject-build-system)
+    (arguments
+     (list
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'ignore-deprecations
+            (lambda _
+              (substitute* "pyproject.toml"
+                (("\"error::") "\"ignore::"))))
+          (add-after 'unpack 'remove-pep517
+            (lambda _
+              (rename-file "backends/pep517.py" "backends/dummypep517.py")
+              (substitute* "pyproject.toml"
+                (("pep517") "dummypep517"))
+              (setenv "PYTHONPATH"
+                      (string-append (getcwd) ":"
+                                     (getcwd) "/backends:"
+                                     (getenv "GUIX_PYTHONPATH")))))
+          (add-after 'unpack 'fix-reference-to-groff
+            (lambda* (#:key inputs #:allow-other-keys)
+              (substitute* "awscli/help.py"
+                (("if self._exists_on_path\\('groff'\\):") "if True:")
+                (("cmdline = \\['groff'")
+                 (string-append "cmdline = ['"
+                                (search-input-file inputs "bin/groff")
+                                "'")))))
+          (replace 'check
+            (lambda* (#:key tests? #:allow-other-keys)
+              (when tests?
+                (substitute* "scripts/ci/run-tests"
+                  (("--numprocesses=auto --dist=loadfile --maxprocesses=4") ""))
+                ;; For an unknown reason pytest receives SIGTERM and no tests
+                ;; are run..
+                #;
+                (invoke "python" "scripts/ci/run-tests")))))))
+    (inputs
+     (list groff
+           python-awscrt-for-awscli
+           python-colorama
+           python-botocore
+           python-cryptography
+           python-dateutil
+           python-docutils
+           python-jmespath
+           python-prompt-toolkit
+           python-ruamel.yaml-0.16
+           python-ruamel.yaml.clib
+           python-urllib3))
+    (native-inputs
+     (list python-distro
+           python-flit
+           python-pytest
+           python-wheel))))
+
 ;; This is not an official release of awscli version 2, so it should not be
 ;; named awscli.
-(define-public awscli-2
+(define-public awscliv2
   (package
     (inherit awscli)
     (name "awscliv2")
-- 
2.46.0

[bug#74345] [PATCH 2/9] gnu: python-botocore: Update to 1.35.59.

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 430 bytes --]

https://git.savannah.gnu.org/cgit/guix.git/log/?h=python-team&ofs=50

Majority tests were passed successfully

https://git.savannah.gnu.org/cgit/guix.git/commit/?h=python-team&id=f4d1cbd70149df83f1d8045f27bea924b58d21ff

My plan on python team branch to fix all faults introduced by
 pyproject build system update, cherry pick "Adjust inputs"
 commits and, squash them into one, which may merge to master directly.

Thanks,

Oleg

[-- Attachment #2: Type: text/html, Size: 994 bytes --]

[bug#74345] Update awscli

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 636 bytes --]


Hi Ricardo,

> Would you like me to rebase my patches on top of python-team?  I'm
> asking because these package updates cause only very few rebuilds, and
> I'd rather not wait for a long time until python-team is merged.

Sounds like a plan for me :-) then I can skip applied commits during
rebase on master. I'm not expecting python-team is merged in upcoming
week or moth.

I've checked AWS CLI related updates after all dependencies passed
build/check phases and the command looks working. There was a reason
behind it when Nix's package became faulty for me it gave me some
encouragement to update Guix's one ^.^

--
Thanks,
Oleg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#74345] Update awscli

[Vincent Legoll]

[-- Attachment #1: Type: text/plain, Size: 316 bytes --]

Hello Ricardo,

just reading the patches, I saw in:

[PATCH 5/9] gnu: awscli-2: Add missing inputs.

That you added python-pytest to native-inputs, but the tests are
disabled, if I'm not mistaken, reading the context lines just above.

Maybe this could warrant an explanation or comment.

WDYT ?

-- 
Vincent Legoll

[-- Attachment #2: Type: text/html, Size: 618 bytes --]

[bug#74345] Update awscli

[Ricardo Wurmus]

Sharlatan Hellseher <sharlatanus@gmail.com> writes:

>> Would you like me to rebase my patches on top of python-team?  I'm
>> asking because these package updates cause only very few rebuilds, and
>> I'd rather not wait for a long time until python-team is merged.
>
> Sounds like a plan for me :-) then I can skip applied commits during
> rebase on master. I'm not expecting python-team is merged in upcoming
> week or moth.

Sorry, I'm not following.  What's the plan now?  I'd like to apply the
patches to the master branch soon, not in a month.

Do you want me to cherry pick your patches from python-team first, then
apply my updates on top of master?

-- 
Ricardo

[bug#74345] [PATCH v2 00/14] Update awscli

[Sharlatan Hellseher]

Hi Ricardo,

I've cherry picked some related commits from python-team and applies yours on
top of them. Build/Check phases passed for all of them sequentially.

Thanks,
Update awscli
Oleg

Ricardo Wurmus (8):
  gnu: python-botocore: Update to 1.35.59.
  gnu: python-boto3: Update to 1.35.59.
  gnu: awscli: Update to 1.36.0.
  gnu: awscli-2: Add missing inputs.
  gnu: awscli-2: Update to 2.3.1.
  gnu: Add python-awscrt.
  gnu: Add python-awscrt-for-awscli.
  gnu: Add awscli-2.

Sharlatan Hellseher (6):
  gnu: python-botocore: Update to 1.35.54.
  gnu: python-s3transfer: Update to 0.10.3.
  gnu: python-boto3: Update to 1.35.54.
  gnu: awscli: Update to 1.35.20.
  gnu: Remove python-boto.
  gnu: Remove python-colorama-for-awscli.

 gnu/packages/python-web.scm | 298 +++++++++++++++++++++++++++++-------
 gnu/packages/python-xyz.scm | 116 ++++++--------
 2 files changed, 288 insertions(+), 126 deletions(-)


base-commit: 9aae575828f263d71fdb16efcaebe4d8e9de7e41
-- 
2.46.0

[bug#74345] [PATCH v2 01/14] gnu: python-botocore: Update to 1.35.54.

[Sharlatan Hellseher]

* gnu/packages/python-xyz.scm (python-botocore): Update to 1.35.54.
[arguments]<tests>: Enable tests. <test-flags>: Disable some failing
tests.
[native-inputs]: Add python-jsonschema, python-pytest,
python-pytest-xdist, python-setuptools, and python-wheel.

Change-Id: I7730f85d5ace5b8643f8e9201ea46e6723b1a022
---
 gnu/packages/python-xyz.scm | 31 ++++++++++++++++++++++---------
 1 file changed, 22 insertions(+), 9 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 5083473c08..c491883431 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -19364,24 +19364,37 @@ (define-public python-botocore
   ;; are compatible.
   (package
     (name "python-botocore")
-    (version "1.24.35")
+    (version "1.35.54")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "botocore" version))
        (sha256
-        (base32
-         "0rv8mvhq5s373zdjs2yb45hzvqcqdh2lp2rbb21jjc8ciwnl5d9n"))))
-    (build-system python-build-system)
+        (base32 "0xhsi4gfmzh1r96zlaa55p9i3kvh5lj4grp866dr72lwwnfba6qk"))))
+    (build-system pyproject-build-system)
     (arguments
-     ;; FIXME: Many tests are failing.
-     '(#:tests? #f))
+     (list
+      #:test-flags
+      #~(list "--numprocesses" "auto"
+              ;; It strugles to find 'botocore'.
+              "--ignore" "tests/functional/leak/test_resource_leaks.py"
+              ;; Tests require networking.
+              "--ignore" "tests/integration")))
+    (native-inputs
+     (list python-jsonschema
+           python-pytest
+           python-pytest-xdist
+           python-setuptools
+           python-wheel))
     (propagated-inputs
-     (list python-dateutil python-jmespath python-urllib3))
+     (list python-dateutil
+           python-jmespath
+           python-urllib3))
     (home-page "https://github.com/boto/botocore")
     (synopsis "Low-level interface to AWS")
-    (description "Botocore is a Python library that provides a low-level
-interface to the Amazon Web Services (AWS) API.")
+    (description
+     "Botocore is a Python library that provides a low-level interface to the
+Amazon Web Services (AWS) API.")
     (license license:asl2.0)))
 
 (define-public python-boto3
-- 
2.46.0

[bug#74345] [PATCH v2 02/14] gnu: python-s3transfer: Update to 0.10.3.

[Sharlatan Hellseher]

* gnu/packages/python-web.scm (python-s3transfer): Update to 0.10.3.
[build-system]: Swap to pyprojct-build-system.
[native-inputs]: Add python-pytest, python-pytest-xdist,
python-setuptools, and python-wheel.

Change-Id: I41d35dc89ea6f9ff24f33f6bfcc1fc6a22ebe890
---
 gnu/packages/python-web.scm | 46 ++++++++++++++++++++-----------------
 1 file changed, 25 insertions(+), 21 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 7bbf05f368..b18248e472 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4718,31 +4718,35 @@ (define-public python-betamax-serializers
 (define-public python-s3transfer
   (package
     (name "python-s3transfer")
-    (version "0.5.0")
-    (source (origin
-              (method url-fetch)
-              (uri (pypi-uri "s3transfer" version))
-              (sha256
-               (base32
-                "0k6sc956yrrv9b4laa0r79jhxajpyxr21jcd1ka8m1n53lz85vah"))))
-    (build-system python-build-system)
+    (version "0.10.3")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "s3transfer" version))
+       (sha256
+        (base32 "032bjky1q8r5x80mvb0ah60g0zq4snwf0xa4c7779m44mdsfsl2g"))))
+    (build-system pyproject-build-system)
     (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (replace 'check
-           (lambda* (#:key tests? #:allow-other-keys)
-             (when tests?
-               ;; Some of the 'integration' tests require network access or
-               ;; login credentials.
-               (invoke "nosetests" "--exclude=integration")))))))
+     (list
+      #:test-flags
+      #~(list "--numprocesses" "auto"
+              ;; Tests require networking.
+              "--ignore" "tests/integration")))
     (native-inputs
-     (list python-docutils python-mock python-nose))
+     (list python-docutils
+           python-mock
+           python-nose
+           python-pytest
+           python-pytest-xdist
+           python-setuptools
+           python-wheel))
     (propagated-inputs
-     (list python-botocore python-urllib3))
-    (synopsis "Amazon S3 Transfer Manager")
-    (description "S3transfer is a Python library for managing Amazon S3
-transfers.")
+     (list python-botocore
+           python-urllib3))
     (home-page "https://github.com/boto/s3transfer")
+    (synopsis "Amazon S3 Transfer Manager")
+    (description
+     "S3transfer is a Python library for managing Amazon S3 transfers.")
     (license license:asl2.0)))
 
 (define-public python-flask-jwt
-- 
2.46.0

[bug#74345] [PATCH v2 03/14] gnu: python-boto3: Update to 1.35.54.

[Sharlatan Hellseher]

* gnu/packages/python-xyz.scm (python-boto3): Update to 1.35.54. Improve
package style.
[bulid-system]: Swap to pyproject-build-system.
[native-inputs]: Add python-pytest-xdist, python-setuptools, and
python-wheel.

Change-Id: I413fcfc599d0d3ccf8587136629648c8a04cd364
---
 gnu/packages/python-xyz.scm | 48 ++++++++++++++++++++-----------------
 1 file changed, 26 insertions(+), 22 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index c491883431..9ddc4ea6c2 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -19400,31 +19400,35 @@ (define-public python-botocore
 (define-public python-boto3
   (package
     (name "python-boto3")
-    (version "1.21.35")
-    (home-page "https://github.com/boto/boto3")
-    (source (origin
-              (method git-fetch)
-              (uri (git-reference (url home-page) (commit version)))
-              (file-name (git-file-name name version))
-              (sha256
-               (base32
-                "1kdyf238rpfldnpzs0rdh3nhjn6hwfym4faskyhzlgzkf1smmbg1"))))
+    (version "1.35.54")
+    (source
+     (origin
+       (method git-fetch)               ; no tests in PyPI release
+       (uri (git-reference
+             (url "https://github.com/boto/boto3")
+             (commit version)))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "0w0k719i8pyzkvd2snv6z91r57k6cxkq1i3phygdpwzk0k6acvj2"))))
+    (build-system pyproject-build-system)
     (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'delete-network-tests
-           ;; Deleting integration tests because they are trying to connect to AWS.
-           (lambda _
-             (delete-file-recursively "tests/integration")))
-         (replace 'check
-           (lambda* (#:key tests? #:allow-other-keys)
-             (when tests?
-               (invoke "pytest" "-v")))))))
-    (build-system python-build-system)
+     (list
+      #:test-flags
+      #~(list "--numprocesses" "auto"
+              ;; Tests require networking.
+              "--ignore" "tests/integration")))
     (native-inputs
-     (list python-nose python-mock python-pytest))
+     (list python-mock
+           python-nose
+           python-pytest
+           python-pytest-xdist
+           python-setuptools
+           python-wheel))
     (propagated-inputs
-     (list python-botocore python-jmespath python-s3transfer))
+     (list python-botocore
+           python-jmespath
+           python-s3transfer))
+    (home-page "https://github.com/boto/boto3")
     (synopsis "AWS SDK for Python")
     (description
      "Boto3 is a Python library for writing programs that interact with
-- 
2.46.0

[bug#74345] [PATCH v2 04/14] gnu: awscli: Update to 1.35.20.

[Sharlatan Hellseher]

* gnu/packages/python-web.scm (awscli): Update to 1.35.20. Use
G-expressions.
[source]: Swap to git checkout as it contains tests.
[build-system]: Swap to pyproject-build-system.
[phases]: Delete use-recent-pyyaml phase; add set-invorenment phase.
[inputs]: Remove groff, python-colorama-for-awscli, and python-pyyaml-5;
add groff-minimal, python-colorama, and python-pyyaml.
[native-inputs]: Add python-pytest, python-pytest-xdist,
python-setuptools, and python-wheel.

Change-Id: Ib7542a783fe65bb93d9f4cbfb89241190813fde4
---
 gnu/packages/python-web.scm | 81 ++++++++++++++++++++++---------------
 1 file changed, 49 insertions(+), 32 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index b18248e472..1421936eb6 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4136,47 +4136,64 @@ (define-public awscli
   (package
     ;; Note: updating awscli typically requires updating botocore as well.
     (name "awscli")
-    (version "1.22.90")
+    (version "1.35.20")
     (source
      (origin
-       (method url-fetch)
-       (uri (pypi-uri name version))
+       (method git-fetch)               ; no tests in PyPI release
+       (uri (git-reference
+             (url "https://github.com/aws/aws-cli")
+             (commit version)))
+       (file-name (git-file-name name version))
        (sha256
-        (base32
-         "0ky4ax4xh7s8w1l0hwc7w9ii8afvh9nib3kz09qhiqdinxzrlv54"))))
-    (build-system python-build-system)
+        (base32 "1hj1hj374hdwb8wq2xw20ywjyrv37s65nfsjzs6k9wa0f629alkf"))))
+    (build-system pyproject-build-system)
     (arguments
-     ;; FIXME: The 'pypi' release does not contain tests.
-     '(#:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'use-recent-pyyaml
-           (lambda _
-             (substitute* '("awscli.egg-info/requires.txt"
-                            "setup.cfg"
-                            "setup.py")
-               (("<5.5") "<=6"))))
-         (add-after 'unpack 'fix-reference-to-groff
-           (lambda* (#:key inputs #:allow-other-keys)
-             (substitute* "awscli/help.py"
-               (("if not self._exists_on_path\\('groff'\\):") "")
-               (("raise ExecutableNotFoundError\\('groff'\\)") "")
-               (("cmdline = \\['groff'")
-                (string-append "cmdline = ['"
-                               (search-input-file inputs "bin/groff")
-                               "'"))))))))
+     (list
+      #:test-flags
+      #~(list "--numprocesses" "auto"
+              ;; Tests require networking.
+              "--ignore" "tests/integration"
+              ;; It strugles to set PYTHONPATH.
+              ;;
+              ;; AssertionError: 'argument operation: Invalid choice, valid
+              ;; choices are:' not found in '
+              "-k" (string-append "not test_subscribe_to_shard_removed"
+                                  " and not test_start_conversation_removed"))
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'fix-reference-to-groff
+            (lambda _
+              ;; XXX: Consider to use wrap-program instead, it tries to parse
+              ;; the PATH.
+              (substitute* "awscli/help.py"
+                (("if self._exists_on_path\\('groff'\\):") "if 'groff':")
+                (("raise ExecutableNotFoundError\\('groff'\\)") "")
+                (("cmdline = \\['groff'")
+                 (format #f "cmdline = ['~a/bin/groff'"
+                         #$(this-package-input "groff-minimal"))))))
+          (add-before 'check 'set-environment
+            (lambda _
+              ;; PermissionError: [Errno 13] Permission denied:
+              ;; '/homeless-shelter'
+              (setenv "HOME" "/tmp"))))))
+    (native-inputs
+     (list python-pytest
+           python-pytest-xdist
+           python-setuptools
+           python-wheel))
     (inputs
-     (list groff
-           python-colorama-for-awscli
+     (list groff-minimal
            python-botocore
-           python-s3transfer
+           python-colorama
            python-docutils-0.15
-           python-pyyaml-5
-           python-rsa))
+           python-pyyaml
+           python-rsa
+           python-s3transfer))
     (home-page "https://aws.amazon.com/cli/")
     (synopsis "Command line client for AWS")
-    (description "AWS CLI provides a unified command line interface to the
-Amazon Web Services (AWS) API.")
+    (description
+     "AWS CLI provides a unified command line interface to the Amazon Web
+Services (AWS) API.")
     (license license:asl2.0)))
 
 (define-public awscli-2
-- 
2.46.0

[bug#74345] [PATCH v2 05/14] gnu: Remove python-boto.

[Sharlatan Hellseher]

* gnu/packages/python-xyz.scm (python-boto): Delete variable.

Change-Id: I4ff3f200734bce027334383d1f56c28134808a9d
---
 gnu/packages/python-xyz.scm | 27 ---------------------------
 1 file changed, 27 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 9ddc4ea6c2..ebf7e89225 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -19332,33 +19332,6 @@ (define-public python-bokeh
 browser from Python.")
     (license license:bsd-3)))
 
-(define-public python-boto
-  (package
-    (name "python-boto")
-    (version "2.49.0")
-    (source (origin
-              (method url-fetch)
-              (uri (pypi-uri "boto" version))
-              (sha256
-               (base32
-                "0njy09c4wjx7ipxhwi6vv404nflyiasl78vwwxxpclnql903n3ga"))))
-    (build-system python-build-system)
-    (arguments
-     ;; XXX: This package is unmaintained and has problems with newer versions
-     ;; of Python 3 as well as test libraries.  'python-moto' still uses a
-     ;; subset of this library, so keep it around for now, but disable tests.
-     '(#:tests? #f))
-    (propagated-inputs
-     (list python-paramiko python-requests))
-    (home-page "https://github.com/boto/boto")
-    (synopsis "Python interfaces for Amazon Web Services")
-    (description
-     "This package provides various facilities for interacting with Amazon
-Web Services through Python.
-
-This software is unmaintained, and new projects should use @code{boto3} instead.")
-    (license license:expat)))
-
 (define-public python-botocore
   ;; Note: When updating botocore, also make sure that boto3 and awscli
   ;; are compatible.
-- 
2.46.0

[bug#74345] [PATCH v2 06/14] gnu: Remove python-colorama-for-awscli.

[Sharlatan Hellseher]

* gnu/packages/python-xyz.scm (python-colorama-for-awscli): Delete variable.

Change-Id: Idb10200319c71fde3fcb78f1b4b0d6f9554166a0
---
 gnu/packages/python-xyz.scm | 12 ------------
 1 file changed, 12 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index ebf7e89225..73b645b1a1 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -18914,18 +18914,6 @@ (define-public python-colorama
    (home-page "https://pypi.org/project/colorama/")
    (license license:bsd-3)))
 
-;; awscli and botocore do not accept version 0.4.4
-(define-public python-colorama-for-awscli
-  (package
-    (inherit python-colorama)
-    (version "0.4.3")
-    (source
-     (origin
-       (method url-fetch)
-       (uri (pypi-uri "colorama" version))
-       (sha256
-        (base32 "189n8hpijy14jfan4ha9f5n06mnl33cxz7ay92wjqgkr639s0vg9"))))))
-
 (define-public python-monthdelta
   (package
     (name "python-monthdelta")
-- 
2.46.0

[bug#74345] [PATCH v2 07/14] gnu: python-botocore: Update to 1.35.59.

[Sharlatan Hellseher]

From: Ricardo Wurmus <rekado@elephly.net>

* gnu/packages/python-xyz.scm (python-botocore): Update to 1.35.59.

Change-Id: Iee873e6a84d582271fcbcd39e78521583574fb40
---
 gnu/packages/python-xyz.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 73b645b1a1..da945e12fe 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -19325,13 +19325,14 @@ (define-public python-botocore
   ;; are compatible.
   (package
     (name "python-botocore")
-    (version "1.35.54")
+    (version "1.35.59")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "botocore" version))
        (sha256
-        (base32 "0xhsi4gfmzh1r96zlaa55p9i3kvh5lj4grp866dr72lwwnfba6qk"))))
+        (base32
+         "161wp1ribgkc23w6wcfs6zzig2j84ava7ylxhs3jrh6zzrayc36y"))))
     (build-system pyproject-build-system)
     (arguments
      (list
-- 
2.46.0

[bug#74345] [PATCH v2 08/14] gnu: python-boto3: Update to 1.35.59.

[Sharlatan Hellseher]

From: Ricardo Wurmus <rekado@elephly.net>

* gnu/packages/python-xyz.scm (python-boto3): Update to 1.35.59.
[native-inputs]: Remove python-nose.

Change-Id: Iafc22bafbf6ef185c2d9f4907386bd904ebaecbd
---
 gnu/packages/python-xyz.scm | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index da945e12fe..ca021c2776 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -19362,7 +19362,7 @@ (define-public python-botocore
 (define-public python-boto3
   (package
     (name "python-boto3")
-    (version "1.35.54")
+    (version "1.35.59")
     (source
      (origin
        (method git-fetch)               ; no tests in PyPI release
@@ -19371,17 +19371,16 @@ (define-public python-boto3
              (commit version)))
        (file-name (git-file-name name version))
        (sha256
-        (base32 "0w0k719i8pyzkvd2snv6z91r57k6cxkq1i3phygdpwzk0k6acvj2"))))
+        (base32 "10bdzdaw7qg2m5n5ivb2zzsdl7wgjmz05xyxajd4cmk629ick95m"))))
     (build-system pyproject-build-system)
     (arguments
      (list
       #:test-flags
       #~(list "--numprocesses" "auto"
-              ;; Tests require networking.
+              ;; Integration tests are trying to connect to AWS.
               "--ignore" "tests/integration")))
     (native-inputs
      (list python-mock
-           python-nose
            python-pytest
            python-pytest-xdist
            python-setuptools
-- 
2.46.0

[bug#74345] [PATCH v2 09/14] gnu: awscli: Update to 1.36.0.

[Sharlatan Hellseher]

From: Ricardo Wurmus <rekado@elephly.net>

* gnu/packages/python-web.scm (awscli): Update to 1.36.0.
[arguments]<test-flags>: Silent more failing tests.

Co-Authored-By: Sharlatan Hellseher <sharlatanus@gmail.com>
Change-Id: Ia06ad5128b13dbc68ec92f93dc025644605cd30c
---
 gnu/packages/python-web.scm | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 1421936eb6..ba689a8a98 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4136,7 +4136,7 @@ (define-public awscli
   (package
     ;; Note: updating awscli typically requires updating botocore as well.
     (name "awscli")
-    (version "1.35.20")
+    (version "1.36.0")
     (source
      (origin
        (method git-fetch)               ; no tests in PyPI release
@@ -4145,7 +4145,7 @@ (define-public awscli
              (commit version)))
        (file-name (git-file-name name version))
        (sha256
-        (base32 "1hj1hj374hdwb8wq2xw20ywjyrv37s65nfsjzs6k9wa0f629alkf"))))
+        (base32 "1iwivc5kj2h5619rvbncrh4649lalxj7bxndzvrjw398vv7cixp5"))))
     (build-system pyproject-build-system)
     (arguments
      (list
@@ -4158,6 +4158,10 @@ (define-public awscli
               ;; AssertionError: 'argument operation: Invalid choice, valid
               ;; choices are:' not found in '
               "-k" (string-append "not test_subscribe_to_shard_removed"
+                                  ;; Tests fail during mocking.
+                                  " and not test_expected_runtime_dependencies"
+                                  " and not test_expected_unbounded_runtime_dependencies"
+                                  " and not test_no_groff_or_mandoc_exists"
                                   " and not test_start_conversation_removed"))
       #:phases
       #~(modify-phases %standard-phases
-- 
2.46.0

[bug#74345] [PATCH v2 10/14] gnu: awscli-2: Add missing inputs.

[Sharlatan Hellseher]

From: Ricardo Wurmus <rekado@elephly.net>

* gnu/packages/python-web.scm (awscli-2)[native-inputs]: Add
python-poetry-core and python-pytest.

Change-Id: Ia0d160eca47f9e39ba902481c7146ad85b848c2a
---
 gnu/packages/python-web.scm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index ba689a8a98..a80d037bdf 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4217,7 +4217,10 @@ (define-public awscli-2
      '(#:tests? #f))
     (inputs
      (list python-importlib-resources
-           python-executor))))
+           python-executor))
+    (native-inputs
+     (list python-poetry-core
+           python-pytest))))
 
 
 (define-public python-wsgiproxy2
-- 
2.46.0

[bug#74345] [PATCH v2 11/14] gnu: awscli-2: Update to 2.3.1.

[Sharlatan Hellseher]

From: Ricardo Wurmus <rekado@elephly.net>

* gnu/packages/python-web.scm (awscli-2): Update to 2.3.1.
[name]: Rename to awscliv2.

Change-Id: I5fcc7afe66490513c6769c19f961a5fcb497d422
---
 gnu/packages/python-web.scm | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index a80d037bdf..32662503a1 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4200,18 +4200,20 @@ (define-public awscli
 Services (AWS) API.")
     (license license:asl2.0)))
 
+;; This is not an official release of awscli version 2, so it should not be
+;; named awscli.
 (define-public awscli-2
   (package
     (inherit awscli)
-    (name "awscli")
-    (version "2.2.0")
+    (name "awscliv2")
+    (version "2.3.1")
     (source
      (origin
        (method url-fetch)
-       (uri (pypi-uri (string-append name "v2") version))
+       (uri (pypi-uri name version))
        (sha256
         (base32
-         "0g1icsy2l4n540gnhliypy830dfp08hpfc3rk12dlxgc9v3ra4wl"))))
+         "1bpp6kmb75qdhgzsx69ki04345bfkzwnmg84y5x6nyfpph2g3fsz"))))
     (arguments
      ;; FIXME: The 'pypi' release does not contain tests.
      '(#:tests? #f))
-- 
2.46.0

[bug#74345] [PATCH v2 12/14] gnu: Add python-awscrt.

[Sharlatan Hellseher]

From: Ricardo Wurmus <rekado@elephly.net>

* gnu/packages/python-web.scm (python-awscrt): New variable.

Change-Id: I1c08a06fdb1ea9811c183d3711f092edd12cdca7
---
 gnu/packages/python-web.scm | 74 +++++++++++++++++++++++++++++++++++++
 1 file changed, 74 insertions(+)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 32662503a1..eab9a7efc5 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -100,6 +100,7 @@ (define-module (gnu packages python-web)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages certs)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages cmake)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages crates-io)
   #:use-module (gnu packages crates-web)
@@ -4132,6 +4133,79 @@ (define-public python-urllib3-next
 supports url redirection and retries, and also gzip and deflate decoding.")
     (license license:expat)))
 
+(define-public python-awscrt
+  (package
+    (name "python-awscrt")
+    (version "0.23.0")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "awscrt" version))
+       (sha256
+        (base32 "0a669xxfmgw3g6xpcnm64pbmlrbxw5wf3jcrivixscl2glapdxgx"))))
+    (build-system pyproject-build-system)
+    (arguments
+     (list
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'disable-broken-tests
+            (lambda _
+              ;; Disable broken tests.  These tests fail because the
+              ;; certificate bundle at the default location does not exist.
+              (substitute* "test/test_auth.py"
+                (("def test_default_provider")
+                 "def _test_default_provider"))
+              (substitute* "test/test_http_client.py"
+                (("def test_h2_client")
+                 "def _test_h2_client"))
+              (substitute* "test/test_s3.py"
+                (("def test_sanity")
+                 "def _test_sanity")
+                (("def test_sanity_secure")
+                 "def _test_sanity_secure")
+                (("def test_wait_shutdown")
+                 "def _test_wait_shutdown"))))
+          (add-after 'unpack 'override-cert-bundle-location
+            (lambda* (#:key inputs #:allow-other-keys)
+              (let ((bundle (search-input-file inputs
+                                               "/etc/ssl/certs/ca-certificates.crt")))
+                (setenv "SSL_CERT_FILE" bundle)
+                (substitute* "awscrt/io.py"
+                  (("( +)opt = TlsContextOptions\\(\\)" m indent)
+                   (string-append m "\n"
+                                  indent "import os\n"
+                                  indent "\
+opt.override_default_trust_store_from_path(None, os.getenv('SSL_CERT_FILE')) if os.getenv('SSL_CERT_FILE') else None\n")))
+                (substitute* "test/appexit_http.py"
+                  (("( +)tls_ctx_opt = awscrt.io.TlsContextOptions.*" m indent)
+                   (string-append m indent
+                                  "tls_ctx_opt.override_default_trust_store_from_path(None, '"
+                                  bundle "')\n")))
+                (substitute* "test/test_io.py"
+                  (("( +)opt = TlsContextOptions\\(\\).*" m indent)
+                   (string-append m indent
+                                  "opt.override_default_trust_store_from_path(None, '"
+                                  bundle "')\n"))))))
+          (add-after 'unpack 'use-system-libraries
+            (lambda _
+              (setenv "AWS_CRT_BUILD_USE_SYSTEM_LIBCRYPTO" "1")))
+          (replace 'check
+            (lambda* (#:key tests? #:allow-other-keys)
+              (when tests?
+                (invoke "python3" "-m" "unittest"
+                        "discover" "--verbose")))))))
+    (inputs (list openssl))
+    (native-inputs (list cmake-minimal
+                         ;; For tests only
+                         nss-certs-for-test
+                         python-boto3
+                         python-websockets))
+    (home-page "https://github.com/awslabs/aws-crt-python")
+    (synopsis "Common runtime for AWS Python projects")
+    (description
+     "This package provides a common runtime for AWS Python projects.")
+    (license license:asl2.0)))
+
 (define-public awscli
   (package
     ;; Note: updating awscli typically requires updating botocore as well.
-- 
2.46.0

[bug#74345] [PATCH v2 13/14] gnu: Add python-awscrt-for-awscli.

[Sharlatan Hellseher]

From: Ricardo Wurmus <rekado@elephly.net>

* gnu/packages/python-web.scm (python-awscrt-for-awscli): New variable.

Change-Id: Iea7f6467d01eee447d6605390be1066589537aae
---
 gnu/packages/python-web.scm | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index eab9a7efc5..d3ef984a07 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4206,6 +4206,17 @@ (define-public python-awscrt
      "This package provides a common runtime for AWS Python projects.")
     (license license:asl2.0)))
 
+(define-public python-awscrt-for-awscli
+  (package
+    (inherit python-awscrt)
+    (version "0.22.0")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "awscrt" version))
+       (sha256
+        (base32 "0w6pw42jbznrxh92cd97p96dg2nz698mcbfy7md3zw18jfsb18jc"))))))
+
 (define-public awscli
   (package
     ;; Note: updating awscli typically requires updating botocore as well.
-- 
2.46.0

[bug#74345] [PATCH v2 14/14] gnu: Add awscli-2.

[Sharlatan Hellseher]

From: Ricardo Wurmus <rekado@elephly.net>

* gnu/packages/python-web.scm (awscli-2): Rename to...
(awscliv2): ...this new variable.
(awscli-2): New variable.

Change-Id: Id5c356ae0e6af53167d4acde7934f88c2f527a00
---
 gnu/packages/python-web.scm | 71 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 70 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index d3ef984a07..bc18c282c6 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -4285,9 +4285,78 @@ (define-public awscli
 Services (AWS) API.")
     (license license:asl2.0)))
 
+(define-public awscli-2
+  (package
+    (inherit awscli)
+    (name "awscli")
+    (version "2.20.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/aws/aws-cli")
+             (commit version)))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0hyr9gmcfk7nzkgs0v6wgkh8k15dyhknqzfymbc9a9sa2dblc40q"))))
+    (build-system pyproject-build-system)
+    (arguments
+     (list
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'ignore-deprecations
+            (lambda _
+              (substitute* "pyproject.toml"
+                (("\"error::") "\"ignore::"))))
+          (add-after 'unpack 'remove-pep517
+            (lambda _
+              (rename-file "backends/pep517.py" "backends/dummypep517.py")
+              (substitute* "pyproject.toml"
+                (("pep517") "dummypep517"))
+              (setenv "PYTHONPATH"
+                      (string-append (getcwd) ":"
+                                     (getcwd) "/backends:"
+                                     (getenv "GUIX_PYTHONPATH")))))
+          (add-after 'unpack 'fix-reference-to-groff
+            (lambda* (#:key inputs #:allow-other-keys)
+              (substitute* "awscli/help.py"
+                (("if self._exists_on_path\\('groff'\\):") "if True:")
+                (("cmdline = \\['groff'")
+                 (string-append "cmdline = ['"
+                                (search-input-file inputs "bin/groff")
+                                "'")))))
+          (replace 'check
+            (lambda* (#:key tests? #:allow-other-keys)
+              (when tests?
+                (substitute* "scripts/ci/run-tests"
+                  (("--numprocesses=auto --dist=loadfile --maxprocesses=4") ""))
+                ;; For an unknown reason pytest receives SIGTERM and no tests
+                ;; are run..
+                #;
+                (invoke "python" "scripts/ci/run-tests")))))))
+    (inputs
+     (list groff
+           python-awscrt-for-awscli
+           python-colorama
+           python-botocore
+           python-cryptography
+           python-dateutil
+           python-docutils
+           python-jmespath
+           python-prompt-toolkit
+           python-ruamel.yaml-0.16
+           python-ruamel.yaml.clib
+           python-urllib3))
+    (native-inputs
+     (list python-distro
+           python-flit
+           python-pytest
+           python-wheel))))
+
 ;; This is not an official release of awscli version 2, so it should not be
 ;; named awscli.
-(define-public awscli-2
+(define-public awscliv2
   (package
     (inherit awscli)
     (name "awscliv2")
-- 
2.46.0

[bug#74345] Update awscli

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 109 bytes --]


Hi Ricardo,

Would like to wait for QA picks up the patches or we may apply them?

--
Thanks,
Oleg 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#74345] Update awscli

[Ricardo Wurmus]

> Would like to wait for QA picks up the patches or we may apply them?

I think it's fine to apply them.  Thanks!

-- 
Ricardo

bug#74345: Update awscli

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 1753 bytes --]


Hi,

Pushed as ea6c381cac..391de92713 to master.

- [X] pateches [9/9]
  - [X] skip [PATCH 1/9] gnu: python-s3transfer: Update to 0.10.3.
  - [X] apply [PATCH 2/9] gnu: python-botocore: Update to 1.35.59.
  - [X] apply [PATCH 3/9] gnu: python-boto3: Update to 1.35.59.
  - [X] apply [PATCH 4/9] gnu: awscli: Update to 1.36.0.
  - [X] apply [PATCH 5/9] gnu: awscli-2: Add missing inputs.
  - [X] apply [PATCH 6/9] gnu: awscli-2: Update to 2.3.1.
    - manually applied failed with am for some reason
  - [X] apply [PATCH 7/9] gnu: Add python-awscrt.
  - [X] apply [PATCH 8/9] gnu: Add python-awscrt-for-awscli.
  - [X] apply [PATCH 9/9] gnu: Add awscli-2.
    - manually applied failed with am for some reason
- [X] issues [7/7]
  - [X] awscli@2.2.0 downloads and runs a docker container from a large american corporation
    <https://issues.guix.gnu.org/68264>
    submitted on <5 Jan 15:40 +0100>
  - [X] [PATCH 0/3] Update awscli to 1.28.0
    <https://issues.guix.gnu.org/64474>
    submitted on <5 Jul 2023 13:02>
  - [X] [PATCH 0/4] Updating awscli and dependencies
     <https://issues.guix.gnu.org/69767>
     submitted on <13 Mar 02:18 +0100>
  - [X] [PATCH 1/4] gnu: python-xyz: Update python-botocore to 1.34.61.
     <https://issues.guix.gnu.org/69769>
     submitted on <13 Mar 02:09 +0100>
  - [X] [PATCH 2/4] gnu: python-xyz: Update python-boto3 to 1.34.61.
     <https://issues.guix.gnu.org/69766>
    submitted on <13 Mar 02:10 +0100>
  - [X] [PATCH 3/4] gnu: python-web: Update python-s3transfer to 0.10.0.
    <https://issues.guix.gnu.org/69765>
    submitted on <13 Mar 02:11 +0100>
  - [X] [PATCH 4/4] gnu: python-web: Update awscli to 1.32.61.
    <https://issues.guix.gnu.org/69768>
    submitted on <13 Mar 02:12 +0100>

--
Oleg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]