[bug#65114] [PATCH 0/6] Update Scrapy.

[bug#65114] [PATCH 0/6] Update Scrapy.

[Vinicius Monego]

Vinicius Monego (6):
  gnu: python-w3lib: Update to 2.1.2.
  gnu: python-cssselect: Update to 1.2.0.
  gnu: python-parsel: Update to 1.8.1.
  gnu: python-itemadapter: Update to 0.8.0.
  gnu: python-itemloaders: Update to 1.1.0.
  gnu: python-scrapy: Update to 2.10.0.

 gnu/local.mk                                  |  1 -
 .../python-w3lib-fix-test-failure.patch       | 60 --------------
 gnu/packages/python-web.scm                   | 80 +++++++------------
 gnu/packages/python-xyz.scm                   | 13 +--
 4 files changed, 38 insertions(+), 116 deletions(-)
 delete mode 100644 gnu/packages/patches/python-w3lib-fix-test-failure.patch

-- 
2.34.1

[bug#65114] [PATCH 1/6] gnu: python-w3lib: Update to 2.1.2.

[Vinicius Monego]

* gnu/packages/python-web.scm (python-w3lib): Update to 2.1.2.
[source]: Remove patch.
[build-system]: Use pyproject-build-system.
[arguments]: Do not override the check phase.
[native-inputs]: Remove python-six.
* gnu/packages/patches/python-w3lib-fix-test-failure.patch: Delete file.
* gnu/local.mk (dist_patch_DATA): Remove it.
---
 gnu/local.mk                                  |  1 -
 .../python-w3lib-fix-test-failure.patch       | 60 -------------------
 gnu/packages/python-web.scm                   | 16 ++---
 3 files changed, 4 insertions(+), 73 deletions(-)
 delete mode 100644 gnu/packages/patches/python-w3lib-fix-test-failure.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 77707127a7..e091b310c2 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1730,7 +1730,6 @@ dist_patch_DATA =						\
   %D%/packages/patches/python-shiboken-2-compat.patch		\
   %D%/packages/patches/python-telingo-fix-comparison.patch	\
   %D%/packages/patches/python-typeguard-python3.10.patch	\
-  %D%/packages/patches/python-w3lib-fix-test-failure.patch	\
   %D%/packages/patches/python-wxwidgets-type-errors.patch	\
   %D%/packages/patches/quodlibet-fix-invalid-glob.patch	\
   %D%/packages/patches/quodlibet-fix-mtime-tests.patch		\
diff --git a/gnu/packages/patches/python-w3lib-fix-test-failure.patch b/gnu/packages/patches/python-w3lib-fix-test-failure.patch
deleted file mode 100644
index f38fc749df..0000000000
--- a/gnu/packages/patches/python-w3lib-fix-test-failure.patch
+++ /dev/null
@@ -1,60 +0,0 @@
-From fae6cc40e112cd13697cb0e8d79976f32c72491d Mon Sep 17 00:00:00 2001
-From: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
-Date: Wed, 10 Mar 2021 12:31:05 -0300
-Subject: [PATCH] [CI] Mark single add_or_replace_parameter test as xfail
-
----
-This patch is based on upstream commit
-fae6cc40e112cd13697cb0e8d79976f32c72491d which is part of
-https://github.com/scrapy/w3lib/pull/166.
-
- .gitignore        |  3 ++-
- tests/test_url.py | 25 ++++++++++++++-----------
- 2 files changed, 16 insertions(+), 12 deletions(-)
-
-diff --git a/tests/test_url.py b/tests/test_url.py
-index 8b07c00..0f7458e 100644
---- a/tests/test_url.py
-+++ b/tests/test_url.py
-@@ -1,12 +1,14 @@
--# -*- coding: utf-8 -*-
- from __future__ import absolute_import
- import os
- import unittest
-+
-+import pytest
-+from six.moves.urllib.parse import urlparse
-+
- from w3lib.url import (is_url, safe_url_string, safe_download_url,
-     url_query_parameter, add_or_replace_parameter, url_query_cleaner,
-     file_uri_to_path, parse_data_uri, path_to_file_uri, any_to_uri,
-     urljoin_rfc, canonicalize_url, parse_url, add_or_replace_parameters)
--from six.moves.urllib.parse import urlparse
- 
- 
- class UrlTests(unittest.TestCase):
-@@ -310,10 +311,6 @@ def test_add_or_replace_parameter(self):
-         self.assertEqual(add_or_replace_parameter(url, 'arg3', 'nv3'),
-                          'http://domain/test?arg1=v1&arg2=v2&arg3=nv3')
- 
--        url = 'http://domain/test?arg1=v1;arg2=v2'
--        self.assertEqual(add_or_replace_parameter(url, 'arg1', 'v3'),
--                         'http://domain/test?arg1=v3&arg2=v2')
--
-         self.assertEqual(add_or_replace_parameter("http://domain/moreInfo.asp?prodID=", 'prodID', '20'),
-                          'http://domain/moreInfo.asp?prodID=20')
-         url = 'http://rmc-offers.co.uk/productlist.asp?BCat=2%2C60&CatID=60'
-@@ -338,6 +335,13 @@ def test_add_or_replace_parameter(self):
-         self.assertEqual(add_or_replace_parameter(url, 'arg1', 'v3'),
-                          'http://domain/test?arg1=v3&arg2=v2')
- 
-+    @pytest.mark.xfail(reason="https://github.com/scrapy/w3lib/issues/164")
-+    def test_add_or_replace_parameter_fail(self):
-+        self.assertEqual(
-+            add_or_replace_parameter('http://domain/test?arg1=v1;arg2=v2', 'arg1', 'v3'),
-+            'http://domain/test?arg1=v3&arg2=v2'
-+        )
-+
-     def test_add_or_replace_parameters(self):
-         url = 'http://domain/test'
-         self.assertEqual(add_or_replace_parameters(url, {'arg': 'v'}),
diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 17c9563cfd..24071a8553 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -7752,25 +7752,17 @@ (define-public python-smart-open
 (define-public python-w3lib
   (package
     (name "python-w3lib")
-    (version "1.22.0")
+    (version "2.1.2")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "w3lib" version))
-       (patches (search-patches "python-w3lib-fix-test-failure.patch"))
        (sha256
         (base32
-         "1pv02lvvmgz2qb61vz1jkjc04fgm4hpfvaj5zm4i3mjp64hd1mha"))))
-    (build-system python-build-system)
-    (arguments
-     `(#:phases
-        (modify-phases %standard-phases
-          (replace 'check
-            (lambda* (#:key tests? #:allow-other-keys)
-              (when tests?
-                (invoke "pytest")))))))
+         "1cd4b3w5g3pfccsg79kjj27fwi216ip927rjq7isp8pfjzlp8nzd"))))
+    (build-system pyproject-build-system)
     (native-inputs
-     (list python-pytest python-six))
+     (list python-pytest))
     (home-page "https://github.com/scrapy/w3lib")
     (synopsis "Python library of web-related functions")
     (description
-- 
2.34.1

[bug#65114] [PATCH 2/6] gnu: python-cssselect: Update to 1.2.0.

[Vinicius Monego]

* gnu/packages/python-web.scm (python-cssselect): Update to 1.2.0.
[build-system]: Use pyproject-build-system.
[arguments]: Do not override the check phase.
---
 gnu/packages/python-web.scm | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 24071a8553..9ecd9f35a3 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -2095,7 +2095,7 @@ (define-public python-css-parser
 (define-public python-cssselect
   (package
     (name "python-cssselect")
-    (version "1.1.0")
+    (version "1.2.0")
     (source (origin
               ;; The PyPI release does not contain tests.
               (method git-fetch)
@@ -2105,13 +2105,8 @@ (define-public python-cssselect
               (file-name (git-file-name name version))
               (sha256
                (base32
-                "0xslrnhbrmgakp4xg6k26qffay3kqffp3a2z2sk27c65rwxa79kc"))))
-    (build-system python-build-system)
-    (arguments
-     `(#:phases (modify-phases %standard-phases
-                  (replace 'check
-                    (lambda _
-                      (invoke "pytest" "-vv"))))))
+                "1x4nrvb1p1byi1whmspik7lbh303akdlh762dayfxam3hycsh5kk"))))
+    (build-system pyproject-build-system)
     (native-inputs
      (list python-lxml python-pytest))
     (home-page "https://github.com/scrapy/cssselect")
-- 
2.34.1

[bug#65114] [PATCH 3/6] gnu: python-parsel: Update to 1.8.1.

[Vinicius Monego]

* gnu/packages/python-web.scm (python-parsel): Update to 1.8.1.
[build-system]: Use pyproject-build-system.
[propagated-inputs]: Add python-jmespath, python-typing-extensions.  Remove
python-six.
[native-inputs]: Add python-psutil.  Remove python-pytest-runner.
---
 gnu/packages/python-web.scm | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 9ecd9f35a3..70a1ab082f 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -8039,21 +8039,22 @@ (define-public python-protego
 (define-public python-parsel
   (package
     (name "python-parsel")
-    (version "1.6.0")
+    (version "1.8.1")
     (source
       (origin
         (method url-fetch)
         (uri (pypi-uri "parsel" version))
         (sha256
-          (base32 "0yawf9r3r863lwxj0n89i7h3n8xjbsl5b7n6xg76r68scl5yzvvh"))))
-    (build-system python-build-system)
+          (base32 "0f8yh30y3961a7kqwcnp4j3s7044ilakykiavc0skwdkr5l8xwmg"))))
+    (build-system pyproject-build-system)
     (propagated-inputs
       (list python-cssselect
+            python-jmespath
             python-lxml
-            python-six
+            python-typing-extensions
             python-w3lib))
     (native-inputs
-      (list python-pytest python-pytest-runner))
+      (list python-psutil python-pytest))
     (home-page "https://github.com/scrapy/parsel")
     (synopsis "Extract data from HTML and XML using XPath and CSS selectors")
     (description "Parsel is a library to extract and remove data from
-- 
2.34.1

[bug#65114] [PATCH 4/6] gnu: python-itemadapter: Update to 0.8.0.

[Vinicius Monego]

* gnu/packages/python-xyz.scm (python-itemadapter): Update to 0.8.0.
---
 gnu/packages/python-xyz.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 884c37e975..e6cbe7d333 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -32504,13 +32504,13 @@ (define-public python-posix-ipc
 (define-public python-itemadapter
   (package
     (name "python-itemadapter")
-    (version "0.5.0")
+    (version "0.8.0")
     (source
       (origin
         (method url-fetch)
         (uri (pypi-uri "itemadapter" version))
         (sha256
-          (base32 "083wp3h2brh8x19jbdr8rz3biqwp3jlqd0rfzcyrjyhssffsgdh5"))))
+          (base32 "1aa898gjgwy3axxfrgsh4kdvhp6n6wz3ccdishq0gh8azf2q8xbp"))))
     (build-system python-build-system)
     (home-page "https://github.com/scrapy/itemadapter")
     (synopsis "Common interface for data container classes")
-- 
2.34.1

[bug#65114] [PATCH 5/6] gnu: python-itemloaders: Update to 1.1.0.

[Vinicius Monego]

* gnu/packages/python-xyz.scm (python-itemloaders): Update to 1.1.0.
[build-system]: Use pyproject-build-system.
[native-inputs]: Add python-pytest.
---
 gnu/packages/python-xyz.scm | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index e6cbe7d333..c7bce1b135 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -32534,16 +32534,17 @@ (define-public python-itemadapter
 (define-public python-itemloaders
   (package
     (name "python-itemloaders")
-    (version "1.0.4")
+    (version "1.1.0")
     (source
       (origin
         (method url-fetch)
         (uri (pypi-uri "itemloaders" version))
         (sha256
-          (base32 "15hc78h90qhwass1bga1c3xar2dd6j8sxg61zg6jvh74lf6csxqj"))))
-    (build-system python-build-system)
+          (base32 "0j2aw4ipalj208594x80blpgkh1i63gqqa4nb67b823av9hirn11"))))
+    (build-system pyproject-build-system)
+    (native-inputs (list python-pytest))
     (propagated-inputs
-      (list python-itemadapter python-jmespath python-parsel python-w3lib))
+     (list python-itemadapter python-jmespath python-parsel python-w3lib))
     (home-page "https://github.com/scrapy/itemloaders")
     (synopsis "Base library for scrapy's ItemLoader")
     (description "Itemloaders is a library that helps you collect data
-- 
2.34.1

[bug#65114] [PATCH 6/6] gnu: python-scrapy: Update to 2.10.0.

[Vinicius Monego]

* gnu/packages/python-web.scm (python-scrapy): Update to 2.10.0.
[build-system]: Use pyproject-build-system.
[arguments]: Use G-Expressions.  Do not override the check phase, skip tests
in #:test-flags instead.  Run tests sequentially and skip fewer tests.
---
 gnu/packages/python-web.scm | 42 ++++++++++++++++---------------------
 1 file changed, 18 insertions(+), 24 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 70a1ab082f..c12740b587 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -8065,36 +8065,30 @@ (define-public python-parsel
 (define-public python-scrapy
   (package
     (name "python-scrapy")
-    (version "2.7.1")
+    (version "2.10.0")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "Scrapy" version))
        (sha256
-        (base32 "0kpi3hg2ycs6s8cg41r2zc1axd0rpnps8bnzg7wisjyjaf1l1yih"))))
-    (build-system python-build-system)
+        (base32 "0fdf1kx39zy7b0v5gkwwqvm58ww63z5bvm78yjymhfn0r0zs65jf"))))
+    (build-system pyproject-build-system)
     (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (replace 'check
-           (lambda* (#:key tests? #:allow-other-keys)
-             (when tests?
-               (invoke "pytest"
-                       "-n" (number->string (parallel-job-count))
-                       ;; These tests fail when run in parallel (see:
-                       ;; https://github.com/scrapy/scrapy/issues/5502).
-                       "--ignore" "tests/test_engine.py"
-                       "--ignore" "tests/test_engine_stop_download_bytes.py"
-                       "--ignore" "tests/test_engine_stop_download_headers.py"
-                       ;; This test require network access.
-                       "--ignore" "tests/test_command_check.py"
-                       "-k"
-                       (string-append
-                        ;; The followin tests fail for unknown reasons.
-                        "not test_server_set_cookie_domain_suffix_public_private"
-                        " and not test_user_set_cookie_domain_suffix_public_private"
-                        " and not test_pformat")
-                       "tests")))))))
+     (list #:test-flags
+           ;; Tests fail with DNS lookup or need a display.
+           #~(list "-k" (string-append
+                         "not test_SCRAPY_CHECK_set"
+                         " and not "
+                         (string-join
+                          (list "test_check_all_default_contracts"
+                                "test_check_cb_kwargs_contract"
+                                "test_check_returns_items_contract"
+                                "test_check_returns_requests_contract"
+                                "test_check_scrapes_contract"
+                                "test_pformat"
+                                "test_pformat_old_windows"
+                                "test_pformat_windows")
+                          " and not ")))))
     (propagated-inputs
      (list python-botocore              ; Optional: For S3FeedStorage class.
            python-cryptography
-- 
2.34.1

[bug#65114] [PATCH 0/6] Update Scrapy.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 771 bytes --]


Vinicius Monego <monego@posteo.net> writes:

> Vinicius Monego (6):
>   gnu: python-w3lib: Update to 2.1.2.
>   gnu: python-cssselect: Update to 1.2.0.
>   gnu: python-parsel: Update to 1.8.1.
>   gnu: python-itemadapter: Update to 0.8.0.
>   gnu: python-itemloaders: Update to 1.1.0.
>   gnu: python-scrapy: Update to 2.10.0.
>
>  gnu/local.mk                                  |  1 -
>  .../python-w3lib-fix-test-failure.patch       | 60 --------------
>  gnu/packages/python-web.scm                   | 80 +++++++------------
>  gnu/packages/python-xyz.scm                   | 13 +--
>  4 files changed, 38 insertions(+), 116 deletions(-)
>  delete mode 100644 gnu/packages/patches/python-w3lib-fix-test-failure.patch

These changes look good to me +1

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

bug#65114: [PATCH 0/6] Update Scrapy.

[Vinicius Monego]

Em dom, 2023-08-27 às 19:14 +0100, Christopher Baines escreveu:


[...]

> These changes look good to me +1
> 

[...]

Applied with a minor update to Scrapy, thanks.

Vinicius