[bug#58530] [PATCH 0/4]: gnu: Add parfive

[bug#58530] [PATCH 0/4]: gnu: Add parfive

[Sharlatan Hellseher]

Hi Guix team!

This parch series adds parfive - Python module and CLI command for
parallel download from HTTP and FTP. This project comes as part of
inputs to some of Astronomical package I'm working on right now and
useful by it's own.

One thing to mention, tests for aioftp and siosocks produces
depreciation warnings which may be solved by upgrading
python-pytest-asynio to 0.19.0 (I'm not dare to touch as it's part of
600+ inputs), see the issues:

- https://github.com/aio-libs/aioftp/issues/159
- https://github.com/pohmelie/siosocks/issues/7

> ./pre-inst-env guix build --rounds=2 python-siosocks python-aioftp python-pytest-socket parfive
/gnu/store/6kj27ljj8js9cn94kfzqn7ld1q5p7kql-parfive-2.0.1
/gnu/store/bzzzi0h42yqvb67x6d486gnwxxdpbrcr-python-pytest-socket-0.5.1
/gnu/store/iyh72cbvq4k6zcsyadpfp1y28nyz27wz-python-aioftp-0.21.4
/gnu/store/1vv1fipiv7vjc4g5akrx8rl2c1p8bz0v-python-siosocks-0.3.0

Sharlatan Hellseher (4):
  gnu: Add siosocks
  gnu: Add aioftp
  gnu: Add pytest-socket
  gnu: Add parfive

 gnu/packages/python-check.scm |  49 +++++++++++++++
 gnu/packages/python-web.scm   | 115 ++++++++++++++++++++++++++++++++++
 2 files changed, 164 insertions(+)

base-commit: cd0d27f30dc2274fbf8477f44275d9184521f044

-- 
… наш разум - превосходная объяснительная машина которая способна
найти смысл почти в чем угодно, истолковать любой феномен, но
совершенно не в состоянии принять мысль о непредсказуемости.

[bug#58530] [PATCH 1/4]: gnu: Add siosocks

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 356 bytes --]

-- 
… наш разум - превосходная объяснительная машина которая способна
найти смысл почти в чем угодно, истолковать любой феномен, но
совершенно не в состоянии принять мысль о непредсказуемости.

[-- Attachment #2: 0001-gnu-Add-siosocks.patch --]
[-- Type: text/x-patch, Size: 2233 bytes --]

From ac31ebaf1111f89802a33ed1b0fe85e8a49299b3 Mon Sep 17 00:00:00 2001
From: Sharlatan Hellseher <sharlatanus@gmail.com>
Date: Fri, 14 Oct 2022 20:59:11 +0100
Subject: [PATCH 1/4] gnu: Add siosocks

* gnu/packages/python-web.scm (python-siosocks): New variable.
---
 gnu/packages/python-web.scm | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 6a9d6ee61d..a00c8b7683 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -6674,6 +6674,43 @@ (define-public python-socksio
 SOCKS protocols.  It can be paired with any I/O library.")
     (license license:expat)))
 
+(define-public python-siosocks
+  (package
+    (name "python-siosocks")
+    (version "0.3.0")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "siosocks" version))
+              (sha256
+               (base32
+                "0qqxy8wl5mrmlkblzjq9nsg0cbm5jwgj409mhnhq6gd1ypvbndms"))))
+    (build-system python-build-system)
+    (arguments
+     (list #:phases #~(modify-phases %standard-phases
+                        (replace 'check
+                          (lambda* (#:key tests? #:allow-other-keys)
+                            (when tests?
+                              (invoke "pytest" "-vvv")))))))
+    (native-inputs (list python-pytest python-pytest-asyncio python-pytest-cov
+                         python-pytest-trio))
+    (propagated-inputs (list python-trio))
+    (home-page "https://github.com/pohmelie/siosocks")
+    (synopsis "SOCKSv4 & SOCKSv5 TCP proxy protocol implementation in Python")
+    (description
+     "This package provides a Python module and framework for sans-io socks proxy
+client/server with couple io backends.
+
+Features:
+@itemize
+@item Only TCP connect (no BIND, no UDP)
+@item Both client and server
+@item SOCKS versions: 4, 4a, 5
+@item SOCKSv5 auth: no auth, username/password
+@item Couple io backends: @code{asyncio}, @code{trio}, @code{socketserver}
+@item One-shot socks server (@code{python -m siosocks})
+@end itemize")
+    (license license:expat)))
+
 (define-public python-msrest
   (package
     (name "python-msrest")
-- 
2.37.3

[bug#58530] [PATCH 2/4]: gnu: Add aioftp

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 356 bytes --]

-- 
… наш разум - превосходная объяснительная машина которая способна
найти смысл почти в чем угодно, истолковать любой феномен, но
совершенно не в состоянии принять мысль о непредсказуемости.

[-- Attachment #2: 0002-gnu-Add-aioftp.patch --]
[-- Type: text/x-patch, Size: 2297 bytes --]

From 274047ed5622650a6e1d232cd5cec39cd9a4f516 Mon Sep 17 00:00:00 2001
From: Sharlatan Hellseher <sharlatanus@gmail.com>
Date: Fri, 14 Oct 2022 21:04:56 +0100
Subject: [PATCH 2/4] gnu: Add aioftp

* gnu/packages/python-web.scm (python-aioftp): New variable.
---
 gnu/packages/python-web.scm | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index a00c8b7683..3c4d179dcc 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -3037,6 +3037,42 @@ (define-public python-requests-ftp
 adapter for use with the Requests library.")
     (license license:asl2.0)))
 
+(define-public python-aioftp
+  (package
+    (name "python-aioftp")
+    (version "0.21.4")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "aioftp" version))
+              (sha256
+               (base32
+                "1f8vql2j2b3ykqyh5bxzsp8x5f2if2c1ya232ld3hz3cc7a2dfr8"))))
+    (build-system python-build-system)
+    (arguments
+     (list #:phases #~(modify-phases %standard-phases
+                        (replace 'check
+                          (lambda* (#:key tests? #:allow-other-keys)
+                            (when tests?
+                              (invoke "pytest" "-vvv")))))))
+    (native-inputs (list python-async-timeout python-pytest
+                         python-pytest-asyncio python-pytest-cov
+                         python-trustme))
+    (propagated-inputs (list python-siosocks))
+    (home-page "https://aioftp.readthedocs.io/")
+    (synopsis "FTP client/server for asyncio in Python")
+    (description
+     "FTP client and server for asyncio (Python 3) Library implementing FTP
+protocol, both client and server for Python asyncio module.
+
+ Supported commands as client: USER, PASS, ACCT, PWD, CWD, CDUP, MKD, RMD,
+ MLSD, MLST, RNFR, RNTO, DELE, STOR, APPE, RETR, TYPE, PASV, ABOR, QUIT,
+ REST, LIST (as fallback).
+
+ Supported commands as server: USER, PASS, QUIT, PWD, CWD, CDUP, MKD, RMD,
+ MLSD, LIST (non-standard), MLST, RNFR, RNTO, DELE, STOR, RETR,
+ TYPE (\"I\" and \"A\"), PASV, ABOR, APPE, REST.")
+    (license license:asl2.0)))
+
 (define-public python-msal
   (package
     (name "python-msal")
-- 
2.37.3

[bug#58530] [PATCH 3/4]: gnu: Add pytest-socket

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 356 bytes --]

-- 
… наш разум - превосходная объяснительная машина которая способна
найти смысл почти в чем угодно, истолковать любой феномен, но
совершенно не в состоянии принять мысль о непредсказуемости.

[-- Attachment #2: 0003-gnu-Add-pytest-socket.patch --]
[-- Type: text/x-patch, Size: 3041 bytes --]

From f5e774473b65f1973756d197d09db415bcb0311c Mon Sep 17 00:00:00 2001
From: Sharlatan Hellseher <sharlatanus@gmail.com>
Date: Fri, 14 Oct 2022 21:42:06 +0100
Subject: [PATCH 3/4] gnu: Add pytest-socket

* gnu/packages/python-check.scm (python-pytest-socket): New variable.
---
 gnu/packages/python-check.scm | 49 +++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index 2358e7448d..fd88fc480e 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -395,6 +395,55 @@ (define-public python-vcrpy
 interactions, which will update them to correspond to the new API.")
     (license license:expat)))
 
+(define-public python-pytest-socket
+  (package
+    (name "python-pytest-socket")
+    (version "0.5.1")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "pytest-socket" version))
+              (sha256
+               (base32
+                "1dkr86nxkxc0ka3rdnpmk335m8gl1zh1sy8i7w4w1jsidbf82jvw"))))
+    (build-system python-build-system)
+    (arguments
+     ;; FIXME: Tests fail a lot, probably requiring Internet access.
+     (list #:tests? #f
+           #:phases #~(modify-phases %standard-phases
+                        (replace 'build
+                          (lambda _
+                            (setenv "SETUPTOOLS_SCM_PRETEND_VERSION"
+                                    #$version)
+                            (setenv "SOURCE_DATE_EPOCH" "315532800")
+                            (invoke "python"
+                                    "-m"
+                                    "build"
+                                    "--wheel"
+                                    "--no-isolation"
+                                    ".")))
+                        (add-before 'check 'disable-unsupported-test
+                          (lambda _
+                            (substitute* "tests/test_async.py"
+                              (("def test_asynctest")
+                               "def __off_test_asynctest"))))
+                        (replace 'check
+                          (lambda* (#:key tests? #:allow-other-keys)
+                            (when tests?
+                              (invoke "python" "-m" "pytest" "-vvv")))))))
+    (native-inputs (list python-httpx
+                         python-poetry-core
+                         python-pypa-build
+                         python-pytest
+                         python-pytest-httpbin
+                         python-pytest-randomly
+                         python-starlette))
+    (home-page "https://pypi.org/project/pytest-socket/")
+    (synopsis "Pytest plugin to disable socket calls during tests")
+    (description
+     "This package provides Pytest extension which disables all network calls flowing
+through Python's socket interface")
+    (license license:expat)))
+
 (define-public python-pytest-ordering
   (package
     (name "python-pytest-ordering")
-- 
2.37.3

[bug#58530] [PATCH 4/4]: gnu: Add parfive

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 356 bytes --]

-- 
… наш разум - превосходная объяснительная машина которая способна
найти смысл почти в чем угодно, истолковать любой феномен, но
совершенно не в состоянии принять мысль о непредсказуемости.

[-- Attachment #2: 0004-gnu-Add-parfive.patch --]
[-- Type: text/x-patch, Size: 2715 bytes --]

From c51bba6b106a31f773d9dfe31dc4bc6d3c1050aa Mon Sep 17 00:00:00 2001
From: Sharlatan Hellseher <sharlatanus@gmail.com>
Date: Fri, 14 Oct 2022 21:45:52 +0100
Subject: [PATCH 4/4] gnu: Add parfive

* gnu/packages/python-web.scm (parfive): Newv variable
---
 gnu/packages/python-web.scm | 42 +++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 3c4d179dcc..c29be41eff 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -983,6 +983,48 @@ (define-public httpie
     (properties `((lint-hidden-cve . ("CVE-2019-10751"))))
     (license license:bsd-3)))
 
+(define-public parfive
+  (package
+    (name "parfive")
+    (version "2.0.1")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "parfive" version))
+              (sha256
+               (base32
+                "19dcbb6g56l5s3ih0bhs3p4acgc0gf4zdzpj4w87m69li2nhmgpx"))))
+    (build-system python-build-system)
+    (arguments
+     (list #:phases #~(modify-phases %standard-phases
+                        (add-before 'check 'disable-test-requiring-network
+                          (lambda _
+                            (substitute* "parfive/tests/test_downloader.py"
+                              (("def test_ftp")
+                               "def __off_test_ftp"))))
+                        (replace 'check
+                          (lambda* (#:key tests? #:allow-other-keys)
+                            (when tests?
+                              (invoke "python" "-m" "pytest" "-vvv" "parfive")))))))
+    (propagated-inputs (list python-aiofiles python-aioftp python-aiohttp
+                             python-tqdm))
+    (native-inputs (list python-pytest
+                         python-pytest-asyncio
+                         python-pytest-cov
+                         python-pytest-localserver
+                         python-pytest-socket
+                         python-setuptools-scm))
+    (home-page "https://parfive.readthedocs.io/")
+    (synopsis "HTTP and FTP parallel file downloader")
+    (description
+     "This package provides CLI tool and Python library @code{parallel} file
+downloader using asyncio. parfive can handle downloading multiple files in
+parallel as well as downloading each file in a number of chunks.
+
+asciicast demo of parfive parfive works by creating a downloader object,
+appending files to it and then running the download. parfive has a synchronous
+API, but uses asyncio to paralellise downloading the files.")
+    (license license:expat)))
+
 (define-public python-html2text
   (package
     (name "python-html2text")
-- 
2.37.3

[bug#58530]

[Sharlatan Hellseher]

Hi,

A gentle ping on status for these patches.

I've stabilized build for SunPy, the astronomical project for data
analysis for the Sun activity research which has parfive as a hard
dependence.

Thanks,
Oleg

--
… наш разум - превосходная объяснительная машина которая способна
найти смысл почти в чем угодно, истолковать любой феномен, но
совершенно не в состоянии принять мысль о непредсказуемости.

[bug#58530]

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]


Sharlatan Hellseher <sharlatanus@gmail.com> writes:

> A gentle ping on status for these patches.
>
> I've stabilized build for SunPy, the astronomical project for data
> analysis for the Sun activity research which has parfive as a hard
> dependence.

Apologies for the delay, I've had a look now and they seem good to me.

I tweaked the commit messages, adding some fullstops in places (just to
match the general Guix convention) and pushed to master as
472917ea6c479b3546ffaea02f755ed9c6e8d46b.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]