[bug#73109] [PATCH 0/2] Add python-transformers

[bug#73109] [PATCH 0/2] Add python-transformers

[Nicolas Graves via Guix-patches via]

This patch adds the popular python-transformers package.

Nicolas Graves (2):
  gnu: python-huggingface-hub: Update to 0.23.2.
  gnu: Add python-transformers.

 gnu/packages/machine-learning.scm | 60 +++++++++++++++++++++++++++++++
 gnu/packages/python-web.scm       | 10 ++++--
 2 files changed, 67 insertions(+), 3 deletions(-)

-- 
2.45.2

[bug#73109] [PATCH 1/2] gnu: python-huggingface-hub: Update to 0.23.2.

[Nicolas Graves via Guix-patches via]

* gnu/packages/python-web.scm (python-huggingface-hub): Update to 0.23.2.

Change-Id: I06989e883f54d9f0fc8db002d0a580008f6c6b67
---
 gnu/packages/python-web.scm | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index bb1b912c0c..2a7dc8f8b3 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -143,7 +143,7 @@ (define-module (gnu packages python-web)
 (define-public python-huggingface-hub
   (package
     (name "python-huggingface-hub")
-    (version "0.20.3")
+    (version "0.23.2")
     (source
      (origin
        (method git-fetch)
@@ -152,7 +152,7 @@ (define-public python-huggingface-hub
              (commit (string-append "v" version))))
        (file-name (git-file-name name version))
        (sha256
-        (base32 "00indl9labvqvm4m0y5jbzl68cgj8i60a6qy498gpnjj2pqk4l6v"))))
+        (base32 "0hygxqcixkc1d9sr47j2km6z0p17aj4k1dzm4cvpddrvhrgqayq5"))))
     (build-system pyproject-build-system)
     (arguments
      (list
@@ -185,7 +185,11 @@ (define-public python-huggingface-hub
              "--ignore=tests/test_utils_git_credentials.py"
              "--ignore=tests/test_utils_http.py"
              "--ignore=tests/test_utils_pagination.py"
-             "--ignore=tests/test_webhooks_server.py")
+             "--ignore=tests/test_webhooks_server.py"
+             "-k" (string-append
+                   "not test_push_to_hub"
+                   " and not test_from_pretrained_model_id_only"
+                   " and not test_from_pretrained_model_id_and_revision"))
       #:phases
       '(modify-phases %standard-phases
          (add-before 'check 'pre-check
-- 
2.45.2

[bug#73109] [PATCH 2/2] gnu: Add python-transformers.

[Nicolas Graves via Guix-patches via]

* gnu/packages/machine-learning.scm (python-transformers): New variable.

Change-Id: Ifd7fa3a0f4611d3298ab76ceb44b3aea1397b824
---
 gnu/packages/machine-learning.scm | 60 +++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)

diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 412499d424..42842d7d61 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -5846,6 +5846,66 @@ (define-public python-tokenizers
 tokenizers, @code{rust-tokenizers}.")
     (license license:asl2.0)))
 
+(define-public python-transformers
+  (package
+    (name "python-transformers")
+    (version "4.44.2")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "transformers" version))
+       (sha256
+        (base32 "09h84wqqk2bgi4vr9d1m3dsliard99l53n96wic405gfjb61gain"))))
+    (build-system pyproject-build-system)
+    ;; The imported package contains ~60 more inputs, but they don't seem
+    ;; necessary to build a minimal version of the package.
+    (propagated-inputs
+     (list python-filelock
+           python-huggingface-hub
+           python-numpy
+           python-pytorch
+           python-pyyaml
+           python-regex
+           python-requests
+           python-safetensors
+           python-tokenizers
+           python-tqdm
+           tensorflow))
+    (home-page "https://github.com/huggingface/transformers")
+    (synopsis "Machine Learning for PyTorch and TensorFlow")
+    (description
+     "This package provides easy download of thousands of pretrained models to
+perform tasks on different modalities such as text, vision, and audio.
+
+These models can be applied on:
+@itemize
+@item Text, for tasks like text classification, information extraction,
+question answering, summarization, translation, and text generation, in over
+100 languages.
+@item Images, for tasks like image classification, object detection, and
+segmentation.
+@item Audio, for tasks like speech recognition and audio classification.
+@end itemize
+
+Transformer models can also perform tasks on several modalities combined, such
+as table question answering, optical character recognition, information
+extraction from scanned documents, video classification, and visual question
+answering.
+
+This package provides APIs to quickly download and use those pretrained models
+on a given text, fine-tune them on your own datasets and then share them with
+the community on our model hub.  At the same time, each python module defining
+an architecture is fully standalone and can be modified to enable quick
+research experiments.
+
+Transformers is backed by the three most popular deep learning libraries —
+Jax, PyTorch and TensorFlow — with a seamless integration between them.  It's
+straightforward to train your models with one before loading them for
+inference with the other.
+
+Note: This version doesn't support integration with JAX.")
+    (license license:asl2.0)))
+
 (define-public python-hmmlearn
   (package
     (name "python-hmmlearn")
-- 
2.45.2

[bug#73069] [PATCH] gnu: python-huggingface-hub: Disable failing tests.

[Nicolas Graves via Guix-patches via]

* gnu/packages/python-web.scm (python-huggingface-hub):
  [arguments]<#:test-flags>: Disable failing test (requires network
  connection).

Change-Id: I7273c4f0bcc3b8cf3d34633a0d6c1df4d76d9170
---
 gnu/packages/python-web.scm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index e02c0d8729..bb1b912c0c 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -175,6 +175,7 @@ (define-public python-huggingface-hub
              "--ignore=tests/test_inference_api.py"
              "--ignore=tests/test_inference_async_client.py"
              "--ignore=tests/test_inference_client.py"
+             "--ignore=tests/test_inference_text_generation.py"
              "--ignore=tests/test_login_utils.py"
              "--ignore=tests/test_offline_utils.py"
              "--ignore=tests/test_repocard.py"
-- 
2.45.2

bug#73069: [PATCH] gnu: python-huggingface-hub: Disable failing tests.

[jgart via Guix-patches via]

applied, thanks!

-- 
all the best,
jgart

[bug#73109] [PATCH 0/2] Add python-transformers

[jgart via Guix-patches via]

Hi,

I applied the huggingface-hub update but the rest no longer applies.

Can you rebase on latest and send a v2 with just python-transformers?

Thanks!

-- 
all the best,
jgart

[bug#73109] [PATCH 0/2] Add python-transformers

[Nicolas Graves via Guix-patches via]

On 2024-09-10 00:07, jgart via Guix-patches via wrote:

> Hi,

Hi jgart, thanks!

>
> I applied the huggingface-hub update but the rest no longer applies.
>
> Can you rebase on latest and send a v2 with just python-transformers?
>

The python-tokenizers [73106] patches are also necessary before merging
python-transformers.

Herman Rimm has already reviewed the python-safetensors patches, these
are the same kind.

I'll resend a patch series for python-transformers once these blocking
patch series will be merged ;)

> Thanks!

-- 
Best regards,
Nicolas Graves