From: zimoun <zimon.toutoune@gmail.com>
To: Guix Devel <guix-devel@gnu.org>
Subject: Be careful with PyPI
Date: Fri, 06 Jan 2023 15:36:38 +0100 [thread overview]
Message-ID: <87sfgnivbd.fsf@gmail.com> (raw)
Hi,
If the origin does not exist upstream, then Guix try other servers as
fallback. For instance,
--8<---------------cut here---------------start------------->8---
Starting download of /gnu/store/lb0kb4c212f9f789ixd1c18bcm8qbsqi-Keras-2.11.0.tar.gz
From https://files.pythonhosted.org/packages/source/K/Keras/Keras-2.11.0.tar.gz...
download failed "https://files.pythonhosted.org/packages/source/K/Keras/Keras-2.11.0.tar.gz" 404 "Not Found"
Starting download of /gnu/store/lb0kb4c212f9f789ixd1c18bcm8qbsqi-Keras-2.11.0.tar.gz
From https://ci.guix.gnu.org/file/Keras-2.11.0.tar.gz/sha256/1j8bsqzh49vjdxy6l1k4iwax5vpjzniynyd041xjavdzvfii1dlh...
download failed "https://ci.guix.gnu.org/file/Keras-2.11.0.tar.gz/sha256/1j8bsqzh49vjdxy6l1k4iwax5vpjzniynyd041xjavdzvfii1dlh" 404 "Not Found"
Starting download of /gnu/store/lb0kb4c212f9f789ixd1c18bcm8qbsqi-Keras-2.11.0.tar.gz
From https://tarballs.nixos.org/sha256/1j8bsqzh49vjdxy6l1k4iwax5vpjzniynyd041xjavdzvfii1dlh...
following redirection to `https://tarballs.nixos.org/sha512/0d57cb8e0af234a619ba3dec245c2ab73cbd0566194ed6b59377231183b2ef916514ba87abe04ec7e518770c9e0cb157747db87cedf0ebeced4ae0f56be401c3'...
downloading from https://tarballs.nixos.org/sha256/1j8bsqzh49vjdxy6l1k4iwax5vpjzniynyd041xjavdzvfii1dlh ...
1j8bsqzh49vjdxy6l1k4iwax5vpjzniynyd041xjavdzvfii1dlh 748KiB 67.1MiB/s 00:00 [##################] 100.0%
successfully built /gnu/store/75g4aq5b25g4j1qwd3ggl1sys24q63la-Keras-2.11.0.tar.gz.drv
--8<---------------cut here---------------end--------------->8---
One potential issue is that the tarballs.nixos.org is using the checksum
as lookup key. Therefore, when modifying only the version and not the
checksum, the something is returned with an inconsistent name/content.
First, let get the source of the current Keras (v2.2.4).
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build python-keras -S
0.5 MB will be downloaded:
/gnu/store/k2wxrqzmr29rjy1w5b0nhd4k36sv8szb-Keras-2.2.4.tar.xz
substituting /gnu/store/k2wxrqzmr29rjy1w5b0nhd4k36sv8szb-Keras-2.2.4.tar.xz...
downloading from https://ci.guix.gnu.org/nar/k2wxrqzmr29rjy1w5b0nhd4k36sv8szb-Keras-2.2.4.tar.xz ...
Keras-2.2.4.tar.xz 521KiB 3.6MiB/s 00:00 [##################] 100.0%
/gnu/store/k2wxrqzmr29rjy1w5b0nhd4k36sv8szb-Keras-2.2.4.tar.xz
--8<---------------cut here---------------end--------------->8---
Then, tweak only the version (update to 2.11.0) and get the source.
--8<---------------cut here---------------start------------->8---
$ git diff
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 75cc100851..2cbe0eddd6 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -2831,7 +2831,7 @@ (define-public python-keras-preprocessing
(define-public python-keras
(package
(name "python-keras")
- (version "2.2.4")
+ (version "2.11.0")
(source
(origin
(method url-fetch)
$ ./pre-inst-env guix build python-keras -S
;;; note: source file /home/simon/src/guix/guix/gnu/packages/machine-learning.scm
;;; newer than compiled /home/simon/src/guix/guix/gnu/packages/machine-learning.go
;;; note: source file /home/simon/src/guix/guix/gnu/packages/machine-learning.scm
;;; newer than compiled /home/simon/src/guix/guix/gnu/packages/machine-learning.go
;;; note: source file /home/simon/src/guix/guix/gnu/packages/machine-learning.scm
;;; newer than compiled /home/simon/.config/guix/profiles/emacs/emacs/lib/guile/3.0/site-ccache/gnu/packages/machine-learning.go
;;; note: source file /home/simon/src/guix/guix/gnu/packages/machine-learning.scm
;;; newer than compiled /home/simon/.config/guix/profiles/emacs/emacs/lib/guile/3.0/site-ccache/gnu/packages/machine-learning.go
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
The following derivations will be built:
/gnu/store/by6pgdambmv4wrg2h40l287d7zhbwc17-Keras-2.11.0.tar.xz.drv
/gnu/store/75g4aq5b25g4j1qwd3ggl1sys24q63la-Keras-2.11.0.tar.gz.drv
building /gnu/store/75g4aq5b25g4j1qwd3ggl1sys24q63la-Keras-2.11.0.tar.gz.drv...
Starting download of /gnu/store/lb0kb4c212f9f789ixd1c18bcm8qbsqi-Keras-2.11.0.tar.gz
From https://files.pythonhosted.org/packages/source/K/Keras/Keras-2.11.0.tar.gz...
download failed "https://files.pythonhosted.org/packages/source/K/Keras/Keras-2.11.0.tar.gz" 404 "Not Found"
Starting download of /gnu/store/lb0kb4c212f9f789ixd1c18bcm8qbsqi-Keras-2.11.0.tar.gz
From https://ci.guix.gnu.org/file/Keras-2.11.0.tar.gz/sha256/1j8bsqzh49vjdxy6l1k4iwax5vpjzniynyd041xjavdzvfii1dlh...
download failed "https://ci.guix.gnu.org/file/Keras-2.11.0.tar.gz/sha256/1j8bsqzh49vjdxy6l1k4iwax5vpjzniynyd041xjavdzvfii1dlh" 404 "Not Found"
Starting download of /gnu/store/lb0kb4c212f9f789ixd1c18bcm8qbsqi-Keras-2.11.0.tar.gz
From https://tarballs.nixos.org/sha256/1j8bsqzh49vjdxy6l1k4iwax5vpjzniynyd041xjavdzvfii1dlh...
following redirection to `https://tarballs.nixos.org/sha512/0d57cb8e0af234a619ba3dec245c2ab73cbd0566194ed6b59377231183b2ef916514ba87abe04ec7e518770c9e0cb157747db87cedf0ebeced4ae0f56be401c3'...
downloading from https://tarballs.nixos.org/sha256/1j8bsqzh49vjdxy6l1k4iwax5vpjzniynyd041xjavdzvfii1dlh ...
1j8bsqzh49vjdxy6l1k4iwax5vpjzniynyd041xjavdzvfii1dlh 748KiB 67.1MiB/s 00:00 [##################] 100.0%
successfully built /gnu/store/75g4aq5b25g4j1qwd3ggl1sys24q63la-Keras-2.11.0.tar.gz.drv
building /gnu/store/by6pgdambmv4wrg2h40l287d7zhbwc17-Keras-2.11.0.tar.xz.drv...
Keras-2.2.4/
[...]
Keras-2.2.4/Keras.egg-info/dependency_links.txt
patching file tests/integration_tests/applications_test.py
Hunk #1 succeeded at 64 (offset 6 lines).
/gnu/store/g2ajyl8xk9aarxrgjbng2hkj3qm2v0z2-tar-1.34/bin/tar: Option --mtime: Treating date '@1' as 1970-01-01 00:00:01
Keras-2.2.4/
[...]
Keras-2.2.4/tests/test_multiprocessing.py
source is at 'Keras-2.2.4'
applying '/gnu/store/xbzvc0ij9z7d13mmjk3pzfsnfwsimlm7-python-keras-integration-test.patch'...
successfully built /gnu/store/by6pgdambmv4wrg2h40l287d7zhbwc17-Keras-2.11.0.tar.xz.drv
/gnu/store/pxj6cnk8bis14jiz79igmp0k2813v21d-Keras-2.11.0.tar.xz
--8<---------------cut here---------------end--------------->8---
It is not Keras-2.11.0 but Keras-2.2.4.
--8<---------------cut here---------------start------------->8---
$ sha256sum /gnu/store/k2wxrqzmr29rjy1w5b0nhd4k36sv8szb-Keras-2.2.4.tar.xz /gnu/store/pxj6cnk8bis14jiz79igmp0k2813v21d-Keras-2.11.0.tar.xz
09220b37e2a8dddcf9db1ea0a1d77d710bf084086fa9339e9278b30eac59b6b7 /gnu/store/k2wxrqzmr29rjy1w5b0nhd4k36sv8szb-Keras-2.2.4.tar.xz
09220b37e2a8dddcf9db1ea0a1d77d710bf084086fa9339e9278b30eac59b6b7 /gnu/store/pxj6cnk8bis14jiz79igmp0k2813v21d-Keras-2.11.0.tar.xz
--8<---------------cut here---------------end--------------->8---
Well, the core of the issue is that
https://files.pythonhosted.org/packages/source/K/Keras/Keras-2.11.0.tar.gz
does not exist, so the fallback uses the checksum (of Keras v2.2.4).
Note that Keras 2.11.0 is available on PyPI,
https://pypi.org/project/keras/2.11.0/
but not the source. They removed the source after 2.6.0,
https://pypi.org/project/keras/2.6.0/#files
Arf!
Well, IMHO, we cannot do better except been really careful when fetching
from PyPI. (And aside this kind of troubles [1]).
Cheers,
simon
1: <https://pytorch.org/blog/compromised-nightly-dependency/>
next reply other threads:[~2023-01-06 16:30 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-06 14:36 zimoun [this message]
2023-01-08 18:47 ` Be careful with PyPI Leo Famulari
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87sfgnivbd.fsf@gmail.com \
--to=zimon.toutoune@gmail.com \
--cc=guix-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.