* Python subprocess fails to execute Pip-installed module @ 2022-10-16 11:38 Pradana Adrinusa AUMARS 2022-10-16 15:21 ` Ricardo Wurmus 0 siblings, 1 reply; 6+ messages in thread From: Pradana Adrinusa AUMARS @ 2022-10-16 11:38 UTC (permalink / raw) To: help-guix I need to use Ray (https://www.ray.io) for a project. Because Guix does not have a python-ray package, and I don't really have time to package one myself, I installed Ray using Pip (user installation is the default, since system installation isn't possible on Guix). Running: import ray ray.init() fails since Python's subprocess needs to run an executable located in ~/.local/lib/python3.9/site-packages/ray/core/src/ray/gcs/gcs_server So the error is narrowed down to: import subprocess subprocess.Popen(["/home/user/.local/lib/python3.9/site-packages/ray/core/src/ray/gcs/gcs_server"]) It doesn't seem to be a $PATH issue. $PATH does not have ~/.local/bin and import subprocess subprocess.Popen(["/home/user/.local/bin/any_binary"]) works normally. I've also tried adding /home/user/.local/lib/python3.9/site-packages/ray/core/src/ray/gcs to $PATH, and it doesn't work. I could package Ray, which would be the permenant solution, but I'm limited on time right now. Does anyone have a temporary solution to fix this issue? Thanks. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Python subprocess fails to execute Pip-installed module 2022-10-16 11:38 Python subprocess fails to execute Pip-installed module Pradana Adrinusa AUMARS @ 2022-10-16 15:21 ` Ricardo Wurmus 2022-10-16 15:37 ` Pradana Adrinusa AUMARS 0 siblings, 1 reply; 6+ messages in thread From: Ricardo Wurmus @ 2022-10-16 15:21 UTC (permalink / raw) To: Pradana Adrinusa AUMARS; +Cc: help-guix Pradana Adrinusa AUMARS <paumars@courrier.dev> writes: > Running: > > import ray > ray.init() > > fails since Python's subprocess needs to run an executable located in > > ~/.local/lib/python3.9/site-packages/ray/core/src/ray/gcs/gcs_server > > So the error is narrowed down to: > > import subprocess > subprocess.Popen(["/home/user/.local/lib/python3.9/site-packages/ray/core/src/ray/gcs/gcs_server"]) Do you get an error message? Does this executable exist? -- Ricardo ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Python subprocess fails to execute Pip-installed module 2022-10-16 15:21 ` Ricardo Wurmus @ 2022-10-16 15:37 ` Pradana Adrinusa AUMARS 2022-10-16 15:50 ` Tobias Geerinckx-Rice 0 siblings, 1 reply; 6+ messages in thread From: Pradana Adrinusa AUMARS @ 2022-10-16 15:37 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: help-guix Sorry, I forgot to add the error message in the original mail... It throws a FileNotFoundError as FileNotFoundError: [Errno 2] No such file or directory: '/home/user/.local/lib/python3.9/site-packages/ray/core/src/ray/gcs/gcs_server' The executable exists, accessible by Python's os import os os.path.exists('/home/user/.local/lib/python3.9/site-packages/ray/core/src/ray/gcs/gcs_server') returns True Its file type is ELF 64-bit LSB pie executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=59c99f77d9fc6566e51a86cbd63edf9f095abb70, not stripped Strangely enough, I cannot execute this binary directly, $ ./home/user/.local/lib/python3.9/site-packages/ray/core/src/ray/gcs/gcs_server outputs that bash cannot find this file. Le dimanche 16 octobre 2022 à 17:21 +0200, Ricardo Wurmus a écrit : > > Pradana Adrinusa AUMARS <paumars@courrier.dev> writes: > > > Running: > > > > import ray > > ray.init() > > > > fails since Python's subprocess needs to run an executable located > > in > > > > ~/.local/lib/python3.9/site-packages/ray/core/src/ray/gcs/gcs_server > > > > So the error is narrowed down to: > > > > import subprocess > > subprocess.Popen(["/home/user/.local/lib/python3.9/site-packages/ray/core/src/ray/gcs/gcs_server"]) > > Do you get an error message? > Does this executable exist? > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Python subprocess fails to execute Pip-installed module 2022-10-16 15:37 ` Pradana Adrinusa AUMARS @ 2022-10-16 15:50 ` Tobias Geerinckx-Rice 2022-10-16 18:53 ` Pradana Adrinusa AUMARS 0 siblings, 1 reply; 6+ messages in thread From: Tobias Geerinckx-Rice @ 2022-10-16 15:50 UTC (permalink / raw) To: Pradana Adrinusa AUMARS; +Cc: Ricardo Wurmus, help-guix [-- Attachment #1: Type: text/plain, Size: 788 bytes --] Hi Pradana, Pradana Adrinusa AUMARS 写道: > dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for And this file doesn't exist. Prebuilt binary blobs don't mix well with Guix for this reason: they hard-code file names such as this one. One (brand-)new work-around is $ guix shell -CF [PACKAGE…] [-- your command line] E.g., $ guix shell -CF -- ./hello # random binary downloaded from Debian Hello, world! which creates a virtual, backwards-compatible directory layout within the container. This is a (glorious) hack, but it's no substitute for proper Guix packaging! > outputs that bash cannot find this file. That error is criminally misleading. ‘A’ file, but not ‘this’ file. :-/ Kind regards, T G-R [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Python subprocess fails to execute Pip-installed module 2022-10-16 15:50 ` Tobias Geerinckx-Rice @ 2022-10-16 18:53 ` Pradana Adrinusa AUMARS 2022-11-12 22:46 ` Python subprocess fails to execute Pip-installed module / Packaging Python Ray Pradana Adrinusa AUMARS 0 siblings, 1 reply; 6+ messages in thread From: Pradana Adrinusa AUMARS @ 2022-10-16 18:53 UTC (permalink / raw) To: Tobias Geerinckx-Rice; +Cc: Ricardo Wurmus, help-guix Thank you for the explanation, Ricardo, Thank you for the explanation Tobias, I tried a simple solution by finding where my "ld" is located with "whereis" and a symbolic link to "/lib64/ld-linux-x86-64.so.2". Unfortunately, I get an Error 80 which means it used a corrupted shared library. I think it's better to package Ray natively into Guix, but it's time- consuming since Ray is not a simple Python package. I was hoping for a temporary solution, but I don't think I can find one. I'll try to use another machine to make this project run. Thank you anyways. Le dimanche 16 octobre 2022 à 17:50 +0200, Tobias Geerinckx-Rice a écrit : > Hi Pradana, > > Pradana Adrinusa AUMARS 写道: > > dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for > > And this file doesn't exist. > > Prebuilt binary blobs don't mix well with Guix for this reason: > they hard-code file names such as this one. > > One (brand-)new work-around is > > $ guix shell -CF [PACKAGE…] [-- your command line] > > E.g., > > $ guix shell -CF -- ./hello # random binary downloaded from > Debian > Hello, world! > > which creates a virtual, backwards-compatible directory layout > within the container. > > This is a (glorious) hack, but it's no substitute for proper Guix > packaging! > > > outputs that bash cannot find this file. > > That error is criminally misleading. ‘A’ file, but not ‘this’ > file. :-/ > > Kind regards, > > T G-R ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Python subprocess fails to execute Pip-installed module / Packaging Python Ray 2022-10-16 18:53 ` Pradana Adrinusa AUMARS @ 2022-11-12 22:46 ` Pradana Adrinusa AUMARS 0 siblings, 0 replies; 6+ messages in thread From: Pradana Adrinusa AUMARS @ 2022-11-12 22:46 UTC (permalink / raw) To: Tobias Geerinckx-Rice; +Cc: Ricardo Wurmus, help-guix I'm in the process of packaging Ray for Python. Here is the current SCM configuration: (use-modules (guix packages) (guix download) ((guix licenses) #:prefix license:) (gnu packages python) (guix utils) (guix build-system python) (gnu packages linux) (gnu packages check) (gnu packages rpc) (gnu packages python-build) (gnu packages protobuf) (gnu packages machine-learning) (gnu packages python-science) (gnu packages python-web) (gnu packages python-xyz)) (define python-setproctitle (package (name "python-setproctitle") (version "1.2.2") (source (origin (method url-fetch) (uri (pypi-uri "setproctitle" version)) (sha256 (base32 "1pwp1lb9mf0kgg3sbf1z8dkfnmwcqvixc0by00s3sh2ji0n4gyvx")))) (build-system python-build-system) (arguments '(#:phases (modify-phases %standard-phases (replace 'check (lambda* (#:key tests? #:allow-other-keys) (setenv "PYTHON" (or (which "python3") (which "python"))) (setenv "PYCONFIG" (if (which "python3-config") "python3-config --embed" "python-config")) (substitute* "tests/conftest.py" (("cc") "gcc")) (when tests? (invoke "pytest" "tests/"))))))) (native-inputs (list procps python-pytest)) ; required for tests (home-page "") (synopsis "") (description "") (license license:bsd-3))) (package (name "python-ray") (version "2.0.0") (source (origin (method url-fetch) (uri (string-append "https://github.com/ray-project/ray/archive/refs/tags/ray-" version ".tar.gz")) (sha256 (base32 "1gqp2h9fngn2miavksvd041b7pd19vf1lamarc5qwcmdx3b42f1k")))) (build-system python-build-system) (arguments '(#:phases (modify-phases %standard-phases (replace 'build (lambda _ (invoke "python3" "python/setup.py" "build")))))) (native-inputs (list python-attrs python-click python-dataclasses python-filelock python-grpcio python-jsonschema python-msgpack python-numpy python-packaging python-pyyaml python-aiosignal python-frozenlist python-requests python-virtualenv python-wheel python-cython python-setproctitle python-colorama python-psutil)) (home-page "") (synopsis "") (description "") (license license:asl2.0)) When I build the package, pip tries to download from pypi.org/simple/psutil, even though it's written as an input in this configuration. The output gives this error: warning: build_py: byte-compiling is disabled, skipping. running build_ext WARNING: The directory '/homeless-shelter/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag. WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7ffff5a81b20>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/psutil/ ... ERROR: Could not find a version that satisfies the requirement psutil (from versions: none) ERROR: No matching distribution found for psutil Has there been a similar issue with Python packages with psutil as an input or with pip not properly finding dependencies like this? Thanks Le dimanche 16 octobre 2022 à 20:53 +0200, Pradana Adrinusa AUMARS a écrit : > Thank you for the explanation, Ricardo, > > Thank you for the explanation Tobias, > > I tried a simple solution by finding where my "ld" is located with > "whereis" and a symbolic link to "/lib64/ld-linux-x86-64.so.2". > > Unfortunately, I get an Error 80 which means it used a corrupted shared > library. > > I think it's better to package Ray natively into Guix, but it's time- > consuming since Ray is not a simple Python package. I was hoping for a > temporary solution, but I don't think I can find one. I'll try to use > another machine to make this project run. > > Thank you anyways. > > Le dimanche 16 octobre 2022 à 17:50 +0200, Tobias Geerinckx-Rice a > écrit : > > Hi Pradana, > > > > Pradana Adrinusa AUMARS 写道: > > > dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for > > > > And this file doesn't exist. > > > > Prebuilt binary blobs don't mix well with Guix for this reason: > > they hard-code file names such as this one. > > > > One (brand-)new work-around is > > > > $ guix shell -CF [PACKAGE…] [-- your command line] > > > > E.g., > > > > $ guix shell -CF -- ./hello # random binary downloaded from > > Debian > > Hello, world! > > > > which creates a virtual, backwards-compatible directory layout > > within the container. > > > > This is a (glorious) hack, but it's no substitute for proper Guix > > packaging! > > > > > outputs that bash cannot find this file. > > > > That error is criminally misleading. ‘A’ file, but not ‘this’ > > file. :-/ > > > > Kind regards, > > > > T G-R > > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-11-12 22:47 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-10-16 11:38 Python subprocess fails to execute Pip-installed module Pradana Adrinusa AUMARS 2022-10-16 15:21 ` Ricardo Wurmus 2022-10-16 15:37 ` Pradana Adrinusa AUMARS 2022-10-16 15:50 ` Tobias Geerinckx-Rice 2022-10-16 18:53 ` Pradana Adrinusa AUMARS 2022-11-12 22:46 ` Python subprocess fails to execute Pip-installed module / Packaging Python Ray Pradana Adrinusa AUMARS
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).