Re: guix import error + New: setting up a jupyterlab environment

[Zelphir Kaltstahl <zelphirkaltstahl@posteo.de>]

Hello Efraim!

Thank you! Your idea of replacing "-" with "_" seems to partially help:

The package is downloaded and byte compiled. However, afterwards there
is a problem, when it tries to copy the byte-compiled files in the "wrap
phase" (comments inserted in some places):

~~~~START~~~~
### ... a lot of output before this ...

byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/__init__.py to __init__.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/_version.py to _version.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/app.py to app.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/handlers.py to handlers.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/listings_handler.py to listings_handler.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/process.py to process.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/process_app.py to process_app.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/server.py to server.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/servertest.py to servertest.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/settings_handler.py to settings_handler.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/themes_handler.py to themes_handler.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/workspaces_handler.py to workspaces_handler.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/tests/__init__.py to __init__.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/tests/test_settings_api.py to test_settings_api.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/tests/test_workspaces_api.py to test_workspaces_api.cpython-38.pyc
byte-compiling /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server/tests/utils.py to utils.cpython-38.pyc

### ... seems byte-compiling went without problems, but not comes the problem ...

running install_egg_info
Copying jupyterlab_server.egg-info to /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/lib/python3.8/site-packages/jupyterlab_server-1.2.0-py3.8.egg-info
running install_scripts
phase `install' succeeded after 0.3 seconds
starting phase `wrap'
find-files: /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/bin: No such file or directory
find-files: /gnu/store/yr7q21bws1xgx3p2dy9cfdiknjpsajm6-python-jupyterlab-server-1.2.0/sbin: No such file or directory
phase `wrap' succeeded after 0.0 seconds
starting phase `check'
running "python setup.py" with command "test" and parameters ()
running test
Searching for ipykernel
Reading https://pypi.org/simple/ipykernel/
Download error on https://pypi.org/simple/ipykernel/: [Errno -2] Name or service not known -- Some packages may not be found!
Couldn't find index page for 'ipykernel' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.org/simple/
Download error on https://pypi.org/simple/: [Errno -2] Name or service not known -- Some packages may not be found!
No local packages or working download links found for ipykernel
error: Could not find suitable distribution for Requirement.parse('ipykernel')
command "python" "-c" "import setuptools, tokenize;__file__='setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))" "test" failed with status 1
~~~~~END~~~~~

So it seems, that while the download URL is corrected, there is another
place, where Guix has an error now, resulting in a final error shown in
command line:

~~~~START~~~~
$ guix time-machine --channels=channels.scm -- environment --manifest=manifest.scm --load=jupyterlab.scm
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following derivations will be built:
   /gnu/store/wzmfn9m45bmsbrx094rv3lf3w0rpmw49-python-pytest-check-links-0.3.0.drv
   /gnu/store/96xf5mrg134mcsp01b0m1qs1lxkb401a-python-jupyterlab-server-1.2.0.drv
   /gnu/store/32a28hd7aainwn0hfsnf3blh8prmfz61-jupyterlab_server-1.2.0.tar.gz.drv

3,9 MB will be downloaded
downloading from https://ci.guix.gnu.org/nar/lzip/54kpl9psx16jwic4gpnjwh44kim04wlk-python-notebook-5.7.4 ...
 python-notebook-5.7.4  3.7MiB                                                                                                                                                                                                        5.1MiB/s 00:01 [##################] 100.0%

building /gnu/store/32a28hd7aainwn0hfsnf3blh8prmfz61-jupyterlab_server-1.2.0.tar.gz.drv...
downloading from https://files.pythonhosted.org/packages/source/j/jupyterlab_server/jupyterlab_server-1.2.0.tar.gz ...
building /gnu/store/wzmfn9m45bmsbrx094rv3lf3w0rpmw49-python-pytest-check-links-0.3.0.drv...
building /gnu/store/96xf5mrg134mcsp01b0m1qs1lxkb401a-python-jupyterlab-server-1.2.0.drv...
\ 'check' phasebuilder for `/gnu/store/96xf5mrg134mcsp01b0m1qs1lxkb401a-python-jupyterlab-server-1.2.0.drv' failed with exit code 1
build of /gnu/store/96xf5mrg134mcsp01b0m1qs1lxkb401a-python-jupyterlab-server-1.2.0.drv failed
View build log at '/var/log/guix/drvs/96/xf5mrg134mcsp01b0m1qs1lxkb401a-python-jupyterlab-server-1.2.0.drv.bz2'.
guix environment: error: build of `/gnu/store/96xf5mrg134mcsp01b0m1qs1lxkb401a-python-jupyterlab-server-1.2.0.drv' failed
~~~~~END~~~~~

Apparently ipykernel is not found, but when I visit
https://pypi.org/simple/ipykernel/, I see packages there.

From the logs:

~~~~START~~~~
Download error on https://pypi.org/simple/ipykernel/: [Errno -2] Name or service not known -- Some packages may not be found!
Couldn't find index page for 'ipykernel' (maybe misspelled?)
~~~~~END~~~~~

Do I need to add another input in my `jupyterlab.scm`? (How?) Or perhaps
I need to create a separate `ipykernel.scm` which contains the output of
`guix import pypi --recursive ipykernel`?

However, it somehow seems to fail for PyPI simple in general (from the
logs again):

~~~~START~~~~
Reading https://pypi.org/simple/
Download error on https://pypi.org/simple/: [Errno -2] Name or service not known -- Some packages may not be found!
No local packages or working download links found for ipykernel
error: Could not find suitable distribution for Requirement.parse('ipykernel')
command "python" "-c" "import setuptools, tokenize;__file__='setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))" "test" failed with status 1
~~~~~END~~~~~

Thanks,
Zelphir


On 10/18/20 6:35 PM, Efraim Flashner wrote:
> On Sun, Oct 18, 2020 at 01:55:49PM +0200, Zelphir Kaltstahl wrote:
>> Hello Guix Users, hello Zimoun and Tobias!
>>
>> Thanks for your information regarding the importer syntax thing.
>>
>> I continued trying to get a jupyterlab environment today.
>>
>> Here is what I did so far:
>>
>> (1) Create my manifest file:
>>
>> ~~~~START~~~~
>> (specifications->manifest
>>  '("python"))
>> ~~~~~END~~~~~
>>
>> (2) Create my channels file:
>>
>> ~~~~START~~~~
>> $ guix describe --format=channels > channels.scm
>> $ cat channels.scm
>> (list (channel
>>         (name 'guix)
>>         (url "https://git.savannah.gnu.org/git/guix.git")
>>         (commit
>>           "6ca673b30dac281ba27a8dac242c0ac83e5e354b")
>>         (introduction
>>           (make-channel-introduction
>>             "9edb3f66fd807b096b48283debdcddccfea34bad"
>>             (openpgp-fingerprint
>>               "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA")))))
>> ~~~~~END~~~~~
>>
>> (3) Create my jupyterlab PyPI import file, this time around using the
>> "/" instead of "@" for specifying the version:
>>
>> ~~~~START~~~~
>> $ guix import pypi --recursive jupyterlab/2.1.5 > jupyterlab.scm
>> ~~~~~END~~~~~
>>
>> (4) Add missing import expressions and add output expression at the end:
>>
>> ~~~~START~~~~
>> ;; add module imports
>> (use-modules
>>  (guix packages)
>>  (guix download)
>>  (guix build-system python)
>>  ((guix licenses) #:prefix license:)
>>  ;; use `guix environment --ad-hoc python-jinja2 -- guix package -A python-jinja2` to find the location of python-jinja2, which is gnu/packages/python-xyz.scm
> This can be just 'guix package -A python-jinja2', no need for the
> environment.
>
>>  (gnu packages python-xyz)
>>  ;; for python-tornado
>>  (gnu packages python-web)
>>  ;; for python-pytest
>>  (gnu packages check))
>>
>> ;; QUESTION: Why are those required packages not imported by guix automatically?
>> ;; QUESTION: Why are the manually imported packages not actually in files with their own name, but instead jinja2 -> python-xyz, tornado -> python-web, and pytest -> check?
>> ;; QUESTION: Is my process of finding the files in which the packages are correct or just working by chance?
>>
>> (define-public python-jupyterlab-server
>>   (package
>>     (name "python-jupyterlab-server")
>>     (version "1.2.0")
>>     (source
>>       (origin
>>         (method url-fetch)
>>         (uri (pypi-uri "jupyterlab-server" version))
>>         (sha256
>>           (base32
>>             "132xby7531rbrjg9bqvsx86birr1blynjxy8gi5kcnb6x7fxjcal"))))
>>     (build-system python-build-system)
>>     (propagated-inputs
>>       `(("python-jinja2" ,python-jinja2)
>>         ("python-json5" ,python-json5)
>>         ("python-jsonschema" ,python-jsonschema)
>>         ("python-notebook" ,python-notebook)
>>         ("python-requests" ,python-requests)))
>>     (native-inputs
>>       `(("python-pytest" ,python-pytest)
>>         ("python-requests" ,python-requests)))
>>     (home-page "https://jupyter.org")
>>     (synopsis "JupyterLab Server")
>>     (description "JupyterLab Server")
>>     (license license:bsd-3)))
>>
>> (define-public python-jupyterlab
>>   (package
>>     (name "python-jupyterlab")
>>     (version "2.1.5")
>>     (source
>>       (origin
>>         (method url-fetch)
>>         (uri (pypi-uri "jupyterlab" version))
>>         (sha256
>>           (base32
>>             "162jn51cg36fsn4l2zhnb5n4nbkhm9wlv974ggcnmdij3i4r4yya"))))
>>     (build-system python-build-system)
>>     (propagated-inputs
>>       `(("python-jinja2" ,python-jinja2)
>>         ("python-jupyterlab-server"
>>          ,python-jupyterlab-server)
>>         ("python-notebook" ,python-notebook)
>>         ("python-tornado" ,python-tornado)))
>>     (native-inputs
>>       `(("python-pytest" ,python-pytest)
>>         ("python-pytest-check-links"
>>          ,python-pytest-check-links)
>>         ("python-requests" ,python-requests)
>>         ("python-virtualenv" ,python-virtualenv)
>>         ("python-wheel" ,python-wheel)))
>>     (home-page "http://jupyter.org")
>>     (synopsis
>>       "The JupyterLab notebook server extension.")
>>     (description
>>       "The JupyterLab notebook server extension.")
>>     (license license:bsd-3)))
>>
>> python-jupyterlab
>> ~~~~~END~~~~~
>>
>> And as noted in the comments, I have 3 questions at this point:
>>
>> (Q1) Why are those required packages not imported by guix automatically?
>> (My guess: This functionality simply has not yet been implemented.)
>>
>> (Q2) Why are the manually imported packages not actually in files with
>> their own name, but instead jinja2 -> python-xyt, tornado -> python-web,
>> and pytest -> check? Why not (gnu packages python pytest) for example?
>>
>> (Q3) Is my process of finding the files in which the packages are
>> correct or just working by chance? Basically I am creating temporary
>> environments, in which I then lookup "where would the package be
>> installed" and then exit the temporary environment to use that
>> information in my imported package file's import expressions /
>> use-modules expression. This seems to be a bit of a convoluted process,
>> though worth it, if it always works in the end.
>>
>> (4) Now I run my time-machine command:
>>
>> ~~~~START~~~~
>> $ guix time-machine --channels=channels.scm -- environment --manifest=manifest.scm --load=jupyterlab.scm
>> ~~~~~END~~~~~
>>
>> This take a moment and then I get an error (the full output is longer,
>> but when running it multiple times, already successful steps are not all
>> done repeatedly, so the output is then reduced to what fails):
>>
>> ~~~~START~~~~
>> $ guix time-machine --channels=channels.scm -- environment --manifest=manifest.scm --load=jupyterlab.scm
>> Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
>> The following derivations will be built:
>>    /gnu/store/wzmfn9m45bmsbrx094rv3lf3w0rpmw49-python-pytest-check-links-0.3.0.drv
>>    /gnu/store/n9i421lrbwrr8d0947w5vma7rhy3iyba-python-jupyterlab-server-1.2.0.drv
>>    /gnu/store/s7jc2dbb0bnrahxklmszmpfv8klgw3sd-jupyterlab-server-1.2.0.tar.gz.drv
>>
>> 4,2 MB will be downloaded
>> downloading from https://ci.guix.gnu.org/nar/lzip/2nahbs5v8q073gjmy224gk0h5lsh1kkw-python-nbconvert-5.0.0b1 ...
>>  python-nbconvert-5.0.0b1  301KiB                                                                                                                                                                                                     2.0MiB/s 00:00 [##################] 100.0%
>>
>> building /gnu/store/s7jc2dbb0bnrahxklmszmpfv8klgw3sd-jupyterlab-server-1.2.0.tar.gz.drv...
>> |builder for `/gnu/store/s7jc2dbb0bnrahxklmszmpfv8klgw3sd-jupyterlab-server-1.2.0.tar.gz.drv' failed to produce output path `/gnu/store/92md7gh966n4fzj6al267lqq4xjx0ncf-jupyterlab-server-1.2.0.tar.gz'
>> build of /gnu/store/s7jc2dbb0bnrahxklmszmpfv8klgw3sd-jupyterlab-server-1.2.0.tar.gz.drv failed
>> View build log at '/var/log/guix/drvs/s7/jc2dbb0bnrahxklmszmpfv8klgw3sd-jupyterlab-server-1.2.0.tar.gz.drv.bz2'.
>> cannot build derivation `/gnu/store/n9i421lrbwrr8d0947w5vma7rhy3iyba-python-jupyterlab-server-1.2.0.drv': 1 dependencies couldn't be built
>> killing process 7384
>> guix environment: error: build of `/gnu/store/n9i421lrbwrr8d0947w5vma7rhy3iyba-python-jupyterlab-server-1.2.0.drv' failed
>> ~~~~~END~~~~~
>>
>> OK, an error, perhaps I can understand it. Lets look at that log file:
>>
>> ~~~~START~~~~
>> $ cp /var/log/guix/drvs/s7/jc2dbb0bnrahxklmszmpfv8klgw3sd-jupyterlab-server-1.2.0.tar.gz.drv.bz2 .
>> $ bzip2 --decompress jc2dbb0bnrahxklmszmpfv8klgw3sd-jupyterlab-server-1.2.0.tar.gz.drv.bz2
>> $ cat jc2dbb0bnrahxklmszmpfv8klgw3sd-jupyterlab-server-1.2.0.tar.gz.drv
>>
>> Starting download of /gnu/store/92md7gh966n4fzj6al267lqq4xjx0ncf-jupyterlab-server-1.2.0.tar.gz
>> From https://files.pythonhosted.org/packages/source/j/jupyterlab-server/jupyterlab-server-1.2.0.tar.gz...
>> download failed "https://files.pythonhosted.org/packages/source/j/jupyterlab-server/jupyterlab-server-1.2.0.tar.gz" 404 "Not Found"
>>
>> Starting download of /gnu/store/92md7gh966n4fzj6al267lqq4xjx0ncf-jupyterlab-server-1.2.0.tar.gz
>> From https://ci.guix.gnu.org/file/jupyterlab-server-1.2.0.tar.gz/sha256/132xby7531rbrjg9bqvsx86birr1blynjxy8gi5kcnb6x7fxjcal...
>> download failed "https://ci.guix.gnu.org/file/jupyterlab-server-1.2.0.tar.gz/sha256/132xby7531rbrjg9bqvsx86birr1blynjxy8gi5kcnb6x7fxjcal" 404 "Not Found"
>>
>> Starting download of /gnu/store/92md7gh966n4fzj6al267lqq4xjx0ncf-jupyterlab-server-1.2.0.tar.gz
>> From https://tarballs.nixos.org/sha256/132xby7531rbrjg9bqvsx86birr1blynjxy8gi5kcnb6x7fxjcal...
>> download failed "https://tarballs.nixos.org/sha256/132xby7531rbrjg9bqvsx86birr1blynjxy8gi5kcnb6x7fxjcal" 404 "Not Found"
>>
>> Starting download of /gnu/store/92md7gh966n4fzj6al267lqq4xjx0ncf-jupyterlab-server-1.2.0.tar.gz
>> From https://archive.softwareheritage.org/api/1/content/sha256:5431d9dde96659364b7cc877693d5d21e7b80cea7ae3959ecc2b87518e5f5d8c/raw/...
>> download failed "https://archive.softwareheritage.org/api/1/content/sha256:5431d9dde96659364b7cc877693d5d21e7b80cea7ae3959ecc2b87518e5f5d8c/raw/" 404 "Not Found"
>> failed to download "/gnu/store/92md7gh966n4fzj6al267lqq4xjx0ncf-jupyterlab-server-1.2.0.tar.gz" from "https://files.pythonhosted.org/packages/source/j/jupyterlab-server/jupyterlab-server-1.2.0.tar.gz"
>> ~~~~~END~~~~~
>>
>> So it seems, that the jupyterlab-server file is simply not downloadable
>> any longer!
>>
>> Trying using wget:
>>
>> ~~~~START~~~~
>> $ wget https://files.pythonhosted.org/packages/source/j/jupyterlab-server/jupyterlab-server-1.2.0.tar.gz
>> --2020-10-18 13:49:52--  https://files.pythonhosted.org/packages/source/j/jupyterlab-server/jupyterlab-server-1.2.0.tar.gz
>> Resolving files.pythonhosted.org (files.pythonhosted.org)... 2a04:4e42:9::319, 151.101.37.63
>> Connecting to files.pythonhosted.org (files.pythonhosted.org)|2a04:4e42:9::319|:443... failed: Connection refused.
>> Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.37.63|:443... connected.
>> HTTP request sent, awaiting response... 404 Not Found
>> 2020-10-18 13:49:53 ERROR 404: Not Found.
>> ~~~~~END~~~~~
>>
>> So now I am out of ideas. It seems the file is not available any longer
>> and the definition of jupyterlab needs to be updated somewhere, so that
>> it does not try to download an old or non-existent version of
>> jupyter-labserver. But how is this usually done? Can I specify it
>> somewhere in one of the 3 files I have, or am I dependent on someone
>> fixing the jupyterlab-server dependency / input for juypterlab?
>>
>> Thanks for your help so far! Tell me, if I need to provide any more
>> information to solve the problem.
> Try chaning the URL line to (uri (pypi-uri "jupyterlab_server" version)),
> with an underscore (_). I don't have any special suggestions for knowing
> that it should be an underscore instead of a dash. The best I have is
> the output from 'guix import pypi jupyterlab-server:
>
> (ins)efraim@E5400 ~$ guix import pypi jupyterlab-server
>
> Starting download of /tmp/guix-file.ayJ41F
> From https://files.pythonhosted.org/packages/92/54/5f321ca4cebfb644ac59eb21601b46a2ff9831ffe1b4e5a588f2a7e98d48/jupyterlab_server-1.2.0.tar.gz...
>  …er-1.2.0.tar.gz  22KiB               57KiB/s 00:00 [##################] 100.0%
>
>
>
>> Best regards,
>> Zelphir
>>
>>
>> On 10/16/20 2:25 PM, Tobias Geerinckx-Rice wrote:
>>> Zelphir,
>>>
>>> Zelphir Kaltstahl 写道：
>>>> $ guix import pypi 'jupyterlab@2.1.5' --recursive
>>> It will work if you substitute ‘/’ for ‘@’.  Yes, this is a happy
>>> accident specific to the PyPI importer, and I'm sure you can guess why
>>> it happens to work.
>>>
>>>>> guix import hackage mtl@2.1.3.1
>>>> But that is in the "hackage" import section. Is the same not available
>>>> for PyPI?
>>> [...]
>>>
>>>> Do the various importers work differently?
>>> Unfortunately so.  It would be great(TM) if PACKAGE[@VERSION] were
>>> standard.  Each importer could handle it properly (e.g., by turning it
>>> into "/") or return a friendlier ‘the foo importer does not yet
>>> support versions’ error.
>>>
>>> I don't have the time, but this sounds like a good intermediate-level
>>> hack for someone who does :-)
>>>
>>> Kind regards,
>>>
>>> T G-R
>> -- 
>> repositories: https://notabug.org/ZelphirKaltstahl
>>
-- 
repositories: https://notabug.org/ZelphirKaltstahl