IPython native kernel missing?

IPython native kernel missing?

[Ricardo Wurmus]

Hi Guix,

I’m trying to package Jupyter (an improved version of IPython, from what
I’ve heard) and noticed that the tests for jupyter-client fail because
the native “python3” (or “python2”) kernel cannot be found.  According
to the documentation, this “kernel” should always be available and
doesn’t need to be installed.

Looking around in the sources of python-ipython (an input to the Jupyter
packages I’m working on) I see that this file

    $out/lib/python2.7/site-packages/IPython/kernel/kernelspec.py

contains hard-coded system paths:

    SYSTEM_KERNEL_DIRS = ["/usr/share/jupyter/kernels",
                          "/usr/local/share/jupyter/kernels",
                         ]

Yet I have not found any directory named “kernels” in the output of the
python-ipython package.  Nor have I found any “kernel.json” files.  It
seems to me that IPython should install at least the native kernel at
some point, but currently doesn’t seem to include it.

I would like to fix this, but I really don’t know anything about IPython
and I don’t even know what the word “kernel” describes in this context.
Could someone with an understanding of IPython please take a look at
this?

~~ Ricardo

Re: IPython native kernel missing?

[Craig Barneses]

[-- Attachment #1: Type: text/plain, Size: 1819 bytes --]

Hi Ricardo,


On 21/09/15 16:28, Ricardo Wurmus wrote:
> Hi Guix,
>
> I’m trying to package Jupyter (an improved version of IPython, from what
> I’ve heard) and noticed that the tests for jupyter-client fail because
> the native “python3” (or “python2”) kernel cannot be found.  According
> to the documentation, this “kernel” should always be available and
> doesn’t need to be installed.
>
> Looking around in the sources of python-ipython (an input to the Jupyter
> packages I’m working on) I see that this file
>
>     $out/lib/python2.7/site-packages/IPython/kernel/kernelspec.py
>
> contains hard-coded system paths:
>
>     SYSTEM_KERNEL_DIRS = ["/usr/share/jupyter/kernels",
>                           "/usr/local/share/jupyter/kernels",
>                          ]
>
> Yet I have not found any directory named “kernels” in the output of the
> python-ipython package.  Nor have I found any “kernel.json” files.  It
> seems to me that IPython should install at least the native kernel at
> some point, but currently doesn’t seem to include it.
>
> I would like to fix this, but I really don’t know anything about IPython
> and I don’t even know what the word “kernel” describes in this context.
> Could someone with an understanding of IPython please take a look at
> this?

looking at setup.py [0] it seems to be a separate module that is pulled
in when setup.py is run. 

Are we depending on requirements.txt to give us this information, as
this seems to be absent from this project?

if so, I've attached a requirements.txt for python 2 and 3 generated
from version in pypi.  I expect the reason that it is not included in
the package is that it's different depending on the python version.


Regards

Craig

[0] https://github.com/jupyter/jupyter/blob/master/setup.py

[-- Attachment #2: python-requirements.txt --]
[-- Type: text/plain, Size: 458 bytes --]

decorator==4.0.2
ipykernel==4.0.3
ipython==4.0.0
ipython-genutils==0.1.0
ipywidgets==4.0.3
Jinja2==2.8
jsonschema==2.5.1
jupyter==1.0.0
jupyter-client==4.0.0
jupyter-console==4.0.2
jupyter-core==4.0.6
MarkupSafe==0.23
mistune==0.7.1
nbconvert==4.0.0
nbformat==4.0.0
notebook==4.0.4
path.py==8.1.1
pexpect==3.3
pickleshare==0.5
ptyprocess==0.5
Pygments==2.0.2
pyzmq==14.7.0
qtconsole==4.0.1
simplegeneric==0.8.1
terminado==0.5
tornado==4.2.1
traitlets==4.0.0

[-- Attachment #3: python2-requirements.txt --]
[-- Type: text/plain, Size: 541 bytes --]

backports.ssl-match-hostname==3.4.0.2
certifi==2015.9.6.2
decorator==4.0.2
functools32==3.2.3.post2
ipykernel==4.0.3
ipython==4.0.0
ipython-genutils==0.1.0
ipywidgets==4.0.3
Jinja2==2.8
jsonschema==2.5.1
jupyter==1.0.0
jupyter-client==4.0.0
jupyter-console==4.0.2
jupyter-core==4.0.6
MarkupSafe==0.23
mistune==0.7.1
nbconvert==4.0.0
nbformat==4.0.0
notebook==4.0.4
path.py==8.1.1
pexpect==3.3
pickleshare==0.5
ptyprocess==0.5
Pygments==2.0.2
pyzmq==14.7.0
qtconsole==4.0.1
simplegeneric==0.8.1
terminado==0.5
tornado==4.2.1
traitlets==4.0.0

Re: IPython native kernel missing?

[Craig Barnes]

[-- Attachment #1: Type: text/plain, Size: 1775 bytes --]

Hi Ricardo,

On 21/09/15 16:28, Ricardo Wurmus wrote:
> Hi Guix,
>
> I’m trying to package Jupyter (an improved version of IPython, from what
> I’ve heard) and noticed that the tests for jupyter-client fail because
> the native “python3” (or “python2”) kernel cannot be found.  According
> to the documentation, this “kernel” should always be available and
> doesn’t need to be installed.
>
> Looking around in the sources of python-ipython (an input to the Jupyter
> packages I’m working on) I see that this file
>
>     $out/lib/python2.7/site-packages/IPython/kernel/kernelspec.py
>
> contains hard-coded system paths:
>
>     SYSTEM_KERNEL_DIRS = ["/usr/share/jupyter/kernels",
>                           "/usr/local/share/jupyter/kernels",
>                          ]
>
> Yet I have not found any directory named “kernels” in the output of the
> python-ipython package.  Nor have I found any “kernel.json” files.  It
> seems to me that IPython should install at least the native kernel at
> some point, but currently doesn’t seem to include it.
>
> I would like to fix this, but I really don’t know anything about IPython
> and I don’t even know what the word “kernel” describes in this context.
> Could someone with an understanding of IPython please take a look at
> this?
IIRC we are using requirements.txt to determine all dependencies, which
it seems is not used by this project.  This is possibly because the
output is slightly different depending on the python version.

setup.py includes the ipykernel package, but to get a full list of
dependencies we need to install using pip in a virtualenv, then run pip
freeze to generate requirements.txt.

I've attached what I got as a result from python 2 and 3.


Cheers

Craig


[-- Attachment #2: python-requirements.txt --]
[-- Type: text/plain, Size: 458 bytes --]

decorator==4.0.2
ipykernel==4.0.3
ipython==4.0.0
ipython-genutils==0.1.0
ipywidgets==4.0.3
Jinja2==2.8
jsonschema==2.5.1
jupyter==1.0.0
jupyter-client==4.0.0
jupyter-console==4.0.2
jupyter-core==4.0.6
MarkupSafe==0.23
mistune==0.7.1
nbconvert==4.0.0
nbformat==4.0.0
notebook==4.0.4
path.py==8.1.1
pexpect==3.3
pickleshare==0.5
ptyprocess==0.5
Pygments==2.0.2
pyzmq==14.7.0
qtconsole==4.0.1
simplegeneric==0.8.1
terminado==0.5
tornado==4.2.1
traitlets==4.0.0

[-- Attachment #3: python2-requirements.txt --]
[-- Type: text/plain, Size: 541 bytes --]

backports.ssl-match-hostname==3.4.0.2
certifi==2015.9.6.2
decorator==4.0.2
functools32==3.2.3.post2
ipykernel==4.0.3
ipython==4.0.0
ipython-genutils==0.1.0
ipywidgets==4.0.3
Jinja2==2.8
jsonschema==2.5.1
jupyter==1.0.0
jupyter-client==4.0.0
jupyter-console==4.0.2
jupyter-core==4.0.6
MarkupSafe==0.23
mistune==0.7.1
nbconvert==4.0.0
nbformat==4.0.0
notebook==4.0.4
path.py==8.1.1
pexpect==3.3
pickleshare==0.5
ptyprocess==0.5
Pygments==2.0.2
pyzmq==14.7.0
qtconsole==4.0.1
simplegeneric==0.8.1
terminado==0.5
tornado==4.2.1
traitlets==4.0.0

Re: IPython native kernel missing?

[Ricardo Wurmus]

Hi Craig,

>> I’m trying to package Jupyter (an improved version of IPython, from what
>> I’ve heard) and noticed that the tests for jupyter-client fail because
>> the native “python3” (or “python2”) kernel cannot be found.  According
>> to the documentation, this “kernel” should always be available and
>> doesn’t need to be installed.
>>
>> Looking around in the sources of python-ipython (an input to the Jupyter
>> packages I’m working on) I see that this file
>>
>>     $out/lib/python2.7/site-packages/IPython/kernel/kernelspec.py
>>
>> contains hard-coded system paths:
>>
>>     SYSTEM_KERNEL_DIRS = ["/usr/share/jupyter/kernels",
>>                           "/usr/local/share/jupyter/kernels",
>>                          ]
>>
>> Yet I have not found any directory named “kernels” in the output of the
>> python-ipython package.  Nor have I found any “kernel.json” files.  It
>> seems to me that IPython should install at least the native kernel at
>> some point, but currently doesn’t seem to include it.
>>
>> I would like to fix this, but I really don’t know anything about IPython
>> and I don’t even know what the word “kernel” describes in this context.
>> Could someone with an understanding of IPython please take a look at
>> this?
> IIRC we are using requirements.txt to determine all dependencies, which
> it seems is not used by this project.  This is possibly because the
> output is slightly different depending on the python version.
>
> setup.py includes the ipykernel package, but to get a full list of
> dependencies we need to install using pip in a virtualenv, then run pip
> freeze to generate requirements.txt.
>
> I've attached what I got as a result from python 2 and 3.

These are the requirements for Jupyter, right?  They seem familiar
because I have packaged a couple of those in the list.  I’ll try to
package the missing dependencies (in particular “ipykernel”) and add it
to the inputs to see if this makes a difference.

Thank you!

~~ Ricardo

Re: IPython native kernel missing?

[Craig Barnes]

Hi Ricardo,
>>> I’m trying to package Jupyter (an improved version of IPython, from what
>>> I’ve heard) and noticed that the tests for jupyter-client fail because
>>> the native “python3” (or “python2”) kernel cannot be found.  According
>>> to the documentation, this “kernel” should always be available and
>>> doesn’t need to be installed.
>>>
>>> Looking around in the sources of python-ipython (an input to the Jupyter
>>> packages I’m working on) I see that this file
>>>
>>>     $out/lib/python2.7/site-packages/IPython/kernel/kernelspec.py
>>>
>>> contains hard-coded system paths:
>>>
>>>     SYSTEM_KERNEL_DIRS = ["/usr/share/jupyter/kernels",
>>>                           "/usr/local/share/jupyter/kernels",
>>>                          ]
>>>
>>> Yet I have not found any directory named “kernels” in the output of the
>>> python-ipython package.  Nor have I found any “kernel.json” files.  It
>>> seems to me that IPython should install at least the native kernel at
>>> some point, but currently doesn’t seem to include it.
>>>
>>> I would like to fix this, but I really don’t know anything about IPython
>>> and I don’t even know what the word “kernel” describes in this context.
>>> Could someone with an understanding of IPython please take a look at
>>> this?
>> IIRC we are using requirements.txt to determine all dependencies, which
>> it seems is not used by this project.  This is possibly because the
>> output is slightly different depending on the python version.
>>
>> setup.py includes the ipykernel package, but to get a full list of
>> dependencies we need to install using pip in a virtualenv, then run pip
>> freeze to generate requirements.txt.
>>
>> I've attached what I got as a result from python 2 and 3.
> These are the requirements for Jupyter, right?  They seem familiar
> because I have packaged a couple of those in the list.  I’ll try to
> package the missing dependencies (in particular “ipykernel”) and add it
> to the inputs to see if this makes a difference.
yes only Jupyter, the steps I used where

mkvirtualenv jupyter
pip install jupyter
# lots of busyness ...
pip freeze > requirements.txt

Of course you probably only need to make sure that the requirements
listed in setup.py are in you package as the rest are dependencies of
those.

Thank you for packaging this.


Craig