bug#46333: sbcl-common-lisp-jupyter does not install kernel.json

bug#46333: sbcl-common-lisp-jupyter does not install kernel.json

[Jack Hill]

[-- Attachment #1: Type: text/plain, Size: 2110 bytes --]

Hi Guix,

The sbcl-common-lisp-jupyter package does not install a kernel.json file. 
That's the file that tells Jupyter about the kernel and how to run it, and 
should be installed in /share/jupyter/kernels/<kernel-name>/kernel.json.

sbcl-common-lisp-jupyter doesn't come with a kernel.json file to install, 
but it can generate one with the following command line:

`sbcl --eval '(require "asdf")' --eval '(require :common-lisp-jupyter)' 
--eval '(cl-jupyter:install)' --eval '(exit)'`

(please pardon any awkwardness with the sbcl command line, I'm new to 
Common Lisp, and just wanted to play around with it in Jupyter)

That produces the following kernel.json in $HOME/.local/… (I've pretty 
printed it for clarity here:

```
{
   "interrupt_method": "message",
   "language": "common-lisp",
   "display_name": "Common Lisp",
   "argv": [
     "sbcl",
     "--eval",
     "(ql:quickload :common-lisp-jupyter)",
     "--eval",
     "(jupyter:run-kernel 'common-lisp-jupyter:kernel #\"{connection_file}\")"
   ]
}
```

Unfortunately that won't work out of the box, as we don't have quicklisp, 
but changing it to:

```
{
   "interrupt_method": "message",
   "language": "common-lisp",
   "display_name": "Common Lisp",
   "argv": [
     "sbcl",
     "--eval",
     "(require \"asdf\")",
     "--eval",
     "(require :common-lisp-jupyter)",
     "--eval",
     "(jupyter:run-kernel 'common-lisp-jupyter:kernel #\"{connection_file}\")"
   ]
}
```

allows Jupyter to run the kernel. We would of course need to also 
substitute the full store path for sbcl as well.

Is it worth having sbcl-common-lisp-jupyter generate the kernel.json, and 
then make many changes to it? Perhaps it would be better to just write out 
the correct definition of the file from Guix.

A final note is that the other Common Lisp implementation, like 
ecl-common-lisp-jupyter, also have this problem because they are created 
as transformation of the sbcl package. I'm not sure if the kernel.json is 
portable across the implementation or in general how to best to accomplish 
this change for our Common Lisp packages.

Best,
Jack

bug#46333: sbcl-common-lisp-jupyter does not install kernel.json

[Jack Hill]

Sharlatan,

Thanks for your recent work updated sbcl-common-lisp-jupyter. I was 
wondering if you had any thoughts on the best way to install the 
kernel.json file [0]. The last time I looked at it, I wasn't sure what the 
best solution would be. Do you have any ideas?

[0] https://issues.guix.gnu.org/46333

Best,
Jack

bug#46333: sbcl-common-lisp-jupyter does not install kernel.json

[Guillaume Le Vaillant]

[-- Attachment #1: Type: text/plain, Size: 315 bytes --]

Hi Jack,

I guess it will be easier to just add a phase writing the "kernel.json"
file in the right place. In this build phase, to know if the package is
being built for SBCL or ECL, the '(%lisp-type)' function that will
return "sbcl" or "ecl" can be used. There's an example in the
sbcl-trivial-backtrace package.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

bug#46333: sbcl-common-lisp-jupyter does not install kernel.json

[Sharlatan Hellseher]

Hi,

I've checked the r-irkernel and it's coping existing kernelspec ,
which is not useful in this case.

As Guillaume mentioned we could tweak it before installation phase by
using cl-jupyter:install, so here is my draft:

--8<---------------cut here---------------start------------->8---
    (arguments
     `(#:phases
       (modify-phases %standard-phases
         (add-before 'install 'generate-kernelspec
           (lambda* (#:key outputs #:allow-other-keys)
             (let* ((out (assoc-ref outputs "out"))
                    (kernelspec (string-append out
"/share/cl-jupyter/kernelspec")))
               (mkdir-p kernelspec)
               (invoke "sbcl"
                       "--eval" "\"(require :asdf)\""
                       "--eval" "\"(require :common-lisp-jupyter)\""
                       "--eval"
                       (string-append
                        "\"(cl-jupyter:install"
                        ":bin-path" (string-append
                                     (assoc-ref %build-inputs "sbcl")
"/bin/sbcl")
                        ":prefix" out ")\"")
                       "--eval" "\"(exit)\""))
             #t))
         (add-after 'install 'install-kernelspec
           (lambda* (#:key outputs #:allow-other-keys)
             (let ((out (assoc-ref outputs "out"))
                   (kernelspec (string-append out
"/share/cl-jupyter/kernelspec")))
               (invoke "jupyter" "kernelspec" "install"
                       "--name" "cl-jupyter"
                       "--prefix" out
                       kernelspec)
               #t))))))
--8<---------------cut here---------------end--------------->8---

But there could be a potential blocking issue with :prefix key

https://github.com/yitzchak/common-lisp-jupyter/issues/78

On Tue, 18 May 2021 at 16:58, Guillaume Le Vaillant <glv@posteo.net> wrote:
>
> Hi Jack,
>
> I guess it will be easier to just add a phase writing the "kernel.json"
> file in the right place. In this build phase, to know if the package is
> being built for SBCL or ECL, the '(%lisp-type)' function that will
> return "sbcl" or "ecl" can be used. There's an example in the
> sbcl-trivial-backtrace package.



--

… наш разум - превосходная объяснительная машина которая способна
найти смысл почти в чем угодно, истолковать любой феномен, но
совершенно не в состоянии принять мысль о непредсказуемости.

bug#46333: sbcl-common-lisp-jupyter does not install kernel.json

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 8610 bytes --]

Hi,

I played with cl-jupyter:install but it heavily depends on Quicklisp
but what basically does - generates simple JSON with CL implementation
https://github.com/yitzchak/common-lisp-jupyter/issues/78

First I tried to do the same during build phase by evaluationg
arbitrary Lisp but could not manged it to work.
Then after checking the kernel.json I just simply formated it with
right %lispt-type path and copied with the same evaluation as R
Jupyter package, but I've got some error during coping into Jupyter
directory:

--8<---------------cut here---------------start------------->8---
phase `generate-kernelspec' succeeded after 0.0 seconds
starting phase `install-kernelspec'
command "jupyter" "kernelspec" "install" "--name" "cl-jupyter"
"--prefix" "/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7"
"/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7/share/cl-jupyter/kernelspec"
failed with status 127
builder for `/gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv'
failed with exit code 1
build of /gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv
failed
View build log at
'/var/log/guix/drvs/az/l65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv.bz2'.
guix build: error: build of
`/gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv'
failed
(guix/linux-gnu)[sharlatan@guxtil ~/Projects/prj/guix-channel]$:
jupyter kernelspec install --name cl-jupyter --prefix
/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7
/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7/share/cl-jupyter/kernelspec
Traceback (most recent call last):
  File "/gnu/store/30ydqwp1xccqzn5s4rcq4clpqzcaz3p1-python-jupyter-client-6.1.12/bin/.jupyter-kernelspec-real",
line 11, in <module>
    load_entry_point('jupyter-client==6.1.12', 'console_scripts',
'jupyter-kernelspec')()
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 663, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-2>", line 2, in initialize
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 297, in initialize
    self.parse_command_line(argv)
  File "<decorator-gen-4>", line 2, in parse_command_line
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 515, in parse_command_line
    return self.initialize_subcommand(subc, subargv)
  File "<decorator-gen-3>", line 2, in initialize_subcommand
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 453, in initialize_subcommand
    self.subapp.initialize(argv)
  File "<decorator-gen-6>", line 2, in initialize
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/gnu/store/yj346rpx2mf1afa6jxl2kh9i335an27y-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py",
line 229, in initialize
    self.migrate_config()
  File "/gnu/store/yj346rpx2mf1afa6jxl2kh9i335an27y-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py",
line 155, in migrate_config
    migrate()
  File "/gnu/store/yj346rpx2mf1afa6jxl2kh9i335an27y-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/migrate.py",
line 244, in migrate
    ensure_dir_exists(env['jupyter_config'])
  File "/gnu/store/yj346rpx2mf1afa6jxl2kh9i335an27y-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/utils/__init__.py",
line 11, in ensure_dir_exists
    os.makedirs(path, mode=mode)
  File "/gnu/store/hq7qr7nc2j29z3pivm3azfjy6jq3d7nx-python-3.8.2/lib/python3.8/os.py",
line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/gnu/store/hq7qr7nc2j29z3pivm3azfjy6jq3d7nx-python-3.8.2/lib/python3.8/os.py",
line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/gnu/store/hq7qr7nc2j29z3pivm3azfjy6jq3d7nx-python-3.8.2/lib/python3.8/os.py",
line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  [Previous line repeated 12 more times]
  File "/gnu/store/hq7qr7nc2j29z3pivm3azfjy6jq3d7nx-python-3.8.2/lib/python3.8/os.py",
line 223, in makedirs
    mkdir(name, mode)
--8<---------------cut here---------------end--------------->8---

On Wed, 19 May 2021 at 22:23, Sharlatan Hellseher <sharlatanus@gmail.com> wrote:
>
> Hi,
>
> I've checked the r-irkernel and it's coping existing kernelspec ,
> which is not useful in this case.
>
> As Guillaume mentioned we could tweak it before installation phase by
> using cl-jupyter:install, so here is my draft:
>
> --8<---------------cut here---------------start------------->8---
>     (arguments
>      `(#:phases
>        (modify-phases %standard-phases
>          (add-before 'install 'generate-kernelspec
>            (lambda* (#:key outputs #:allow-other-keys)
>              (let* ((out (assoc-ref outputs "out"))
>                     (kernelspec (string-append out
> "/share/cl-jupyter/kernelspec")))
>                (mkdir-p kernelspec)
>                (invoke "sbcl"
>                        "--eval" "\"(require :asdf)\""
>                        "--eval" "\"(require :common-lisp-jupyter)\""
>                        "--eval"
>                        (string-append
>                         "\"(cl-jupyter:install"
>                         ":bin-path" (string-append
>                                      (assoc-ref %build-inputs "sbcl")
> "/bin/sbcl")
>                         ":prefix" out ")\"")
>                        "--eval" "\"(exit)\""))
>              #t))
>          (add-after 'install 'install-kernelspec
>            (lambda* (#:key outputs #:allow-other-keys)
>              (let ((out (assoc-ref outputs "out"))
>                    (kernelspec (string-append out
> "/share/cl-jupyter/kernelspec")))
>                (invoke "jupyter" "kernelspec" "install"
>                        "--name" "cl-jupyter"
>                        "--prefix" out
>                        kernelspec)
>                #t))))))
> --8<---------------cut here---------------end--------------->8---
>
> But there could be a potential blocking issue with :prefix key
>
> https://github.com/yitzchak/common-lisp-jupyter/issues/78
>
> On Tue, 18 May 2021 at 16:58, Guillaume Le Vaillant <glv@posteo.net> wrote:
> >
> > Hi Jack,
> >
> > I guess it will be easier to just add a phase writing the "kernel.json"
> > file in the right place. In this build phase, to know if the package is
> > being built for SBCL or ECL, the '(%lisp-type)' function that will
> > return "sbcl" or "ecl" can be used. There's an example in the
> > sbcl-trivial-backtrace package.
>
>
>
> --
>
> … наш разум - превосходная объяснительная машина которая способна
> найти смысл почти в чем угодно, истолковать любой феномен, но
> совершенно не в состоянии принять мысль о непредсказуемости.



-- 

… наш разум - превосходная объяснительная машина которая способна
найти смысл почти в чем угодно, истолковать любой феномен, но
совершенно не в состоянии принять мысль о непредсказуемости.

[-- Attachment #2: 0001-gnu-common-lisp-jupyter-Format-kernelspec.patch --]
[-- Type: text/x-patch, Size: 2392 bytes --]

From cbc17c767069e8fa5175f2f0f30fc961a2be319a Mon Sep 17 00:00:00 2001
From: Sharlatan Hellseher <sharlatanus@gmail.com>
Date: Mon, 24 May 2021 22:18:28 +0100
Subject: [PATCH] gnu: common-lisp-jupyter: Format kernelspec

---
 gnu/packages/lisp-xyz.scm | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/gnu/packages/lisp-xyz.scm b/gnu/packages/lisp-xyz.scm
index 4a1e9064d5..91370c62c4 100644
--- a/gnu/packages/lisp-xyz.scm
+++ b/gnu/packages/lisp-xyz.scm
@@ -14897,6 +14897,41 @@ and @code{doseq*}.")
          (sha256
           (base32 "0si69xfzi769dprwfy7gp1x3bl7lxz6d4n98sa26w9r41wvay5ja"))))
       (build-system asdf-build-system/sbcl)
+      (arguments
+       `(#:phases
+         (modify-phases %standard-phases
+           (add-after 'create-asdf-configuration 'generate-kernelspec
+             (lambda* (#:key outputs #:allow-other-keys)
+               (let* ((out (assoc-ref outputs "out"))
+                      (kernelspec (string-append out "/share/cl-jupyter/kernelspec")))
+                 (mkdir-p kernelspec)
+                 (copy-file (string-append "res/" (%lisp-type) "/logo-64x64.png")
+                            (string-append kernelspec "/logo-64x64.png"))
+                 (with-output-to-file (string-append kernelspec "/kernel.json")
+                   (lambda ()
+                     ;; FIXME: Use of guile-json would beneficial here
+                     (format #t "{ \"argv\": [ ~s,
+ \"--eval\",
+\"(require :asdf)\",
+ \"--eval\",
+\"(asdf:load-system :common-lisp-jupyter)\",
+ \"--eval\",
+\"(jupyter:run-kernel 'common-lisp-jupyter:kernel)\",
+\"{connection_file}\"
+],
+\"display_name\": \"Common Lisp\",
+\"language\": \"common-lisp\",
+\"interrupt_mode\": \"message\"
+}"
+                             (string-append (assoc-ref %build-inputs (%lisp-type)) "/bin/" (%lisp-type))))))))
+      (add-after 'generate-kernelspec 'install-kernelspec
+        (lambda* (#:key outputs #:allow-other-keys)
+          (let* ((out (assoc-ref outputs "out"))
+                (kernelspec (string-append out "/share/cl-jupyter/kernelspec")))
+            (invoke "jupyter" "kernelspec" "install"
+                    "--name" "cl-jupyter"
+                    "--prefix" out
+                    kernelspec)))))))
       (inputs
        `(("alexandria" ,sbcl-alexandria)
          ("babel" ,sbcl-babel)
-- 
2.31.1

bug#46333: sbcl-common-lisp-jupyter does not install kernel.json

[Guillaume Le Vaillant]

[-- Attachment #1: Type: text/plain, Size: 1818 bytes --]

Sharlatan Hellseher <sharlatanus@gmail.com> skribis:

> I played with cl-jupyter:install but it heavily depends on Quicklisp
> but what basically does - generates simple JSON with CL implementation
> https://github.com/yitzchak/common-lisp-jupyter/issues/78
>
> First I tried to do the same during build phase by evaluationg
> arbitrary Lisp but could not manged it to work.
> Then after checking the kernel.json I just simply formated it with
> right %lispt-type path and copied with the same evaluation as R
> Jupyter package, but I've got some error during coping into Jupyter
> directory:
>
> --8<---------------cut here---------------start------------->8---
> phase `generate-kernelspec' succeeded after 0.0 seconds
> starting phase `install-kernelspec'
> command "jupyter" "kernelspec" "install" "--name" "cl-jupyter"
> "--prefix" "/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7"
> "/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7/share/cl-jupyter/kernelspec"
> failed with status 127
> builder for `/gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv'
> failed with exit code 1
> build of /gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv
> failed
> [...]
> --8<---------------cut here---------------end--------------->8---

I think this error comes from the fact that the jupyter package is
missing as native input, therefore the 'jupyter ...' command can't work.

It lools like the 'install-kernelspec' phase just copies the kernel you
generated in "share/cl-jupyter/kernelspec/" to
"share/jupyter/kernels/cl-jupyter/". Wouldn't it be easier to generate
the kernel in the final directory directly? Then the
'install-kernelspec' phase would not be necessary.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

bug#46333: Install Command Line

[Tarn Burton]

[-- Attachment #1: Type: text/plain, Size: 541 bytes --]

The command line that you should be using is

sbcl --non-interacive --eval "(require :asdf)" --eval "(asdf:load-system
:common-lisp-jupyter)" --eval "(clj:install :use-implementation :prefix
\"bla\" :system t)"

The sbcl should not have quicklisp installed so that the installer knows
that this is not a quicklisp install. If the user's sbcl binary is located
in a different place then you may need to add :bin-path

https://github.com/yitzchak/common-lisp-jupyter/blob/master/README.md#installing-via-quicklispasdf

-Tarn Burton/CLJ Author

[-- Attachment #2: Type: text/html, Size: 1131 bytes --]

bug#46333: Typos

[Tarn Burton]

[-- Attachment #1: Type: text/plain, Size: 225 bytes --]

Looks like I had some typos in the command.

sbcl --non-interactive --eval "(require :asdf)" --eval
"(asdf:load-system:common-lisp-jupyter)"
--eval "(clj:install :use-implementation t :prefix\"bla\" :system t)"

-Tarn Burton

[-- Attachment #2: Type: text/html, Size: 510 bytes --]