From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id iO5NNm9qU2EeEwAAgWs5BA (envelope-from ) for ; Tue, 28 Sep 2021 21:18:07 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id +woLMm9qU2HYewAA1q6Kng (envelope-from ) for ; Tue, 28 Sep 2021 19:18:07 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5BC3129CC for ; Tue, 28 Sep 2021 21:18:07 +0200 (CEST) Received: from localhost ([::1]:32990 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mVIcA-00029V-GJ for larch@yhetil.org; Tue, 28 Sep 2021 15:18:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42868) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mVIc6-00029N-9e for guix-patches@gnu.org; Tue, 28 Sep 2021 15:18:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:35480) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mVIc6-0006iU-1z for guix-patches@gnu.org; Tue, 28 Sep 2021 15:18:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mVIc5-0005tg-Q5 for guix-patches@gnu.org; Tue, 28 Sep 2021 15:18:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50708] [PATCH] gnu: web: Add jupyter-service References: <565a1b21-aefa-4129-a024-52517fdff9bd@gmail.com> Resent-From: Xinglu Chen Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 28 Sep 2021 19:18:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50708 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Jesse , 50708@debbugs.gnu.org Received: via spool by 50708-submit@debbugs.gnu.org id=B50708.163285665822639 (code B ref 50708); Tue, 28 Sep 2021 19:18:01 +0000 Received: (at 50708) by debbugs.gnu.org; 28 Sep 2021 19:17:38 +0000 Received: from localhost ([127.0.0.1]:47026 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mVIbh-0005t4-Br for submit@debbugs.gnu.org; Tue, 28 Sep 2021 15:17:38 -0400 Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:35436 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mVIbf-0005sr-2a for 50708@debbugs.gnu.org; Tue, 28 Sep 2021 15:17:36 -0400 From: Xinglu Chen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz; s=mail; t=1632856644; bh=gbF0IEUYTXNx0QXjC2vodx6M76RCGo+BUf+8zh0uHuQ=; h=From:To:Subject:In-Reply-To:Date; b=gSpg9ZfgDGh3VURUzSjtI1FWAXV3ann1u+XI6GFqVNawjOXwUVOmiBlwiwGjLHbZZ cgzCNjetsTlV7nKxBmUdA8FJOSpASnlkEauhGzeWBbGr2LZVISGD1cJu08VpCjr8mf nT+NK/+duq+plv4b2xZSEtvFbYkE1VOjyav1KI04= In-Reply-To: Date: Tue, 28 Sep 2021 21:17:23 +0200 Message-ID: <87bl4c33y4.fsf@yoctocell.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1632856687; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=Wu+VthX7Y1aDmfU+2uU9QaP0jAYBq4J9UYHXwXWjjcc=; b=qAhl8ttFHEyiUp6Eds1o5kQaJuhXx7yMXjkfzVPnxH18Cg/fqWmEedjbLcVapY8rtoIB3I akgIe8lQ96nC/7MIFfgy3VntP0DXWUxR5zo4L/SziNosWGyrTo3ykn2bkyUxhLCUfcBWnH EGmyNWyPlZlhxK00O+3AZGEabJPpGjSnHYKUOHJ9j5D4UKA92ZLyMQrN85hNy3LG54NQ3J WNMdpjE493ygzXj2/4juBUYlV0pDI1r5ceqj+E7gDlDFlc4o2HXqgfGINSd+NkmLf/WzCm shN/5KF7y4gGm5jGOH/NCVMJ1HQJVAUaQEUnGjKpxE49qJUXa9cF6ewji4BdLA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1632856687; a=rsa-sha256; cv=none; b=KTJnLUMv949c2QWgR0c1mnXffsrV+EFsaJpaLqLYi65tIOspVCMMPLkk/Q2pQD5GmpU89I zPY0+VzBsKdr87wnOAy7O4j4s2NXm6AxlDhlsIGnuMJ9dYt7loTbaz1AqJpJDZTNMlHvEB beHsWW/1XzXnxtLd+12fVktyj9/4rmeGw27Q+XeNB9m7ldooMMuzzR+Zw+3q3Gcnh9yf5O yAGMKAA1vX3sqRkNf/eAAZVLTP4m2EOAZi8Z60OMF+q//h8s8UXU05NCCB9EbQ3gaj388Q U5YOaCAUpIB1cElvyGRt6g0FfnNPkDffOkQyZbb6NO40U1BGswtYkz3Y1WTLfw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=yoctocell.xyz header.s=mail header.b=gSpg9Zfg; dmarc=fail reason="SPF not aligned (relaxed)" header.from=yoctocell.xyz (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: -1.90 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=yoctocell.xyz header.s=mail header.b=gSpg9Zfg; dmarc=fail reason="SPF not aligned (relaxed)" header.from=yoctocell.xyz (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 5BC3129CC X-Spam-Score: -1.90 X-Migadu-Scanner: scn0.migadu.com X-TUID: PX2V1jXtm2u8 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Wed, Sep 22 2021, Jesse wrote: > Attached is yet another updated patch fixing a mistake in the added=20 > documentation. > > On 9/21/21 7:26 PM, Jesse wrote: >> Attached is an updated patch. > From d264da0811480a8d2acd5e73f58d320e15bfa9f3 Mon Sep 17 00:00:00 2001 > From: Jesse > Date: Mon, 20 Sep 2021 16:01:22 -0600 > Subject: [PATCH] gnu: web: Add jupyter-service > > * gnu/services/web.scm (gitile-service-type): New variable. > * doc/guix.texi (Version Control Services): Document it. > --- > doc/guix.texi | 52 +++++++++++++++++++ > gnu/services/web.scm | 120 +++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 172 insertions(+) It would also be good to have a test for the service. You can look at at (gnu tests ...) modules for some inspiration. > diff --git a/doc/guix.texi b/doc/guix.texi > index 6436e83a7c..0f8aaca413 100644 > --- a/doc/guix.texi > +++ b/doc/guix.texi > @@ -26197,6 +26197,58 @@ The file which should store the logging output o= f Agate. > @end table > @end deftp >=20=20 > +@cindex jupyter > +The Jupyter Notebook is a web application that allows you to create > +and share documents that contain live code, equations, visualizations > +and explanatory text. Uses include: data cleaning and transformation, > +numerical simulation, statistical modeling, machine learning and much > +more. > + > +@deffn (Scheme Variable) jupyter-service-type > +This is the type of the agate service, whose value should be an Agate service? The Jupyter service doesn=E2=80=99t seem to use Agate anywh= ere. > +@code{jupyter-service-type} object, as in this example: > + > +@lisp > +(service jupyter-service-type > + (jupyter-configuration > + (log-file "/var/log/jupyter.log") > + (server-config > + (kernels > + (list jupyter-guile-kernel))))) > + > +@end lisp The indentation looks incorrect. Also the trailing newline isn=E2=80=99t necessary. Something like =2D-8<---------------cut here---------------start------------->8--- (service jupyter-service-type (jupyter-configuration (log-file "/var/log/jupyter.log") (server-config (kernels (list jupyter-guile-kernel))))) =2D-8<---------------cut here---------------end--------------->8--- > +The example above tells the the Jupyter service to provide the > +@code{jupyter-guile-kernel} as an optional kernel. > +@end deffn > + > +@deftp {Data Type} jupyter-configuration > +Data type representing the configuration of Agate. s/Agate/Jupyter/ ? > +@table @asis > +@item @code{jupyter-package} (default: @code{jupyter}) > +The jupyter package to use. =E2=80=9Cjupyter=E2=80=9D should be capitalized. > + > + > +@item @code{kernels} (default: @code{'()} > +A list of kernel packages to use with the jupyter service. > + > +@item @code{root-dir} (default: @file{"/var/lib/jupyter/notebooks"}) > +The directory where the jupyter server looks for jupyter notebooks. > + > +@item @code{log-file} (default: @code{#f}) > +The location of the log file. If #f is given, no log is produced. ^ There should be two spaces after a period. > +@item @code{shell} (default: @code{(file-append bash "/bin/bash")} > +The location of the shell used when starting a web terminal (if it is co= nfigured to allow terminals). Line should be kept at <=3D80 chars. > +@item @code{server-config} (default: @code{#f}) > +A file contianing the jupyter server's config file. If @code{#f} is give= n, an empty file is produced, and the default settings are used. Likewise (<=3D80 line length). What exactly does =E2=80=9CA file=E2=80=9D = mean, do you mean =E2=80=9Cfile-like objects=E2=80=9D (see =E2=80=9C8.10 G-Expressions= =E2=80=9D in the manual)? The example config doesn=E2=80=99t seem to provide anything =E2=80=9CA file= =E2=80=9D either =2D-8<---------------cut here---------------start------------->8--- (service jupyter-service-type (jupyter-configuration (log-file "/var/log/jupyter.log") (server-config (kernels (list jupyter-guile-kernel))))) =2D-8<---------------cut here---------------end--------------->8--- =E2=80=98kernels=E2=80=99 doesn=E2=80=99t seem to be a procedure or macro, = but rather a field in the =E2=80=98jupyter-configuration=E2=80=99 record, so the example needs to= be fixed as well. > + > +@end table > +@end deftp > + > + > @node Certificate Services > @subsection Certificate Services >=20=20 > diff --git a/gnu/services/web.scm b/gnu/services/web.scm > index 6a093cf4e4..2d2946f7a7 100644 > --- a/gnu/services/web.scm > +++ b/gnu/services/web.scm > @@ -15,6 +15,7 @@ > ;;; Copyright =C2=A9 2020 Arun Isaac > ;;; Copyright =C2=A9 2020 Oleg Pykhalov > ;;; Copyright =C2=A9 2020, 2021 Alexandru-Sergiu Marton > +;;; Copyright =C2=A9 2020, 2021 Jesse Gibbons > ;;; > ;;; This file is part of GNU Guix. > ;;; > @@ -41,11 +42,13 @@ > #:use-module (gnu system shadow) > #:use-module (gnu packages admin) > #:use-module (gnu packages base) > + #:use-module (gnu packages bash) > #:use-module (gnu packages databases) > #:use-module (gnu packages web) > #:use-module (gnu packages patchutils) > #:use-module (gnu packages php) > #:use-module (gnu packages python) > + #:use-module (gnu packages python-xyz) > #:use-module (gnu packages gnupg) > #:use-module (gnu packages guile) > #:use-module (gnu packages logging) > @@ -91,6 +94,8 @@ >=20=20 > httpd-service-type >=20=20 > + jupyter-configuration > + > nginx-configuration > nginx-configuration? > nginx-configuration-nginx > @@ -1994,3 +1999,118 @@ root=3D/srv/gemini > (service-extension shepherd-root-service-type > agate-shepherd-service))) > (default-value (agate-configuration)))) > + > + > +;;; Jupyter configuration > +;;; Jupyter server configuration > +;;; -> includes a list of kernels to allow in the jupyter server > +;;; -> includes a list of configuration options specifically for the jup= yter server > +;;; --> These options can be concatenated to a python file > +;;; --> Nested alist > +;;; ---> '((NotebookApp (allow-remote-access #t))) -> "c.NotebookApp.all= ow_remote_access =3D True" > +;;; -> Include some settings specifying how to run the daemon > +;;; --> location of log file > +;;; --> If a console should be allowed, the location of the shell to be = used. > +;;; --> The package containing the jupyter server itself, default jupyter What do these arrows mean? Also, keep lines <=3D80 chars wide. > +(define-public default-jupyter-config "#") Does the default config have to contain a =E2=80=9C#=E2=80=9D, can=E2=80=99= t it just be an empty string? > + > +(define-record-type* jupyter-configuration > + this-jupyter-configuration jupyter-configuration? > + (jupyter-package jupyter-configuration-jupyter-pack= age > + (default jupyter)) > + (kernels jupyter-configuration-kernels > + (default '())) > + (root-dir jupyter-configuration-root-dir > + (default "/var/lib/jupyter/notebooks")) > + (log-file jupyter-configuration-log-file > + (default #f)) I think it would be good to have a log file by default. > + (shell jupyter-configuration-shell > + (default (file-append bash "/bin/bash"))) > + (server-config jupyter-configuration-server-config > + (default #f))); TODO: Make configura= tion DSL. > + > + > +(define (search-path-string search-path-pair) > + (string-append (search-path-specification-variable (car search-path-pa= ir)) > + "=3D" > + (cdr search-path-pair))) Prefer =E2=80=98match=E2=80=99 over =E2=80=98car=E2=80=99 and =E2=80=98cdr= =E2=80=99. > + > +;;;TODO: Add actions to list jupyter servers, change passwords, etc. > +(define (jupyter-shepherd-service config) > + (list (shepherd-service > + (provision '(jupyter)) ;TODO: Add magic to allow multiple Jupy= ter servers > + (requirement '(loopback)) > + (start #~(make-forkexec-constructor > + (list "/run/current-system/profile/bin/jupyter" > + "notebook" > + (string-append "--config=3D" > + #$(or (jupyter-configuration-server-config config) > + (plain-file "jupyter_notebook_config.py" > + default-jupyter-config)) > + #$(jupyter-configuration-root-dir config))) Prefer =E2=80=98match=E2=80=99 instead of using accessor procedures. Somet= hing like =2D-8<---------------cut here---------------start------------->8--- (match config (($ _ _ root-dir log-file _ server-config) (list (shepherd-service ...)))) =2D-8<---------------cut here---------------end--------------->8--- > + #:user "jupyter" > + #:group "jupyter" > + #:environment-variables ; TODO use search-paths mag= ic instead of hard-coding these things. > + (list "GI_TYPELIB_PATH=3D/run/current-system/profil= e/lib/girepository-1.0" > + "GUILE_LOAD_COMPILED_PATH=3D/run/current-syst= em/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guil= e/site/3.0" > + "GUILE_LOAD_PATH=3D/run/current-system/profil= e/share/guile/site/3.0" > + "HOME=3D/var/lib/jupyter" > + "JUPYTER_PATH=3D/run/current-system/profile/s= hare/jupyter" > + "PATH=3D/run/current-system/profile/bin:/run/= current-system/profile/sbin" > + "PYTHONPATH=3D/run/current-system/profile/lib= /python3.8/site-packages" > + "R_LIBS_SITE=3D/run/current-system/profile/si= te-library/" > + "TEXMF=3D/run/current-system/profile/share/te= xmf-dist" > + "TEXMFCNF=3D/run/current-system/profile//shar= e/texmf-dist/web2c" > + "XDG_DATA_DIRS=3D/run/current-system/profile/= share") > + > + #:directory #$(jupyter-configuration-root-dir confi= g) > + #:log-file #$(jupyter-configuration-log-file config= ))) > + (stop #~(make-kill-destructor)) > + (documentation "Runs a Jupyter Notebook server. A Jupyter Note= book is a web application that allows you to create and share documents tha= t contain live code, equations, visualizations, and explanatory text.")))) Watch the line length! :-) > + > +(define (jupyter-account config) > + (list > + (user-account > + (name "jupyter") > + (group "jupyter") > + (comment "Jupyter Notebook Server") > + (home-directory "/var/lib/jupyter") > + (shell (jupyter-configuration-shell config)) > + (system? #t)) > + (user-group > + (name "jupyter") > + (system? #t)))) > + > +(define (jupyter-profile config) > + (cons* > + (jupyter-configuration-jupyter-package config) > + (jupyter-configuration-kernels config))) Prefer =E2=80=98match=E2=80=99 instead of using accessor procedures. > + > +(define (jupyter-activation config) > + #~(begin > + (let ((root-dir #$(jupyter-configuration-root-dir config)) > + (pw (getpwnam "jupyter"))) > + (unless (file-exists? root-dir) > + (mkdir root-dir) > + (chown root-dir (passwd:uid pw) > + (passwd:gid pw)) > + (chmod root-dir #o700))))) > + > +;;; Jupyter service type > +;;; -> Information vital to settinng up the server, like the port and ac= cepted parameters > +;;; -> list of kernels considered permissible. > +;;; -> a shepherd service extension that runs the jupyter server > +;;; --> shepherd service to list the running servers, set the password, = etc. > +;;; --> Make a log file only readable by root? > +;;; -> an accounts service extension describing the user that runs the a= ccounts > +;;; --> group "jupyter" > +(define-public jupyter-service-type > + (service-type > + (name "jupyter") > + (extensions (list > + (service-extension shepherd-root-service-type jupyter-= shepherd-service) > + (service-extension account-service-type jupyter-accoun= t) > + (service-extension activation-service-type jupyter-act= ivation) > + (service-extension profile-service-type jupyter-profil= e))) > + (description "Runs a Jupyter Notebook server. A Jupyter Notebook is = a web application that allows you to create and share documents that contai= n live code, equations, visualizations, and explanatory text.") Keep lines <=3D80 chars, and use double-spacing. > + (default-value (jupyter-configuration)))) > --=20 > 2.33.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmFTakMVHHB1YmxpY0B5 b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x5ZeEQALpe1vcjYaUg+tpPsIRIcrgG6AGX uy44G76f6iodJ61ORImldfIQ7fJWpBRWtFecPOhfuhgFYdoNaGf7poNSF4mRYtgp 0ZPVcQbgOqcxxFy7RASsREtDiSqDdWQwoptEkjryAP3L5hxc7G9rLCVIFKV8WqLj NHQwSV4JLhik003j2awu1SvGbYHQlANwQt9uC3Yh18vmrBEzucqrB1d7iIyevR8e PGAz7Z6TYCRf5x3JDYK4CGq4bsgpPHjjyiioAkQ5Pgb8OMIeHkb0zWIt10o1Cv1p aVinhx3Naf/w1emaad3TL7rG42aN4aPDlLCykLDUwJqlgLJ0hz2BAE4CfhCA2Sl/ uudwNmN14O8slEK8pU/HgzgAXxuYG8iLgx7/zY6RkKxqLmfKLl5N8zx9uE1iImLE 8maGetj9/x8yMfbnlZHMBYT0RQsUEbo/srMGfFNcBQlzoF1lKABIVqkRnBpH+NMt ogdolWuSqdpLIdKnNGdxoCbk/bXH1YLrX+aHJZz3qLrPbodtm97WR2KnV5/bf04Z 1ns96yMUPX2pcu//3fkU2yrYjBPNL/DOo96jbD88BlmRxIq+rWS1k0PuKD0otjYE lKjKrSSKWwFoAwyKJkdde7vttHVFFtpqSc3fbeCS1hPoZR5w/RqOSKwaikrabCw2 2LSBikYlfHUKb6Cv =+0pq -----END PGP SIGNATURE----- --=-=-=--