From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id QOUlHqjkS2HEFAEAgWs5BA (envelope-from ) for ; Thu, 23 Sep 2021 04:21:28 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id yMzxGajkS2FVZAAA1q6Kng (envelope-from ) for ; Thu, 23 Sep 2021 02:21:28 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id BAB7918CF1 for ; Thu, 23 Sep 2021 04:21:27 +0200 (CEST) Received: from localhost ([::1]:59976 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mTEMY-00014C-Dh for larch@yhetil.org; Wed, 22 Sep 2021 22:21:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37682) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mTEMA-000143-RT for guix-patches@gnu.org; Wed, 22 Sep 2021 22:21:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:40734) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mTEMA-0008QT-Bl for guix-patches@gnu.org; Wed, 22 Sep 2021 22:21:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mTEMA-0000Sb-43 for guix-patches@gnu.org; Wed, 22 Sep 2021 22:21:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50708] [PATCH] gnu: web: Add jupyter-service Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 23 Sep 2021 02:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50708 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Jesse Cc: 50708@debbugs.gnu.org Received: via spool by 50708-submit@debbugs.gnu.org id=B50708.16323636081654 (code B ref 50708); Thu, 23 Sep 2021 02:21:02 +0000 Received: (at 50708) by debbugs.gnu.org; 23 Sep 2021 02:20:08 +0000 Received: from localhost ([127.0.0.1]:52280 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mTELH-0000Qb-8J for submit@debbugs.gnu.org; Wed, 22 Sep 2021 22:20:07 -0400 Received: from mail-qt1-f176.google.com ([209.85.160.176]:46050) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mTELD-0000Py-Ug for 50708@debbugs.gnu.org; Wed, 22 Sep 2021 22:20:06 -0400 Received: by mail-qt1-f176.google.com with SMTP id r1so4773705qta.12 for <50708@debbugs.gnu.org>; Wed, 22 Sep 2021 19:20:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=q1qyW2iFeX4Y3DN1kQwWyFgQH13xNqJtZCr3CEDdiqI=; b=jo13dQwht0nwS6aZzTLe1MB+c+H1RHzwoL7z2deUxrgGf2vRIR4NvlHai6EL1lnOEd lNwJRXlgDKOJ+Xkv0KuhrEyUUQUbxFB5HiyYG4Sb+N0P8CCh4Jl++rcuRgk55WAItnP3 K/WmB1Uy9tuQeYKKx8ZV4w4ivNwD/9TpXNX1zyFK4ivjyRM7kkCq6t5DNhSSvOfSwAtP sFqmkaoNtfom0WraDf9alfddGXOd+qJ8qwogO+61iV73LuTdZcQeIgc9CisODsqVEfBU gee+JC4SbwCstnPGu8Y7fOthqblSSfGVWD4NKN6/evo1bRBCFdAaV9FVGEDXlwE+Y/rq Si3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=q1qyW2iFeX4Y3DN1kQwWyFgQH13xNqJtZCr3CEDdiqI=; b=FjQnwCwr5InrnqQpFzgcnP7203pAjm6doKC/50Yflx+qKjaYnCGudFi7MWouBDOVqn eWSM9qqONv82sRIvjkYccaoXQgdRJXxyjXHuml9QdZRT/Uff9O988FDGAQzbC0wS5e1W EbTYf1pOWeJ7AcZ+fDnpJ5Rznn8svaPuPGPh9QgDSQ9z4wRl8iWj4BaWXYICK8z5OS/j 0jrSvnOZraFASMMqU4ux0o4EWOc/BboBN8yOu8cgPALRh/m1VfBvjuVNMvoQ/Wy7qs1t AFArmd9roW8p5t1GzYSaqwuUYhxyVxlskbkZivjIRxm7JB2JV3xf88f5b/tjtLy240Mk +ZKA== X-Gm-Message-State: AOAM532wJLNUwI9MFJwIGYBmC3xou6UhEHxQ+KJ0/R1Q/iKJLaBRJ6KP l6/lh2G5ShIcR1fisFfQsszIPgqrFSg= X-Google-Smtp-Source: ABdhPJxfiUe5obR6okYcEZqcQXFD4fPp3NhEIc+zEDdqCgkS5yiyN9+5Ecgp/BWyR23nCp1yvKIqXA== X-Received: by 2002:ac8:72d3:: with SMTP id o19mr2694046qtp.19.1632363598267; Wed, 22 Sep 2021 19:19:58 -0700 (PDT) Received: from hurd (dsl-157-23.b2b2c.ca. [66.158.157.23]) by smtp.gmail.com with ESMTPSA id i67sm3216474qkd.90.2021.09.22.19.19.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Sep 2021 19:19:57 -0700 (PDT) From: Maxim Cournoyer References: <565a1b21-aefa-4129-a024-52517fdff9bd@gmail.com> <74f78a5c-3f79-7485-0cf2-ae1fbb1969dd@gmail.com> Date: Wed, 22 Sep 2021 22:19:56 -0400 In-Reply-To: <74f78a5c-3f79-7485-0cf2-ae1fbb1969dd@gmail.com> (Jesse's message of "Tue, 21 Sep 2021 19:26:11 -0600") Message-ID: <87pmt0kp83.fsf_-_@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1632363687; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=q1qyW2iFeX4Y3DN1kQwWyFgQH13xNqJtZCr3CEDdiqI=; b=QjV52LpHF37J4MsUnRjdUlpO0W/RRFRhuC5FEcH95RBkAZsGtmgivhKUD1qxrltqV8r49i c3wXjvS1DvoqOX5RVLPAIN+hj6LsJg5QfwAvNuf9rBi3FbTdNn3phMLF07+HbtdOK8S7og jhIOSQgRrEl1KHvd1TV1nM4ll6fQi1a0qvxA9+N+I80rVafCWMgBdix7UNb1LuDQBxsvjU ug4xKz3tSIqmLG1lRRmxk2L28xGUfWv5U4dL8OwW6r4kL4n4Ye7crxL7sVkXuNdPCG/Pw4 upi5eqlOUz70IBOS4pcZPT95KwgjoYcC2PQ/2PCp7zc/Du7dP2cQxyEb010Cmg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1632363687; a=rsa-sha256; cv=none; b=UBRVXDWURtX67szU5TiPtZ9NVNTTiMsKKNxVQ+nekgIBgkS5H3A8tOF/mtfsOobnuW7Oru wk6XnFYAThlmy/BtUNkB3k8ZRg9yYodx4kU4diU7wbdcw9Ko4RN+YHlOsZDHpFaRTC4RWo 2RndnspRlBRbYCVXyJ9S2vZS7dNgiZhcVRUOilOOKHArRdAUQxvoG6P3SBNlbpznmJLcTy 3Dyn7kIn2LoZCRRBNcmFJctql2YFhLxcyEJUDrosm8qb1NgWCb/43slf6oA4meqwRn2zT+ 2mBK/S72RaJwT2qFmJiOKPFK2ao0tnqYlQ/8UdEqhQSDTHUlljgCR9dpSdlryQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=jo13dQwh; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: -1.29 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=jo13dQwh; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: BAB7918CF1 X-Spam-Score: -1.29 X-Migadu-Scanner: scn0.migadu.com X-TUID: fI6kfygQ0N2D Hello Jesse, Jesse writes: > Attached is an updated patch. > > From d264da0811480a8d2acd5e73f58d320e15bfa9f3 Mon Sep 17 00:00:00 2001 > From: Jesse > Date: Mon, 20 Sep 2021 16:01:22 -0600 > Subject: [PATCH] gnu: web: Add jupyter-service > > * gnu/services/web.scm (gitile-service-type): New variable. > * doc/guix.texi (Version Control Services): Document it. > --- > doc/guix.texi | 52 +++++++++++++++++++ > gnu/services/web.scm | 120 +++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 172 insertions(+) > > diff --git a/doc/guix.texi b/doc/guix.texi > index 6436e83a7c..0f8aaca413 100644 > --- a/doc/guix.texi > +++ b/doc/guix.texi > @@ -26197,6 +26197,58 @@ The file which should store the logging output o= f Agate. > @end table > @end deftp >=20=20 > +@cindex jupyter > +The Jupyter Notebook is a web application that allows you to create > +and share documents that contain live code, equations, visualizations > +and explanatory text. Uses include: data cleaning and transformation, > +numerical simulation, statistical modeling, machine learning and much > +more. > + > +@deffn (Scheme Variable) jupyter-service-type > +This is the type of the agate service, whose value should be an ^ Agate I'm a bit confused by this service referring to Agate; seems it'd be clearer for the neophyte to simply refer to Jupyter in its place, even if it's technically an Agate server powering it? > +@code{jupyter-service-type} object, as in this example: > + > +@lisp > +(service jupyter-service-type > + (jupyter-configuration > + (log-file "/var/log/jupyter.log") > + (server-config > + (kernels > + (list jupyter-guile-kernel))))) > + > +@end lisp I'd put the (list jupyter-guile-kernel) on the same line as (kernels ...) > +The example above tells the the Jupyter service to provide the ^ typo =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20 > +@code{jupyter-guile-kernel} as an optional kernel. > +@end deffn > + > +@deftp {Data Type} jupyter-configuration > +Data type representing the configuration of Agate. > + > +@table @asis > +@item @code{jupyter-package} (default: @code{jupyter}) > +The jupyter package to use. ^ Jupyter I'd capitalize Jupyter in other places as well where it's not decorated as a command or code, since it's a proper name. > + > +@item @code{kernels} (default: @code{'()} > +A list of kernel packages to use with the jupyter service. ^ here > + > +@item @code{root-dir} (default: @file{"/var/lib/jupyter/notebooks"}) > +The directory where the jupyter server looks for jupyter notebooks. ^ and here > +@item @code{log-file} (default: @code{#f}) > +The location of the log file. If #f is given, no log is produced. > > +@item @code{shell} (default: @code{(file-append bash "/bin/bash")} > +The location of the shell used when starting a web terminal (if it is co= nfigured to allow terminals). > + > +@item @code{server-config} (default: @code{#f}) > +A file contianing the jupyter server's config file. If @code{#f} is give= n, an empty file is produced, ^ containing > and the default settings are used. > + > +@end table > +@end deftp > + > + > @node Certificate Services > @subsection Certificate Services >=20=20 > diff --git a/gnu/services/web.scm b/gnu/services/web.scm > index 6a093cf4e4..2d2946f7a7 100644 > --- a/gnu/services/web.scm > +++ b/gnu/services/web.scm > @@ -15,6 +15,7 @@ > ;;; Copyright =C2=A9 2020 Arun Isaac > ;;; Copyright =C2=A9 2020 Oleg Pykhalov > ;;; Copyright =C2=A9 2020, 2021 Alexandru-Sergiu Marton > +;;; Copyright =C2=A9 2020, 2021 Jesse Gibbons > ;;; > ;;; This file is part of GNU Guix. > ;;; > @@ -41,11 +42,13 @@ > #:use-module (gnu system shadow) > #:use-module (gnu packages admin) > #:use-module (gnu packages base) > + #:use-module (gnu packages bash) > #:use-module (gnu packages databases) > #:use-module (gnu packages web) > #:use-module (gnu packages patchutils) > #:use-module (gnu packages php) > #:use-module (gnu packages python) > + #:use-module (gnu packages python-xyz) > #:use-module (gnu packages gnupg) > #:use-module (gnu packages guile) > #:use-module (gnu packages logging) > @@ -91,6 +94,8 @@ >=20=20 > httpd-service-type >=20=20 > + jupyter-configuration > + > nginx-configuration > nginx-configuration? > nginx-configuration-nginx > @@ -1994,3 +1999,118 @@ root=3D/srv/gemini > (service-extension shepherd-root-service-type > agate-shepherd-service))) > (default-value (agate-configuration)))) > + > + > +;;; Jupyter configuration > +;;; Jupyter server configuration > +;;; -> includes a list of kernels to allow in the jupyter server > +;;; -> includes a list of configuration options specifically for the jup= yter server > +;;; --> These options can be concatenated to a python file > +;;; --> Nested alist > +;;; ---> '((NotebookApp (allow-remote-access #t))) -> "c.NotebookApp.all= ow_remote_access =3D True" > +;;; -> Include some settings specifying how to run the daemon > +;;; --> location of log file > +;;; --> If a console should be allowed, the location of the shell to be = used. > +;;; --> The package containing the jupyter server itself, default > jupyter What is the meaning of the arrows in the above comment? It seems to me that they were meant to be used during development, not in the final result? Otherwise, the types of the fields could be self-documented and controlled by the use of the 'define-configuration' procedure and friends from the (gnu services configuration) module. > +(define-public default-jupyter-config "#") > + > +(define-record-type* jupyter-configuration > + this-jupyter-configuration jupyter-configuration? > + (jupyter-package jupyter-configuration-jupyter-pack= age > + (default jupyter)) > + (kernels jupyter-configuration-kernels > + (default '())) > + (root-dir jupyter-configuration-root-dir > + (default "/var/lib/jupyter/notebooks")) > + (log-file jupyter-configuration-log-file > + (default #f)) > + (shell jupyter-configuration-shell > + (default (file-append bash "/bin/bash"))) > + (server-config jupyter-configuration-server-config > + (default #f))); TODO: Make configura= tion DSL. > + > + > +(define (search-path-string search-path-pair) > + (string-append (search-path-specification-variable (car search-path-pa= ir)) > + "=3D" > + (cdr search-path-pair))) > + > +;;;TODO: Add actions to list jupyter servers, change passwords, etc. > +(define (jupyter-shepherd-service config) > + (list (shepherd-service > + (provision '(jupyter)) ;TODO: Add magic to allow multiple Jupy= ter servers > + (requirement '(loopback)) > + (start #~(make-forkexec-constructor > + (list "/run/current-system/profile/bin/jupyter" > + "notebook" > + (string-append "--config=3D" > + #$(or (jupyter-configuration-server-config config) > + (plain-file "jupyter_notebook_config.py" > + default-jupyter-config)) > + #$(jupyter-configuration-root-dir config))) > + #:user "jupyter" > + #:group "jupyter" > + #:environment-variables ; TODO use search-paths mag= ic instead of hard-coding these things. > + (list "GI_TYPELIB_PATH=3D/run/current-system/profil= e/lib/girepository-1.0" > + "GUILE_LOAD_COMPILED_PATH=3D/run/current-syst= em/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guil= e/site/3.0" > + "GUILE_LOAD_PATH=3D/run/current-system/profil= e/share/guile/site/3.0" > + "HOME=3D/var/lib/jupyter" > + "JUPYTER_PATH=3D/run/current-system/profile/s= hare/jupyter" > + "PATH=3D/run/current-system/profile/bin:/run/= current-system/profile/sbin" > + "PYTHONPATH=3D/run/current-system/profile/lib= /python3.8/site-packages" > + "R_LIBS_SITE=3D/run/current-system/profile/si= te-library/" > + "TEXMF=3D/run/current-system/profile/share/te= xmf-dist" > + "TEXMFCNF=3D/run/current-system/profile//shar= e/texmf-dist/web2c" ^ extra s= lash =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 > + "XDG_DATA_DIRS=3D/run/current-system/profile/= share") > + Hmm. Would it work if search paths were set on the 'jupyter' package instead? The above is too fragile, with hard coded versions baked in. > + #:directory #$(jupyter-configuration-root-dir confi= g) > + #:log-file #$(jupyter-configuration-log-file config= ))) > + (stop #~(make-kill-destructor)) > + (documentation "Runs a Jupyter Notebook server. A Jupyter Note= book is ^ Use two spaces after pe= riod. >a web application that allows you to create and share documents that conta= in live code, equations, visualizations, and explanatory text.")))) > + > +(define (jupyter-account config) > + (list > + (user-account > + (name "jupyter") > + (group "jupyter") > + (comment "Jupyter Notebook Server") > + (home-directory "/var/lib/jupyter") > + (shell (jupyter-configuration-shell config)) > + (system? #t)) > + (user-group > + (name "jupyter") > + (system? #t)))) > + > +(define (jupyter-profile config) > + (cons* > + (jupyter-configuration-jupyter-package config) > + (jupyter-configuration-kernels config))) > + > +(define (jupyter-activation config) > + #~(begin > + (let ((root-dir #$(jupyter-configuration-root-dir config)) > + (pw (getpwnam "jupyter"))) > + (unless (file-exists? root-dir) > + (mkdir root-dir) > + (chown root-dir (passwd:uid pw) > + (passwd:gid pw)) > + (chmod root-dir #o700))))) > + > +;;; Jupyter service type > +;;; -> Information vital to settinng up the server, like the port and ac= cepted parameters > +;;; -> list of kernels considered permissible. > +;;; -> a shepherd service extension that runs the jupyter server > +;;; --> shepherd service to list the running servers, set the password, = etc. > +;;; --> Make a log file only readable by root? > +;;; -> an accounts service extension describing the user that runs the a= ccounts > +;;; --> group "jupyter" Same comment as earlier. > +(define-public jupyter-service-type > + (service-type > + (name "jupyter") > + (extensions (list > + (service-extension shepherd-root-service-type jupyter-= shepherd-service) > + (service-extension account-service-type jupyter-accoun= t) > + (service-extension activation-service-type jupyter-act= ivation) > + (service-extension profile-service-type jupyter-profil= e))) > + (description "Runs a Jupyter Notebook server. A Jupyter Notebook is = a web application that allows you to create and share documents that contai= n live code, equations, visualizations, and explanatory text.") > + (default-value (jupyter-configuration)))) I haven't tried it, but it looks promising. If we can find a better solution than the hard coded version values in the environment variables list, I think it'd be in good shape for inclusion, although a system test would be a nice to have. Thank you! Maxim