* [bug#39765] Add package JupyterLab
@ 2020-02-24 10:18 Lars-Dominik Braun
2020-03-26 22:55 ` Ludovic Courtès
0 siblings, 1 reply; 5+ messages in thread
From: Lars-Dominik Braun @ 2020-02-24 10:18 UTC (permalink / raw)
To: 39765
[-- Attachment #1: Type: text/plain, Size: 651 bytes --]
Hi,
this patch series adds Jupyter’s JupyterLab, which is the new frontend for
Jupyter Notebooks. The software works fine, but there are a few caveats
1) it comes with bundled pre-compiled JavaScript, which cannot be removed until
we have proper support for importing from NPM
2) it contains an extension manager, that downloads arbitrary packages from NPM
(`jupyter lab build`). This works, but is less than optimal imo. We should
figure out how to package extensions in guix.
3) also it is required to install the package `jupyter`, otherwise installed
kernels cannot be found and the `jupyter` command does not work.
Cheers,
Lars
[-- Attachment #2: 0001-gnu-Add-package-python-pytest-check-links.patch --]
[-- Type: text/x-diff, Size: 1688 bytes --]
From 4a5862e2add1d537770a5ea466dbb8a4851afad9 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Fri, 7 Feb 2020 08:38:32 +0100
Subject: [PATCH 1/5] gnu: Add package python-pytest-check-links
* gnu/packages/python-xyz.scm (python-pytest-check-links): New variable.
---
gnu/packages/python-xyz.scm | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 84b70954bd..902ca5030b 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -17570,3 +17570,31 @@ sequences.")
(define-public python2-fuzzywuzzy
(package-with-python2 python-fuzzywuzzy))
+
+(define-public python-pytest-check-links
+ (package
+ (name "python-pytest-check-links")
+ (version "0.3.0")
+ (source
+ (origin
+ (method url-fetch)
+ ;; URI uses underscores
+ (uri (pypi-uri "pytest_check_links" version))
+ (sha256
+ (base32
+ "12x3wmrdzm6wgk0vz02hb769h68nr49q47w5q1pj95pc89hsa34v"))))
+ (build-system python-build-system)
+ (propagated-inputs
+ `(("python-docutils" ,python-docutils)
+ ("python-html5lib" ,python-html5lib)
+ ("python-nbconvert" ,python-nbconvert)
+ ("python-nbformat" ,python-nbformat)
+ ("python-pytest" ,python-pytest)
+ ("python-six" ,python-six)))
+ (native-inputs
+ `(("python-pbr-minimal" ,python-pbr-minimal)))
+ (home-page
+ "https://github.com/minrk/pytest-check-links")
+ (synopsis "Check links in files")
+ (description "Plugin for pytest that checks URLs for HTML-containing files")
+ (license license:bsd-3)))
--
2.20.1
[-- Attachment #3: 0002-gnu-Add-package-python-json5.patch --]
[-- Type: text/x-diff, Size: 1651 bytes --]
From 690d45cba7d1a21fa6ae97fbe4fec6e1abae3635 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Fri, 7 Feb 2020 08:39:55 +0100
Subject: [PATCH 2/5] gnu: Add package python-json5
* gnu/packages/python-xyz.scm (python-json5): New variable.
---
gnu/packages/python-xyz.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 902ca5030b..ad1cd5fbd4 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -17598,3 +17598,27 @@ sequences.")
(synopsis "Check links in files")
(description "Plugin for pytest that checks URLs for HTML-containing files")
(license license:bsd-3)))
+
+(define-public python-json5
+ (package
+ (name "python-json5")
+ (version "0.8.5")
+ (source
+ (origin
+ ;; sample.json5 is missing from PyPi source tarball
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/dpranke/pyjson5.git")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "0nyngj18jlkgvm1177lc3cj47wm4yh3dqigygvcvw7xkyryafsqn"))))
+ (build-system python-build-system)
+ (home-page "https://github.com/dpranke/pyjson5")
+ (synopsis
+ "Python implementation of the JSON5 data format")
+ (description
+ "JSON5 extends the JSON data interchange format to make it slightly more
+usable as a configuration language. This Python package implements parsing and
+dumping of JSON5 data structures.")
+ (license license:asl2.0)))
--
2.20.1
[-- Attachment #4: 0003-gnu-Add-package-python-jupyterlab-server.patch --]
[-- Type: text/x-diff, Size: 1927 bytes --]
From a4535f6002a618444171d5a92146fb7a7e7e8243 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Fri, 7 Feb 2020 08:40:41 +0100
Subject: [PATCH 3/5] gnu: Add package python-jupyterlab-server
* gnu/packages/python-xyz.scm (python-jupyterlab-server): New variable.
---
gnu/packages/python-xyz.scm | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index ad1cd5fbd4..92ee53fe6f 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -17622,3 +17622,37 @@ sequences.")
usable as a configuration language. This Python package implements parsing and
dumping of JSON5 data structures.")
(license license:asl2.0)))
+
+(define-public python-jupyterlab-server
+ (package
+ (name "python-jupyterlab-server")
+ (version "1.0.6")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "jupyterlab_server" version))
+ (sha256
+ (base32
+ "1bax8iqwcc5p02h5ysdc48zvx7ll5jfzfsybhb3lfvyfpwkpb5yh"))))
+ (build-system python-build-system)
+ (propagated-inputs
+ `(("python-jinja2" ,python-jinja2)
+ ("python-json5" ,python-json5)
+ ("python-jsonschema" ,python-jsonschema)
+ ("python-notebook" ,python-notebook)))
+ (native-inputs
+ `(("python-pytest" ,python-pytest)
+ ("python-requests" ,python-requests)
+ ("python-ipykernel" ,python-ipykernel)))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ ;; python setup.py test does not invoke pytest?
+ (replace 'check
+ (lambda _
+ (invoke "pytest" "-vv"))))))
+ (home-page "https://jupyter.org")
+ (synopsis "JupyterLab Server")
+ (description "A set of server components for JupyterLab and JupyterLab like
+applications")
+ (license license:bsd-3)))
--
2.20.1
[-- Attachment #5: 0004-gnu-Add-package-python-jupyterlab.patch --]
[-- Type: text/x-diff, Size: 5193 bytes --]
From c50fc3ec734cb94e78168a4a29c9aff070f4ae9f Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Fri, 7 Feb 2020 08:41:24 +0100
Subject: [PATCH 4/5] gnu: Add package python-jupyterlab
* gnu/packages/python-xyz.scm (python-jupyterlab): New variable.
---
.../python-jupyterlab-copy-nometa.patch | 33 +++++++++++
gnu/packages/python-xyz.scm | 55 +++++++++++++++++++
2 files changed, 88 insertions(+)
create mode 100644 gnu/packages/patches/python-jupyterlab-copy-nometa.patch
diff --git a/gnu/packages/patches/python-jupyterlab-copy-nometa.patch b/gnu/packages/patches/python-jupyterlab-copy-nometa.patch
new file mode 100644
index 0000000000..5e770c0f47
--- /dev/null
+++ b/gnu/packages/patches/python-jupyterlab-copy-nometa.patch
@@ -0,0 +1,33 @@
+diff '--exclude=*.swp' -Naur jupyterlab-1.2.6.orig/jupyterlab/commands.py jupyterlab-1.2.6/jupyterlab/commands.py
+--- jupyterlab-1.2.6.orig/jupyterlab/commands.py 2020-01-24 17:25:16.238353500 +0100
++++ jupyterlab-1.2.6/jupyterlab/commands.py 2020-01-31 12:36:10.497375982 +0100
+@@ -1100,7 +1100,7 @@
+ 'webpack.prod.minimize.config.js',
+ '.yarnrc', 'yarn.js']:
+ target = pjoin(staging, fname)
+- shutil.copy(pjoin(HERE, 'staging', fname), target)
++ shutil.copyfile(pjoin(HERE, 'staging', fname), target)
+
+ # Ensure a clean templates directory
+ templates = pjoin(staging, 'templates')
+@@ -1108,7 +1108,10 @@
+ _rmtree(templates, self.logger)
+
+ try:
+- shutil.copytree(pjoin(HERE, 'staging', 'templates'), templates)
++ shutil.copytree(pjoin(HERE, 'staging', 'templates'), templates,
++ copy_function=shutil.copyfile)
++ # cannot replace or disable copytree’s call to .copystat
++ os.chmod (templates, 0o755)
+ except shutil.Error as error:
+ # `copytree` throws an error if copying to + from NFS even though
+ # the copy is successful (see https://bugs.python.org/issue24564
+@@ -1178,7 +1181,7 @@
+ with open(lock_path, 'w', encoding='utf-8') as f:
+ f.write(template)
+ elif not osp.exists(lock_path):
+- shutil.copy(lock_template, lock_path)
++ shutil.copyfile(lock_template, lock_path)
+
+ def _get_package_template(self, silent=False):
+ """Get the template the for staging package.json file.
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 92ee53fe6f..232841ccb1 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -126,6 +126,7 @@
#:use-module (gnu packages multiprecision)
#:use-module (gnu packages networking)
#:use-module (gnu packages ncurses)
+ #:use-module (gnu packages node)
#:use-module (gnu packages openstack)
#:use-module (gnu packages pcre)
#:use-module (gnu packages perl)
@@ -17656,3 +17657,57 @@ dumping of JSON5 data structures.")
(description "A set of server components for JupyterLab and JupyterLab like
applications")
(license license:bsd-3)))
+
+(define-public python-jupyterlab
+ (package
+ (name "python-jupyterlab")
+ (version "1.2.6")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "jupyterlab" version))
+ (sha256
+ (base32
+ "0mc3nrj7fc5q2ajr09m261j386jsp8qjljg8anghlh8czc9ln4s2"))
+ (patches (search-patches "python-jupyterlab-copy-nometa.patch"))))
+ (build-system python-build-system)
+ (propagated-inputs
+ `(("python-jinja2" ,python-jinja2)
+ ("python-jupyterlab-server"
+ ,python-jupyterlab-server)
+ ("python-notebook" ,python-notebook)
+ ("python-tornado" ,python-tornado)
+ ("node" ,node)))
+ (native-inputs
+ `(("python-pytest" ,python-pytest)
+ ("python-pytest-check-links"
+ ,python-pytest-check-links)
+ ("python-requests" ,python-requests)
+ ("python-ipykernel" ,python-ipykernel)))
+ (arguments
+ ;; testing requires npm, so disabled for now
+ '(#:tests? #f
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'patch-syspath
+ (lambda* (#:key outputs inputs configure-flags #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out")))
+ (substitute* "jupyterlab/commands.py"
+ ;; sys.prefix defaults to Python’s prefix in the store, not
+ ;; jupyterlab’s. Fix that.
+ (("sys\\.prefix")
+ (string-append "'" out "'"))))
+ #t))
+ ;; 'build does not respect configure-flags
+ (replace 'build
+ (lambda _
+ (invoke "python" "setup.py" "build" "--skip-npm"))))
+ #:configure-flags (list "--skip-npm")))
+ (home-page "https://jupyter.org")
+ (synopsis
+ "The JupyterLab notebook server extension")
+ (description
+ "An extensible environment for interactive and reproducible computing,
+based on the Jupyter Notebook and Architecture.")
+ (license license:bsd-3)))
+
--
2.20.1
[-- Attachment #6: 0005-gnu-python-notebook-Support-UNIX-domain-sockets.patch --]
[-- Type: text/x-diff, Size: 26768 bytes --]
From a47fd94aa6f3e62b77f3b7208c4e6757e3a9ee08 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Thu, 12 Dec 2019 08:53:39 +0100
Subject: [PATCH 5/5] gnu: python-notebook: Support UNIX domain sockets
* gnu/packages/python-xyz.scm (python-notebook): Add patch from upstream
https://github.com/jupyter/notebook/pull/4835
(python-requests-unixsocket) New variable
---
...pyter-unix-domain-sockets-4835-5.7.4.patch | 591 ++++++++++++++++++
gnu/packages/python-xyz.scm | 35 +-
2 files changed, 624 insertions(+), 2 deletions(-)
create mode 100644 gnu/packages/patches/jupyter-unix-domain-sockets-4835-5.7.4.patch
diff --git a/gnu/packages/patches/jupyter-unix-domain-sockets-4835-5.7.4.patch b/gnu/packages/patches/jupyter-unix-domain-sockets-4835-5.7.4.patch
new file mode 100644
index 0000000000..134d3ad2b8
--- /dev/null
+++ b/gnu/packages/patches/jupyter-unix-domain-sockets-4835-5.7.4.patch
@@ -0,0 +1,591 @@
+diff -Naur notebook-5.7.4/notebook/base/handlers.py notebook-5.7.4.patched/notebook/base/handlers.py
+--- notebook-5.7.4/notebook/base/handlers.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/notebook/base/handlers.py 2019-11-18 12:16:58.315065024 +0100
+@@ -40,7 +40,7 @@
+ import notebook
+ from notebook._tz import utcnow
+ from notebook.i18n import combine_translations
+-from notebook.utils import is_hidden, url_path_join, url_is_absolute, url_escape
++from notebook.utils import is_hidden, url_path_join, url_is_absolute, url_escape, urldecode_unix_socket_path
+ from notebook.services.security import csp_report_uri
+
+ #-----------------------------------------------------------------------------
+@@ -426,13 +426,18 @@
+ # ip_address only accepts unicode on Python 2
+ host = host.decode('utf8', 'replace')
+
+- try:
+- addr = ipaddress.ip_address(host)
+- except ValueError:
+- # Not an IP address: check against hostnames
+- allow = host in self.settings.get('local_hostnames', ['localhost'])
++ # UNIX socket handling
++ check_host = urldecode_unix_socket_path(host)
++ if check_host.startswith('/') and os.path.exists(check_host):
++ allow = True
+ else:
+- allow = addr.is_loopback
++ try:
++ addr = ipaddress.ip_address(host)
++ except ValueError:
++ # Not an IP address: check against hostnames
++ allow = host in self.settings.get('local_hostnames', ['localhost'])
++ else:
++ allow = addr.is_loopback
+
+ if not allow:
+ self.log.warning(
+diff -Naur notebook-5.7.4/notebook/__init__.py notebook-5.7.4.patched/notebook/__init__.py
+--- notebook-5.7.4/notebook/__init__.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/notebook/__init__.py 2019-11-18 12:16:58.315065024 +0100
+@@ -20,6 +20,8 @@
+ os.path.join(os.path.dirname(__file__), "templates"),
+ ]
+
++DEFAULT_NOTEBOOK_PORT = 8888
++
+ del os
+
+ from .nbextensions import install_nbextension
+diff -Naur notebook-5.7.4/notebook/notebookapp.py notebook-5.7.4.patched/notebook/notebookapp.py
+--- notebook-5.7.4/notebook/notebookapp.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/notebook/notebookapp.py 2019-11-18 12:21:34.975072928 +0100
+@@ -63,8 +63,11 @@
+ from tornado import web
+ from tornado.httputil import url_concat
+ from tornado.log import LogFormatter, app_log, access_log, gen_log
++if not sys.platform.startswith('win'):
++ from tornado.netutil import bind_unix_socket
+
+ from notebook import (
++ DEFAULT_NOTEBOOK_PORT,
+ DEFAULT_STATIC_FILES_PATH,
+ DEFAULT_TEMPLATE_PATH_LIST,
+ __version__,
+@@ -108,7 +111,16 @@
+ from notebook._sysinfo import get_sys_info
+
+ from ._tz import utcnow, utcfromtimestamp
+-from .utils import url_path_join, check_pid, url_escape, urljoin, pathname2url
++from .utils import (
++ check_pid,
++ pathname2url,
++ url_escape,
++ url_path_join,
++ urldecode_unix_socket_path,
++ urlencode_unix_socket,
++ urlencode_unix_socket_path,
++ urljoin,
++)
+
+ #-----------------------------------------------------------------------------
+ # Module globals
+@@ -212,7 +224,7 @@
+ warnings.warn(_("The `ignore_minified_js` flag is deprecated and will be removed in Notebook 6.0"), DeprecationWarning)
+
+ now = utcnow()
+-
++
+ root_dir = contents_manager.root_dir
+ home = os.path.expanduser('~')
+ if root_dir.startswith(home + os.path.sep):
+@@ -385,6 +397,7 @@
+ set_password(config_file=self.config_file)
+ self.log.info("Wrote hashed password to %s" % self.config_file)
+
++
+ def shutdown_server(server_info, timeout=5, log=None):
+ """Shutdown a notebook server in a separate process.
+
+@@ -397,14 +410,39 @@
+ Returns True if the server was stopped by any means, False if stopping it
+ failed (on Windows).
+ """
+- from tornado.httpclient import HTTPClient, HTTPRequest
++ from tornado import gen
++ from tornado.httpclient import AsyncHTTPClient, HTTPClient, HTTPRequest
++ from tornado.netutil import bind_unix_socket, Resolver
+ url = server_info['url']
+ pid = server_info['pid']
++ resolver = None
++
++ # UNIX Socket handling.
++ if url.startswith('http+unix://'):
++ # This library doesn't understand our URI form, but it's just HTTP.
++ url = url.replace('http+unix://', 'http://')
++
++ class UnixSocketResolver(Resolver):
++ def initialize(self, resolver):
++ self.resolver = resolver
++
++ def close(self):
++ self.resolver.close()
++
++ @gen.coroutine
++ def resolve(self, host, port, *args, **kwargs):
++ raise gen.Return([
++ (socket.AF_UNIX, urldecode_unix_socket_path(host))
++ ])
++
++ resolver = UnixSocketResolver(resolver=Resolver())
++
+ req = HTTPRequest(url + 'api/shutdown', method='POST', body=b'', headers={
+ 'Authorization': 'token ' + server_info['token']
+ })
+ if log: log.debug("POST request to %sapi/shutdown", url)
+- HTTPClient().fetch(req)
++ AsyncHTTPClient.configure(None, resolver=resolver)
++ HTTPClient(AsyncHTTPClient).fetch(req)
+
+ # Poll to see if it shut down.
+ for _ in range(timeout*10):
+@@ -435,13 +473,20 @@
+ version = __version__
+ description="Stop currently running notebook server for a given port"
+
+- port = Integer(8888, config=True,
+- help="Port of the server to be killed. Default 8888")
++ port = Integer(DEFAULT_NOTEBOOK_PORT, config=True,
++ help="Port of the server to be killed. Default %s" % DEFAULT_NOTEBOOK_PORT)
++
++ sock = Unicode(u'', config=True,
++ help="UNIX socket of the server to be killed.")
+
+ def parse_command_line(self, argv=None):
+ super(NbserverStopApp, self).parse_command_line(argv)
+ if self.extra_args:
+- self.port=int(self.extra_args[0])
++ try:
++ self.port = int(self.extra_args[0])
++ except ValueError:
++ # self.extra_args[0] was not an int, so it must be a string (unix socket).
++ self.sock = self.extra_args[0]
+
+ def shutdown_server(self, server):
+ return shutdown_server(server, log=self.log)
+@@ -451,16 +496,16 @@
+ if not servers:
+ self.exit("There are no running servers")
+ for server in servers:
+- if server['port'] == self.port:
+- print("Shutting down server on port", self.port, "...")
++ if server.get('sock') == self.sock or server['port'] == self.port:
++ print("Shutting down server on %s..." % self.sock or self.port)
+ if not self.shutdown_server(server):
+ sys.exit("Could not stop server")
+ return
+ else:
+ print("There is currently no server running on port {}".format(self.port), file=sys.stderr)
+- print("Ports currently in use:", file=sys.stderr)
++ print("Ports/sockets currently in use:", file=sys.stderr)
+ for server in servers:
+- print(" - {}".format(server['port']), file=sys.stderr)
++ print(" - {}".format(server.get('sock', server['port'])), file=sys.stderr)
+ self.exit(1)
+
+
+@@ -540,6 +585,8 @@
+ 'ip': 'NotebookApp.ip',
+ 'port': 'NotebookApp.port',
+ 'port-retries': 'NotebookApp.port_retries',
++ 'sock': 'NotebookApp.sock',
++ 'sock-umask': 'NotebookApp.sock_umask',
+ 'transport': 'KernelManager.transport',
+ 'keyfile': 'NotebookApp.keyfile',
+ 'certfile': 'NotebookApp.certfile',
+@@ -678,10 +725,18 @@
+ or containerized setups for example).""")
+ )
+
+- port = Integer(8888, config=True,
++ port = Integer(DEFAULT_NOTEBOOK_PORT, config=True,
+ help=_("The port the notebook server will listen on.")
+ )
+
++ sock = Unicode(u'', config=True,
++ help=_("The UNIX socket the notebook server will listen on.")
++ )
++
++ sock_umask = Unicode(u'0600', config=True,
++ help=_("The UNIX socket umask to set on creation (default: 0600).")
++ )
++
+ port_retries = Integer(50, config=True,
+ help=_("The number of additional ports to try if the specified port is not available.")
+ )
+@@ -1370,6 +1425,27 @@
+ self.log.critical(_("\t$ python -m notebook.auth password"))
+ sys.exit(1)
+
++ # Socket options validation.
++ if self.sock:
++ if self.port != DEFAULT_NOTEBOOK_PORT:
++ self.log.critical(
++ _('Options --port and --sock are mutually exclusive. Aborting.'),
++ )
++ sys.exit(1)
++
++ if self.open_browser:
++ # If we're bound to a UNIX socket, we can't reliably connect from a browser.
++ self.log.critical(
++ _('Options --open-browser and --sock are mutually exclusive. Aborting.'),
++ )
++ sys.exit(1)
++
++ if sys.platform.startswith('win'):
++ self.log.critical(
++ _('Option --sock is not supported on Windows, but got value of %s. Aborting.' % self.sock),
++ )
++ sys.exit(1)
++
+ self.web_app = NotebookWebApplication(
+ self, self.kernel_manager, self.contents_manager,
+ self.session_manager, self.kernel_spec_manager,
+@@ -1401,6 +1477,32 @@
+ max_body_size=self.max_body_size,
+ max_buffer_size=self.max_buffer_size)
+
++ success = self._bind_http_server()
++ if not success:
++ self.log.critical(_('ERROR: the notebook server could not be started because '
++ 'no available port could be found.'))
++ self.exit(1)
++
++ def _bind_http_server(self):
++ return self._bind_http_server_unix() if self.sock else self._bind_http_server_tcp()
++
++ def _bind_http_server_unix(self):
++ try:
++ sock = bind_unix_socket(self.sock, mode=int(self.sock_umask.encode(), 8))
++ self.http_server.add_socket(sock)
++ except socket.error as e:
++ if e.errno == errno.EADDRINUSE:
++ self.log.info(_('The socket %s is already in use.') % self.sock)
++ return False
++ elif e.errno in (errno.EACCES, getattr(errno, 'WSAEACCES', errno.EACCES)):
++ self.log.warning(_("Permission to listen on sock %s denied") % self.sock)
++ return False
++ else:
++ raise
++ else:
++ return True
++
++ def _bind_http_server_tcp(self):
+ success = None
+ for port in random_ports(self.port, self.port_retries+1):
+ try:
+@@ -1418,10 +1520,11 @@
+ self.port = port
+ success = True
+ break
+- if not success:
+- self.log.critical(_('ERROR: the notebook server could not be started because '
+- 'no available port could be found.'))
+- self.exit(1)
++ return success
++
++ def _concat_token(self, url):
++ token = self.token if self._token_generated else '...'
++ return url_concat(url, {'token': token})
+
+ @property
+ def display_url(self):
+@@ -1429,26 +1532,33 @@
+ url = self.custom_display_url
+ if not url.endswith('/'):
+ url += '/'
++ elif self.sock:
++ url = self._unix_sock_url()
+ else:
+ if self.ip in ('', '0.0.0.0'):
+ ip = "(%s or 127.0.0.1)" % socket.gethostname()
+ else:
+ ip = self.ip
+- url = self._url(ip)
+- if self.token:
+- # Don't log full token if it came from config
+- token = self.token if self._token_generated else '...'
+- url = url_concat(url, {'token': token})
++ url = self._tcp_url(ip)
++ if self.token and not self.sock:
++ url = self._concat_token(url)
++ url += '\n or %s' % self._concat_token(self._tcp_url('127.0.0.1'))
+ return url
+
+ @property
+ def connection_url(self):
+- ip = self.ip if self.ip else 'localhost'
+- return self._url(ip)
++ if self.sock:
++ return self._unix_sock_url()
++ else:
++ ip = self.ip if self.ip else 'localhost'
++ return self._tcp_url(ip)
+
+- def _url(self, ip):
++ def _unix_sock_url(self, token=None):
++ return '%s%s' % (urlencode_unix_socket(self.sock), self.base_url)
++
++ def _tcp_url(self, ip, port=None):
+ proto = 'https' if self.certfile else 'http'
+- return "%s://%s:%i%s" % (proto, ip, self.port, self.base_url)
++ return "%s://%s:%i%s" % (proto, ip, port or self.port, self.base_url)
+
+ def init_terminals(self):
+ if not self.terminals_enabled:
+@@ -1660,6 +1770,7 @@
+ return {'url': self.connection_url,
+ 'hostname': self.ip if self.ip else 'localhost',
+ 'port': self.port,
++ 'sock': self.sock,
+ 'secure': bool(self.certfile),
+ 'base_url': self.base_url,
+ 'token': self.token,
+@@ -1780,19 +1891,31 @@
+ self.write_server_info_file()
+ self.write_browser_open_file()
+
+- if self.open_browser or self.file_to_run:
++ if (self.open_browser or self.file_to_run) and not self.sock:
+ self.launch_browser()
+
+ if self.token and self._token_generated:
+ # log full URL with generated token, so there's a copy/pasteable link
+ # with auth info.
+- self.log.critical('\n'.join([
+- '\n',
+- 'To access the notebook, open this file in a browser:',
+- ' %s' % urljoin('file:', pathname2url(self.browser_open_file)),
+- 'Or copy and paste one of these URLs:',
+- ' %s' % self.display_url,
+- ]))
++ if self.sock:
++ self.log.critical('\n'.join([
++ '\n',
++ 'Notebook is listening on %s' % self.display_url,
++ '',
++ (
++ 'UNIX sockets are not browser-connectable, but you can tunnel to '
++ 'the instance via e.g.`ssh -L 8888:%s -N user@this_host` and then '
++ 'opening e.g. %s in a browser.'
++ ) % (self.sock, self._concat_token(self._tcp_url('localhost', 8888)))
++ ]))
++ else:
++ self.log.critical('\n'.join([
++ '\n',
++ 'To access the notebook, open this file in a browser:',
++ ' %s' % urljoin('file:', pathname2url(self.browser_open_file)),
++ 'Or copy and paste one of these URLs:',
++ ' %s' % self.display_url,
++ ]))
+
+ self.io_loop = ioloop.IOLoop.current()
+ if sys.platform.startswith('win'):
+diff -Naur notebook-5.7.4/notebook/tests/launchnotebook.py notebook-5.7.4.patched/notebook/tests/launchnotebook.py
+--- notebook-5.7.4/notebook/tests/launchnotebook.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/notebook/tests/launchnotebook.py 2019-11-18 12:22:25.931074384 +0100
+@@ -19,12 +19,13 @@
+ from mock import patch #py2
+
+ import requests
++import requests_unixsocket
+ from tornado.ioloop import IOLoop
+ import zmq
+
+ import jupyter_core.paths
+ from traitlets.config import Config
+-from ..notebookapp import NotebookApp
++from ..notebookapp import NotebookApp, urlencode_unix_socket
+ from ..utils import url_path_join
+ from ipython_genutils.tempdir import TemporaryDirectory
+
+@@ -55,7 +56,7 @@
+ url = cls.base_url() + 'api/contents'
+ for _ in range(int(MAX_WAITTIME/POLL_INTERVAL)):
+ try:
+- requests.get(url)
++ cls.fetch_url(url)
+ except Exception as e:
+ if not cls.notebook_thread.is_alive():
+ raise RuntimeError("The notebook server failed to start")
+@@ -79,6 +80,10 @@
+ headers['Authorization'] = 'token %s' % cls.token
+ return headers
+
++ @staticmethod
++ def fetch_url(url):
++ return requests.get(url)
++
+ @classmethod
+ def request(cls, verb, path, **kwargs):
+ """Send a request to my server
+@@ -93,6 +98,10 @@
+ return response
+
+ @classmethod
++ def get_bind_args(cls):
++ return dict(port=cls.port)
++
++ @classmethod
+ def setup_class(cls):
+ cls.tmp_dir = TemporaryDirectory()
+ def tmp(*parts):
+@@ -103,7 +112,7 @@
+ if e.errno != errno.EEXIST:
+ raise
+ return path
+-
++
+ cls.home_dir = tmp('home')
+ data_dir = cls.data_dir = tmp('data')
+ config_dir = cls.config_dir = tmp('config')
+@@ -138,8 +147,8 @@
+ if 'asyncio' in sys.modules:
+ import asyncio
+ asyncio.set_event_loop(asyncio.new_event_loop())
++ bind_args = cls.get_bind_args()
+ app = cls.notebook = NotebookApp(
+- port=cls.port,
+ port_retries=0,
+ open_browser=False,
+ config_dir=cls.config_dir,
+@@ -150,6 +159,7 @@
+ config=config,
+ allow_root=True,
+ token=cls.token,
++ **bind_args
+ )
+ # don't register signal handler during tests
+ app.init_signal = lambda : None
+@@ -197,6 +207,25 @@
+ return 'http://localhost:%i%s' % (cls.port, cls.url_prefix)
+
+
++class UNIXSocketNotebookTestBase(NotebookTestBase):
++ # Rely on `/tmp` to avoid any Linux socket length max buffer
++ # issues. Key on PID for process-wise concurrency.
++ sock = '/tmp/.notebook.%i.sock' % os.getpid()
++
++ @classmethod
++ def get_bind_args(cls):
++ return dict(sock=cls.sock)
++
++ @classmethod
++ def base_url(cls):
++ return '%s%s' % (urlencode_unix_socket(cls.sock), cls.url_prefix)
++
++ @staticmethod
++ def fetch_url(url):
++ with requests_unixsocket.monkeypatch():
++ return requests.get(url)
++
++
+ @contextmanager
+ def assert_http_error(status, msg=None):
+ try:
+diff -Naur notebook-5.7.4/notebook/tests/test_notebookapp_integration.py notebook-5.7.4.patched/notebook/tests/test_notebookapp_integration.py
+--- notebook-5.7.4/notebook/tests/test_notebookapp_integration.py 1970-01-01 01:00:00.000000000 +0100
++++ notebook-5.7.4.patched/notebook/tests/test_notebookapp_integration.py 2019-11-18 12:16:58.319065025 +0100
+@@ -0,0 +1,39 @@
++import os
++import stat
++import subprocess
++import time
++
++from ipython_genutils.testing.decorators import skip_win32
++
++from .launchnotebook import UNIXSocketNotebookTestBase
++from ..utils import urlencode_unix_socket, urlencode_unix_socket_path
++
++
++@skip_win32
++def test_shutdown_sock_server_integration():
++ sock = UNIXSocketNotebookTestBase.sock
++ url = urlencode_unix_socket(sock)
++ encoded_sock_path = urlencode_unix_socket_path(sock)
++
++ p = subprocess.Popen(
++ ['jupyter', 'notebook', '--no-browser', '--sock=%s' % sock],
++ stdout=subprocess.PIPE, stderr=subprocess.PIPE
++ )
++
++ for line in iter(p.stderr.readline, b''):
++ if url.encode() in line:
++ complete = True
++ break
++
++ assert complete, 'did not find socket URL in stdout when launching notebook'
++
++ assert encoded_sock_path.encode() in subprocess.check_output(['jupyter', 'notebook', 'list'])
++
++ # Ensure default umask is properly applied.
++ assert stat.S_IMODE(os.lstat(sock).st_mode) == 0o600
++
++ subprocess.check_output(['jupyter', 'notebook', 'stop', sock])
++
++ assert encoded_sock_path.encode() not in subprocess.check_output(['jupyter', 'notebook', 'list'])
++
++ p.wait()
+diff -Naur notebook-5.7.4/notebook/tests/test_notebookapp.py notebook-5.7.4.patched/notebook/tests/test_notebookapp.py
+--- notebook-5.7.4/notebook/tests/test_notebookapp.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/notebook/tests/test_notebookapp.py 2019-11-18 12:16:58.319065025 +0100
+@@ -25,7 +25,7 @@
+ from notebook.auth.security import passwd_check
+ NotebookApp = notebookapp.NotebookApp
+
+-from .launchnotebook import NotebookTestBase
++from .launchnotebook import NotebookTestBase, UNIXSocketNotebookTestBase
+
+
+ def test_help_output():
+@@ -192,3 +192,15 @@
+ servers = list(notebookapp.list_running_servers())
+ assert len(servers) >= 1
+ assert self.port in {info['port'] for info in servers}
++
++
++# UNIX sockets aren't available on Windows.
++if not sys.platform.startswith('win'):
++ class NotebookUnixSocketTests(UNIXSocketNotebookTestBase):
++ def test_run(self):
++ self.fetch_url(self.base_url() + 'api/contents')
++
++ def test_list_running_sock_servers(self):
++ servers = list(notebookapp.list_running_servers())
++ assert len(servers) >= 1
++ assert self.sock in {info['sock'] for info in servers}
+diff -Naur notebook-5.7.4/notebook/utils.py notebook-5.7.4.patched/notebook/utils.py
+--- notebook-5.7.4/notebook/utils.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/notebook/utils.py 2019-11-18 12:23:05.231075507 +0100
+@@ -306,3 +306,18 @@
+ check_pid = _check_pid_win32
+ else:
+ check_pid = _check_pid_posix
++
++def urlencode_unix_socket_path(socket_path):
++ """Encodes a UNIX socket path string from a socket path for the `http+unix` URI form."""
++ return socket_path.replace('/', '%2F')
++
++
++def urldecode_unix_socket_path(socket_path):
++ """Decodes a UNIX sock path string from an encoded sock path for the `http+unix` URI form."""
++ return socket_path.replace('%2F', '/')
++
++
++def urlencode_unix_socket(socket_path):
++ """Encodes a UNIX socket URL from a socket path for the `http+unix` URI form."""
++ return 'http+unix://%s' % urlencode_unix_socket_path(socket_path)
++
+diff -Naur notebook-5.7.4/setup.py notebook-5.7.4.patched/setup.py
+--- notebook-5.7.4/setup.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/setup.py 2019-11-18 12:23:33.851076325 +0100
+@@ -98,7 +98,8 @@
+ ':python_version == "2.7"': ['ipaddress'],
+ 'test:python_version == "2.7"': ['mock'],
+ 'test': ['nose', 'coverage', 'requests', 'nose_warnings_filters',
+- 'nbval', 'nose-exclude', 'selenium'],
++ 'nbval', 'nose-exclude', 'selenium',
++ 'requests-unixsocket'],
+ 'test:sys_platform == "win32"': ['nose-exclude'],
+ },
+ entry_points = {
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 232841ccb1..4263a33c6b 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -7811,7 +7811,8 @@ convert an @code{.ipynb} notebook file into various static formats including:
(uri (pypi-uri "notebook" version))
(sha256
(base32
- "0jm7324mbxljmn9hgapj66q7swyz5ai92blmr0jpcy0h80x6f26r"))))
+ "0jm7324mbxljmn9hgapj66q7swyz5ai92blmr0jpcy0h80x6f26r"))
+ (patches (search-patches "jupyter-unix-domain-sockets-4835-5.7.4.patch"))))
(build-system python-build-system)
(arguments
`(#:phases
@@ -7834,7 +7835,8 @@ convert an @code{.ipynb} notebook file into various static formats including:
("python-nbconvert" ,python-nbconvert)
("python-prometheus-client" ,python-prometheus-client)
("python-send2trash" ,python-send2trash)
- ("python-terminado" ,python-terminado)))
+ ("python-terminado" ,python-terminado)
+ ("python-requests-unixsocket" ,python-requests-unixsocket)))
(native-inputs
`(("python-nose" ,python-nose)
("python-sphinx" ,python-sphinx)
@@ -17711,3 +17713,32 @@ applications")
based on the Jupyter Notebook and Architecture.")
(license license:bsd-3)))
+(define-public python-requests-unixsocket
+ (package
+ (name "python-requests-unixsocket")
+ (version "0.2.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "requests-unixsocket" version))
+ (sha256
+ (base32
+ "1sn12y4fw1qki5gxy9wg45gmdrxhrndwfndfjxhpiky3mwh1lp4y"))))
+ (build-system python-build-system)
+ (native-inputs
+ ;; pbr is required for setup only
+ `(("python-pbr" ,python-pbr)))
+ (propagated-inputs
+ `(("python-requests" ,python-requests)
+ ("python-urllib3" ,python-urllib3)))
+ (arguments
+ ;; tests depend on very specific package version, which are not available in guix
+ '(#:tests? #f))
+ (home-page
+ "https://github.com/msabramo/requests-unixsocket")
+ (synopsis
+ "Use requests to talk HTTP via a UNIX domain socket")
+ (description
+ "Use requests to talk HTTP via a UNIX domain socket")
+ (license license:asl2.0)))
+
--
2.20.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [bug#39765] Add package JupyterLab
2020-02-24 10:18 [bug#39765] Add package JupyterLab Lars-Dominik Braun
@ 2020-03-26 22:55 ` Ludovic Courtès
2020-03-27 7:30 ` Lars-Dominik Braun
0 siblings, 1 reply; 5+ messages in thread
From: Ludovic Courtès @ 2020-03-26 22:55 UTC (permalink / raw)
To: Lars-Dominik Braun; +Cc: 39765
Hi Lars,
Sorry for the late reply.
Lars-Dominik Braun <ldb@leibniz-psychology.org> skribis:
> this patch series adds Jupyter’s JupyterLab, which is the new frontend for
> Jupyter Notebooks. The software works fine, but there are a few caveats
>
> 1) it comes with bundled pre-compiled JavaScript, which cannot be removed until
> we have proper support for importing from NPM
> 2) it contains an extension manager, that downloads arbitrary packages from NPM
> (`jupyter lab build`). This works, but is less than optimal imo. We should
> figure out how to package extensions in guix.
> 3) also it is required to install the package `jupyter`, otherwise installed
> kernels cannot be found and the `jupyter` command does not work.
#2 should be quite easy to address: we could arrange to have that
feature disabled by default, so that users don’t find themselves
unknowingly downloading arbitrary code from npm.
#3 is OK.
#1 is a showstopper. :-/ I suppose that’s a lot of code that would
need to be imported from npm, right?
It’s sad because all this is free software, but we practically can’t get
the corresponding source.
I’ve pushed the first two patches of the series (python-json5 and
python-pytest-check-links).
Comments on the other bits that are readily applicable:
>>From a47fd94aa6f3e62b77f3b7208c4e6757e3a9ee08 Mon Sep 17 00:00:00 2001
> From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
> Date: Thu, 12 Dec 2019 08:53:39 +0100
> Subject: [PATCH 5/5] gnu: python-notebook: Support UNIX domain sockets
>
> * gnu/packages/python-xyz.scm (python-notebook): Add patch from upstream
> https://github.com/jupyter/notebook/pull/4835
> (python-requests-unixsocket) New variable
> ---
> ...pyter-unix-domain-sockets-4835-5.7.4.patch | 591 ++++++++++++++++++
> gnu/packages/python-xyz.scm | 35 +-
> 2 files changed, 624 insertions(+), 2 deletions(-)
> create mode 100644 gnu/packages/patches/jupyter-unix-domain-sockets-4835-5.7.4.patch
>
> diff --git a/gnu/packages/patches/jupyter-unix-domain-sockets-4835-5.7.4.patch b/gnu/packages/patches/jupyter-unix-domain-sockets-4835-5.7.4.patch
> new file mode 100644
> index 0000000000..134d3ad2b8
> --- /dev/null
> +++ b/gnu/packages/patches/jupyter-unix-domain-sockets-4835-5.7.4.patch
> @@ -0,0 +1,591 @@
Please add provenance info at the top of the patch (such as the URL of
the upstream commit), as well as a line or two explaining what it does.
You can omit “-4835-5.7.4” from the file name.
Make sure to add the file to ‘gnu/local.mk’.
That said, it’s a big patch, so it would be even better if we didn’t
have to carry it. Will the next version of ‘notebook’ include it?
Last, ‘python-requests-unixsocket’ should be added in a separate patch.
[...]
> + (arguments
> + ;; tests depend on very specific package version, which are not available in guix
> + '(#:tests? #f))
Perhaps add a “FIXME” and clarify which packages we’re talking about
(the “not available” bit is bound to become outdated :-)).
> + (home-page
> + "https://github.com/msabramo/requests-unixsocket")
> + (synopsis
> + "Use requests to talk HTTP via a UNIX domain socket")
> + (description
> + "Use requests to talk HTTP via a UNIX domain socket")
Please follow the synopsis/description guidelines (info "(guix) Synopses
and Descriptions").
Thank you for this endeavor!
Ludo’.
^ permalink raw reply [flat|nested] 5+ messages in thread
* [bug#39765] Add package JupyterLab
2020-03-26 22:55 ` Ludovic Courtès
@ 2020-03-27 7:30 ` Lars-Dominik Braun
2020-03-29 14:37 ` Ludovic Courtès
0 siblings, 1 reply; 5+ messages in thread
From: Lars-Dominik Braun @ 2020-03-27 7:30 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 39765
[-- Attachment #1: Type: text/plain, Size: 1313 bytes --]
Hi Ludo,
> #2 should be quite easy to address: we could arrange to have that
> feature disabled by default, so that users don’t find themselves
> unknowingly downloading arbitrary code from npm.
it’s “disabled” by default, because it is considered experimental in this
version of JupyterLab. But a user can re-enable it. And the last part is
entirely client-side, so we cannot disable it completely until we fix #1.
> #1 is a showstopper. :-/ I suppose that’s a lot of code that would
> need to be imported from npm, right?
`jupyter build` downloads about 600 NPM packages, as far as I remember.
> I’ve pushed the first two patches of the series (python-json5 and
> python-pytest-check-links).
Thank you!
> That said, it’s a big patch, so it would be even better if we didn’t
> have to carry it. Will the next version of ‘notebook’ include it?
Does not look like it. The pull request[1] has been open for a few months now.
It’s vital to our use-case and (probably) everyone hosting notebooks, but not
very useful to the casual home user. So, executive decision: Do you want it in
guix proper? I’ll just maintain it in my channel[2] otherwise.
Lars
[1] https://github.com/jupyter/notebook/pull/4835
[2] https://github.com/leibniz-psychology/guix-zpid
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* [bug#39765] Add package JupyterLab
2020-03-27 7:30 ` Lars-Dominik Braun
@ 2020-03-29 14:37 ` Ludovic Courtès
2020-03-30 6:10 ` Lars-Dominik Braun
0 siblings, 1 reply; 5+ messages in thread
From: Ludovic Courtès @ 2020-03-29 14:37 UTC (permalink / raw)
To: Lars-Dominik Braun; +Cc: 39765
Hi,
Lars-Dominik Braun <ldb@leibniz-psychology.org> skribis:
>> #2 should be quite easy to address: we could arrange to have that
>> feature disabled by default, so that users don’t find themselves
>> unknowingly downloading arbitrary code from npm.
> it’s “disabled” by default, because it is considered experimental in this
> version of JupyterLab. But a user can re-enable it. And the last part is
> entirely client-side, so we cannot disable it completely until we fix #1.
>
>> #1 is a showstopper. :-/ I suppose that’s a lot of code that would
>> need to be imported from npm, right?
> `jupyter build` downloads about 600 NPM packages, as far as I remember.
OK.
>> That said, it’s a big patch, so it would be even better if we didn’t
>> have to carry it. Will the next version of ‘notebook’ include it?
> Does not look like it. The pull request[1] has been open for a few months now.
> It’s vital to our use-case and (probably) everyone hosting notebooks, but not
> very useful to the casual home user. So, executive decision: Do you want it in
> guix proper? I’ll just maintain it in my channel[2] otherwise.
(It’s not about what I personally want or don’t want, of course. :-))
In general, the guideline is to have patches that are either included
upstream, just not in a published release, or are Guix-specific and thus
are not meant to be included upstream.
This patch doesn’t seem to fall in any of these two categories, so I
would prefer not to have it, at least not until upstream has included
it.
WDYT?
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 5+ messages in thread
* [bug#39765] Add package JupyterLab
2020-03-29 14:37 ` Ludovic Courtès
@ 2020-03-30 6:10 ` Lars-Dominik Braun
0 siblings, 0 replies; 5+ messages in thread
From: Lars-Dominik Braun @ 2020-03-30 6:10 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 39765
Hi Ludo,
> (It’s not about what I personally want or don’t want, of course. :-))
> In general, the guideline is to have patches that are either included
> upstream, just not in a published release, or are Guix-specific and thus
> are not meant to be included upstream.
>
> This patch doesn’t seem to fall in any of these two categories, so I
> would prefer not to have it, at least not until upstream has included
> it.
sure, I can see that :)
Lars
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-03-30 6:12 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-02-24 10:18 [bug#39765] Add package JupyterLab Lars-Dominik Braun
2020-03-26 22:55 ` Ludovic Courtès
2020-03-27 7:30 ` Lars-Dominik Braun
2020-03-29 14:37 ` Ludovic Courtès
2020-03-30 6:10 ` Lars-Dominik Braun
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).