From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
To: 39765@debbugs.gnu.org
Subject: [bug#39765] Add package JupyterLab
Date: Mon, 24 Feb 2020 11:18:10 +0100 [thread overview]
Message-ID: <20200224101810.GA9010@zpidnp36> (raw)
[-- Attachment #1: Type: text/plain, Size: 651 bytes --]
Hi,
this patch series adds Jupyter’s JupyterLab, which is the new frontend for
Jupyter Notebooks. The software works fine, but there are a few caveats
1) it comes with bundled pre-compiled JavaScript, which cannot be removed until
we have proper support for importing from NPM
2) it contains an extension manager, that downloads arbitrary packages from NPM
(`jupyter lab build`). This works, but is less than optimal imo. We should
figure out how to package extensions in guix.
3) also it is required to install the package `jupyter`, otherwise installed
kernels cannot be found and the `jupyter` command does not work.
Cheers,
Lars
[-- Attachment #2: 0001-gnu-Add-package-python-pytest-check-links.patch --]
[-- Type: text/x-diff, Size: 1688 bytes --]
From 4a5862e2add1d537770a5ea466dbb8a4851afad9 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Fri, 7 Feb 2020 08:38:32 +0100
Subject: [PATCH 1/5] gnu: Add package python-pytest-check-links
* gnu/packages/python-xyz.scm (python-pytest-check-links): New variable.
---
gnu/packages/python-xyz.scm | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 84b70954bd..902ca5030b 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -17570,3 +17570,31 @@ sequences.")
(define-public python2-fuzzywuzzy
(package-with-python2 python-fuzzywuzzy))
+
+(define-public python-pytest-check-links
+ (package
+ (name "python-pytest-check-links")
+ (version "0.3.0")
+ (source
+ (origin
+ (method url-fetch)
+ ;; URI uses underscores
+ (uri (pypi-uri "pytest_check_links" version))
+ (sha256
+ (base32
+ "12x3wmrdzm6wgk0vz02hb769h68nr49q47w5q1pj95pc89hsa34v"))))
+ (build-system python-build-system)
+ (propagated-inputs
+ `(("python-docutils" ,python-docutils)
+ ("python-html5lib" ,python-html5lib)
+ ("python-nbconvert" ,python-nbconvert)
+ ("python-nbformat" ,python-nbformat)
+ ("python-pytest" ,python-pytest)
+ ("python-six" ,python-six)))
+ (native-inputs
+ `(("python-pbr-minimal" ,python-pbr-minimal)))
+ (home-page
+ "https://github.com/minrk/pytest-check-links")
+ (synopsis "Check links in files")
+ (description "Plugin for pytest that checks URLs for HTML-containing files")
+ (license license:bsd-3)))
--
2.20.1
[-- Attachment #3: 0002-gnu-Add-package-python-json5.patch --]
[-- Type: text/x-diff, Size: 1651 bytes --]
From 690d45cba7d1a21fa6ae97fbe4fec6e1abae3635 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Fri, 7 Feb 2020 08:39:55 +0100
Subject: [PATCH 2/5] gnu: Add package python-json5
* gnu/packages/python-xyz.scm (python-json5): New variable.
---
gnu/packages/python-xyz.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 902ca5030b..ad1cd5fbd4 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -17598,3 +17598,27 @@ sequences.")
(synopsis "Check links in files")
(description "Plugin for pytest that checks URLs for HTML-containing files")
(license license:bsd-3)))
+
+(define-public python-json5
+ (package
+ (name "python-json5")
+ (version "0.8.5")
+ (source
+ (origin
+ ;; sample.json5 is missing from PyPi source tarball
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/dpranke/pyjson5.git")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "0nyngj18jlkgvm1177lc3cj47wm4yh3dqigygvcvw7xkyryafsqn"))))
+ (build-system python-build-system)
+ (home-page "https://github.com/dpranke/pyjson5")
+ (synopsis
+ "Python implementation of the JSON5 data format")
+ (description
+ "JSON5 extends the JSON data interchange format to make it slightly more
+usable as a configuration language. This Python package implements parsing and
+dumping of JSON5 data structures.")
+ (license license:asl2.0)))
--
2.20.1
[-- Attachment #4: 0003-gnu-Add-package-python-jupyterlab-server.patch --]
[-- Type: text/x-diff, Size: 1927 bytes --]
From a4535f6002a618444171d5a92146fb7a7e7e8243 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Fri, 7 Feb 2020 08:40:41 +0100
Subject: [PATCH 3/5] gnu: Add package python-jupyterlab-server
* gnu/packages/python-xyz.scm (python-jupyterlab-server): New variable.
---
gnu/packages/python-xyz.scm | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index ad1cd5fbd4..92ee53fe6f 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -17622,3 +17622,37 @@ sequences.")
usable as a configuration language. This Python package implements parsing and
dumping of JSON5 data structures.")
(license license:asl2.0)))
+
+(define-public python-jupyterlab-server
+ (package
+ (name "python-jupyterlab-server")
+ (version "1.0.6")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "jupyterlab_server" version))
+ (sha256
+ (base32
+ "1bax8iqwcc5p02h5ysdc48zvx7ll5jfzfsybhb3lfvyfpwkpb5yh"))))
+ (build-system python-build-system)
+ (propagated-inputs
+ `(("python-jinja2" ,python-jinja2)
+ ("python-json5" ,python-json5)
+ ("python-jsonschema" ,python-jsonschema)
+ ("python-notebook" ,python-notebook)))
+ (native-inputs
+ `(("python-pytest" ,python-pytest)
+ ("python-requests" ,python-requests)
+ ("python-ipykernel" ,python-ipykernel)))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ ;; python setup.py test does not invoke pytest?
+ (replace 'check
+ (lambda _
+ (invoke "pytest" "-vv"))))))
+ (home-page "https://jupyter.org")
+ (synopsis "JupyterLab Server")
+ (description "A set of server components for JupyterLab and JupyterLab like
+applications")
+ (license license:bsd-3)))
--
2.20.1
[-- Attachment #5: 0004-gnu-Add-package-python-jupyterlab.patch --]
[-- Type: text/x-diff, Size: 5193 bytes --]
From c50fc3ec734cb94e78168a4a29c9aff070f4ae9f Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Fri, 7 Feb 2020 08:41:24 +0100
Subject: [PATCH 4/5] gnu: Add package python-jupyterlab
* gnu/packages/python-xyz.scm (python-jupyterlab): New variable.
---
.../python-jupyterlab-copy-nometa.patch | 33 +++++++++++
gnu/packages/python-xyz.scm | 55 +++++++++++++++++++
2 files changed, 88 insertions(+)
create mode 100644 gnu/packages/patches/python-jupyterlab-copy-nometa.patch
diff --git a/gnu/packages/patches/python-jupyterlab-copy-nometa.patch b/gnu/packages/patches/python-jupyterlab-copy-nometa.patch
new file mode 100644
index 0000000000..5e770c0f47
--- /dev/null
+++ b/gnu/packages/patches/python-jupyterlab-copy-nometa.patch
@@ -0,0 +1,33 @@
+diff '--exclude=*.swp' -Naur jupyterlab-1.2.6.orig/jupyterlab/commands.py jupyterlab-1.2.6/jupyterlab/commands.py
+--- jupyterlab-1.2.6.orig/jupyterlab/commands.py 2020-01-24 17:25:16.238353500 +0100
++++ jupyterlab-1.2.6/jupyterlab/commands.py 2020-01-31 12:36:10.497375982 +0100
+@@ -1100,7 +1100,7 @@
+ 'webpack.prod.minimize.config.js',
+ '.yarnrc', 'yarn.js']:
+ target = pjoin(staging, fname)
+- shutil.copy(pjoin(HERE, 'staging', fname), target)
++ shutil.copyfile(pjoin(HERE, 'staging', fname), target)
+
+ # Ensure a clean templates directory
+ templates = pjoin(staging, 'templates')
+@@ -1108,7 +1108,10 @@
+ _rmtree(templates, self.logger)
+
+ try:
+- shutil.copytree(pjoin(HERE, 'staging', 'templates'), templates)
++ shutil.copytree(pjoin(HERE, 'staging', 'templates'), templates,
++ copy_function=shutil.copyfile)
++ # cannot replace or disable copytree’s call to .copystat
++ os.chmod (templates, 0o755)
+ except shutil.Error as error:
+ # `copytree` throws an error if copying to + from NFS even though
+ # the copy is successful (see https://bugs.python.org/issue24564
+@@ -1178,7 +1181,7 @@
+ with open(lock_path, 'w', encoding='utf-8') as f:
+ f.write(template)
+ elif not osp.exists(lock_path):
+- shutil.copy(lock_template, lock_path)
++ shutil.copyfile(lock_template, lock_path)
+
+ def _get_package_template(self, silent=False):
+ """Get the template the for staging package.json file.
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 92ee53fe6f..232841ccb1 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -126,6 +126,7 @@
#:use-module (gnu packages multiprecision)
#:use-module (gnu packages networking)
#:use-module (gnu packages ncurses)
+ #:use-module (gnu packages node)
#:use-module (gnu packages openstack)
#:use-module (gnu packages pcre)
#:use-module (gnu packages perl)
@@ -17656,3 +17657,57 @@ dumping of JSON5 data structures.")
(description "A set of server components for JupyterLab and JupyterLab like
applications")
(license license:bsd-3)))
+
+(define-public python-jupyterlab
+ (package
+ (name "python-jupyterlab")
+ (version "1.2.6")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "jupyterlab" version))
+ (sha256
+ (base32
+ "0mc3nrj7fc5q2ajr09m261j386jsp8qjljg8anghlh8czc9ln4s2"))
+ (patches (search-patches "python-jupyterlab-copy-nometa.patch"))))
+ (build-system python-build-system)
+ (propagated-inputs
+ `(("python-jinja2" ,python-jinja2)
+ ("python-jupyterlab-server"
+ ,python-jupyterlab-server)
+ ("python-notebook" ,python-notebook)
+ ("python-tornado" ,python-tornado)
+ ("node" ,node)))
+ (native-inputs
+ `(("python-pytest" ,python-pytest)
+ ("python-pytest-check-links"
+ ,python-pytest-check-links)
+ ("python-requests" ,python-requests)
+ ("python-ipykernel" ,python-ipykernel)))
+ (arguments
+ ;; testing requires npm, so disabled for now
+ '(#:tests? #f
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'patch-syspath
+ (lambda* (#:key outputs inputs configure-flags #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out")))
+ (substitute* "jupyterlab/commands.py"
+ ;; sys.prefix defaults to Python’s prefix in the store, not
+ ;; jupyterlab’s. Fix that.
+ (("sys\\.prefix")
+ (string-append "'" out "'"))))
+ #t))
+ ;; 'build does not respect configure-flags
+ (replace 'build
+ (lambda _
+ (invoke "python" "setup.py" "build" "--skip-npm"))))
+ #:configure-flags (list "--skip-npm")))
+ (home-page "https://jupyter.org")
+ (synopsis
+ "The JupyterLab notebook server extension")
+ (description
+ "An extensible environment for interactive and reproducible computing,
+based on the Jupyter Notebook and Architecture.")
+ (license license:bsd-3)))
+
--
2.20.1
[-- Attachment #6: 0005-gnu-python-notebook-Support-UNIX-domain-sockets.patch --]
[-- Type: text/x-diff, Size: 26768 bytes --]
From a47fd94aa6f3e62b77f3b7208c4e6757e3a9ee08 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Thu, 12 Dec 2019 08:53:39 +0100
Subject: [PATCH 5/5] gnu: python-notebook: Support UNIX domain sockets
* gnu/packages/python-xyz.scm (python-notebook): Add patch from upstream
https://github.com/jupyter/notebook/pull/4835
(python-requests-unixsocket) New variable
---
...pyter-unix-domain-sockets-4835-5.7.4.patch | 591 ++++++++++++++++++
gnu/packages/python-xyz.scm | 35 +-
2 files changed, 624 insertions(+), 2 deletions(-)
create mode 100644 gnu/packages/patches/jupyter-unix-domain-sockets-4835-5.7.4.patch
diff --git a/gnu/packages/patches/jupyter-unix-domain-sockets-4835-5.7.4.patch b/gnu/packages/patches/jupyter-unix-domain-sockets-4835-5.7.4.patch
new file mode 100644
index 0000000000..134d3ad2b8
--- /dev/null
+++ b/gnu/packages/patches/jupyter-unix-domain-sockets-4835-5.7.4.patch
@@ -0,0 +1,591 @@
+diff -Naur notebook-5.7.4/notebook/base/handlers.py notebook-5.7.4.patched/notebook/base/handlers.py
+--- notebook-5.7.4/notebook/base/handlers.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/notebook/base/handlers.py 2019-11-18 12:16:58.315065024 +0100
+@@ -40,7 +40,7 @@
+ import notebook
+ from notebook._tz import utcnow
+ from notebook.i18n import combine_translations
+-from notebook.utils import is_hidden, url_path_join, url_is_absolute, url_escape
++from notebook.utils import is_hidden, url_path_join, url_is_absolute, url_escape, urldecode_unix_socket_path
+ from notebook.services.security import csp_report_uri
+
+ #-----------------------------------------------------------------------------
+@@ -426,13 +426,18 @@
+ # ip_address only accepts unicode on Python 2
+ host = host.decode('utf8', 'replace')
+
+- try:
+- addr = ipaddress.ip_address(host)
+- except ValueError:
+- # Not an IP address: check against hostnames
+- allow = host in self.settings.get('local_hostnames', ['localhost'])
++ # UNIX socket handling
++ check_host = urldecode_unix_socket_path(host)
++ if check_host.startswith('/') and os.path.exists(check_host):
++ allow = True
+ else:
+- allow = addr.is_loopback
++ try:
++ addr = ipaddress.ip_address(host)
++ except ValueError:
++ # Not an IP address: check against hostnames
++ allow = host in self.settings.get('local_hostnames', ['localhost'])
++ else:
++ allow = addr.is_loopback
+
+ if not allow:
+ self.log.warning(
+diff -Naur notebook-5.7.4/notebook/__init__.py notebook-5.7.4.patched/notebook/__init__.py
+--- notebook-5.7.4/notebook/__init__.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/notebook/__init__.py 2019-11-18 12:16:58.315065024 +0100
+@@ -20,6 +20,8 @@
+ os.path.join(os.path.dirname(__file__), "templates"),
+ ]
+
++DEFAULT_NOTEBOOK_PORT = 8888
++
+ del os
+
+ from .nbextensions import install_nbextension
+diff -Naur notebook-5.7.4/notebook/notebookapp.py notebook-5.7.4.patched/notebook/notebookapp.py
+--- notebook-5.7.4/notebook/notebookapp.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/notebook/notebookapp.py 2019-11-18 12:21:34.975072928 +0100
+@@ -63,8 +63,11 @@
+ from tornado import web
+ from tornado.httputil import url_concat
+ from tornado.log import LogFormatter, app_log, access_log, gen_log
++if not sys.platform.startswith('win'):
++ from tornado.netutil import bind_unix_socket
+
+ from notebook import (
++ DEFAULT_NOTEBOOK_PORT,
+ DEFAULT_STATIC_FILES_PATH,
+ DEFAULT_TEMPLATE_PATH_LIST,
+ __version__,
+@@ -108,7 +111,16 @@
+ from notebook._sysinfo import get_sys_info
+
+ from ._tz import utcnow, utcfromtimestamp
+-from .utils import url_path_join, check_pid, url_escape, urljoin, pathname2url
++from .utils import (
++ check_pid,
++ pathname2url,
++ url_escape,
++ url_path_join,
++ urldecode_unix_socket_path,
++ urlencode_unix_socket,
++ urlencode_unix_socket_path,
++ urljoin,
++)
+
+ #-----------------------------------------------------------------------------
+ # Module globals
+@@ -212,7 +224,7 @@
+ warnings.warn(_("The `ignore_minified_js` flag is deprecated and will be removed in Notebook 6.0"), DeprecationWarning)
+
+ now = utcnow()
+-
++
+ root_dir = contents_manager.root_dir
+ home = os.path.expanduser('~')
+ if root_dir.startswith(home + os.path.sep):
+@@ -385,6 +397,7 @@
+ set_password(config_file=self.config_file)
+ self.log.info("Wrote hashed password to %s" % self.config_file)
+
++
+ def shutdown_server(server_info, timeout=5, log=None):
+ """Shutdown a notebook server in a separate process.
+
+@@ -397,14 +410,39 @@
+ Returns True if the server was stopped by any means, False if stopping it
+ failed (on Windows).
+ """
+- from tornado.httpclient import HTTPClient, HTTPRequest
++ from tornado import gen
++ from tornado.httpclient import AsyncHTTPClient, HTTPClient, HTTPRequest
++ from tornado.netutil import bind_unix_socket, Resolver
+ url = server_info['url']
+ pid = server_info['pid']
++ resolver = None
++
++ # UNIX Socket handling.
++ if url.startswith('http+unix://'):
++ # This library doesn't understand our URI form, but it's just HTTP.
++ url = url.replace('http+unix://', 'http://')
++
++ class UnixSocketResolver(Resolver):
++ def initialize(self, resolver):
++ self.resolver = resolver
++
++ def close(self):
++ self.resolver.close()
++
++ @gen.coroutine
++ def resolve(self, host, port, *args, **kwargs):
++ raise gen.Return([
++ (socket.AF_UNIX, urldecode_unix_socket_path(host))
++ ])
++
++ resolver = UnixSocketResolver(resolver=Resolver())
++
+ req = HTTPRequest(url + 'api/shutdown', method='POST', body=b'', headers={
+ 'Authorization': 'token ' + server_info['token']
+ })
+ if log: log.debug("POST request to %sapi/shutdown", url)
+- HTTPClient().fetch(req)
++ AsyncHTTPClient.configure(None, resolver=resolver)
++ HTTPClient(AsyncHTTPClient).fetch(req)
+
+ # Poll to see if it shut down.
+ for _ in range(timeout*10):
+@@ -435,13 +473,20 @@
+ version = __version__
+ description="Stop currently running notebook server for a given port"
+
+- port = Integer(8888, config=True,
+- help="Port of the server to be killed. Default 8888")
++ port = Integer(DEFAULT_NOTEBOOK_PORT, config=True,
++ help="Port of the server to be killed. Default %s" % DEFAULT_NOTEBOOK_PORT)
++
++ sock = Unicode(u'', config=True,
++ help="UNIX socket of the server to be killed.")
+
+ def parse_command_line(self, argv=None):
+ super(NbserverStopApp, self).parse_command_line(argv)
+ if self.extra_args:
+- self.port=int(self.extra_args[0])
++ try:
++ self.port = int(self.extra_args[0])
++ except ValueError:
++ # self.extra_args[0] was not an int, so it must be a string (unix socket).
++ self.sock = self.extra_args[0]
+
+ def shutdown_server(self, server):
+ return shutdown_server(server, log=self.log)
+@@ -451,16 +496,16 @@
+ if not servers:
+ self.exit("There are no running servers")
+ for server in servers:
+- if server['port'] == self.port:
+- print("Shutting down server on port", self.port, "...")
++ if server.get('sock') == self.sock or server['port'] == self.port:
++ print("Shutting down server on %s..." % self.sock or self.port)
+ if not self.shutdown_server(server):
+ sys.exit("Could not stop server")
+ return
+ else:
+ print("There is currently no server running on port {}".format(self.port), file=sys.stderr)
+- print("Ports currently in use:", file=sys.stderr)
++ print("Ports/sockets currently in use:", file=sys.stderr)
+ for server in servers:
+- print(" - {}".format(server['port']), file=sys.stderr)
++ print(" - {}".format(server.get('sock', server['port'])), file=sys.stderr)
+ self.exit(1)
+
+
+@@ -540,6 +585,8 @@
+ 'ip': 'NotebookApp.ip',
+ 'port': 'NotebookApp.port',
+ 'port-retries': 'NotebookApp.port_retries',
++ 'sock': 'NotebookApp.sock',
++ 'sock-umask': 'NotebookApp.sock_umask',
+ 'transport': 'KernelManager.transport',
+ 'keyfile': 'NotebookApp.keyfile',
+ 'certfile': 'NotebookApp.certfile',
+@@ -678,10 +725,18 @@
+ or containerized setups for example).""")
+ )
+
+- port = Integer(8888, config=True,
++ port = Integer(DEFAULT_NOTEBOOK_PORT, config=True,
+ help=_("The port the notebook server will listen on.")
+ )
+
++ sock = Unicode(u'', config=True,
++ help=_("The UNIX socket the notebook server will listen on.")
++ )
++
++ sock_umask = Unicode(u'0600', config=True,
++ help=_("The UNIX socket umask to set on creation (default: 0600).")
++ )
++
+ port_retries = Integer(50, config=True,
+ help=_("The number of additional ports to try if the specified port is not available.")
+ )
+@@ -1370,6 +1425,27 @@
+ self.log.critical(_("\t$ python -m notebook.auth password"))
+ sys.exit(1)
+
++ # Socket options validation.
++ if self.sock:
++ if self.port != DEFAULT_NOTEBOOK_PORT:
++ self.log.critical(
++ _('Options --port and --sock are mutually exclusive. Aborting.'),
++ )
++ sys.exit(1)
++
++ if self.open_browser:
++ # If we're bound to a UNIX socket, we can't reliably connect from a browser.
++ self.log.critical(
++ _('Options --open-browser and --sock are mutually exclusive. Aborting.'),
++ )
++ sys.exit(1)
++
++ if sys.platform.startswith('win'):
++ self.log.critical(
++ _('Option --sock is not supported on Windows, but got value of %s. Aborting.' % self.sock),
++ )
++ sys.exit(1)
++
+ self.web_app = NotebookWebApplication(
+ self, self.kernel_manager, self.contents_manager,
+ self.session_manager, self.kernel_spec_manager,
+@@ -1401,6 +1477,32 @@
+ max_body_size=self.max_body_size,
+ max_buffer_size=self.max_buffer_size)
+
++ success = self._bind_http_server()
++ if not success:
++ self.log.critical(_('ERROR: the notebook server could not be started because '
++ 'no available port could be found.'))
++ self.exit(1)
++
++ def _bind_http_server(self):
++ return self._bind_http_server_unix() if self.sock else self._bind_http_server_tcp()
++
++ def _bind_http_server_unix(self):
++ try:
++ sock = bind_unix_socket(self.sock, mode=int(self.sock_umask.encode(), 8))
++ self.http_server.add_socket(sock)
++ except socket.error as e:
++ if e.errno == errno.EADDRINUSE:
++ self.log.info(_('The socket %s is already in use.') % self.sock)
++ return False
++ elif e.errno in (errno.EACCES, getattr(errno, 'WSAEACCES', errno.EACCES)):
++ self.log.warning(_("Permission to listen on sock %s denied") % self.sock)
++ return False
++ else:
++ raise
++ else:
++ return True
++
++ def _bind_http_server_tcp(self):
+ success = None
+ for port in random_ports(self.port, self.port_retries+1):
+ try:
+@@ -1418,10 +1520,11 @@
+ self.port = port
+ success = True
+ break
+- if not success:
+- self.log.critical(_('ERROR: the notebook server could not be started because '
+- 'no available port could be found.'))
+- self.exit(1)
++ return success
++
++ def _concat_token(self, url):
++ token = self.token if self._token_generated else '...'
++ return url_concat(url, {'token': token})
+
+ @property
+ def display_url(self):
+@@ -1429,26 +1532,33 @@
+ url = self.custom_display_url
+ if not url.endswith('/'):
+ url += '/'
++ elif self.sock:
++ url = self._unix_sock_url()
+ else:
+ if self.ip in ('', '0.0.0.0'):
+ ip = "(%s or 127.0.0.1)" % socket.gethostname()
+ else:
+ ip = self.ip
+- url = self._url(ip)
+- if self.token:
+- # Don't log full token if it came from config
+- token = self.token if self._token_generated else '...'
+- url = url_concat(url, {'token': token})
++ url = self._tcp_url(ip)
++ if self.token and not self.sock:
++ url = self._concat_token(url)
++ url += '\n or %s' % self._concat_token(self._tcp_url('127.0.0.1'))
+ return url
+
+ @property
+ def connection_url(self):
+- ip = self.ip if self.ip else 'localhost'
+- return self._url(ip)
++ if self.sock:
++ return self._unix_sock_url()
++ else:
++ ip = self.ip if self.ip else 'localhost'
++ return self._tcp_url(ip)
+
+- def _url(self, ip):
++ def _unix_sock_url(self, token=None):
++ return '%s%s' % (urlencode_unix_socket(self.sock), self.base_url)
++
++ def _tcp_url(self, ip, port=None):
+ proto = 'https' if self.certfile else 'http'
+- return "%s://%s:%i%s" % (proto, ip, self.port, self.base_url)
++ return "%s://%s:%i%s" % (proto, ip, port or self.port, self.base_url)
+
+ def init_terminals(self):
+ if not self.terminals_enabled:
+@@ -1660,6 +1770,7 @@
+ return {'url': self.connection_url,
+ 'hostname': self.ip if self.ip else 'localhost',
+ 'port': self.port,
++ 'sock': self.sock,
+ 'secure': bool(self.certfile),
+ 'base_url': self.base_url,
+ 'token': self.token,
+@@ -1780,19 +1891,31 @@
+ self.write_server_info_file()
+ self.write_browser_open_file()
+
+- if self.open_browser or self.file_to_run:
++ if (self.open_browser or self.file_to_run) and not self.sock:
+ self.launch_browser()
+
+ if self.token and self._token_generated:
+ # log full URL with generated token, so there's a copy/pasteable link
+ # with auth info.
+- self.log.critical('\n'.join([
+- '\n',
+- 'To access the notebook, open this file in a browser:',
+- ' %s' % urljoin('file:', pathname2url(self.browser_open_file)),
+- 'Or copy and paste one of these URLs:',
+- ' %s' % self.display_url,
+- ]))
++ if self.sock:
++ self.log.critical('\n'.join([
++ '\n',
++ 'Notebook is listening on %s' % self.display_url,
++ '',
++ (
++ 'UNIX sockets are not browser-connectable, but you can tunnel to '
++ 'the instance via e.g.`ssh -L 8888:%s -N user@this_host` and then '
++ 'opening e.g. %s in a browser.'
++ ) % (self.sock, self._concat_token(self._tcp_url('localhost', 8888)))
++ ]))
++ else:
++ self.log.critical('\n'.join([
++ '\n',
++ 'To access the notebook, open this file in a browser:',
++ ' %s' % urljoin('file:', pathname2url(self.browser_open_file)),
++ 'Or copy and paste one of these URLs:',
++ ' %s' % self.display_url,
++ ]))
+
+ self.io_loop = ioloop.IOLoop.current()
+ if sys.platform.startswith('win'):
+diff -Naur notebook-5.7.4/notebook/tests/launchnotebook.py notebook-5.7.4.patched/notebook/tests/launchnotebook.py
+--- notebook-5.7.4/notebook/tests/launchnotebook.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/notebook/tests/launchnotebook.py 2019-11-18 12:22:25.931074384 +0100
+@@ -19,12 +19,13 @@
+ from mock import patch #py2
+
+ import requests
++import requests_unixsocket
+ from tornado.ioloop import IOLoop
+ import zmq
+
+ import jupyter_core.paths
+ from traitlets.config import Config
+-from ..notebookapp import NotebookApp
++from ..notebookapp import NotebookApp, urlencode_unix_socket
+ from ..utils import url_path_join
+ from ipython_genutils.tempdir import TemporaryDirectory
+
+@@ -55,7 +56,7 @@
+ url = cls.base_url() + 'api/contents'
+ for _ in range(int(MAX_WAITTIME/POLL_INTERVAL)):
+ try:
+- requests.get(url)
++ cls.fetch_url(url)
+ except Exception as e:
+ if not cls.notebook_thread.is_alive():
+ raise RuntimeError("The notebook server failed to start")
+@@ -79,6 +80,10 @@
+ headers['Authorization'] = 'token %s' % cls.token
+ return headers
+
++ @staticmethod
++ def fetch_url(url):
++ return requests.get(url)
++
+ @classmethod
+ def request(cls, verb, path, **kwargs):
+ """Send a request to my server
+@@ -93,6 +98,10 @@
+ return response
+
+ @classmethod
++ def get_bind_args(cls):
++ return dict(port=cls.port)
++
++ @classmethod
+ def setup_class(cls):
+ cls.tmp_dir = TemporaryDirectory()
+ def tmp(*parts):
+@@ -103,7 +112,7 @@
+ if e.errno != errno.EEXIST:
+ raise
+ return path
+-
++
+ cls.home_dir = tmp('home')
+ data_dir = cls.data_dir = tmp('data')
+ config_dir = cls.config_dir = tmp('config')
+@@ -138,8 +147,8 @@
+ if 'asyncio' in sys.modules:
+ import asyncio
+ asyncio.set_event_loop(asyncio.new_event_loop())
++ bind_args = cls.get_bind_args()
+ app = cls.notebook = NotebookApp(
+- port=cls.port,
+ port_retries=0,
+ open_browser=False,
+ config_dir=cls.config_dir,
+@@ -150,6 +159,7 @@
+ config=config,
+ allow_root=True,
+ token=cls.token,
++ **bind_args
+ )
+ # don't register signal handler during tests
+ app.init_signal = lambda : None
+@@ -197,6 +207,25 @@
+ return 'http://localhost:%i%s' % (cls.port, cls.url_prefix)
+
+
++class UNIXSocketNotebookTestBase(NotebookTestBase):
++ # Rely on `/tmp` to avoid any Linux socket length max buffer
++ # issues. Key on PID for process-wise concurrency.
++ sock = '/tmp/.notebook.%i.sock' % os.getpid()
++
++ @classmethod
++ def get_bind_args(cls):
++ return dict(sock=cls.sock)
++
++ @classmethod
++ def base_url(cls):
++ return '%s%s' % (urlencode_unix_socket(cls.sock), cls.url_prefix)
++
++ @staticmethod
++ def fetch_url(url):
++ with requests_unixsocket.monkeypatch():
++ return requests.get(url)
++
++
+ @contextmanager
+ def assert_http_error(status, msg=None):
+ try:
+diff -Naur notebook-5.7.4/notebook/tests/test_notebookapp_integration.py notebook-5.7.4.patched/notebook/tests/test_notebookapp_integration.py
+--- notebook-5.7.4/notebook/tests/test_notebookapp_integration.py 1970-01-01 01:00:00.000000000 +0100
++++ notebook-5.7.4.patched/notebook/tests/test_notebookapp_integration.py 2019-11-18 12:16:58.319065025 +0100
+@@ -0,0 +1,39 @@
++import os
++import stat
++import subprocess
++import time
++
++from ipython_genutils.testing.decorators import skip_win32
++
++from .launchnotebook import UNIXSocketNotebookTestBase
++from ..utils import urlencode_unix_socket, urlencode_unix_socket_path
++
++
++@skip_win32
++def test_shutdown_sock_server_integration():
++ sock = UNIXSocketNotebookTestBase.sock
++ url = urlencode_unix_socket(sock)
++ encoded_sock_path = urlencode_unix_socket_path(sock)
++
++ p = subprocess.Popen(
++ ['jupyter', 'notebook', '--no-browser', '--sock=%s' % sock],
++ stdout=subprocess.PIPE, stderr=subprocess.PIPE
++ )
++
++ for line in iter(p.stderr.readline, b''):
++ if url.encode() in line:
++ complete = True
++ break
++
++ assert complete, 'did not find socket URL in stdout when launching notebook'
++
++ assert encoded_sock_path.encode() in subprocess.check_output(['jupyter', 'notebook', 'list'])
++
++ # Ensure default umask is properly applied.
++ assert stat.S_IMODE(os.lstat(sock).st_mode) == 0o600
++
++ subprocess.check_output(['jupyter', 'notebook', 'stop', sock])
++
++ assert encoded_sock_path.encode() not in subprocess.check_output(['jupyter', 'notebook', 'list'])
++
++ p.wait()
+diff -Naur notebook-5.7.4/notebook/tests/test_notebookapp.py notebook-5.7.4.patched/notebook/tests/test_notebookapp.py
+--- notebook-5.7.4/notebook/tests/test_notebookapp.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/notebook/tests/test_notebookapp.py 2019-11-18 12:16:58.319065025 +0100
+@@ -25,7 +25,7 @@
+ from notebook.auth.security import passwd_check
+ NotebookApp = notebookapp.NotebookApp
+
+-from .launchnotebook import NotebookTestBase
++from .launchnotebook import NotebookTestBase, UNIXSocketNotebookTestBase
+
+
+ def test_help_output():
+@@ -192,3 +192,15 @@
+ servers = list(notebookapp.list_running_servers())
+ assert len(servers) >= 1
+ assert self.port in {info['port'] for info in servers}
++
++
++# UNIX sockets aren't available on Windows.
++if not sys.platform.startswith('win'):
++ class NotebookUnixSocketTests(UNIXSocketNotebookTestBase):
++ def test_run(self):
++ self.fetch_url(self.base_url() + 'api/contents')
++
++ def test_list_running_sock_servers(self):
++ servers = list(notebookapp.list_running_servers())
++ assert len(servers) >= 1
++ assert self.sock in {info['sock'] for info in servers}
+diff -Naur notebook-5.7.4/notebook/utils.py notebook-5.7.4.patched/notebook/utils.py
+--- notebook-5.7.4/notebook/utils.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/notebook/utils.py 2019-11-18 12:23:05.231075507 +0100
+@@ -306,3 +306,18 @@
+ check_pid = _check_pid_win32
+ else:
+ check_pid = _check_pid_posix
++
++def urlencode_unix_socket_path(socket_path):
++ """Encodes a UNIX socket path string from a socket path for the `http+unix` URI form."""
++ return socket_path.replace('/', '%2F')
++
++
++def urldecode_unix_socket_path(socket_path):
++ """Decodes a UNIX sock path string from an encoded sock path for the `http+unix` URI form."""
++ return socket_path.replace('%2F', '/')
++
++
++def urlencode_unix_socket(socket_path):
++ """Encodes a UNIX socket URL from a socket path for the `http+unix` URI form."""
++ return 'http+unix://%s' % urlencode_unix_socket_path(socket_path)
++
+diff -Naur notebook-5.7.4/setup.py notebook-5.7.4.patched/setup.py
+--- notebook-5.7.4/setup.py 2018-12-17 11:01:51.000000000 +0100
++++ notebook-5.7.4.patched/setup.py 2019-11-18 12:23:33.851076325 +0100
+@@ -98,7 +98,8 @@
+ ':python_version == "2.7"': ['ipaddress'],
+ 'test:python_version == "2.7"': ['mock'],
+ 'test': ['nose', 'coverage', 'requests', 'nose_warnings_filters',
+- 'nbval', 'nose-exclude', 'selenium'],
++ 'nbval', 'nose-exclude', 'selenium',
++ 'requests-unixsocket'],
+ 'test:sys_platform == "win32"': ['nose-exclude'],
+ },
+ entry_points = {
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 232841ccb1..4263a33c6b 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -7811,7 +7811,8 @@ convert an @code{.ipynb} notebook file into various static formats including:
(uri (pypi-uri "notebook" version))
(sha256
(base32
- "0jm7324mbxljmn9hgapj66q7swyz5ai92blmr0jpcy0h80x6f26r"))))
+ "0jm7324mbxljmn9hgapj66q7swyz5ai92blmr0jpcy0h80x6f26r"))
+ (patches (search-patches "jupyter-unix-domain-sockets-4835-5.7.4.patch"))))
(build-system python-build-system)
(arguments
`(#:phases
@@ -7834,7 +7835,8 @@ convert an @code{.ipynb} notebook file into various static formats including:
("python-nbconvert" ,python-nbconvert)
("python-prometheus-client" ,python-prometheus-client)
("python-send2trash" ,python-send2trash)
- ("python-terminado" ,python-terminado)))
+ ("python-terminado" ,python-terminado)
+ ("python-requests-unixsocket" ,python-requests-unixsocket)))
(native-inputs
`(("python-nose" ,python-nose)
("python-sphinx" ,python-sphinx)
@@ -17711,3 +17713,32 @@ applications")
based on the Jupyter Notebook and Architecture.")
(license license:bsd-3)))
+(define-public python-requests-unixsocket
+ (package
+ (name "python-requests-unixsocket")
+ (version "0.2.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "requests-unixsocket" version))
+ (sha256
+ (base32
+ "1sn12y4fw1qki5gxy9wg45gmdrxhrndwfndfjxhpiky3mwh1lp4y"))))
+ (build-system python-build-system)
+ (native-inputs
+ ;; pbr is required for setup only
+ `(("python-pbr" ,python-pbr)))
+ (propagated-inputs
+ `(("python-requests" ,python-requests)
+ ("python-urllib3" ,python-urllib3)))
+ (arguments
+ ;; tests depend on very specific package version, which are not available in guix
+ '(#:tests? #f))
+ (home-page
+ "https://github.com/msabramo/requests-unixsocket")
+ (synopsis
+ "Use requests to talk HTTP via a UNIX domain socket")
+ (description
+ "Use requests to talk HTTP via a UNIX domain socket")
+ (license license:asl2.0)))
+
--
2.20.1
next reply other threads:[~2020-02-24 10:19 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-24 10:18 Lars-Dominik Braun [this message]
2020-03-26 22:55 ` [bug#39765] Add package JupyterLab Ludovic Courtès
2020-03-27 7:30 ` Lars-Dominik Braun
2020-03-29 14:37 ` Ludovic Courtès
2020-03-30 6:10 ` Lars-Dominik Braun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200224101810.GA9010@zpidnp36 \
--to=ldb@leibniz-psychology.org \
--cc=39765@debbugs.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).