bug#41702: `guix environment` performance issues

["Ludovic Courtès" <ludo@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 2565 bytes --]

Hi,

Lars-Dominik Braun <ldb@leibniz-psychology.org> skribis:

> That would have been my best guess too, but it does not seem to be the biggest
> problem right now. Looking at the numbers again (both patches applied) with the
> attached manifest[1], I see that:
>
> ---snip---
> Local UNIX socket with and without --no-grafts:
>     N           Min           Max        Median           Avg        Stddev
> x  10          6.07          6.35         6.145          6.16    0.08232726
> +  10         17.47         17.89        17.545        17.602    0.14351152
> Difference at 99.0% confidence
>         11.442 +/- 0.150576
>         185.747% +/- 4.07133%
>
> Local UNIX socket vs. guix://localhost transport:
>     N           Min           Max        Median           Avg        Stddev
> x  10         17.47         17.89        17.545        17.602    0.14351152
> +  10         17.43          18.1         17.61        17.642    0.20131788
> No difference proven at 99.0% confidence
>
> Local UNIX socket vs ssh://localhost transport:
>     N           Min           Max        Median           Avg        Stddev
> x  10         17.47         17.89        17.545        17.602    0.14351152
> +  10         33.46         35.27        34.315        34.359    0.53873205
> Difference at 99.0% confidence
>         16.757 +/- 0.5074
>         95.1994% +/- 3.13957%
> ---snap---
>
> So I would conclude:
>
> 1) Grafting still takes a lot of time and needs more work

Yes, I noticed that there’s still redundant work being done, so we may
need global caching in addition to what
58bb833365db4e8934a386497d5b00a063cfd27d did.

> 2) Linux optimizes localhost networking pretty well
> 3) Our SSH transport is terribly slow

Yes.

The patch below is a noticeable improvement for me.  On my laptop,

  GUIX_DAEMON_SOCKET=ssh://localhost ./pre-inst-env guix build libreoffice -n 

goes from 5.8s to 3.3s.  It just does the same thing as we do for
guix://.

Could you check what results it gives you?

> So, I guess it would make sense for me to look at the SSH transport itself
> again and see if there are any other low-hanging fruit. Not sure how much I can
> help with profiling guile/guix itself. A different/better RPC protocol is
> probably GSoC/v2.0-worthy?

There’s a project to rewrite the daemon in Scheme, started by Caleb
Ristvedt (reepa) as part of GSoC a few years ago.  This could be an
opportunity to add a new version of the protocol that would support
pipelining.

Thanks,
Ludo’.


[-- Attachment #2: Type: text/x-patch, Size: 3278 bytes --]

diff --git a/guix/ssh.scm b/guix/ssh.scm
index 2d7ca7d01d..a9312b7c8c 100644
--- a/guix/ssh.scm
+++ b/guix/ssh.scm
@@ -98,7 +98,7 @@ actual key does not match."
                                 key type))))))))
 
 (define* (open-ssh-session host #:key user port identity
-                           host-key
+                           host-key open-connection
                            (compression %compression)
                            (timeout 3600))
   "Open an SSH session for HOST and return it.  IDENTITY specifies the file
@@ -110,6 +110,10 @@ When HOST-KEY is true, it must be a string like \"ssh-ed25519 AAAAC3Nz…
 root@example.org\"; the server is authenticated and an error is raised if its
 host key is different from HOST-KEY.
 
+When OPEN-CONNECTION is true, it must be a two-argument procedure; it is
+passed HOST and PORT and must return a socket (a file port).  When
+OPEN-CONNECTION is false, Guile-SSH takes care of opening the connection.
+
 Install TIMEOUT as the maximum time in seconds after which a read or write
 operation on a channel of the returned session is considered as failing.
 
@@ -134,6 +138,13 @@ Throw an error on failure."
     ;; Honor ~/.ssh/config.
     (session-parse-config! session)
 
+    (when open-connection
+      (let* ((sock (open-connection host port)))
+        ;; Since 'session-set!' doesn't increase the revealed count of SOCK
+        ;; (as of Guile-SSH 0.12.0), do it ourselves.
+        (set-port-revealed! sock 1)
+        (session-set! session 'fd sock)))
+
     (match (connect! session)
       ('ok
        (if host-key
diff --git a/guix/store/ssh.scm b/guix/store/ssh.scm
index 09c0832505..d07f0d7467 100644
--- a/guix/store/ssh.scm
+++ b/guix/store/ssh.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2017 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2017, 2020 Ludovic Courtès <ludo@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -19,6 +19,7 @@
 (define-module (guix store ssh)
   #:use-module (guix ssh)
   #:use-module (web uri)
+  #:use-module (ssh session)
   #:export (connect-to-daemon))
 
 ;;; Commentary:
@@ -29,11 +30,27 @@
 ;;;
 ;;; End:
 
+(define (open-connection host port)
+  "Open a connection to HOST and PORT.  Use the standard SSH port if PORT is
+false."
+  (let* ((lst  (getaddrinfo host
+                            (if port (number->string port) "ssh")
+                            (if port AI_NUMERICSERV 0)))
+         (addr (addrinfo:addr (car lst)))
+         (sock (socket (sockaddr:fam addr) SOCK_STREAM 0)))
+    ;; Setting this option makes a dramatic difference because it avoids the
+    ;; "ACK delay" on our RPC messages.
+    (setsockopt sock IPPROTO_TCP TCP_NODELAY 1)
+
+    (connect sock addr)
+    sock))
+
 (define (connect-to-daemon uri)
   "Connect to the SSH daemon at URI, a URI object with the 'ssh' scheme."
   (remote-daemon-channel
    (open-ssh-session (uri-host uri)
                      #:port (or (uri-port uri) 22)
-                     #:user (uri-userinfo uri))))
+                     #:user (uri-userinfo uri)
+                     #:open-connection open-connection)))
 
 ;;; ssh.scm ends here