unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / code / Atom feed
* bug#38411: HTTP pipelining of narinfo requests broken for https://ci.guix.gnu.org
@ 2019-11-28  9:22 Ludovic Courtès
  2019-11-28  9:59 ` Ludovic Courtès
                   ` (3 more replies)
  0 siblings, 4 replies; 7+ messages in thread
From: Ludovic Courtès @ 2019-11-28  9:22 UTC (permalink / raw)
  To: bug-Guix

[-- Attachment #1: Type: text/plain, Size: 318 bytes --]

Starting from a couple of days ago (it seems; roughly around the same
time berlin hit ENOSPC), people have been experiencing issues during the
“updating list of substitutes” phase from https://ci.guix.gnu.org, where
they’d get an ugly backtrace when they’re at 80% or so.

Here’s a small reproducer:


[-- Attachment #2: the reproducer --]
[-- Type: text/plain, Size: 1556 bytes --]

(use-modules (guix scripts substitute)
             (srfi srfi-1)
             (srfi srfi-26)
             (web uri)
             (web request)
             (web response)
             (rnrs io ports))

(define http-multiple-get
  (@@ (guix scripts substitute) http-multiple-get))

(define %base-url
  "https://berlin.guix.gnu.org")

(define %request-count
  ;; Number of requests to send.  Starts failing at 85 (that is, we don't
  ;; receive the 85th response).
  200)

(http-multiple-get (string->uri %base-url)
                   (lambda (request response port result)
                     (let ((len (or (response-content-length response)
                                    0)))
                       (pk 'resp (length result)
                           (uri-path (request-uri request)))
                       (get-bytevector-n port len)
                       (cons result result)))
                   '()
                   (unfold (cut >= <> %request-count)
                           (lambda (n)
                             (build-request
                              (string->uri
                               (string-append
                                %base-url
                                "/"
                                (string-pad (number->string n) 32 #\a)
                                ".narinfo"))
                              #:method 'GET
                              #:headers '((User-Agent . "GNU Guile"))))
                           1+
                           0)
                   #:verify-certificate? #f)

[-- Attachment #3: Type: text/plain, Size: 826 bytes --]


You’ll see that it hangs waiting for a response as soon as you pipeline
85 requests or more.

Note that:

  1. https://bayfront.guix.gnu.org doesn’t have that problem;

  2. http://ci.guix.gnu.org doesn’t have that problem;

  3. when you send 85 requests, it hangs waiting for the 85th response;
     but when you send 200 requests, it hangs waiting for the 160th
     response; so it seems it’s not just a matter of TLS record size.

I suspected something having to do with TLS record size limits, but item
#3 above may invalidate that hypothesis.

The weird thing is that we haven’t change the config of berlin in recent
days.  Berlin runs nginx 1.17.5 on openssl 1.1.1d, while bayfront runs
nginx 1.17.0 on openssl 1.0.2p.

I very much welcome any ideas you may have!

Thanks,
Ludo’.

^ permalink raw reply	[flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-12-26 18:15 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-28  9:22 bug#38411: HTTP pipelining of narinfo requests broken for https://ci.guix.gnu.org Ludovic Courtès
2019-11-28  9:59 ` Ludovic Courtès
2019-11-28 10:18   ` Ludovic Courtès
2019-11-28 10:29   ` Ludovic Courtès
2019-11-28 10:04 ` Ludovic Courtès
2019-12-25 16:50 ` Gábor Boskovits
     [not found] ` <handler.38411.D38411.157729267617189.notifdone@debbugs.gnu.org>
2019-12-26 18:14   ` bug#38411: closed (HTTP pipelining of narinfo requests broken for https://ci.guix.gnu.org) Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).