From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id kNKzL2gKsl92IQAA0tVLHw (envelope-from ) for ; Mon, 16 Nov 2020 05:13:12 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id 8IKsK2gKsl+MCAAA1q6Kng (envelope-from ) for ; Mon, 16 Nov 2020 05:13:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 241A4940657 for ; Mon, 16 Nov 2020 05:13:11 +0000 (UTC) Received: from localhost ([::1]:48146 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1keWpB-0006wx-Av for larch@yhetil.org; Mon, 16 Nov 2020 00:13:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:52004) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1keWp4-0006w5-AH for guix-patches@gnu.org; Mon, 16 Nov 2020 00:13:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:43255) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1keWp4-0000Zv-09 for guix-patches@gnu.org; Mon, 16 Nov 2020 00:13:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1keWp3-0003MK-Ry for guix-patches@gnu.org; Mon, 16 Nov 2020 00:13:01 -0500 Subject: bug#44567: [PATCH] publish: Improve HTTP performance when not using --cache. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-To: guix-patches@gnu.org Resent-Date: Mon, 16 Nov 2020 05:13:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: cc-closed 44567 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 44567-done@debbugs.gnu.org Mail-Followup-To: 44567@debbugs.gnu.org, maxim.cournoyer@gmail.com, maxim.cournoyer@gmail.com Received: via spool by 44567-done@debbugs.gnu.org id=D44567.160550357112892 (code D ref 44567); Mon, 16 Nov 2020 05:13:01 +0000 Received: (at 44567-done) by debbugs.gnu.org; 16 Nov 2020 05:12:51 +0000 Received: from localhost ([127.0.0.1]:54799 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1keWot-0003Lr-2g for submit@debbugs.gnu.org; Mon, 16 Nov 2020 00:12:51 -0500 Received: from mail-il1-f194.google.com ([209.85.166.194]:40026) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1keWop-0003Lc-Of for 44567-done@debbugs.gnu.org; Mon, 16 Nov 2020 00:12:49 -0500 Received: by mail-il1-f194.google.com with SMTP id n5so13960966ile.7 for <44567-done@debbugs.gnu.org>; Sun, 15 Nov 2020 21:12:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=18UNXV4/bLkfxc1521p+JC5Q8qUPWY16NGtW39GUvA0=; b=Si+1TBxIUOCvWW74bKom1ykmzV9WhONretochbCaRqaSLI3u07vMGN7SVqV5TXj8/e 7dSKJOCexAWPjdozEPHyaphZbcYWIzrk3xTP2oWVLYMv1I5lPK5AZTJTNtFZXFd0ZRAN yAVrtnB5yFRCdKNWIgdO62Mtsjpz4jw++/2FaNWMSF43GRH3Czqev3miUaIzPZ3y5lqR qpDrheI34EfaenyP/Nc6pJP3idwoqyP2JSE7Q4m12bom6qy8Dm1GHXbhP2EzTY/SlfCG Qf7HaaLuiF50qpZB8f6zJRWdfboUs5tvlfX5DGhTTJRERpil2cZxrUOJaGauDIYnXTX8 eSJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=18UNXV4/bLkfxc1521p+JC5Q8qUPWY16NGtW39GUvA0=; b=tdDDRPmtZBmbDP7k85J9tO/QlWY6qEwYH+ILNXeFTGHrTFbHiB82gttRzzrq5SEaDb WfoHx8FghpoR1EzE0J2izFnMUJk0J4DN4wBHtUEzTO5Jth3dCuEDsJ3LFSwxdZ86XREU LRs3wN0WYn/c195LR3ISmn8wZdp2XNEoAvIR23YyMTlC0CGQI6zByLvwHYHXMkOfiyYN WJs/H4RbT7SC8J/JlZlittkkCJ2VV5exZ917NuPeGugnoUd3QJCqNFRLNtdj4uEqiEg8 NBN0/MtBnIB31AmlZT2FjFdC8ax2Q0m3MVO26Tp4rma262pbIO7nZ6SDHCPFQKiJx09p L8sQ== X-Gm-Message-State: AOAM533zhHU7LunE9dJdKZk6hJT0JkugqY3rOEhJ5YZ6A2hS6M9Q7WWf k0fzOGU0OaFhIanZ+tVGG4ON5rcRmyF+IA== X-Google-Smtp-Source: ABdhPJwMEtwR0wsJBZoXvVjg1Fk67U/5oBA1krxK9KvZXAaHsUb9kPx9kfHV5qnixgNETk2KjSh3sg== X-Received: by 2002:a05:6e02:12ab:: with SMTP id f11mr6996136ilr.89.1605503561632; Sun, 15 Nov 2020 21:12:41 -0800 (PST) Received: from hurd (dsl-10-128-91.b2b2c.ca. [72.10.128.91]) by smtp.gmail.com with ESMTPSA id f8sm8604705ioc.24.2020.11.15.21.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Nov 2020 21:12:41 -0800 (PST) From: Maxim Cournoyer References: <20201111035727.11184-1-maxim.cournoyer@gmail.com> <87361dcgzc.fsf@gnu.org> Date: Mon, 16 Nov 2020 00:12:40 -0500 In-Reply-To: <87361dcgzc.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Fri, 13 Nov 2020 18:19:03 +0100") Message-ID: <87mtzhdgvr.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: ns3122888.ip-94-23-21.eu Authentication-Results: aspmx1.migadu.com; dkim=fail (headers rsa verify failed) header.d=gmail.com header.s=20161025 header.b=Si+1TBxI; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: JJnZloKYOxvK Hi Ludovic, Ludovic Court=C3=A8s writes: > Hi Maxim, > > Maxim Cournoyer skribis: > >> This change harmonizes the way we configure the buffer sizes and the soc= ket >> options, so that we don't forget to change it at one place like it happe= ned in >> commit 5e3d169945935b53325e6b738a307ba286751259. Using a greater socket >> buffer size reportedly improves throughput tenfold. >> >> The solution was found by Ludovic Court=C3=A8s. >> >> * guix/scripts/publish.scm (%default-buffer-size) >> (%default-socket-options): New variables. >> * guix/scripts/publish.scm (configure-socket): New procedure. >> (compress-nar): Use %default-buffer-size for the buffer size, increased = from >> 128 to 208 KiB. >> (nar-response-port): Likewise, increased from 64 to 208 KiB. >> (http-write): Use configure-socket to set socket options. >> (open-server-socket): Likewise. > > Apologies for not noticing this before pushing > 1cbda46d4aae5ba9bd89a1837f0d81a29653ed7b. What you propose here is > nicer. No worries! >> +(define %default-buffer-size >> + (* 208 1024)) > > Why 208? Did you notice a difference compared to 128KiB? (I=E2=80=99m f= ine > either way, just wondering.) I didn't observe any meaningful difference, it perhaps felt more 'steady'. The value comes from what Linux, the kernel, configures as a max value by default (see 'man tcp' and 'cat /proc/sys/net/core/wmem_max'). > Perhaps add a comment as to what buffer we=E2=80=99re talking about. Done. >> +(define %default-socket-options > > Maybe add: ;; List of options passed to 'setsockopt' when transmitting fi= les. > >> + (list (list SO_SNDBUF %default-buffer-size))) > >> (make-gzip-output-port (response-port response) >> #:level level >> - #:buffer-size (* 64 1024))) >> + #:buffer-size %default-buffer-size)) > > Does the gzip buffer size have to match the TCP buffer size? I=E2=80=99d= say > not necessarily, so perhaps we should have a separate variable for the > gzip buffer size (or keep it as is). It doesn't, but if we're going to pick some value, it seems the network one (SO_SNDBUF) should be the lowest common denominator. Since we're augmenting the previous value, I've just all made them agree. Another variable should be created the day we have a rationale to give them different values, I'd say. > That=E2=80=99s it, thank you! Thanks for the review! Pushed to the 1.2.0-version branch. Maxim