From mboxrd@z Thu Jan 1 00:00:00 1970 From: Leo Famulari Subject: Substituting in parallel to save time? Date: Wed, 7 Nov 2018 16:26:24 -0500 Message-ID: <20181107212624.GA31229@jasmine.lan> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="jI8keyz6grp/JLjh" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:44952) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gKVLf-0008Ke-Uf for guix-devel@gnu.org; Wed, 07 Nov 2018 16:26:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gKVLc-0005Ku-L6 for guix-devel@gnu.org; Wed, 07 Nov 2018 16:26:51 -0500 Received: from out5-smtp.messagingengine.com ([66.111.4.29]:55949) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gKVLc-0004zL-9s for guix-devel@gnu.org; Wed, 07 Nov 2018 16:26:48 -0500 Received: from localhost (c-76-124-202-137.hsd1.pa.comcast.net [76.124.202.137]) by mail.messagingengine.com (Postfix) with ESMTPA id 00F35E427A for ; Wed, 7 Nov 2018 16:26:27 -0500 (EST) Content-Disposition: inline List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: guix-devel@gnu.org --jI8keyz6grp/JLjh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Substitution honors the --max-jobs argument to the guix-daemon, which means that, by default, substitutes are downloaded in sequence, one at a time. I noticed that in many cases, it takes longer to set up the substitution than to actually download the substitute on my 30 Mbps downlink. The effect is that it might take a few minutes to to download a few dozen small substitutes, which may only total 5 megabytes of data, even though my connection to the substitute server is fast enough that it should only take a few seconds. So, it benefits me to increase max-jobs for substitute downloads. However, I rarely want to actually build packages in parallel; one package build may fully load my system. Currently, is it possible to distinguish between substitute downloads and builds? Maybe it would be an overall performance win to download substitutes in parallel, in tune with parallel-job-count. --jI8keyz6grp/JLjh Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlvjWIAACgkQJkb6MLrK fwh34A//U7dSYJB60tEvEw8z7AQ8ZA8keo5inls5ee4NDX0V1xu4wea3zDNHUw3Y 47/ZYDsvG2/RQJ58RyByw7dSEwChl2dpAWpuW8+lsgg1zKTqrDI9+pBcaNPiz9SS ItlW3SzwTucUOYgEMAIzMr94kSd0MBrIHrJtDZ81ACp0LTtLw2JD1vrNVY28h/Ud xoFWSXx/VaGWa8LAHyUy/IIOxrZnLPBKMqFLQrM3KufwuRW5t3gX50jKdnu0FN3C TYlgKHlXnQT2ssh9HHdFMBt+/RTyN+zo5rcUKj9g0v6wAq71m02ScoA/cjDvuHgZ 0nRYerzV3qy0XbH3Tu8rAQzqalZu86kZiIXSZhvLSexuRSQ65BuYU8O5mJVlhiei CLzfNYddLMeQVispWOeFouOiH7wSL+h0GBRS+qYz5Xha9Sdw36gSfl3UIpw4Wy2r 98RFVFPDU1xD0OkjouXNludgBMvisjPn0NtILSlCSBtZRMZTeSJAvkBV+qKJlKtz 91MDRcrCXYAcKYMbU2x4h0Hqv+/9UvfOFNlup3xZXFH93Y/mupAO9zXWYhiX8xWX AsIw2+iaxWw/XjEpnqwsRO3d79HUHErMGVEurxVl6NjKvOh0Y6cZLkirgUlPcy8I tKN4Cz0z8RO6W9fwNXkqEd1BStrStC59Ingftbr7MPQ+kmnJFR8= =ATdY -----END PGP SIGNATURE----- --jI8keyz6grp/JLjh--