From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:45560) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihFWz-0002uw-AE for guix-patches@gnu.org; Tue, 17 Dec 2019 11:17:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ihFWx-0006ja-73 for guix-patches@gnu.org; Tue, 17 Dec 2019 11:17:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:36298) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ihFWw-0006hk-3O for guix-patches@gnu.org; Tue, 17 Dec 2019 11:17:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ihFWv-0000CI-VY for guix-patches@gnu.org; Tue, 17 Dec 2019 11:17:01 -0500 Subject: [bug#38649] [PATCH] Parallelize `guix package` Resent-Message-ID: Message-ID: <303cfa9fb7484874e028e55bb0fb82a9387207a7.camel@student.tugraz.at> From: Leo Prikler Date: Tue, 17 Dec 2019 17:16:03 +0100 In-Reply-To: <09CEFC5C-85EB-4B43-BADD-C4D1920E656A@lepiller.eu> References: <87tv5zrpjp.fsf@gnu.org> <3d0ca2a8b59dd99e15b55033bc89b2e21aa49814.camel@student.tugraz.at> <09CEFC5C-85EB-4B43-BADD-C4D1920E656A@lepiller.eu> Content-Type: text/plain; charset="UTF-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: julien@lepiller.eu, 38649@debbugs.gnu.org, ludo@gnu.org Cc: roptat@lepiller.eu Am Dienstag, den 17.12.2019, 16:50 +0100 schrieb Julien Lepiller: > Le 17 décembre 2019 16:19:34 GMT+01:00, Leo Prikler < > leo.prikler@student.tugraz.at> a écrit : > > Of course, any operation can also fail midway due to some step not > > succeeding. In that case it would be as if one had issued the > > other > > command right after that, which may perhaps not be what one wanted > > to > > do (assuming I install package A, and some guide suggests to also > > build > > related, but not dependency-connected package B, so I end up > > installing > > B without A). However, such cases can easily be fixed by either > > installing a fixed version of A later, using B on its own if it can > > be, > > or rolling back. > > > > Of course, both solutions are flawed in the way that they assume > > user > > intent either way. Perhaps a better one would be to let the user > > specify whether they want to wait or not through a command line > > parameter, using the current behaviour as the default approach. > > > > WDYT? > > I might be missing something. Guix install etc act on a "hidden" > descripcion of the profile. Tgey take the current profile, modify it > as specified (adding a package, renovinh another or upgrading some). > When you run two guix package in parallel, they both work on the same > profile, which creates unexpected results. That's why the lock is claimed first. This way, the second process acts on the profile that the first generated. I've tested this by installing cowsay in parallel to lolcat, but it should work for bigger packages in much the same way. > The expectation behind tge lock is that users will cancel tge ocher > command and fix it before re-running it (e.g. instead of guix install > foo & guix install bar, run guix install foo bar). That is perhaps a reasonable expectation in most cases, but may be annoying in others. Take any package with an absurdly long build time (e.g. icecat) and then think "Oh, but I also wanted this" while it is building. Now you have to either actively wait for icecat to complete or stop it, add the other package and suffer the same build time again, (whereas in the other way, you can wait for icecat to complete and still launch a second process). With the parallel builds of 0002, thing become even better, as you can use bar even before foo is completed in case it manages to grab the lock first. With the long build of icecat against a package with a relatively short build, this could very well be the case and might end up being a game changer. Of course, one could abuse ad-hoc environments as well while waiting for the first process to finish, but I don't think that's how people running into this problem expect to be solving it (especially if they do want both foo and bar in their profiles). Regards, Leo