unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
From: Leo Prikler <leo.prikler@student.tugraz.at>
To: "Ludovic Courtès" <ludo@gnu.org>
Cc: julien lepiller <roptat@lepiller.eu>, 38649@debbugs.gnu.org
Subject: [bug#38649] [PATCH] Parallelize `guix package`
Date: Tue, 17 Dec 2019 16:19:34 +0100	[thread overview]
Message-ID: <3d0ca2a8b59dd99e15b55033bc89b2e21aa49814.camel@student.tugraz.at> (raw)
In-Reply-To: <87tv5zrpjp.fsf@gnu.org>

Hi Ludo’,

Am Dienstag, den 17.12.2019, 15:32 +0100 schrieb Ludovic Courtès:
> Hi Leo,
> 
> (Cc: Julien, who worked on this part.)
> 
> Leo Prikler <leo.prikler@student.tugraz.at> skribis:
> 
> > Yesterday I had an interesting conversation on IRC about the
> > behaviour
> > of multiple `guix package` processes running in parallel. 
> > Specifically, when two transactions target the same profile
> > (usually
> > /var/guix/profiles/per-user/$USER/guix-profile) at the same time,
> > one
> > of them will fail to claim the lock and abort.  0001 makes it so
> > that
> > the process waits for the lock.  0002 makes it so that packages
> > specified via -i can be built in parallel.
> 
> I actually like the current behavior, FWIW.  Julien came up with this
> locking mostly so that people do not inadvertently attempt to perform
> several operations concurrently.
Fair enough and that is an improvement over non-locking behaviour,
where one could spawn multiple profile generations from one, neither of
which is complete.  Perhaps my attempt at doing this in a somewhat
controlled manner is equally harmful, but I will still try my best
arguing for it, as I believe it can make a positive impact.

> The key word here is “inadvertently”: IMO, there’s no reason to run
> multiple ‘guix package’ on the same profile concurrently.  With a
> wait-for-lock policy, the result would be non-deterministic: you
> cannot
> tell which one of the two processes will complete first.
> 
> WDYT?
I think the current policy is wait-for-lock deferred to the user.  The
user has to let the first task complete before they can start the
second.  In this setup, the user can simply launch the setup and trust,
that it will complete later while taking into account the changes the
first one has made.

Let's talk about three classes of operations – installations, removals
and upgrades – and their interactions.  I will not take into account
roll-back, switch-generation and delete-generation, as it is
nonsensical to perform these in parallel to any other action.  Perhaps
we could check for their presence first and acquire the lock with no-
wait semantics in that case.

- any operation on different packages: Either succeeds first and the
other builds on the profile it generates. As there is no collision in
the packages themselves, there will be no harm.
- install same package twice: Either succeeds first, the other will be
a no-op.
- install vs. remove same package: Non-deterministic, but why would you
do that?
- install vs. upgrade same package: Upgrade will be a no-op in either
case.
- remove vs. upgrade same package: Upgrade may inadvertently upgrade
the old package if it happens to come first, but in the final package
it will be removed either way. 

Of course, any operation can also fail midway due to some step not
succeeding.  In that case it would be as if one had issued the other
command right after that, which may perhaps not be what one wanted to
do (assuming I install package A, and some guide suggests to also build
related, but not dependency-connected package B, so I end up installing
B without A).  However, such cases can easily be fixed by either
installing a fixed version of A later, using B on its own if it can be,
or rolling back.

Of course, both solutions are flawed in the way that they assume user
intent either way.  Perhaps a better one would be to let the user
specify whether they want to wait or not through a command line
parameter, using the current behaviour as the default approach.

WDYT?

Regards,
Leo

  reply	other threads:[~2019-12-17 15:20 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-17 14:18 [bug#38649] [PATCH] Parallelize `guix package` Leo Prikler
2019-12-17 14:20 ` Brett Gilio
2019-12-17 14:34   ` Leo Prikler
2019-12-17 14:38     ` Brett Gilio
2019-12-17 14:32 ` Ludovic Courtès
2019-12-17 15:19   ` Leo Prikler [this message]
2019-12-17 15:50     ` Julien Lepiller
2019-12-17 16:16       ` Leo Prikler
2019-12-18 14:37     ` Ludovic Courtès
2021-05-11 14:24       ` Leo Prikler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3d0ca2a8b59dd99e15b55033bc89b2e21aa49814.camel@student.tugraz.at \
    --to=leo.prikler@student.tugraz.at \
    --cc=38649@debbugs.gnu.org \
    --cc=ludo@gnu.org \
    --cc=roptat@lepiller.eu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).