From: Phil Beadling <phil@beadling.co.uk>
To: Ricardo Wurmus <rekado@elephly.net>
Cc: "guix-devel@gnu.org" <guix-devel@gnu.org>
Subject: Re: Parallel guix builds can trample?
Date: Tue, 18 Jan 2022 12:53:48 +0000 [thread overview]
Message-ID: <CAOvsyQuPgH2JckiyS1Oougo4O3tvg0zUpKMN5+Rds51w3GF89g@mail.gmail.com> (raw)
In-Reply-To: <87k0exfj6l.fsf@beadling.co.uk>
[-- Attachment #1: Type: text/plain, Size: 3415 bytes --]
Hi all,
The issue also exists when using --with-commit - see below for a refined
test that makes it trivial to demonstrate with any package where the source
is retrieved from git.
On Tue, 18 Jan 2022 at 10:10, Phil <phil@beadling.co.uk> wrote:
>
> > Philip Beadling <phil@beadling.co.uk> writes:
>
> >No, and we probably should do, even in spite of this issue. We're going
> >to repeat the experiment now using --with-commit, to confirm behaviour
> there.
>
>
Let's try and build 2 package variations - for this experiement we don't
even have to run them at the same time:
mkdir checkout-test
cd checkout-test
mkdir cache1 cache2
XDG_CACHE_HOME=`realpath ./cache1` guix build
--with-commit=simm=*3981f60d81d3c9592d78b3e88545259713a93194
*simm
XDG_CACHE_HOME=`realpath ./cache2` guix build
--with-commit=simm=*ec1357c1fdf78a2ac1408d00c0c3e7d025ab9ade
*simm
Now look at the caches:
$ ll cache1/guix/checkouts
total 4.0K
-rw-rw-r-- 1 phil.beadling phil.beadling 10 Jan 18 12:37
last-expiry-cleanup
drwxrwxr-x 6 phil.beadling phil.beadling 162 Jan 18 12:37
lx4ipn27sk6uvu6b33kelt4tik65oianvl7b7yn5bpng7gpt4pyq
drwxrwxr-x 5 phil.beadling phil.beadling 115 Jan 18 12:37
mqqlbc4ksftu2tkbnxydosm4j6mynxsh5nmhpq3cfrofkktkhexq
drwxrwxr-x 5 phil.beadling phil.beadling 103 Jan 18 12:37
qqzuuoqxvqikivkpenqom43kzk2xr7nlntanbm6fpe7bci6kaenq
drwxrwxr-x 4 phil.beadling phil.beadling 202 Jan 18 12:37
smvbhjf46cbaxxju6fsqj7t6ujktxy5i7vhuhzpvgjt2vhid2vqq
drwxrwxr-x 5 phil.beadling phil.beadling 267 Jan 18 12:37
*xulvhb6agjrsf6x7wknutpn7wkt5i4nesk637czazvfxps357eoq*
$ ll cache2/guix/checkouts
total 4.0K
-rw-rw-r-- 1 phil.beadling phil.beadling 10 Jan 18 12:38
last-expiry-cleanup
drwxrwxr-x 6 phil.beadling phil.beadling 162 Jan 18 12:38
lx4ipn27sk6uvu6b33kelt4tik65oianvl7b7yn5bpng7gpt4pyq
drwxrwxr-x 5 phil.beadling phil.beadling 115 Jan 18 12:38
mqqlbc4ksftu2tkbnxydosm4j6mynxsh5nmhpq3cfrofkktkhexq
drwxrwxr-x 5 phil.beadling phil.beadling 103 Jan 18 12:39
qqzuuoqxvqikivkpenqom43kzk2xr7nlntanbm6fpe7bci6kaenq
drwxrwxr-x 4 phil.beadling phil.beadling 202 Jan 18 12:38
smvbhjf46cbaxxju6fsqj7t6ujktxy5i7vhuhzpvgjt2vhid2vqq
drwxrwxr-x 5 phil.beadling phil.beadling 267 Jan 18 12:38
*xulvhb6agjrsf6x7wknutpn7wkt5i4nesk637czazvfxps357eoq*
$
Despite asking for different variants of the simm package, they are both
stored under the same SHA. If they both used the same cache directory (the
default) - they'd have overwritten each other. Thus it is not safe to run
--with-commit or any other variation in parallel for the same user.
To prove I've matched the right SHA - I can back-out the original commit
ids provided to --with-commit:
$ cd
cache1/guix/checkouts/xulvhb6agjrsf6x7wknutpn7wkt5i4nesk637czazvfxps357eoq
&& git rev-parse HEAD && cd -
*3981f60d81d3c9592d78b3e88545259713a93194*
~/checkout-test
$ cd
cache2/guix/checkouts/xulvhb6agjrsf6x7wknutpn7wkt5i4nesk637czazvfxps357eoq
&& git rev-parse HEAD && cd -
*ec1357c1fdf78a2ac1408d00c0c3e7d025ab9ade*
~/checkout-test
$
Whilst my original setup was a bit of a edge-case, I think this refined
example demonstrates that the problem can occur with typical/reasonable use
of guix switches? I can even think of how this could occur without the
user running in parallel - for example if a package required via nth-order
dependencies to build 2 versions of the same repo - it's possible that a
single "guix build" command could cause the issue?
Thoughts?
[-- Attachment #2: Type: text/html, Size: 4457 bytes --]
prev parent reply other threads:[~2022-01-18 14:02 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-11 21:26 Parallel guix builds can trample? Phil
2022-01-11 23:16 ` Ricardo Wurmus
2022-01-12 7:00 ` Philip Beadling
2022-01-12 8:27 ` Ricardo Wurmus
2022-01-12 9:37 ` Phil
2022-01-17 17:23 ` Phil Beadling
2022-01-17 17:44 ` Maxime Devos
2022-01-18 9:28 ` Phil
2022-01-18 9:36 ` Maxime Devos
2022-01-18 14:59 ` Ludovic Courtès
2022-01-18 10:10 ` Phil
2022-01-18 12:53 ` Phil Beadling [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAOvsyQuPgH2JckiyS1Oougo4O3tvg0zUpKMN5+Rds51w3GF89g@mail.gmail.com \
--to=phil@beadling.co.uk \
--cc=guix-devel@gnu.org \
--cc=rekado@elephly.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).