From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
To: "Ludovic Courtès" <ludo@gnu.org>
Cc: guix-devel <guix-devel@gnu.org>
Subject: Re: Dependency cycle issues when using a Gexp-based snippet
Date: Wed, 02 Sep 2020 11:08:27 -0400 [thread overview]
Message-ID: <87mu286wt0.fsf@gmail.com> (raw)
In-Reply-To: <87wo1nzr6p.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 24 Aug 2020 23:09:50 +0200")
Hi Ludovic!
Thank you for the reply.
Ludovic Courtès <ludo@gnu.org> writes:
> Hello!
>
> maxim.cournoyer@gmail.com skribis:
>
>> While trying to move some of the patching done to qtbase into a snippet,
>> with the goal of having at least the ./configure script runnable in a
>> guix environment without having to manually run patching phases:
>
> [...]
>
>> I encountered the following issue, which seems similar to one
>> encountered by Ricardo in 2016 [0]:
>>
>> ice-9/eval.scm:293:34: error: canonical-package: unbound variable
>> hint: Did you forget a `use-modules' form?
>>
>>
>> The origin can be correctly built at the REPL, so the problem indeed
>> seems to be a dependency cycle.
>
> Indeed: the problem is that when loading this module, we try to resolve
> one of the variables referenced in the snippet, but that variable is not
> defined yet because it comes from a module that’s in a dependency circle
> with the one at hand.
>
>> Attempting a suggested fix by Ludovic in that same conversation [0],
>> namely, making the snippet field of the <origin> record a thunked one:
>>
>> modified guix/packages.scm
>> @@ -250,7 +250,8 @@ as base32. Otherwise, it must be a bytevector."
>> (patches origin-patches ; list of file names
>> (default '()) (delayed))
>>
>> - (snippet origin-snippet (default #f)) ; sexp or #f
>> + (snippet origin-snippet
>> + (default #f) (thunked)) ; sexp or #f
>> (patch-flags origin-patch-flags ; list of strings
>> (default '("-p1")))
>
> We should check what this change costs in CPU and memory, but it’s
> probably worth it. As Marius noted before, the snippets for
> ungoogled-chromium and linux-libre are contrived because of this
> limitation. (Perhaps we can use ‘delayed’ instead of ‘thunked’.)
What is the difference between delayed and thunked? Would a thunked
capture the closure of its environment while delayed not? Is the
closure useful to access record-bound values such as the version field
of a package?
I checked the usage at compilation and run time, using the 'time'
command (aliased to time+ on my system), and didn't find any meaningful
difference whether the snippet is made a thunked or delayed field, or
none (current situation):
On current master:
--8<---------------cut here---------------start------------->8---
time+ make -j8
2436.29user 56.47system 14:29.36elapsed 286%CPU (0avgtext+0avgdata 870828maxresident)k
5480inputs+405952outputs (71major+320522minor)pagefaults 0swaps
time+ ./pre-inst-env guix package -A | wc -l
9.87user 0.24system 0:06.51elapsed 155%CPU (0avgtext+0avgdata 281564maxresident)k
0inputs+0outputs (0major+25636minor)pagefaults 0swaps
14702
--8<---------------cut here---------------end--------------->8---
With delayed source field:
--8<---------------cut here---------------start------------->8---
time+ make -j8
2493.40user 51.16system 16:29.33elapsed 257%CPU (0avgtext+0avgdata 842008maxresident)k
48888inputs+413640outputs (158major+340632minor)pagefaults 0swaps
time+ ./pre-inst-env guix package -A | wc -l
9.68user 0.26system 0:06.68elapsed 148%CPU (0avgtext+0avgdata 293488maxresident)k
1008inputs+0outputs (7major+42943minor)pagefaults 0swaps
14702
--8<---------------cut here---------------end--------------->8---
And finally with thunked source field:
--8<---------------cut here---------------start------------->8---
time+ make -j8
2302.67user 27.44system 22:32.82elapsed 172%CPU (0avgtext+0avgdata 862336maxresident)k
2960inputs+416688outputs (51major+300689minor)pagefaults 0swaps
time+ ./pre-inst-env guix package -A | wc -l
10.02user 0.23system 0:07.46elapsed 137%CPU (0avgtext+0avgdata 283052maxresident)k
2984inputs+0outputs (101major+24685minor)pagefaults 0swaps
14702
--8<---------------cut here---------------end--------------->8---
Based on this, I would choose the most capable 'mode' for the source
field, which I presume is thunked.
>> It now seems a new cycle is introduced because trying to build anything
>> hangs using the CPU with slowly increasing memory usage:
>
> Hmm not sure exactly why, but look:
>
> + (snippet
> + (with-imported-modules '((guix build utils))
> + #~(begin
> + (use-modules (guix build utils))
> + ;; corelib uses bundled harfbuzz, md4, md5, sha3
> + (with-directory-excursion "src/3rdparty"
> + (for-each delete-file-recursively
> + (list "double-conversion" "freetype" "harfbuzz-ng"
> + "libpng" "libjpeg" "pcre2" "sqlite" "xcb"
> + "zlib")))
> +
> + (let ((coreutils #+(canonical-package coreutils)))
> + (substitute* "configure"
> + (("/bin/pwd")
> + (string-append coreutils "/bin/pwd")))
> + (substitute* "src/corelib/global/global.pri"
> + (("/bin/ls")
> + (string-append coreutils "/bin/ls"))))
> + #t)))))
>
> Such substitutions are system-dependent; thus, they should be made in a
> phase, not in a snippet. Perhaps we’ll sidestep the issue altogether?
> :-)
Indeed. I didn't consider this aspect well. Apart from being
inefficient (the sources of a package would be different for each
system) it would still technically work, no?
But yeah, that bloat is probably not worth the extra convenience (of
having a source that can be built straight from 'guix build --source').
Thanks,
Maxim
next prev parent reply other threads:[~2020-09-02 15:08 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-24 2:10 Dependency cycle issues when using a Gexp-based snippet maxim.cournoyer
2020-08-24 21:09 ` Ludovic Courtès
2020-09-02 15:08 ` Maxim Cournoyer [this message]
2020-09-07 9:30 ` Ludovic Courtès
2020-09-14 16:55 ` Maxim Cournoyer
2020-09-16 10:08 ` Ludovic Courtès
2020-09-07 18:30 ` Mark H Weaver
2020-09-16 10:00 ` Ludovic Courtès
2020-09-08 4:07 ` What should "guix build --source" produce? (was Re: Dependency cycle issues when using a Gexp-based snippet) Mark H Weaver
2020-09-08 7:22 ` Andreas Enge
2020-09-11 18:22 ` Maxim Cournoyer
2020-09-11 18:42 ` zimoun
2020-09-11 20:40 ` Andreas Enge
2020-09-11 18:26 ` Maxim Cournoyer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87mu286wt0.fsf@gmail.com \
--to=maxim.cournoyer@gmail.com \
--cc=guix-devel@gnu.org \
--cc=ludo@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).