From: ludo@gnu.org (Ludovic Courtès)
To: "Taylan Ulrich \"Bayırlı/Kammer\"" <taylanbayirli@gmail.com>
Cc: guix-devel@gnu.org
Subject: Re: [PATCH] build: Speed up .go compilation.
Date: Sat, 09 Jan 2016 22:59:18 +0100 [thread overview]
Message-ID: <87bn8u76tl.fsf@gnu.org> (raw)
In-Reply-To: <87egdqy24m.fsf@T420.taylan> ("Taylan Ulrich \=\?utf-8\?Q\?\=5C\=22Bay\=C4\=B1rl\=C4\=B1\=2FKammer\=5C\=22\=22's\?\= message of "Sat, 09 Jan 2016 20:38:33 +0100")
taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:
> ludo@gnu.org (Ludovic Courtès) writes:
>
>> So, what speedup to you get compared to ‘make -jN’?
>
> It should have the same improvement as 'guix pull' since it does
> basically the same thing. I measured to make things concrete, and on my
> machine the run time of 'make -j4' decreases from ~26m to ~6m.
On my 4-core machine, it takes a bit less than 2mn.
> Here the lambda cannot entirely be factored out since the 'mutex' from
> the lexical scope is needed. I still factored out the actual code and
> pass the mutex as an argument to the procedure in a minimal lambda, as
> seen in the new patch below; I hope it's readable this way.
It is.
>> It would be awesome if you could check that ‘make distcheck’ still
>> passes, and also make sure things behave correctly when modifying just
>> one file and running ‘make’, things like that.
>
> I'm having headaches with distcheck. Currently it bails out because I'm
> missing tex. Debian's version is apparently too old, and Guix's version
> is huge and has been downloading for many hours now. I'll report back
> again when I'm done with that.
OK.
> Other notable changes in this version of the patch:
>
> - I noticed guix/config.scm and guix/tests.scm are not in MODULES in
> Makefile.am (intentionally?), so I added them to the make-go rule.
Good catch.
This is indeed intentional, because $(MODULES) goes to the distribution,
but we wouldn’t want to distribute config.scm, which is derived from
config.scm.in at configure-time; tests.scm is not in $(MODULES) because
it is not installed.
> - I removed the MKDIR_P loop in the make-go rule, and do the equivalent
> in the Scheme code now.
>
> - The target host and top source directory are now passed to the script
> via lowercase environment variables, which makes the code a little
> simpler. I hope this is stylistically fine.
Good!
> From 697950b82ea86f7b7438e586bbf4efae3e87d8f8 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?=
> <taylanbayirli@gmail.com>
> Date: Thu, 5 Nov 2015 23:42:45 +0100
> Subject: [PATCH] build: Speed up .go compilation.
>
> * build-aux/compile-all.scm: New file.
> * Makefile.am (EXTRA_DIST): Add it.
> (%.go, make-go): New rules.
The code looks good to me. I’ve tested it, it works fine indeed, and
it’s much faster than ‘make -j4’!
I have one concern: while running it peaked at 300 MiB resident in
‘top’, which is OK on many machines but still quite a lot. I realize we
have the same problem with ‘guix pull’ now, but that’s not great. Part
of the problem may be that modules are not GC’d. But anyway, that’s a
scalability problem.
What do people think?
A lesser concern is the long command-line passed to compile-all.scm,
notably because Emacs’ compilation-mode runs regexps on each line, and
that takes time proportional to the length of the line, so that leads
Emacs to hang for a second when it sees that line. Silly. ;-)
Thanks!
Ludo’.
next prev parent reply other threads:[~2016-01-09 21:59 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-12 16:41 [PATCH] Makefile: Speed up .go compilation Taylan Ulrich Bayırlı/Kammer
2016-01-08 11:48 ` [PATCH] build: " Taylan Ulrich Bayırlı/Kammer
2016-01-08 17:06 ` Ludovic Courtès
2016-01-09 19:38 ` Taylan Ulrich Bayırlı/Kammer
2016-01-09 21:59 ` Ludovic Courtès [this message]
2016-01-10 10:24 ` Taylan Ulrich Bayırlı/Kammer
2016-01-10 17:01 ` Mathieu Lirzin
2016-01-10 20:46 ` Taylan Ulrich Bayırlı/Kammer
2016-01-11 21:16 ` Ludovic Courtès
2016-01-10 13:34 ` Taylan Ulrich Bayırlı/Kammer
2016-01-10 15:11 ` Taylan Ulrich Bayırlı/Kammer
2016-01-10 17:27 ` Mathieu Lirzin
2016-01-10 20:52 ` Taylan Ulrich Bayırlı/Kammer
2016-01-10 21:18 ` Mathieu Lirzin
2016-01-11 21:05 ` Ludovic Courtès
2016-01-11 21:47 ` Taylan Ulrich Bayırlı/Kammer
2016-01-10 16:47 ` Mark H Weaver
2016-01-10 20:33 ` Taylan Ulrich Bayırlı/Kammer
2016-01-11 21:14 ` Ludovic Courtès
2016-01-11 21:56 ` Taylan Ulrich Bayırlı/Kammer
2016-01-14 14:02 ` Ludovic Courtès
2016-01-17 20:16 ` Ludovic Courtès
2016-01-18 8:05 ` Taylan Ulrich Bayırlı/Kammer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87bn8u76tl.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=guix-devel@gnu.org \
--cc=taylanbayirli@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.