From: ludo@gnu.org (Ludovic Courtès)
To: "Taylan Ulrich \"Bayırlı/Kammer\"" <taylanbayirli@gmail.com>
Cc: guix-devel@gnu.org
Subject: Re: [PATCH] build: pull: Compile .scm files in one process.
Date: Sat, 14 Nov 2015 15:54:14 +0100 [thread overview]
Message-ID: <87mvugws15.fsf@gnu.org> (raw)
In-Reply-To: <877flmrn2m.fsf@T420.taylan> ("Taylan Ulrich \=\?utf-8\?Q\?\=5C\=22Bay\=C4\=B1rl\=C4\=B1\=2FKammer\=5C\=22\=22's\?\= message of "Fri, 13 Nov 2015 15:28:01 +0100")
taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:
> ludo@gnu.org (Ludovic Courtès) writes:
[...]
>> I think we’re in trouble if DEBUG-PORT is a real port because ports
>> aren’t thread-safe in 2.0, and ‘compile-file’ might write to it. Void
>> ports seem to be OK because their ‘write’ method doesn’t do anything.
>>
>> So I think we have to do things sequentially when DEBUG-PORT is
>> non-void.
>>
>> I’m also concerned about modules accessed by concurrent calls to
>> ‘compile-file’ because modules and hash tables aren’t thread-safe
>> either.
>>
>> WDYT?
>>
>> Thanks to the two of you! Sounds like we’ll soon have a less slow or
>> even a fast ‘guix pull’. ;-)
>
> Hmm, how about this one? I guess it could still garble output if
> 'compile-file' writes things in small pieces that belong together, but
> at least it's sure not to crash. Compile-file doesn't seem to write a
> lot either; one probably won't see much garbled output in practice.
> Maybe the speed gain makes this acceptable.
It’s not about garbled output; it’s about concurrent modifications of
the port’s internal structure, which can lead to segfaults and whatnot.
> Re. modules, AFAIUI compile-file shouldn't do any further mutations on
> modules, since we load them in advance. Maybe Andy can tell how safe
> this is.
Right, it *seems* reasonable. Andy, Mark?
> From 78be6d09d2d4c0a563be14c66ac2a1a345ff9b1d Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?=
> <taylanbayirli@gmail.com>
> Date: Thu, 5 Nov 2015 23:43:20 +0100
> Subject: [PATCH] build: pull: Compile .scm files in one process.
>
> * guix/build/pull.scm (call-with-process, report-build-progress)
> (p-for-each): Remove.
> (thread-safe-port): New procedure.
> (build-guix): Load and compile files in one process.
[...]
> +(define (thread-safe-port port)
> + (define mutex (make-mutex))
> + (define (wrap proc)
> + (lambda args
> + (with-mutex mutex (apply proc args))))
> + (make-soft-port
> + (vector (wrap write-char) (wrap display) (wrap force-output) #f #f) "w"))
This doesn’t address the fact that internally, things may still go wrong
(see ‘scm_lfwrite’ and the likes.)
I think we simply need to build sequentially when ports are involved,
which is when --debug is used.
Could you address this last bit?
Thanks!
Ludo’.
next prev parent reply other threads:[~2015-11-14 14:54 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-05 17:10 [PATCH] build: pull: Compile .scm files in one process Taylan Ulrich Bayırlı/Kammer
2015-11-05 21:06 ` Ludovic Courtès
2015-11-06 8:56 ` Taylan Ulrich Bayırlı/Kammer
2015-11-06 20:50 ` Ludovic Courtès
2015-11-07 9:54 ` Taylan Ulrich Bayırlı/Kammer
2015-11-10 18:00 ` Ludovic Courtès
2015-11-11 12:12 ` Taylan Ulrich Bayırlı/Kammer
2015-11-11 13:26 ` Ludovic Courtès
2015-11-12 10:23 ` Taylan Ulrich Bayırlı/Kammer
2015-11-12 12:07 ` Andy Wingo
2015-11-12 12:27 ` Taylan Ulrich Bayırlı/Kammer
2015-11-12 12:36 ` Andy Wingo
2015-11-12 13:52 ` Taylan Ulrich Bayırlı/Kammer
2015-11-12 20:44 ` Ludovic Courtès
2015-11-13 14:28 ` Taylan Ulrich Bayırlı/Kammer
2015-11-14 14:54 ` Ludovic Courtès [this message]
2015-11-26 22:20 ` Ludovic Courtès
2015-11-27 8:53 ` Taylan Ulrich Bayırlı/Kammer
2015-11-27 10:07 ` Ludovic Courtès
2015-11-27 15:16 ` Taylan Ulrich Bayırlı/Kammer
2015-11-30 13:04 ` Ludovic Courtès
2015-12-02 8:09 ` Taylan Ulrich Bayırlı/Kammer
2015-12-03 10:44 ` Ludovic Courtès
2015-12-21 9:49 ` Taylan Ulrich Bayırlı/Kammer
2016-01-05 17:56 ` Taylan Ulrich Bayırlı/Kammer
2016-01-05 21:39 ` Ludovic Courtès
2016-01-05 22:32 ` Taylan Ulrich Bayırlı/Kammer
2016-01-07 17:01 ` Leo Famulari
2016-01-07 17:54 ` Alex Sassmannshausen
2016-01-07 18:10 ` Ludovic Courtès
2015-12-03 15:27 ` Mark H Weaver
2015-12-03 15:54 ` Taylan Ulrich Bayırlı/Kammer
2015-12-04 14:11 ` Ludovic Courtès
2015-11-06 16:12 ` Andy Wingo
2015-11-06 16:41 ` Taylan Ulrich Bayırlı/Kammer
2015-11-06 20:48 ` Ludovic Courtès
2015-11-09 7:41 ` Andy Wingo
2015-11-09 8:51 ` Taylan Ulrich Bayırlı/Kammer
2015-11-09 9:07 ` Andy Wingo
2015-11-09 9:50 ` Taylan Ulrich Bayırlı/Kammer
2015-11-09 10:49 ` Andy Wingo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87mvugws15.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=guix-devel@gnu.org \
--cc=taylanbayirli@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.