From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: [PATCH] build: Speed up .go compilation. Date: Mon, 11 Jan 2016 22:14:23 +0100 Message-ID: <87r3hnes40.fsf@gnu.org> References: <87lha3rx04.fsf@T420.taylan> <87mvsgxpef.fsf@T420.taylan> <87ziwgf1b4.fsf@gnu.org> <877fjhjsa8.fsf@netris.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:43467) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aIjnK-0006ws-7Q for guix-devel@gnu.org; Mon, 11 Jan 2016 16:14:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aIjnG-0006sz-WE for guix-devel@gnu.org; Mon, 11 Jan 2016 16:14:30 -0500 In-Reply-To: <877fjhjsa8.fsf@netris.org> (Mark H. Weaver's message of "Sun, 10 Jan 2016 11:47:11 -0500") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Mark H Weaver Cc: guix-devel@gnu.org Mark H Weaver skribis: > I haven't yet investigated, but my strong suspicion is that this is due > to the fact that Guile's module system is not thread safe. More > specifically, when a new module is created, it mutates the global > directory of modules in a way that is not thread safe. > > New modules are created by 'compile-file', both for the module being > compiled and for any imported modules that haven't been previously > loaded. Unfortunately, this means that this approach of compiling files > in multiple threads within a single guile process is not safe. There > are likely to be random crashes and corruptions. Right. This is one of the concerns I raised before I forgot again. ;-) https://lists.gnu.org/archive/html/guix-devel/2015-11/msg00359.html Taylan writes: > The latest version of the patch loads all needed modules first, in the > main thread, and then compiles the files corresponding to the modules in > parallel. Does that still lead to mutation in the directory of modules > in the parallelized segment of the code? It seems to be safe, but we=E2=80=99re treading in a risky zone. We=E2=80=99re pushing =E2=80=98compile-file=E2=80=99 and related code to it= s limits. It would be great to address these issues in Guile itself. Ludo=E2=80=99.