From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: [PATCH] build: pull: Compile .scm files in one process. Date: Sat, 14 Nov 2015 15:54:14 +0100 Message-ID: <87mvugws15.fsf@gnu.org> References: <87si4kxtge.fsf@T420.taylan> <87611gdul8.fsf@gnu.org> <87h9kzy09b.fsf@T420.taylan> <87bnb6c0nh.fsf@gnu.org> <874mgyxhgy.fsf@T420.taylan> <877flpohu6.fsf@gnu.org> <87mvuku444.fsf@T420.taylan> <87pozgfyzt.fsf@gnu.org> <87io57tt2s.fsf@T420.taylan> <876117mnef.fsf@igalia.com> <87egfvtnbw.fsf@T420.taylan> <87y4e3l7hm.fsf@igalia.com> <87a8qjtje8.fsf@T420.taylan> <876117t0ax.fsf@gnu.org> <877flmrn2m.fsf@T420.taylan> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:37701) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZxcDd-0002Al-DJ for guix-devel@gnu.org; Sat, 14 Nov 2015 09:54:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZxcDZ-00074G-A9 for guix-devel@gnu.org; Sat, 14 Nov 2015 09:54:21 -0500 In-Reply-To: <877flmrn2m.fsf@T420.taylan> ("Taylan Ulrich \=\?utf-8\?Q\?\=5C\=22Bay\=C4\=B1rl\=C4\=B1\=2FKammer\=5C\=22\=22's\?\= message of "Fri, 13 Nov 2015 15:28:01 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Taylan Ulrich =?utf-8?Q?=22Bay=C4=B1rl=C4=B1?= =?utf-8?Q?=2FKammer=22?= Cc: guix-devel@gnu.org taylanbayirli@gmail.com (Taylan Ulrich "Bay=C4=B1rl=C4=B1/Kammer") skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) writes: [...] >> I think we=E2=80=99re in trouble if DEBUG-PORT is a real port because po= rts >> aren=E2=80=99t thread-safe in 2.0, and =E2=80=98compile-file=E2=80=99 mi= ght write to it. Void >> ports seem to be OK because their =E2=80=98write=E2=80=99 method doesn= =E2=80=99t do anything. >> >> So I think we have to do things sequentially when DEBUG-PORT is >> non-void. >> >> I=E2=80=99m also concerned about modules accessed by concurrent calls to >> =E2=80=98compile-file=E2=80=99 because modules and hash tables aren=E2= =80=99t thread-safe >> either. >> >> WDYT? >> >> Thanks to the two of you! Sounds like we=E2=80=99ll soon have a less sl= ow or >> even a fast =E2=80=98guix pull=E2=80=99. ;-) > > Hmm, how about this one? I guess it could still garble output if > 'compile-file' writes things in small pieces that belong together, but > at least it's sure not to crash. Compile-file doesn't seem to write a > lot either; one probably won't see much garbled output in practice. > Maybe the speed gain makes this acceptable. It=E2=80=99s not about garbled output; it=E2=80=99s about concurrent modifi= cations of the port=E2=80=99s internal structure, which can lead to segfaults and what= not. > Re. modules, AFAIUI compile-file shouldn't do any further mutations on > modules, since we load them in advance. Maybe Andy can tell how safe > this is. Right, it *seems* reasonable. Andy, Mark? > From 78be6d09d2d4c0a563be14c66ac2a1a345ff9b1d Mon Sep 17 00:00:00 2001 > From: =3D?UTF-8?q?Taylan=3D20Ulrich=3D20Bay=3DC4=3DB1rl=3DC4=3DB1/Kammer?= =3D > > Date: Thu, 5 Nov 2015 23:43:20 +0100 > Subject: [PATCH] build: pull: Compile .scm files in one process. > > * guix/build/pull.scm (call-with-process, report-build-progress) > (p-for-each): Remove. > (thread-safe-port): New procedure. > (build-guix): Load and compile files in one process. [...] > +(define (thread-safe-port port) > + (define mutex (make-mutex)) > + (define (wrap proc) > + (lambda args > + (with-mutex mutex (apply proc args)))) > + (make-soft-port > + (vector (wrap write-char) (wrap display) (wrap force-output) #f #f) "= w")) This doesn=E2=80=99t address the fact that internally, things may still go = wrong (see =E2=80=98scm_lfwrite=E2=80=99 and the likes.) I think we simply need to build sequentially when ports are involved, which is when --debug is used. Could you address this last bit? Thanks! Ludo=E2=80=99.