From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Newsgroups: gmane.lisp.guile.bugs Subject: bug#15602: Compiling several files in the same session [2.0.9] Date: Tue, 21 Jun 2016 14:01:17 +0200 Message-ID: <87eg7qagz6.fsf@gnu.org> References: <8761t19t4f.fsf@gnu.org> <87twgm23dv.fsf@pobox.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1466510637 10221 80.91.229.3 (21 Jun 2016 12:03:57 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 21 Jun 2016 12:03:57 +0000 (UTC) Cc: 15602-done@debbugs.gnu.org To: Andy Wingo Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Tue Jun 21 14:03:45 2016 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bFKP6-0003q0-IT for guile-bugs@m.gmane.org; Tue, 21 Jun 2016 14:03:40 +0200 Original-Received: from localhost ([::1]:51284 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bFKP5-0007G1-Ra for guile-bugs@m.gmane.org; Tue, 21 Jun 2016 08:03:39 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:51491) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bFKNa-0005UU-Ft for bug-guile@gnu.org; Tue, 21 Jun 2016 08:02:10 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bFKNW-0003GG-A7 for bug-guile@gnu.org; Tue, 21 Jun 2016 08:02:05 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:36351) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bFKNW-0003GB-7e for bug-guile@gnu.org; Tue, 21 Jun 2016 08:02:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1bFKNW-0001Nd-1B for bug-guile@gnu.org; Tue, 21 Jun 2016 08:02:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Tue, 21 Jun 2016 12:02:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15602 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 15602-done@debbugs.gnu.org id=D15602.14665104955261 (code D ref 15602); Tue, 21 Jun 2016 12:02:01 +0000 Original-Received: (at 15602-done) by debbugs.gnu.org; 21 Jun 2016 12:01:35 +0000 Original-Received: from localhost ([127.0.0.1]:48688 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bFKN2-0001Mi-7A for submit@debbugs.gnu.org; Tue, 21 Jun 2016 08:01:35 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:59988) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bFKN0-0001MV-AU for 15602-done@debbugs.gnu.org; Tue, 21 Jun 2016 08:01:30 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bFKMs-00030l-4W for 15602-done@debbugs.gnu.org; Tue, 21 Jun 2016 08:01:25 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42411) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bFKMs-000304-0j; Tue, 21 Jun 2016 08:01:22 -0400 Original-Received: from pluto.bordeaux.inria.fr ([193.50.110.57]:41594 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bFKMq-0005YR-3m; Tue, 21 Jun 2016 08:01:20 -0400 X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 4 Messidor an 224 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu In-Reply-To: <87twgm23dv.fsf@pobox.com> (Andy Wingo's message of "Tue, 21 Jun 2016 13:22:04 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:8080 Archived-At: Hi again! Andy Wingo skribis: > On Sun 13 Oct 2013 15:51, ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> $ guile --no-auto-compile -L . -c '(use-modules (system base compile)) (= for-each compile-file (list "one.scm" "two.scm" "three.scm"))' >> >> $ guile --no-auto-compile -L . -c '(use-modules (system base compile)) (= for-each compile-file (list "three.scm" "two.scm" "one.scm"))' >> >> $ guile --no-auto-compile -L . -c '(use-modules (system base compile)) (= for-each compile-file (list "one.scm" "three.scm" "two.scm"))' >> Backtrace: > > I understand this is now fixed in Guix. I think it=E2=80=99s worked around, not fixed. :-) >> I think the right thing would be to use a separate module hierarchy in >> the dynamic extent of =E2=80=98compile-file=E2=80=99, somehow, such that= all module side >> effects are isolated. > > I don't think this is going to happen, for what that's worth :/ What do you mean? Global state held in global variables is Bad. To me, this is clearly an area where things can and should be improved. In a live-hacked, long-running system, that =E2=80=98compile-file=E2=80=99 modifies the globa= l state in arbitrary ways is not just a theoretical problem. WDYT? >> Of course the above can be worked around by running =E2=80=98compile-fil= e=E2=80=99 in a >> child process, but forking alone is more expensive than =E2=80=98compile= -file=E2=80=99, >> so that=E2=80=99s not really a solution when there are many files. > > It's the N^2 expansion that's the problem, not the forking. If you have > N files which depend on each other, then compiling each one will require > the expansion of approximately all N files, so N*N costs in number of > files. Or N log N if you have a tree ordering of your files. Anyway > it's badness. Right. However, when compiling a set of files in a single process, one could expect previously-expanded files to be cached. > Glad to know that Taylan has fixed this one in Guix. I should mention that the hack in Guix to allow parallel compilation is brittle since shared state (module obarrays) can, in some cases, be accessed concurrently, and modules aren=E2=80=99t thread safe. Concretely, building all the files in Guix works well ~95% of the time; when only some files need rebuilding, it=E2=80=99s not uncommon to see it f= ail weirdly (=E2=80=9Cno such language=E2=80=9D or a similarly obscure error.) Any idea how this could be addressed? Thanks! Ludo=E2=80=99.