From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id gPSfGGP32F7qYwAA0tVLHw (envelope-from ) for ; Thu, 04 Jun 2020 13:30:11 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id YH6UFGP32F4/CgAAB5/wlQ (envelope-from ) for ; Thu, 04 Jun 2020 13:30:11 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0316D940058 for ; Thu, 4 Jun 2020 13:30:11 +0000 (UTC) Received: from localhost ([::1]:50384 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgpwg-00044d-0g for larch@yhetil.org; Thu, 04 Jun 2020 09:30:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59982) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgpwX-000449-LG for guix-devel@gnu.org; Thu, 04 Jun 2020 09:30:01 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:60881) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgpwW-0004c7-EN; Thu, 04 Jun 2020 09:30:00 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=45626 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jgpwW-0000ef-1N; Thu, 04 Jun 2020 09:30:00 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Andy Wingo Subject: Re: Compilation time with Guile 3.0.3-to-be References: <87sgfbgthj.fsf@inria.fr> <87ftbbrwgq.fsf@igalia.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 17 Prairial an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 04 Jun 2020 15:29:57 +0200 In-Reply-To: <87ftbbrwgq.fsf@igalia.com> (Andy Wingo's message of "Thu, 04 Jun 2020 11:50:29 +0200") Message-ID: <87tuzr7ycq.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Spam-Score: -1.01 X-TUID: qYARQx607x5U Hello! Andy Wingo skribis: > On Thu 04 Jun 2020 09:50, Ludovic Court=C3=A8s writes: [...] >> For the record, the optimizations currently used in (guix build compile) >> are between the new -O0 and -O1: >> >> (cond ((or (string-contains file "gnu/packages/") >> (string-contains file "gnu/tests/")) >> ;; Level 0 is good enough but partial evaluation helps preserve= the >> ;; "macro writer's bill of rights". >> (override-option #:partial-eval? #t >> (optimizations-for-level 0))) > > Here fwiw I would use -O1. It is basically the same as -O0 except that > it adds partial evaluation and it inlines primcalls. If you are willing > to do partial evaluation, you are probably willing to inline primcalls > too; I think it typically leads to less code and the compilation time is > similar to -O0. Alright, let=E2=80=99s do that. >> ((string-contains file "gnu/services/") >> ;; '-O2 -Ono-letrectify' compiles about ~20% faster than '-O2' = for >> ;; large files like gnu/services/mail.scm. >> (override-option #:letrectify? #f >> (optimizations-for-level 2))) > > Interesting. I think this is probably a bug of some sort that we'll > have to keep working on. Yeah I think memory consumption increases noticeably with letrectification, which in turns means more GC activity. > Note that in 3.0.3 there is also a new phase called "lowering". Before > a compiler from Tree-IL to language X is called, the tree-IL program is > "lowered" -- meaning canonicalized and optionally optimized. > > (define (lower-exp exp env optimization-level opts) > (let ((make-lowerer (language-lowerer (lookup-language 'tree-il)))) > ((make-lowerer optimization-level opts) exp env))) > > Similarly there is an analysis pass for warnings, which runs before > lowering: > > (define (analyze-exp exp env warning-level warnings) > (let ((make-analyzer (language-analyzer (lookup-language 'tree-il)))) > ((make-analyzer warning-level warnings) exp env))) > > These can be interesting to test different phases of the tree-il -> > bytecode path. OK, I=E2=80=99ll take a look. >> The profile looks like this: >> >> scheme@(guile-user)> ,pr (define t (call-with-input-file "gnu/packages/= python-xyz.scm" (lambda (port) (read-and-compile port #:to 'tree-il)))) >> % cumulative self=20=20=20=20=20=20=20=20=20=20=20=20=20 >> time seconds seconds procedure >> 13.16 0.45 0.40 anon #x1136458 [...] >> It=E2=80=99d be great to waive the anonymity of that first lambda. :-) > > I think I just fixed it :) Yay, thank you, it=E2=80=99s much nicer now! --8<---------------cut here---------------start------------->8--- scheme@(guile-user)> ,use(system base compile) scheme@(guile-user)> ,pr (define t (call-with-input-file "gnu/packages/pyt= hon-xyz.scm" (lambda (port) (read-and-compile port #:to 'tree-il)))) % cumulative self=20=20=20=20=20=20=20=20=20=20=20=20=20 time seconds seconds procedure 16.24 0.52 0.49 set-source-properties! 12.82 0.47 0.39 ice-9/boot-9.scm:3128:0:module-gensym 4.27 0.13 0.13 ice-9/popen.scm:168:0:reap-pipes 4.27 0.13 0.13 ice-9/psyntax.scm:749:8:search 3.42 0.13 0.10 ice-9/boot-9.scm:2201:0:%load-announce 3.42 0.10 0.10 ice-9/boot-9.scm:3434:11:b 3.42 0.10 0.10 read 3.42 0.10 0.10 source-properties 2.56 50.34 0.08 ice-9/threads.scm:388:4 2.56 12.56 0.08 ice-9/psyntax.scm:1611:10:parse 2.56 0.08 0.08 module-variable 2.56 0.08 0.08 memoize-expression 2.56 0.08 0.08 ice-9/psyntax.scm:668:4:make-binding-wrap 1.71 0.08 0.05 ice-9/boot-9.scm:2790:0:module-ref-submodule 1.71 0.05 0.05 number->string 1.71 0.05 0.05 ice-9/boot-9.scm:1396:0:symbol-append 1.71 0.05 0.05 append 1.71 0.05 0.05 string-append 0.85 270.25 0.03 ice-9/boot-9.scm:220:5:map1 --8<---------------cut here---------------end--------------->8--- Nowadays I freak out every time I see those weak hash tables show up. ;-) Well, given the size of the file, it=E2=80=99s no surprise that =E2=80=98set-source-properties!=E2=80=99 is called a lot, but it=E2=80=99s = still worrying that it=E2=80=99s #1. (Perhaps an issue similar to = ?) > I think the reap-pipes call is a pretty bad sign, incidentally! Yeah, it was discussed recently and I can=E2=80=99t explain it: https://lists.gnu.org/archive/html/guile-devel/2020-05/msg00019.html Thanks for your feedback! Ludo=E2=80=99.