From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Jean Abou Samra Newsgroups: gmane.lisp.guile.devel Subject: =?UTF-8?Q?Re=3a_=e2=80=9cToo_many_root_sets=e2=80=9d_when_calling_c?= =?UTF-8?Q?ompile_frequently?= Date: Fri, 19 Aug 2022 09:22:11 +0200 Message-ID: References: <8e80efa7-ca95-0472-f1b4-aaa4aabc75ab@abou-samra.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="4515"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.12.0 To: guile-devel@gnu.org Original-X-From: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Fri Aug 19 09:23:08 2022 Return-path: Envelope-to: guile-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oOwLT-00014L-OI for guile-devel@m.gmane-mx.org; Fri, 19 Aug 2022 09:23:07 +0200 Original-Received: from localhost ([::1]:39406 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oOwLS-0007Ek-CX for guile-devel@m.gmane-mx.org; Fri, 19 Aug 2022 03:23:06 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:42790) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oOwKr-0007Dg-5U for guile-devel@gnu.org; Fri, 19 Aug 2022 03:22:29 -0400 Original-Received: from mout.kundenserver.de ([217.72.192.75]:47199) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oOwKp-0002xZ-CU for guile-devel@gnu.org; Fri, 19 Aug 2022 03:22:28 -0400 Original-Received: from [10.4.0.32] ([109.190.253.13]) by mrelayeu.kundenserver.de (mreue106 [212.227.15.184]) with ESMTPSA (Nemesis) id 1MNtjq-1o0mi00ELA-00OKi9 for ; Fri, 19 Aug 2022 09:22:24 +0200 Content-Language: en-US In-Reply-To: X-Provags-ID: V03:K1:khRXPNsKQBZSvvKoEGHA2w0z4MaX6euijCW5xzxSEp7d7Cew9RJ AZd3yhnLwja7CErOSKmkzu/cAjGJKRay1p+XafAexrVDtKyu9A8qTb1mtVIyWyuFroQwSTY EBgnpLmv8NWju8w05otHxdfUOSRowmB6GKKYi57Uid3w6IEYb+LFL/CXg4Qo3up5ix2dnK8 Q3UfDpvy/+nsdi2cu52gg== X-UI-Out-Filterresults: notjunk:1;V03:K0:/i8gtE9Q8Zc=:gRJv+iLWiWqtwivw9Oac+1 Ol5MtjD9UU7DfpQzY5jMcv5MQD3sF21Jiz+GoyV1pmBGX1DfOFEAQOsD0Oc6SrRa0qwtZZX9j uamGjcieKWxnoUEFg9y2zAqxrsuqrRh6Du3z6MJ5VNjN6frJRmzK8dTNqZ56lB8nRvHBLu3Pb 8XJRlqrGHEWxWJHvHyrGcZxVLVwPmcFus8BvAzU8g5kvWID31cKf2yVZcgql48kIYnP5cEls6 NBGbeMSwchhsjBZ1z/nP5e/G8mPjnEutTvDU3V7Ao22hV7mFgoD40t/7BenE7JdXFLhQIbHxX T0Lwv1FLIq69mHDKIIGEgjJ+Vhs92qyqB7/36dDM7P9IoTXNT0HFeZI2e7jgMjw00fQneJ47d DnH36G8T+Aq6pUzR/2gBZSvEw0ycKMOKfI7errlrpDzc6/oyURjOhZvkta98v7dL2it77X/PN wt5K0cTtNwzKutb34xcFWQEhzT80sYiilOx3++tCt4Bn8yuP5zINGjZbeMXmUEka1Eu/MpQts A02jbjYvOF0zej4/UYv8ISegKP5YRuAzhIRpr+cgyizXkLyvkmxTVXFiuLVdXOBjMaUSLc0B0 0ar8nWsRFWAiZK0yn41leQ4ZUdvz9TXZQJBcQYuG42u0LKJ+xhGLlcsqXBtzh7fdshJNfp4TA ukqnZ4jbqmTLAkqVH32/dNshqi3IZtjXe65bduQIT5CSnb4Jch5oFmRh5YixMw4V7VX35H/ZW T+hKvZ/TMMRk9OxkA78PKP6flZIZF7HIWXGcCg== Received-SPF: none client-ip=217.72.192.75; envelope-from=jean@abou-samra.fr; helo=mout.kundenserver.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.io gmane.lisp.guile.devel:21308 Archived-At: Le 19/08/2022 à 00:33, Jean Abou Samra a écrit : > Le 19/08/2022 à 00:18, Jean Abou Samra a écrit : >> Calling the Guile compiler often causes this BDWGC error: “Too >> many root sets”. >> >> scheme@(guile-user)> (define-syntax-rule (repeat n expr expr* ...) >> (do ((i 0 (1+ i))) ((eqv? i n)) expr expr* ...)) >> scheme@(guile-user)> (use-modules (system base compile)) >> scheme@(guile-user)> (repeat 10000 (compile 5)) >> Too many root sets >> Abandon (core dumped) >> >> Any idea what is going on here? Should I report it as a bug? >> Is there a workaround? > > > Interestingly: > > scheme@(guile-user)> (define-syntax-rule (repeat n expr expr* ...) (do > ((i 0 (1+ i))) ((eqv? i n)) expr expr* ...)) > scheme@(guile-user)> (use-modules (system base compile)) > scheme@(guile-user)> (repeat 10000 (compile 5 #:to 'bytecode)) > scheme@(guile-user)> (use-modules (system vm loader)) > scheme@(guile-user)> (repeat 10000 (load-thunk-from-memory (compile 5 > #:to 'bytecode))) > Too many root sets > Abandon (core dumped) > > > So the problem lies in the VM loading infrastructure. (This is > as far as I can investigate for now.) I tried this code: (use-modules (system vm loader)) (define-syntax-rule (repeat n-expr expr expr* ...)   (let ((n n-expr))     (do ((i 0 (1+ i)))       ((eqv? i n))       expr expr* ...))) (let ((code (compile 5 #:to 'bytecode)))   (repeat 10000     (load-thunk-from-memory code)     (display (length (all-mapped-elf-images))) (newline))) For me, the output ends with 8174 8175 8176 8177 8178 8179 8180 8181 Too many root sets Abandon (core dumped) Now, BDWGC defines MAX_ROOT_SETS in include/private/gc_priv.h as /* Root sets.  Logically private to mark_rts.c.  But we don't want the  */ /* tables scanned, so we put them here.                                 */ /* MAX_ROOT_SETS is the maximum number of ranges that can be    */ /* registered as static roots.                                  */ # ifdef LARGE_CONFIG #   define MAX_ROOT_SETS 8192 # elif !defined(SMALL_CONFIG) #   define MAX_ROOT_SETS 2048 # else #   define MAX_ROOT_SETS 512 # endif I am using Fedora, where BDWGC is compiled with --enable-large-config. When the loader ingests a VM code chunk, it does (loader.c)   if (gc_root)     GC_add_roots (gc_root, gc_root + gc_root_size); So each load is really adding a root until this threshold of 8192 is crossed. I have no idea if it is possible to fix and/or work around this other than by not calling compile often like this. (Yes, I know, I should be using eval for one-off code evaluation, but it discards source locations, which is what brought me here in the first place.) Jean