From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Mikael Djurfeldt Newsgroups: gmane.lisp.guile.user Subject: Re: Need help embedding Guile Date: Thu, 23 Dec 2021 10:40:27 +0100 Message-ID: References: <2a789e248ef8d1922caec7af553cf26e9b360619.camel@telenet.be> <87ee65shtv.fsf@laura> Reply-To: mikael@djurfeldt.com Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="12630"; mail-complaints-to="usenet@ciao.gmane.io" Cc: guile-user To: Dimitris Papavasiliou Original-X-From: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Thu Dec 23 10:40:55 2021 Return-path: Envelope-to: guile-user@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1n0Kal-00034Z-7w for guile-user@m.gmane-mx.org; Thu, 23 Dec 2021 10:40:55 +0100 Original-Received: from localhost ([::1]:40368 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n0Kak-0005nt-0H for guile-user@m.gmane-mx.org; Thu, 23 Dec 2021 04:40:54 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:57126) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n0KaU-0005il-8Q for guile-user@gnu.org; Thu, 23 Dec 2021 04:40:38 -0500 Original-Received: from mail-ua1-f49.google.com ([209.85.222.49]:37406) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n0KaR-0006Dy-CF for guile-user@gnu.org; Thu, 23 Dec 2021 04:40:37 -0500 Original-Received: by mail-ua1-f49.google.com with SMTP id o1so9155344uap.4 for ; Thu, 23 Dec 2021 01:40:34 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=klWLmSZNq4J4hrsGMsOvs42e1njuC+RpHkVrwhaUk3I=; b=7C+ML0JOGSrikm0Ntachm2IeI9D6xDQykKLJO5G34z51nRk4WLWOqQ7PeaFsbCSJeV RoCyRkVDPe7XETDu5lAj3syu/3wsRrQ73/yV6+GXI7SfboOhNo/z+Z+ncM3mqlrBxdEz /DybaL+lnP6yuBNo/YEzAFkdn4syw2aH/FN1IuTotYoAr3C/YoVaWDMxeQKuQa9ceqlB J5xOI7qgQtxZKnZur18rboG43mrZ/CWiqLJ+ng0J+/n/h2QwYWXcMQrBXA+KdIYr95VN ZCQes6xGb5GdhrGCCOAee3TminbjhFOfCGRs4MfrDI5umzka40WcvLlKexX58/tpDcc6 5Vyw== X-Gm-Message-State: AOAM533bnGopB5RkqA4Z9eavRiPqfQXFPkf0NsCh2JB2XjCCa44rma3A lQ92htxeWpRe6FplngMfUwYZSjUuKSVlCMnN9ZjmL9US X-Google-Smtp-Source: ABdhPJwxfuAgCfIKlrgBrejnZC6uKGU/uijfEjL0NhJSWANxwAnbNXlHWPYDVQygBO5vDd62Dz2y6ZoNUUcq1gaeZK0= X-Received: by 2002:a9f:218c:: with SMTP id 12mr423570uac.71.1640252434314; Thu, 23 Dec 2021 01:40:34 -0800 (PST) In-Reply-To: Received-SPF: pass client-ip=209.85.222.49; envelope-from=mdjurfeldt@gmail.com; helo=mail-ua1-f49.google.com X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.25, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.io gmane.lisp.guile.user:17908 Archived-At: Den ons 22 dec. 2021 18:37Mikael Djurfeldt skrev: > Hi, > > I think you should give up the idea to GC collect all objects. Due to the > GC being of the conservative type, that can't be guaranteed to happen. Any > attempt to wait until all objects are collected runs the risk of hanging > indefinitely. > > Here's a suggestion for a less "heavy-handed" approach than to kill a > child process. Take it with a grain of salt since I'm not entirely > up-to-date with Guile: > > First, skip Guile finalizers. > Actually, here not being up-to-date bit me. I'm accustomed to extending Guile using SMOB:s. But, as you know, now we are supposed to use the foreign object interface where "finalizer" corresponds to the SMOB free function. So, yes, what I write below then relies on finalizers. Specifically, one of the tasks of the finalizer is to unlink the object. Sorry for the confusion. You could then let the responsibility of finalizing and freeing the C++ > objects be shared between Guile and your C++ application. For example, the > C++ application could have a doubly linked list of the C++ objects. When > Guile collects an object, make it unlink it from the C++ list. Then, when > you want to enter your second phase, you can go through the list, which now > only contains the objects not yet collected, and finalize them. This > creates the following problem: What if some Guile code runs *after* you > have finalized your remnant objects? Then, these objects would be > interpreted as live, but they are, in fact, dead and could cause a > segmentation fault. So, you either need to make sure that no Guile code > runs after this point (maybe somewhat ugly, since it leaves the Guile data > structures in an inconsistent state), or only allow Guile to deallocate the > first level C++ data structure. That data structure could then, e.g., > contain a flag indicating whether the object is dead or alive. > > All of this indicates that it could be nice to have some kind of Guile > shutdown call in the C API. Such a shutdown call could go through live > objects and free them. > > Best regards, > Mikael > > On Wed, Dec 22, 2021 at 12:47 PM Dimitris Papavasiliou < > dpapavas@protonmail.ch> wrote: > >> Hi Olivier, >> >> Thanks for chiming in! >> >> On Wednesday, December 22nd, 2021 at 12:39 AM, Olivier Dion wrote: >> >> > From this description, what I understand is that you want to use Scheme >> > as a configuration file for batching the operations to be done in a >> > second phase in C++. >> >> From an architectural point of view, the situation is probably similar >> to simple configuration parsing, as you say. In practice though, the >> scope of Guile's role won't be nearly as limited as what this might >> imply. >> >> In a way, what I'm working on, can be described as a "compiler" for >> geometry. That is to say, it reads in the description of geometry in >> some language and then evaluates it and dumps it in a format that can >> be used for e.g. 3D printing. Guile (amongst other options) supplies >> the language frontend. For example, one might invoke it on the shell >> like this: >> >> $ compile hollow_cube.scm >> >> Here, `hollow_cube.scm' would contain, say: >> >> (write-stl >> "hollow_cube.stl" >> (difference >> (cube 2 2 2) >> (cube 1 1 1))) >> >> When the `cube' procedure is evaluated, it creates a node in the C++ >> backend that evaluates to the geometry of a cube. Similarly >> `difference' creates a node, with edges to the two cubes, which when >> evaluated computes their boolean difference and so on. Of course, in >> this instance, the Scheme code amounts to little more than what might >> be termed a configuration file, but imagine the code to create a >> complex mechanical part and the situation is (conceptually) quite >> different. >> >> Creating the complete graph before evaluation begins in the second >> phase is probably not necessary (nodes could be evaluated as they're >> created), but it creates the opportunity for certain optimizations >> (like dead code elimination for instance). This makes some sort of >> forcing/ensuring that Guile has terminated desirable. >> >> > However, I failed to see why you need to finalize these objects >> > since you're going to use them in your second phase? >> >> Foreign objects currently come in two categories: >> >> 1. Complete geometric operations such as `cube' and `difference' >> above. These are allocated on the C++ side and a so-called "smart >> pointer" (shared_ptr) is exported to Scheme. Failure to finalize >> this retains a reference on the C++ side, which would prevent >> destroying the operation. Since these can get quite large in terms >> of memory, destroying them after they're no longer needed can be >> essential. >> >> 2. "Complex" arguments to the above operations such as >> transforamtions, planes, points, etc. which are C++ classes, for >> which the destructor needs to be called in one way or the other. >> These are generally passed by value to the operations and can >> therefore be freed once the first phase is complete. >> >> > One way I think of would be to fork the process and create your C++ >> > objects in a shared memory area between the parent and the child. Once >> > Guile is done reading your inputs, the child process dies and all its >> > memory is reclaimed by the OS. >> >> I have considered that but, besides the fact that it seems a bit >> heavy-handed, it still won't allow me to make sure the C++ objects are >> properly finalized. This creates two problems: for one, some object >> might need finalization that's more than just freeing memory and I'd >> have to make sure that this isn't the case for each specific type >> (which typically come from external libraries). And for another, >> these would show up as leaks in tools such as Valgrind, which would >> make checking for memory bugs difficult. >> >> One idea would be to simply call `scm_gc()' and `scm_run_finalizers()' >> until the latter returns 0. As far as I can see, this should ensure >> all finalizers are called, assumming no references to any foreign >> objects remain, but I see no way of ensuring the latter short of >> process termination... >> >> Dimitris >> >> >>