From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: David Kastrup Newsgroups: gmane.lisp.guile.user Subject: Re: Need for scm_remember_upto_here_* in guile-2.0 Date: Wed, 23 Sep 2015 14:27:15 +0200 Organization: Organization?!? Message-ID: <8761319v8c.fsf@fencepost.gnu.org> References: <20150912132725.5a5e539c@bother.homenet> <87d1xamrmz.fsf@gnu.org> <87fv26bhlw.fsf@fencepost.gnu.org> <878u7yfmmw.fsf@netris.org> <87h9mmjgxi.fsf@gnu.org> <20150923003509.69409f72@laptop.homenet> <20150923004838.7215fe7d@laptop.homenet> <87r3lpa95o.fsf@fencepost.gnu.org> <20150923094146.3f66b926@laptop.homenet> <87eghpa52d.fsf@fencepost.gnu.org> <20150923104049.26603786@laptop.homenet> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1443011303 13105 80.91.229.3 (23 Sep 2015 12:28:23 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 23 Sep 2015 12:28:23 +0000 (UTC) To: guile-user@gnu.org Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Wed Sep 23 14:28:15 2015 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Zej9i-0002Bu-7f for guile-user@m.gmane.org; Wed, 23 Sep 2015 14:28:14 +0200 Original-Received: from localhost ([::1]:47537 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zej9h-0005hA-Ia for guile-user@m.gmane.org; Wed, 23 Sep 2015 08:28:13 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:46965) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zej94-00056x-22 for guile-user@gnu.org; Wed, 23 Sep 2015 08:27:35 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zej90-0000BR-PN for guile-user@gnu.org; Wed, 23 Sep 2015 08:27:33 -0400 Original-Received: from plane.gmane.org ([80.91.229.3]:51377) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zej90-0000BH-GG for guile-user@gnu.org; Wed, 23 Sep 2015 08:27:30 -0400 Original-Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Zej8w-0001Mh-Dp for guile-user@gnu.org; Wed, 23 Sep 2015 14:27:26 +0200 Original-Received: from x2f44515.dyn.telefonica.de ([2.244.69.21]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 23 Sep 2015 14:27:26 +0200 Original-Received: from dak by x2f44515.dyn.telefonica.de with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 23 Sep 2015 14:27:26 +0200 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 108 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: x2f44515.dyn.telefonica.de X-Face: 2FEFf>]>q>2iw=B6, xrUubRI>pR&Ml9=ao@P@i)L:\urd*t9M~y1^:+Y]'C0~{mAl`oQuAl \!3KEIp?*w`|bL5qr,H)LFO6Q=qx~iH4DN; i"; /yuIsqbLLCh/!U#X[S~(5eZ41to5f%E@'ELIi$t^ Vc\LWP@J5p^rst0+('>Er0=^1{]M9!p?&:\z]|;&=NP3AhB!B_bi^]Pfkw User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) Cancel-Lock: sha1:KcgzSPtAwKngEAssm54henGsE5E= X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 80.91.229.3 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: guile-user-bounces+guile-user=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.user:12035 Archived-At: Chris Vine writes: > On Wed, 23 Sep 2015 10:54:50 +0200 > David Kastrup wrote: >> Chris Vine writes: >> >> > On Wed, 23 Sep 2015 09:26:27 +0200 >> > David Kastrup wrote: >> > >> >> Also, a pointer to an array (rather than something more opaque like >> >> SCM) is much more likely to be subject to strength reduction and >> >> address arithmetic by the compiler, leading to a situation where >> >> looping through some array does not leave a live pointer in >> >> registers and stack frame in a form recognizable by libgc. >> > >> > However, I haven't understood your point about arrays. If they are >> > allocated with malloc() you need a finaliser and you may need to >> > call scm_remember_upto_here_1() to prevent the smob's finaliser >> > from firing while they are still in use. If they are allocated with >> > scm_gc_malloc() you don't, as I understand it. Were you making a >> > point in addition to that, and if so could you expand on it for me? >> >> The point is that libgc may be less reliable with recognizing a >> typical C/C++ array still being in use than with SCM values because >> the compiler is more likely to mangle array access beyond recognition >> by stack frame inspection. >> >> For example, if you index arrays starting from 1 (or have indexing >> expressions with similar effects), the actual register from which the >> compiler does its indexing may lie strictly outside of the allocated >> area for an array. >> >> A compiler may also choose to convert a loop running from indexes 0 to >> n-1 to one running from -n to -1 in order to have a nicer terminating >> condition. Again, with strength reduction this may lead to the actual >> libgc-recognizable pointer being clear outside of the allocated array. >> >> SCM is defined in a manner where the compiler has far fewer useful >> transformations at its disposal for the normal operations. So the >> likelihood of libgc making mistakes is just lower. > > Your trees look excellent, but your pedagogical approach to the wood > is still a mystery to me. So are you saying: > > (i) libgc's garbage collector is unreliable with arrays allocated with > scm_gc_malloc() (or scm_gc_malloc_pointerless()), so using those > functions for garbage collected arrays should be avoided, and hand > written memory management for arrays using smob finalisers should be > employed instead, Unreliable with arrays? That's like defining what ratio of oak trees as compared to fir trees a wood should have in order to be considered safe from muggings at times 6am to 20pm. Because, you know, foliage and light and density of growth. There are just no hard numbers. Yet you can still pick a place where to build a house. Or a town. Is a naturally protected area safe? That's all a matter of degrees. > or > > (ii) if you are doing hand written memory management for arrays using > smob finalisers instead of using garbage collected arrays, That's not really "hand-written" since it still relies on conservative stack scanning. It's just wiring down the libgc work to the closed subset of SCM marking and collection rather than letting it deal with everything. So libgc is working under better-controlled and tested circumstances. That relies less on the skills of the libgc programmer (which may find a hard limit in the realities of code generation) and more on the skills of the GUILE application programmer. Of course, given bug #19883 it appears that in this case it is more of a question of the libgc programmers getting the hard part of collection right while messing up with finalizers. Fallout from that problem is likely one of the driving factors of GUILE developers actively recommending practices that amount to not relying on finalizers for memory management. GUILE would do better to mark SMOBs as dead, dead, dead (presumably by setting the type to 0) once it calls the finalizer on it instead of continuing to call the mark hooks (in effect, we do something comparable in LilyPond now in order not to mark collected objects). And/or report this problem to the libgc authors. Alas, "This bug report was last modified 205 days ago.". > you should make liberal use of scm_remember_upto_here_* because libgc > is particularly apt to finalise your smob while the arrays are still > in use? You should be aware what you are doing. That's all. One good rule of thumb is to pass data around as SCM as long as possible and not dereference the SCM before it is needed. An SCM in the call frame will usually be sufficient protection even without scm_remember_upto_here_* as long as you don't overwrite it with other values. Or it is passed by register. Which is an architecture-dependent thing so you cannot really rely on code not needing scm_remember_upto_here_* on one platform to run equally well on another. No, I cannot give you hard and fast rules for woods. I still have my preferences, and there are reasons for them. And there may be multiple reasons that are mostly independent. -- David Kastrup