From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 07A44431FD0 for ; Thu, 8 Sep 2011 08:13:34 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sZi5uOgtnYyH for ; Thu, 8 Sep 2011 08:13:33 -0700 (PDT) Received: from dmz-mailsec-scanner-1.mit.edu (DMZ-MAILSEC-SCANNER-1.MIT.EDU [18.9.25.12]) by olra.theworths.org (Postfix) with ESMTP id 73689431FB6 for ; Thu, 8 Sep 2011 08:13:33 -0700 (PDT) X-AuditID: 1209190c-b7b06ae000000aad-5e-4e68db6d6ac6 Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) by dmz-mailsec-scanner-1.mit.edu (Symantec Messaging Gateway) with SMTP id 30.C7.02733.D6BD86E4; Thu, 8 Sep 2011 11:12:45 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id p88FDTpI023371; Thu, 8 Sep 2011 11:13:29 -0400 Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91]) (authenticated bits=0) (User authenticated as amdragon@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id p88FDRYW005253 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT); Thu, 8 Sep 2011 11:13:28 -0400 (EDT) Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.72) (envelope-from ) id 1R1gKb-0003mu-RK; Thu, 08 Sep 2011 11:15:57 -0400 Date: Thu, 8 Sep 2011 11:15:57 -0400 From: Austin Clements To: Sebastian Spaeth Subject: Re: Memory management practices Message-ID: <20110908151557.GM5688@mit.edu> References: <8739h1pbaq.fsf@gmail.com> <87pqjprzu2.fsf@gmail.com> <20110829183010.GA2605@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> <87liucyn7i.fsf@gmail.com> <87aaag3xaf.fsf@gmail.com> <87zkiff8in.fsf@SSpaeth.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87zkiff8in.fsf@SSpaeth.de> User-Agent: Mutt/1.5.20 (2009-06-14) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrHKsWRmVeSWpSXmKPExsUixCmqrZt7O8PP4HuzqEXj33tMFjOf7GW2 WL5KyuL6zZnMFrPmzGN0YPV4vXoaq8fOWXfZPZ5OmMzu8WzVLWaPxV+WsgSwRnHZpKTmZJal FunbJXBldF/vZCs4KVLxoKOPqYHxhEAXIweHhICJxJPLcV2MnECmmMSFe+vZuhi5OIQE9jFK zNl+iQnCWc8ocWLzc2YI5wSTRPejC1CZJYwS717sYwbpZxFQkZjwYwUbiM0moCGxbf9yRhBb REBb4mjLDlaQBmaBrYwS/941sYMkhIGKZj7oBmvgBSr6sesWK8TUF0wSvX8esUIkBCVOznzC AmIzC2hJ3Pj3kgnkcGYBaYnl/zhAwpxAc84/3QR2hCjQEdf2t7NNYBSahaR7FpLuWQjdCxiZ VzHKpuRW6eYmZuYUpybrFicn5uWlFuka6uVmluilppRuYgRHhSTPDsY3B5UOMQpwMCrx8LJN TfcTYk0sK67MPcQoycGkJMrrfCnDT4gvKT+lMiOxOCO+qDQntfgQowQHs5IIb/AJoBxvSmJl VWpRPkxKmoNFSZz34A4HPyGB9MSS1OzU1ILUIpisDAeHkgTvjltAjYJFqempFWmZOSUIaSYO TpDhPEDDn4LU8BYXJOYWZ6ZD5E8x6nK07194nFGIJS8/L1VKnJcVmI6EBECKMkrz4ObAktkr RnGgt4R5n4CM4gEmQrhJr4CWMAEtOZSfCrKkJBEhJdXA2LPs5rK+KdWzkg7M+3vMxCw8U0rP KCAz1fhrv+0qfsWjk56m+AQ8NNkwc8+6O+saA5eyO35u+3Jv5gp1Prtj3ce2VNa7Sgj8NVHg q2d3ct2kZ5799fuFdy9lyhvTxIP9ymNSjZYLN6wwXrZ7y/E/yVknxPIu/w7RCPY5ksg7k/OG bd/h2jMZSizFGYmGWsxFxYkAGifroUEDAAA= Cc: Bertram Felgenhauer , Bart Massey , notmuch X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Sep 2011 15:13:34 -0000 Quoth Sebastian Spaeth on Sep 08 at 3:50 pm: > On Wed, 7 Sep 2011 23:05:19 -0400, Austin Clements wrote: > > Sorry, I went back and re-read your earlier messages and now I see why > > your references were the way they were. I stand by the rest of my > > previous message though. I think the technique used in the Python > > bindings only works because Python's GC happens to finalize in a > > particular order (though I doubt that's guaranteed, and could easily > > not be the case if you stray into the realm of its cycle collector). > > In general, it seems like approach is trying to recreate C-like memory > > management and is fragile as a result, whereas talloc should, I think, > > allow bindings to express their runtime's memory management rather > > naturally. > > Mmmh? Why would the method in python be fragile? Each message object > holds a reference to its parent query object to keep it alive. Are you > saying cycle collectors could kill off the query object nonetheless? > (Assume that I know nothing of GCs which comes close to reality) In general, a garbage collector can't make any guarantees about finalization order. When a collection of objects all become unreachable simultaneously (for example, the last reference to any Messages object is dropped, causing the Query object and the Message object to both become unreachable), the garbage collector *could* finalize the Query first (causing talloc to free the notmuch_messages_t) and then the Messages object (causing it to crash). There's no guarantee in general because, in the presence of cycles, there is no meaningful finalization order. That being said, this approach might be (probably is) fine in Python because Python has an unusual hybrid garbage collector. Long ago, Python had only a reference-count based garbage collector. It now has a cycle detector layered on top of that [1], but that only kicks in if there are reference cycles. Assuming there aren't cycles in the objects created by the Python bindings, you should get the deterministic behavior of the reference counted collector. This isn't the case in Haskell, which has a generational collector that makes no guarantees about finalization order (guarantees it couldn't always keep). [1] One day a student came to Moon and said: "I understand how to make a better garbage collector. We must keep a reference count of the pointers to each cons." Moon patiently told the student the following story: "One day a student came to Moon and said: 'I understand how to make a better garbage collector...