From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Mark H Weaver Newsgroups: gmane.lisp.guile.devel Subject: Re: Mark procedures, LilyPond, and backward compatibility Date: Wed, 04 Nov 2015 12:01:48 -0500 Message-ID: <87oaf9zojn.fsf@netris.org> References: <87vb9ihy6x.fsf@igalia.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1446656533 26816 80.91.229.3 (4 Nov 2015 17:02:13 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 4 Nov 2015 17:02:13 +0000 (UTC) Cc: guile-devel@gnu.org To: Andy Wingo Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Wed Nov 04 18:02:09 2015 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Zu1Rn-0002v0-Rp for guile-devel@m.gmane.org; Wed, 04 Nov 2015 18:02:07 +0100 Original-Received: from localhost ([::1]:56078 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zu1Rm-0007YS-W6 for guile-devel@m.gmane.org; Wed, 04 Nov 2015 12:02:07 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:54436) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zu1Ri-0007Tz-6h for guile-devel@gnu.org; Wed, 04 Nov 2015 12:02:03 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zu1Re-0006vL-3y for guile-devel@gnu.org; Wed, 04 Nov 2015 12:02:02 -0500 Original-Received: from world.peace.net ([50.252.239.5]:52481) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zu1Re-0006uz-1D for guile-devel@gnu.org; Wed, 04 Nov 2015 12:01:58 -0500 Original-Received: from [10.1.10.104] (helo=jojen) by world.peace.net with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1Zu1RV-000159-Gb; Wed, 04 Nov 2015 12:01:49 -0500 In-Reply-To: <87vb9ihy6x.fsf@igalia.com> (Andy Wingo's message of "Wed, 04 Nov 2015 10:10:46 +0000") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 50.252.239.5 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:17970 Archived-At: Hi Andy, Thanks for the detailed analysis of the problems with mark procedures. Andy Wingo writes: > At the end of our IRC conversation yesterday you put the burden on me to > argue against mark procedures, which was fair, but at this point I think > we would need good arguments for keeping them, The most important reason that we need to preserve mark procedures is that we've supported them for most of Guile's existence, and there's a lot of important code out there that depends upon them. Remember LilyPond? As much as we might like to forget that LilyPond exists and move onward with more fun things, the fact remains that as you prepare a 2.1.x pre-release, LilyPond still doesn't work with Guile 2.0.x, and not for lack of effort. This problem is not just going to go away. We _must_ help to get LilyPond working, and we must *not* take more steps that would make that job more difficult. We must demonstrate that Guile is a safe base upon which to build a large application. LilyPond is by far the most important project that has done that, and the ongoing LilyPond+Guile2 disaster is a glaring example that "mistakes were made" in the 1.8 -> 2.0 transition. I think we need to acknowledge those mistakes, learn from them, and pay more attention to backward compatibility now and in the future. What mistakes were made? Among many other things: Guile 2.0 switched to a new macro expander that doesn't support lazy macro expansion, and to a GC with very different finalization semantics. LilyPond has a lot of code that depends on the semantics of our old GC. For starters, it depends heavily upon our long-standing support for mark procedures, and will surely require ongoing support for a non-threaded marking mode as well. While we're on the subject: demonstrated that Boehm GC will invoke marking procedures on objects that have already been finalized. It seems likely that fixing LilyPond will require adding support to Boehm GC for another kind of finalizer with semantics similar to the finalizers in Guile 1.8, where the finalizers were strictly called before the mutator resumes execution. > I think it makes sense to *avoid* adding mark functions to the new > APIs, at least for now. We don't have the use cases right now and so > we would surely specify the wrong thing. We can always add something > later. In summary, if we're going to deprecate and replace an existing API, the new API needs to be designed such that it's straightforward to update existing code to use the new API. Therefore, we must continue to support mark procedures. I would go further and assert that the new API should also include what's needed to support a precise, moving GC. To do otherwise would be to strongly encourage the creation of code that cannot possibly work with a moving GC. Deprecating and replacing the SMOB API is a significant event in the history of Guile. It should not be rushed, nor should it be finalized until it supports existing use cases and anticipated future developments. As an aside, I'm not happy with the fact that this new foreign objects API was pushed to the stable-2.0 branch less than 25 hours after the initial proposal was posted. It feels like establishing "facts on the ground", and sets up a situation where the burden is now on me to argue that it's not ready to be included in 2.0.12, whereas the burden should rather be on you to obtain a consensus that this new major API is ready for release. For now, I would prefer to revert the commits that added this new API, to enable the immediate release of 2.0.12. Then we can have a proper discussion about what a SMOB replacement API should look like, and make sure that both past and future anticipated use cases are covered before releasing it. What do you think? Mark