From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: MPS: Win64 testers? Date: Fri, 23 Aug 2024 17:32:29 +0300 Message-ID: <86v7zrpawy.fsf@gnu.org> References: <87bk1k8mzd.fsf@protonmail.com> <861q2gs8xu.fsf@gnu.org> <877cc88j4p.fsf@protonmail.com> <86seuvre6x.fsf@gnu.org> <8734mv8yb6.fsf@protonmail.com> <8634mvqte0.fsf@gnu.org> <87o75j73ry.fsf@protonmail.com> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="307"; mail-complaints-to="usenet@ciao.gmane.io" Cc: sebastian@sebasmonia.com, kien.n.quang@gmail.com, emacs-devel@gnu.org, yantar92@posteo.net To: Pip Cet Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Aug 23 16:33:48 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1shVMJ-000ASO-NF for ged-emacs-devel@m.gmane-mx.org; Fri, 23 Aug 2024 16:33:47 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1shVLc-000436-0Q; Fri, 23 Aug 2024 10:33:04 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1shVLZ-00042A-Op for emacs-devel@gnu.org; Fri, 23 Aug 2024 10:33:02 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1shVLV-00077k-2t; Fri, 23 Aug 2024 10:33:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=dmL2EtYxO7a8Qvm6tlPQXfhSj3BUJ5hCbDbnK++0XBc=; b=EQW55a3soj2G mGd8+GWDPCsFzGtlR+4yAB/M60lk+Wu09HN9BtVusl8Sqn4u1zx9DTCu9a3tDflGBP4b+21QXG032 y5+hPqIGW4LuiqvS9KdeCdjKbGeaazQRvDxtGmzqq/1w/WacPAwa2Un2yFYxuOf1d+Upj1VDQC+Vh vwuztyQNI/DVD46sf7oaRsKnpWdNfYKK/Iya4ngJPpQx+aJuR0bK++/3hUWY57jfmJG41hFoRnDPw hQtQrIHJC+Lf0A8vZDFOVFxnboM4phKrTuvdbAsAL0IJSoXEgn7MNBu3O6l5iVC/Ll+x330o/Xpav HtUvjGyTag1FPWviWBKMiw==; In-Reply-To: <87o75j73ry.fsf@protonmail.com> (message from Pip Cet on Fri, 23 Aug 2024 13:44:04 +0000) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:323088 Archived-At: > Date: Fri, 23 Aug 2024 13:44:04 +0000 > From: Pip Cet > Cc: sebastian@sebasmonia.com, kien.n.quang@gmail.com, emacs-devel@gnu.org, yantar92@posteo.net > > > But my bother is that we will need to similarly pin many other > > pointers and objects, or risk segfaults. > > Indeed. There are quite a few changes along the same lines in the > scratch/igc branch, and it makes me think we need a slightly more > general infrastructure to turn a pointer (or Lisp_Object) known to Emacs > into a void * suitable for passing as a cookie to an external library, > and then for turning the cookie passed to a callback by that external > library back into a pointer (or Lisp_Object) the callback (in Emacs) can > use. > > > Maybe it means that MPS and similar igc systems are not suitable for > > Emacs. > > I believe said infrastructure would be generally useful, because it > allows us to catch another class of subtle bugs that may appear without > MPS: passing a pointer to an external library which is freed by our > traditional GC, which would cause very similar segfaults (if anything, > much trickier to debug) without MPS. > > IOW, it's always worth our time to think about why something we pass as > a cookie to an external callback cannot be garbage-collected and/or > copied, regardless of which garbage collector is in use. The price we'd > pay is to require such cookies to be explicitly deregistered, but we'd > gain a very useful feature (maybe for --enable-checking, maybe enabled > by default): a clear and loud warning informing us that a cookie is > still registered for an object we just tried to GC. > > So I think this could improve Emacs in general, even for people who will > never use MPS. We are not coding Emacs from scratch. Emacs is already a very large body of code, most of it tested by time and gobs of users and use patterns. To start rewriting too much of that means we are losing all that hard-earned stability, because many corners in Emacs code are not well known to those who are currently active in development and maintenance. So from where I stand, it is definitely a significant problem to make such vast changes, and the fact that it will help us eventually doesn't help. Because if we destabilize Emacs enough, we will lose our users, which are the only reason Emacs is still alive and kicking. > > EnumFontFamiliesEx (dc, &match_data->pattern, > > (FONTENUMPROC) add_font_entity_to_list, > > (LPARAM) match_data, 0); > > > > where add_font_entity_to_list is our callback which is called for > > every font family and decides whether it matches the font spec; if so, > > the callback adds the family to a list that is then used by the caller > > of EnumFontFamiliesEx. > > This particular call is safe. 'match_data' lives on the stack, MPS > treats the stack conservatively as pinning every object to which it > might possibly refer, thus 'match_data->pattern' cannot be moved. IOW, we are lucky. Because match_data could well be a global variable allocated off the heap, or something else. > >> The problem here is that 'proc' might move. One solution is to make the > >> cookie a pointer to memory known to MPS, which contains the actual > >> pointer to the Lisp_Process. Note that it is not strictly necessary for > >> the pointer to be allocated with igc_xzalloc_ambig, we could use the > >> _raw_exact variant. > > > > Why not tell MPS not to move this pointer instead? It sounds like a > > simpler solution than the change you proposed. > > The change I proposed is redundant: it tells MPS not to move the > pointer, and it uses a pointer-to-a-pointer to catch potential moves. > Only one part of that is needed, but in both cases, we need to remember > to un-pin/free something. If the change is redundant, why did you propose it? More importantly, we need to have boilerplate code to pin a pointer, because this will be needed a lot. The fact that we still don't have it is unfortunate. > > The solution you propose will complicate large parts of the Emacs code > > by using double indirection; > > Indeed, which is why I'm seriously proposing that cookies for use by > external libraries be generated via an opaque API that has different > implementations: > > 1. for standard Emacs, where it just returns the pointer > 2. for MPS, where it returns a pointer to the pointer > 3. for debug builds, where it registers the object so we warn if we try > to GC it, then returns the pointer. > > > pinning a pointer is much simpler. Or am I missing something? > > Pinning it certainly is, remembering to un-pin it when we're done with > it is the hard part. Then we need to figure that out, IMO. > And also something we should always do, since > forgetting to unpin it would result in memory leaks and slower GCs for > the lifetime of the Emacs session. If unpinning is automatic, it won't be forgotten. > >> There are other solutions which may be better. > > > > I certainly hope so, since auditing all of our sources for such > > gotchas is an impossible job. > > I think I disagree that it's impossible. We've already caught many of > them I think we have many, many more. > and auditing our usage of such callbacks is necessary anyway > because traditional GC might free (but not move) pointers prematurely, > resulting in subtle bugs which might not be caught for years. With stack marking, I think those bugs are largely a myth, as is proven by decades of use. I didn't have a production Emacs crash on me for the last 20 years at least. > I believe the cookies are generally of one of the following types: > > - void * > - gpointer > - LPARAM > - WPARAM? Why do you think this is all the list?