From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Pip Cet Newsgroups: gmane.emacs.devel Subject: Re: MPS: Win64 testers? Date: Fri, 23 Aug 2024 15:51:41 +0000 Message-ID: <87ed6f6xvc.fsf@protonmail.com> References: <87bk1k8mzd.fsf@protonmail.com> <861q2gs8xu.fsf@gnu.org> <877cc88j4p.fsf@protonmail.com> <86seuvre6x.fsf@gnu.org> <8734mv8yb6.fsf@protonmail.com> <8634mvqte0.fsf@gnu.org> <87o75j73ry.fsf@protonmail.com> <86v7zrpawy.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="2433"; mail-complaints-to="usenet@ciao.gmane.io" Cc: sebastian@sebasmonia.com, kien.n.quang@gmail.com, emacs-devel@gnu.org, yantar92@posteo.net To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Aug 23 17:59:49 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1shWhZ-0000Uu-5a for ged-emacs-devel@m.gmane-mx.org; Fri, 23 Aug 2024 17:59:49 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1shWgv-00030K-Kf; Fri, 23 Aug 2024 11:59:09 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1shWZr-0001xN-2K for emacs-devel@gnu.org; Fri, 23 Aug 2024 11:51:51 -0400 Original-Received: from mail-4322.protonmail.ch ([185.70.43.22]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1shWZo-0000kT-H0; Fri, 23 Aug 2024 11:51:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1724428304; x=1724687504; bh=wecRC7XsLWVfIou27NZyepUZEAGlRk9KjhYoAkgyIWk=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=a489N448kokblww35bo+UkOhn4mOtb6quDnT4OrfoTVlWEJVGEJt5fmrpldRrG3wM WpTGe0Rula+Xw77S051YVShkEkqpUEUpFDycVeQ+xHIeagWa87w4Z08K+iDLeiahfG 9aSncrRUednrNIXGafLqfxylEiM5we/x6WJ3cQk8hirHgw3dXzbYm20QG3iY8K9Hjg OxwcGF9TnBesRywR7hzlszjQ+rFVwuEMnJQufZCLsZOBLh7OcWudwfG3xiQ/oD04ge UQSlPm3rJYSv3glhijQIgDHelj4oVTndfruZ/eE+Edy0o9ynYjV8Z2HsRe3yQvNBC6 /QaaXQUz4i1JA== In-Reply-To: <86v7zrpawy.fsf@gnu.org> Feedback-ID: 112775352:user:proton X-Pm-Message-ID: 36c05056f4e48ce56407ff7ddcee3c2ca6ec2725 Received-SPF: pass client-ip=185.70.43.22; envelope-from=pipcet@protonmail.com; helo=mail-4322.protonmail.ch X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, FREEMAIL_REPLY=1, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 23 Aug 2024 11:59:07 -0400 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:323091 Archived-At: "Eli Zaretskii" writes: >> Date: Fri, 23 Aug 2024 13:44:04 +0000 >> From: Pip Cet >> Cc: sebastian@sebasmonia.com, kien.n.quang@gmail.com, emacs-devel@gnu.or= g, yantar92@posteo.net >> >> > But my bother is that we will need to similarly pin many other >> > pointers and objects, or risk segfaults. >> >> Indeed. There are quite a few changes along the same lines in the >> scratch/igc branch, and it makes me think we need a slightly more >> general infrastructure to turn a pointer (or Lisp_Object) known to Emacs >> into a void * suitable for passing as a cookie to an external library, >> and then for turning the cookie passed to a callback by that external >> library back into a pointer (or Lisp_Object) the callback (in Emacs) can >> use. >> >> > Maybe it means that MPS and similar igc systems are not suitable for >> > Emacs. >> >> I believe said infrastructure would be generally useful, because it >> allows us to catch another class of subtle bugs that may appear without >> MPS: passing a pointer to an external library which is freed by our >> traditional GC, which would cause very similar segfaults (if anything, >> much trickier to debug) without MPS. >> >> IOW, it's always worth our time to think about why something we pass as >> a cookie to an external callback cannot be garbage-collected and/or >> copied, regardless of which garbage collector is in use. The price we'd >> pay is to require such cookies to be explicitly deregistered, but we'd >> gain a very useful feature (maybe for --enable-checking, maybe enabled >> by default): a clear and loud warning informing us that a cookie is >> still registered for an object we just tried to GC. >> >> So I think this could improve Emacs in general, even for people who will >> never use MPS. > > We are not coding Emacs from scratch. Emacs is already a very large > body of code, most of it tested by time and gobs of users and use > patterns. To start rewriting too much of that means we are losing all > that hard-earned stability, because many corners in Emacs code are not > well known to those who are currently active in development and > maintenance. > > So from where I stand, it is definitely a significant problem to make > such vast changes, and the fact that it will help us eventually > doesn't help. Because if we destabilize Emacs enough, we will lose > our users, which are the only reason Emacs is still alive and kicking. I don't see how we'd destabilize the existing Emacs code by making such changes; any unconverted code would continue working as it did before without MPS, and continue not working with MPS. If your point is that it might be a long time until MPS is stable enough for very unusual configurations, that is certainly true, and affects when it's a good idea to enable it for general use. >> > EnumFontFamiliesEx (dc, &match_data->pattern, >> > (FONTENUMPROC) add_font_entity_to_list, >> > (LPARAM) match_data, 0); >> > >> > where add_font_entity_to_list is our callback which is called for >> > every font family and decides whether it matches the font spec; if so, >> > the callback adds the family to a list that is then used by the caller >> > of EnumFontFamiliesEx. >> >> This particular call is safe. 'match_data' lives on the stack, MPS >> treats the stack conservatively as pinning every object to which it >> might possibly refer, thus 'match_data->pattern' cannot be moved. > > IOW, we are lucky. Because match_data could well be a global variable > allocated off the heap, or something else. Which would be a bug both with and without MPS, as nothing protects match_data.list from GC if it doesn't live on the stack. >> >> The problem here is that 'proc' might move. One solution is to make = the >> >> cookie a pointer to memory known to MPS, which contains the actual >> >> pointer to the Lisp_Process. Note that it is not strictly necessary = for >> >> the pointer to be allocated with igc_xzalloc_ambig, we could use the >> >> _raw_exact variant. >> > >> > Why not tell MPS not to move this pointer instead? It sounds like a >> > simpler solution than the change you proposed. >> >> The change I proposed is redundant: it tells MPS not to move the >> pointer, and it uses a pointer-to-a-pointer to catch potential moves. >> Only one part of that is needed, but in both cases, we need to remember >> to un-pin/free something. > > If the change is redundant, why did you propose it? Better to solve a problem twice than not to solve it at all? I don't see a problem there, to be honest. (It's morally equivalent to writing if (p) xfree (p); even though that's redundant as well.) > More importantly, we need to have boilerplate code to pin a pointer, > because this will be needed a lot. The fact that we still don't have > it is unfortunate. I think it's better to have boilerplate code to turn a (potentially MPS-managed) pointer into a cookie, and back, and deregister it. Of course it's undesirable for pointers to be unnecessarily pinned, so it's likely we'd end up merely registering a pointer to the pointer with MPS, and making sure that is traced exactly rather than ambiguously so the object pointed to can move. >> > pinning a pointer is much simpler. Or am I missing something? I should have been clearer: if we remember to un-pin the pointer, it's no longer any simpler to pin it than it is just to use a pointer to a pointer, and the latter approach lets us know about memory leaks and is better from a GC point of view. >> Pinning it certainly is, remembering to un-pin it when we're done with >> it is the hard part. > > Then we need to figure that out, IMO. That requires knowledge of the library in question, AFAICT. To be honest, I'm not entirely sure I trust the GTK code to get it right. >> And also something we should always do, since >> forgetting to unpin it would result in memory leaks and slower GCs for >> the lifetime of the Emacs session. > > If unpinning is automatic, it won't be forgotten. But that requires knowing when the library stops using the cookie, which is all but impossible. right? >> >> There are other solutions which may be better. >> > >> > I certainly hope so, since auditing all of our sources for such >> > gotchas is an impossible job. >> >> I think I disagree that it's impossible. We've already caught many of >> them > > I think we have many, many more. We may have to clear the object files one by one, then, and warn about unaudited object files in MPS builds. >> and auditing our usage of such callbacks is necessary anyway >> because traditional GC might free (but not move) pointers prematurely, >> resulting in subtle bugs which might not be caught for years. > > With stack marking, I think those bugs are largely a myth, as is > proven by decades of use. I didn't have a production Emacs crash on > me for the last 20 years at least. I'm not sure what you're trying to get at here. Stack marking, of course, has the same protective effect with and without MPS, since MPS marks the stack conservatively. >> I believe the cookies are generally of one of the following types: >> >> - void * >> - gpointer >> - LPARAM >> - WPARAM? > > Why do you think this is all the list? I don't, why do you think I think that? Pip